BENCHMARKS IN COMPUTATIONAL MOLECULAR BIOLOGY
Matthew Witten
mwitten at chpc.utexas.edu
Fri Dec 20 17:45:56 EST 1991
The following post organizes a number of benchmarks completed
at the University of Texas System Center For High Performance Computing
by the Computational Molecular Biology and Genetics Group.
Individuals interested in contacting this group should contact
GENTOOLS at CHPC.UTEXAS.EDU
If you are interested in more details on the actual benchmarks, contact
ANNE at CHPC.UTEXAS.EDU
We welcome comments, suggestions, etc.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1.
Software: gm (gene modeler)
Author: C.A. Soderlund
Computing Research Laboratory
New Mexico State University, Las Cruces, NM 88003-0001
Version: 1.0 (1-1-90)
Language: C
Purpose: Identification of candidate genes in anonymous,
genomic DNA sequence data.
Changes: only minor changes, so that the program runs on all
three machines. I did not try to optimize the program
to obtain better performance !
Date: 10-31-91
12-19-91 timings for Panasonic Solbourne Laptop
added.
inputfiles col-1.in, myo-2.in, unc-15.in as included in tar-file.
col-1.in : 1759 residues
myo-2.in : 10780 residues
unc-15.in : 12087 residues
|Compiler| col-1.in | myo-2.in | unc-15.in |
Machine | Option | user system | user system | user system |
--------+--------+----------------+----------------+----------------|
SPARC | -- | 1.10 0.22 | 13.82 0.70 | 9.56 0.66 |
station |-----------------------------------------------------------|
IPC | -g | 1.12 0.21 | 13.87 0.79 | 9.60 0.58 |
|-----------------------------------------------------------|
| -O | 0.75 0.23 | 9.37 0.78 | 6.13 0.71 |
|-----------------------------------------------------------|
| -O4 | 0.55 0.26 | 7.13 0.85 | 4.71 0.61 |
--------|-----------------------------------------------------------|
Pana- | -- | 1.05 0.24 | 12.01 0.82 | 8.59 0.94 |
sonic |-----------------------------------------------------------|
Sol- | -g | 1.05 0.30 | 11.97 0.97 | 8.70 0.87 |
bourne |-----------------------------------------------------------|
Laptop | -O | 0.78 0.25 | 8.82 0.89 | 6.29 0.90 |
|-----------------------------------------------------------|
| -O4 | 0.53 0.24 | 6.24 1.03 | 4.56 0.89 |
--------------------------------------------------------------------|
IBM | -- | 0.25 0.09 | 2.98 0.78 | 1.79 0.63 |
RS/6000 |-----------------------------------------------------------|
| -g | 0.19 0.17 | 2.79 0.77 | 1.76 0.61 |
|-----------------------------------------------------------|
| -O | 0.22 0.15 | 2.91 0.72 | 1.76 0.76 |
--------|-----------------------------------------------------------|
Cray | -- | 0.29 0.02 |* 4.45 0.15 |* 2.18 0.12 |
Y-MP |-----------------------------------------------------------|
8/864 | -g | 0.60 0.02 |* 7.82 0.20 |* 4.09 0.14 |
|-----------------------------------------------------------|
| -h | 0.29 0.02 |* 4.41 0.12 |* 2.19 0.10 |
|vector0 | | | |
|-----------------------------------------------------------|
| -h | 0.28 0.02 |* 4.31 0.14 |* 2.08 0.10 |
|inline3 | | | |
--------------------------------------------------------------------+
All times are measured in seconds (CPU times).
In each case the times are the best result of three runs.
Although it is looking like the -g option produces the fastest
code on IBM RS/6000, that is not true. The times vary from run
to run. In some runs the -g code is faster, sometimes the -O code
is faster. Probably they are the same.
* : you cannot compare these times against the SPARC/IBM times
because in these cases the Cray program led to different
results. (reason: Cray does not use IEEE format)
speed ups:
comparison between the versions without compiler options
on each machine and comparison between the fastest
versions on each machine (user times).
| SPARC | SPARC | Cray | Cray |
| no options | -O4 | no options | -h inline3 |
--------------------------------------------------------------|
IBM | 4.4 - 5.3 | | 1.16 | |
no opt. | | | | |
--------------------------------------------------------------|
IBM | | 2.6-2.9 | | 1.47 |
-g | | | | |
--------------------------------------------------------------|
Cray | 3.8 | | | |
no opt. | | | | |
--------------------------------------------------------------|
Cray | | 2.0 | | |
inline3 | | | | |
--------------------------------------------------------------+
2.
Software: blastn, tblastn (sequential versions)
Author: Dr. Stephen Altschul, et. al
National Center for Biotechnology Information
National Library of Medicine
Bethesda, MD 20894-0001
Versions: blastn: 1.1.21 (9-27-91)
tblastn: 1.1.19 (9-19-91)
Language: C
Purpose: compare query sequences against databases.
Changes: no changes. I did not try to optimize the program
to obtain better performance.
Date: 10-9-91
12-20-91 timings for database gbinv.wrp (GenBank,
release 68) added, including timings for
Panasonic Solbourne Laptop.
all times measured in seconds (CPU time).
program blastn :
----------------
query sequences used: col-1.nt, myo-2.nt, unc-15.nt
(nucleotide sequences as used in gm in col-1.in, myo-2.in
and unc-15.in).
databases used:
1. gbinv.wrp (GenBank, rel.64, contains the query sequences )
2. gbpri.wrp (GenBank, rel.64, largest gb64 file)
3. gbinv.wrp (GenBank, rel.68, contains the query sequences)
1. database gbinv.wrp:
|Compiler| col-1.nt | myo-2.nt | unc-15.nt |
Machine | Option | user system | user system | user system |
--------+--------+----------------+----------------+----------------|
SPARC | -O4 | 3.68 2.26 | 8.76 2.10 | 11.03 2.48 |
station | | | | |
IPC | | | | |
--------------------------------------------------------------------|
IBM | -O | 1.45 0.28 | 3.05 0.22 | 3.93 0.27 |
RS/6000 | | | | |
--------|-----------------------------------------------------------|
Cray | -O | 1:46 0.28 | 4.48 0.92 | 5.73 0.79 |
Y-MP | | | | |
8/864 | | | | |
--------+-----------------------------------------------------------+
speed ups:
IBM vs. Sparc : 2.54 - 2.87
Cray vs. Sparc : 1.92 - 2.52
IBM vs. Cray : 1.01 - 1.47
2. database gbpri.wrp:
|Compiler| col-1.nt | myo-2.nt | unc-15.nt |
Machine | Option | user system | user system | user system |
--------+--------+----------------+----------------+----------------|
SPARC | -O4 | 6.68 3.23 | 13.23 2.23 | 17.71 4.38 |
station | | | | |
IPC | | | | |
--------|-----------------------------------------------------------|
IBM | -O | 2.60 0.36 | 4.74 0.25 | 6.55 0.36 |
RS/6000 | | | | |
--------|-----------------------------------------------------------|
Cray | -O | 3.11 1.01 | 6.63 0.83 | 9.03 2.05 |
Y-MP | | | | |
8/864 | | | | |
--------+-----------------------------------------------------------+
speed ups:
IBM vs. Sparc : 2.57 - 2.79
Cray vs. Sparc : 1.96 - 2.15
IBM vs. Cray : 1.12 - 1.40
3. database gbinv.wrp (release 68):
|Compiler| col-1.nt | myo-2.nt | unc-15.nt |
Machine | Option | user system | user system | user system |
--------+--------+----------------+----------------+----------------|
SPARC | -O4 | 3.53 1.38 | 7.78 1.63 | 10.06 1.71 |
station | | | | |
IPC | | | | |
--------|-----------------------------------------------------------|
Pana- | -O4 | 3.40 1.23 | 6.90 1.16 | 8.68 1.70 |
sonic | | | | |
Solbourne Laptop | | | |
--------|-----------------------------------------------------------|
IBM | -O | 1.43 0.22 | 3.24 0.27 | 4.24 0.31 |
RS/6000 | | | | |
--------|-----------------------------------------------------------|
Cray | -O | 1.77 0.48 | 4.24 0.66 | 5.49 0.61 |
Y-MP | | | | |
8/864 | | | | |
--------+-----------------------------------------------------------+
These times are the best results of several runs. The
variation was relatively high, e.g. col-1.nt on IBM
1.43 (best) - 1.64 (worst) or unc-15.nt on Panasonic
8.68 (best) - 8.93 (worst).
program tblastn :
----------------
query sequences used: col-1.aa.g1, myo-2.aa.g1, unc-15.aa.g1
(genes obtained by program gm)
databases used:
1. gbinv.wrp (GenBank, rel.64, contains the query sequences )
2. gbpri.wrp (GenBank, rel.64, largest gb64 file)
3. gbinv.wrp (GenBank, rel.68, contains the query sequences)
1. database gbinv.wrp:
|Compiler| col-1.nt | myo-2.nt | unc-15.nt |
Machine | Option | user system | user system | user system |
--------+--------+----------------+----------------+----------------|
SPARC | -O4 | 87.85 5.30 | 314.06 4.11 | 139.51 3.48 |
station | | | | |
IPC | | | | |
--------------------------------------------------------------------|
IBM | -O | 36.90 0.44 | 141.90 0.34 | 61.26 0.31 |
RS/6000 | | | | |
--------|-----------------------------------------------------------|
Cray | -O | 61.19 2.69 | 224.86 11.18 | 99.32 4.15 |
Y-MP | | | | |
8/864 | | | | |
--------+-----------------------------------------------------------+
speed ups:
IBM vs. Sparc : 2.21 - 2.38
Cray vs. Sparc : 1.40 - 1.44
IBM vs. Cray : 1.58 - 1.66
2. database gbpri.wrp:
|Compiler| col-1.nt | myo-2.nt | unc-15.nt |
Machine | Option | user system | user system | user system |
--------+--------+----------------+----------------+----------------|
SPARC | -O4 | 334.50 3.48 | 648.28 5.76 | 281.20 4.65 |
station | | | | |
IPC | | | | |
--------------------------------------------------------------------|
IBM | -O | 141.46 1.30 | 280.20 0.40 | 118.67 0.40 |
RS/6000 | | | | |
--------|-----------------------------------------------------------|
Cray | -O | 231.25 12.36 | 445.25 20.85 | 195.51 19.29 |
Y-MP | | | | |
8/864 | | | | |
--------+-----------------------------------------------------------+
speed ups:
IBM vs. Sparc : 2.31 - 2.37
Cray vs. Sparc : 1.44 - 1.50
IBM vs. Cray : 1.59 - 1.65
3. database gbinv.wrp (release 68):
|Compiler| col-1.nt | myo-2.nt | unc-15.nt |
Machine | Option | user system | user system | user system |
--------+--------+----------------+----------------+----------------|
SPARC | -O4 | 90.96 4.03 | 328.60 2.30 | 144.56 2.26 |
station | | | | |
IPC | | | | |
--------|-----------------------------------------------------------|
Pana- | -O4 | 83.26 3.18 | 295.91 2.03 | 128.73 2.23 |
sonic | | | | |
Solbourne Laptop | | | |
--------|-----------------------------------------------------------|
IBM | -O | 37.07 0.52 | 143.76 0.36 | 61.53 0.22 |
RS/6000 | | | | |
--------|-----------------------------------------------------------|
Cray | -O | 60.80 2.34 | 222.80 8.65 | 98.45 12.95 |
Y-MP | | | | |
8/864 | | | | |
--------+-----------------------------------------------------------+
3.
Software: blastp, blastn, blastx, tblastn, blast3 (parallel versions)
Author: Dr. Stephen Altschul, et. al
National Center for Biotechnology Information
National Library of Medicine
Bethesda, MD 20894-0001
Versions: blastp: 1.1.22MP (9-25-91)
blastn: 1.1.21MP (9-27-91)
blastx: 1.1.20MP (9-25-91)
tblastn: 1.1.19MP (9-19-91)
blast3: 1.1.18MP (9-25-91)
Language: C
Purpose: compare query sequences against databases.
Changes: Cray compiler directives added instead of Silicon
Graphics parallel instructions. goto instructions
replaced by other constructs, because they prevent
tasking. No performance optimization.
Date: 10-30-91 - 11-5-91
all times are user times, measured in seconds
The value Concurrent CPUs (Avg.) is not a speedup factor, but
rather an indication of average overlap. It is probably close
to the speedup factor, which may be a little bit higher.
program blastn :
----------------
query sequence used: unc-15.nt
database used: gbpri.wrp (GenBank, release 68)
sequential program: 8.09
parallel program, 1 processor used: 8.37
part before parallel region: <0.3
part after parallel region: <0.03
results obtained by run in parallel queue,
using the dedcpu command, 4 cpus:
(Concurrent CPUs * Connect seconds = CPU seconds)
--------------- --------------- -----------
1 * 0.3277 = 0.3277
2 * 0.8394 = 1.6788
3 * 0.9866 = 2.9599
4 * 0.8549 = 3.4195
(Concurrent CPUs * Connect seconds = CPU seconds)
(Avg.) (total) (total)
--------------- -------------- -----------
2.79 * 3.0086 = 8.3859
program tblastn :
----------------
query sequence used: col-1.aa.g1
database used: gbinv.wrp (GenBank, release 68)
sequential program: 58.73
parallel program, 1 processor used: 59.35
part before parallel region: <0.2
part after parallel region: <0.1
results obtained by run in parallel queue,
using the dedcpu command, 4 cpus:
(Concurrent CPUs * Connect seconds = CPU seconds)
--------------- --------------- -----------
1 * 0.2713 = 0.2713
2 * 1.7599 = 3.5198
3 * 4.3181 = 12.9543
4 * 10.7542 = 43.0169
(Concurrent CPUs * Connect seconds = CPU seconds)
(Avg.) (total) (total)
--------------- -------------- -----------
3.49 * 17.1036 = 59.7624
program blastp :
----------------
query sequence used: col-1.aa.g1
database used: swiss-prot_all.wrp (Swiss-Prot, release 14)
sequential program: 46.86
parallel program, 1 processor used: 47.53
part before parallel region: <0.2
part after parallel region: <0.1
results obtained by run in parallel queue,
using the dedcpu command, 4 cpus:
(Concurrent CPUs * Connect seconds = CPU seconds)
--------------- --------------- -----------
1 * 0.2604 = 0.2604
2 * 4.8022 = 9.6043
3 * 2.3360 = 7.0081
4 * 7.6686 = 30.6743
(Concurrent CPUs * Connect seconds = CPU seconds)
(Avg.) (total) (total)
--------------- -------------- -----------
3.16 * 15.0672 = 47.5471
program blastx :
----------------
query sequence used: col-1.nt
database used: swiss-prot_all.wrp (Swiss-Prot, release 14)
sequential program: 224.57
parallel program, 1 processor used: 226.72
part before parallel region: <0.9
part after parallel region: <0.1
results obtained by run in parallel queue,
using the dedcpu command, 4 cpus:
(Concurrent CPUs * Connect seconds = CPU seconds)
--------------- --------------- -----------
1 * 1.2393 = 1.2393
2 * 3.6862 = 7.3725
3 * 8.1599 = 24.4798
4 * 48.5807 = 194.3226
(Concurrent CPUs * Connect seconds = CPU seconds)
(Avg.) (total) (total)
--------------- -------------- -----------
3.69 * 61.6662 = 227.4143
program blast3 :
----------------
query sequence used: col-1.aa.g1
database used: swiss-prot_all.wrp (Swiss-Prot, release 14)
sequential program: 172.78
parallel program, 1 processor used: 177.67
part before 1. parallel region: 0.24
part between parallel regions: 0.43
part after 2. parallel region: 0.22
results obtained by run in parallel queue,
using the dedcpu command, 4 cpus:
(Concurrent CPUs * Connect seconds = CPU seconds)
--------------- --------------- -----------
1 * 1.0258 = 1.0258
2 * 5.5568 = 11.1135
3 * 6.4635 = 19.3906
4 * 36.9139 = 147.6557
(Concurrent CPUs * Connect seconds = CPU seconds)
(Avg.) (total) (total)
--------------- -------------- -----------
3.59 * 49.9600 = 179.1856
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
_____________________________________________________________________
Matthew Witten, Ph.D. MED
Director, Applications Research & Development
Associate Director,
UT System Center For High Performance Computing
Balcones Research Center, 1.154 CMS
10100 Burnet Road, Austin, TX 78758-4497 USA
Phone: (512) 471-2472 FAX: (512) 471-2445
E-MAIL MWITTEN at CHPC.UTEXAS.EDU
or
MWITTEN at UTHERMES.BITNET
"some intellectuals make their living by creating
obscurities for the rest of us to puzzle over"
_____________________________________________________________________
More information about the Bioforum
mailing list