BENCHMARKS IN COMPUTATIONAL MOLECULAR BIOLOGY

Matthew Witten mwitten at chpc.utexas.edu
Fri Dec 20 17:45:56 EST 1991


The following post organizes a number of benchmarks completed
at the University of Texas System Center For High Performance Computing
by the Computational Molecular Biology and Genetics Group. 

Individuals interested in contacting this group should contact

               GENTOOLS at CHPC.UTEXAS.EDU

If you are interested in more details on the actual benchmarks, contact

               ANNE at CHPC.UTEXAS.EDU  

We welcome comments, suggestions, etc.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

1.

Software:   gm (gene modeler)
Author:     C.A. Soderlund
            Computing Research Laboratory
            New Mexico State University, Las Cruces, NM 88003-0001
Version:    1.0 (1-1-90)
Language:   C
Purpose:    Identification of candidate genes in anonymous, 
            genomic DNA sequence data.
Changes:    only minor changes, so that the program runs on all
            three machines. I did not try to optimize the program
            to obtain better performance !
Date:       10-31-91
            12-19-91  timings for Panasonic Solbourne Laptop
                      added.


inputfiles col-1.in, myo-2.in, unc-15.in as included in tar-file.

col-1.in  :  1759 residues
myo-2.in  : 10780 residues
unc-15.in : 12087 residues



        |Compiler|    col-1.in    |    myo-2.in    |   unc-15.in    |
Machine | Option | user    system | user    system | user    system |
--------+--------+----------------+----------------+----------------|
SPARC   |  --    |   1.10    0.22 |  13.82    0.70 |   9.56    0.66 | 
station |-----------------------------------------------------------| 
IPC     |  -g    |   1.12    0.21 |  13.87    0.79 |   9.60    0.58 | 
        |-----------------------------------------------------------|
        |  -O    |   0.75    0.23 |   9.37    0.78 |   6.13    0.71 |
        |-----------------------------------------------------------|
        |  -O4   |   0.55    0.26 |   7.13    0.85 |   4.71    0.61 |
--------|-----------------------------------------------------------|
Pana-   |  --    |   1.05    0.24 |  12.01    0.82 |   8.59    0.94 | 
sonic   |-----------------------------------------------------------| 
Sol-    |  -g    |   1.05    0.30 |  11.97    0.97 |   8.70    0.87 | 
bourne  |-----------------------------------------------------------|
Laptop  |  -O    |   0.78    0.25 |   8.82    0.89 |   6.29    0.90 |
        |-----------------------------------------------------------|
        |  -O4   |   0.53    0.24 |   6.24    1.03 |   4.56    0.89 |
--------------------------------------------------------------------|
IBM     |  --    |   0.25    0.09 |   2.98    0.78 |   1.79    0.63 |
RS/6000 |-----------------------------------------------------------|
        |  -g    |   0.19    0.17 |   2.79    0.77 |   1.76    0.61 |
        |-----------------------------------------------------------|
        |  -O    |   0.22    0.15 |   2.91    0.72 |   1.76    0.76 |
--------|-----------------------------------------------------------|
Cray    |  --    |   0.29    0.02 |*  4.45    0.15 |*  2.18    0.12 |
Y-MP    |-----------------------------------------------------------|
8/864   |  -g    |   0.60    0.02 |*  7.82    0.20 |*  4.09    0.14 |
        |-----------------------------------------------------------|
        |  -h    |   0.29    0.02 |*  4.41    0.12 |*  2.19    0.10 |
        |vector0 |                |                |                |
        |-----------------------------------------------------------|
        |  -h    |   0.28    0.02 |*  4.31    0.14 |*  2.08    0.10 |
        |inline3 |                |                |                |
--------------------------------------------------------------------+


All times are measured in seconds (CPU times).
In each case the times are the best result of three runs.

Although it is looking like the -g option produces the fastest 
code on IBM RS/6000, that is not true. The times vary from run
to run. In some runs the -g code is faster, sometimes the -O code
is faster. Probably they are the same.

* : you cannot compare these times against the SPARC/IBM times
    because in these cases the Cray program led to different
    results. (reason: Cray does not use IEEE format)


speed ups:
        comparison between the versions without compiler options
        on each machine and comparison between the fastest 
        versions on each machine (user times). 

        |   SPARC     |  SPARC   |    Cray     |    Cray      |
        | no options  |   -O4    | no options  |  -h inline3  |
--------------------------------------------------------------|
IBM     | 4.4 - 5.3   |          |   1.16      |              |
no opt. |             |          |             |              |
--------------------------------------------------------------|
IBM     |             | 2.6-2.9  |             |     1.47     |
-g      |             |          |             |              |
--------------------------------------------------------------|
Cray    |     3.8     |          |             |              |
no opt. |             |          |             |              |
--------------------------------------------------------------|
Cray    |             |    2.0   |             |              |
inline3 |             |          |             |              |
--------------------------------------------------------------+

2.

Software:   blastn, tblastn  (sequential versions)
Author:     Dr. Stephen Altschul, et. al
            National Center for Biotechnology Information
            National Library of Medicine
            Bethesda, MD 20894-0001                               
Versions:   blastn: 1.1.21 (9-27-91)
            tblastn: 1.1.19 (9-19-91)
Language:   C
Purpose:    compare query sequences against databases.          
Changes:    no changes. I did not try to optimize the program
            to obtain better performance.
Date:       10-9-91
            12-20-91 timings for database gbinv.wrp (GenBank,
                     release 68) added, including timings for
                     Panasonic Solbourne Laptop.


all times measured in seconds (CPU time).
                                        

program blastn :
----------------
        query sequences used: col-1.nt, myo-2.nt, unc-15.nt
        (nucleotide sequences as used in gm in col-1.in, myo-2.in
        and unc-15.in).
        databases used: 
        1. gbinv.wrp (GenBank, rel.64, contains the query sequences )
        2. gbpri.wrp (GenBank, rel.64, largest gb64 file)
        3. gbinv.wrp (GenBank, rel.68, contains the query sequences)


1. database gbinv.wrp:

        |Compiler|    col-1.nt    |    myo-2.nt    |   unc-15.nt    |
Machine | Option | user    system | user    system | user    system |
--------+--------+----------------+----------------+----------------|
SPARC   |  -O4   |   3.68    2.26 |   8.76    2.10 |  11.03    2.48 | 
station |        |                |                |                | 
IPC     |        |                |                |                | 
--------------------------------------------------------------------|
IBM     |  -O    |   1.45    0.28 |   3.05    0.22 |   3.93    0.27 |
RS/6000 |        |                |                |                |
--------|-----------------------------------------------------------|
Cray    |  -O    |   1:46    0.28 |   4.48    0.92 |   5.73    0.79 |
Y-MP    |        |                |                |                |
8/864   |        |                |                |                |
--------+-----------------------------------------------------------+

speed ups:

      IBM  vs. Sparc : 2.54 - 2.87
      Cray vs. Sparc : 1.92 - 2.52
      IBM  vs. Cray  : 1.01 - 1.47


2. database gbpri.wrp:

        |Compiler|    col-1.nt    |    myo-2.nt    |   unc-15.nt    |
Machine | Option | user    system | user    system | user    system |
--------+--------+----------------+----------------+----------------|
SPARC   |  -O4   |   6.68    3.23 |  13.23    2.23 |  17.71    4.38 | 
station |        |                |                |                | 
IPC     |        |                |                |                | 
--------|-----------------------------------------------------------|
IBM     |  -O    |   2.60    0.36 |   4.74    0.25 |   6.55    0.36 |
RS/6000 |        |                |                |                |
--------|-----------------------------------------------------------|
Cray    |  -O    |   3.11    1.01 |   6.63    0.83 |   9.03    2.05 |
Y-MP    |        |                |                |                |
8/864   |        |                |                |                |
--------+-----------------------------------------------------------+

speed ups:

      IBM  vs. Sparc : 2.57 - 2.79
      Cray vs. Sparc : 1.96 - 2.15
      IBM  vs. Cray  : 1.12 - 1.40  


3. database gbinv.wrp (release 68):

        |Compiler|    col-1.nt    |    myo-2.nt    |   unc-15.nt    |
Machine | Option | user    system | user    system | user    system |
--------+--------+----------------+----------------+----------------|
SPARC   |  -O4   |   3.53    1.38 |   7.78    1.63 |  10.06    1.71 | 
station |        |                |                |                | 
IPC     |        |                |                |                | 
--------|-----------------------------------------------------------|
Pana-   |  -O4   |   3.40    1.23 |   6.90    1.16 |   8.68    1.70 |
sonic   |        |                |                |                |
Solbourne Laptop |                |                |                |
--------|-----------------------------------------------------------|
IBM     |  -O    |   1.43    0.22 |   3.24    0.27 |   4.24    0.31 |
RS/6000 |        |                |                |                |
--------|-----------------------------------------------------------|
Cray    |  -O    |   1.77    0.48 |   4.24    0.66 |   5.49    0.61 |
Y-MP    |        |                |                |                |
8/864   |        |                |                |                |
--------+-----------------------------------------------------------+

        These times are the best results of several runs. The 
        variation was relatively high, e.g. col-1.nt on IBM
        1.43 (best) - 1.64 (worst) or unc-15.nt on Panasonic
        8.68 (best) - 8.93 (worst).


program tblastn :
----------------
        query sequences used: col-1.aa.g1, myo-2.aa.g1, unc-15.aa.g1
        (genes obtained by program gm)
        databases used: 
        1. gbinv.wrp (GenBank, rel.64, contains the query sequences )
        2. gbpri.wrp (GenBank, rel.64, largest gb64 file)
        3. gbinv.wrp (GenBank, rel.68, contains the query sequences)


1. database gbinv.wrp:

        |Compiler|    col-1.nt    |    myo-2.nt    |   unc-15.nt    |
Machine | Option | user    system | user    system | user    system |
--------+--------+----------------+----------------+----------------|
SPARC   |  -O4   |  87.85    5.30 | 314.06    4.11 | 139.51    3.48 | 
station |        |                |                |                | 
IPC     |        |                |                |                | 
--------------------------------------------------------------------|
IBM     |  -O    |  36.90    0.44 | 141.90    0.34 |  61.26    0.31 |
RS/6000 |        |                |                |                |
--------|-----------------------------------------------------------|
Cray    |  -O    |  61.19    2.69 | 224.86   11.18 |  99.32    4.15 |
Y-MP    |        |                |                |                |
8/864   |        |                |                |                |
--------+-----------------------------------------------------------+

speed ups:

      IBM  vs. Sparc : 2.21 - 2.38
      Cray vs. Sparc : 1.40 - 1.44
      IBM  vs. Cray  : 1.58 - 1.66  


2. database gbpri.wrp:

        |Compiler|    col-1.nt    |    myo-2.nt    |   unc-15.nt    |
Machine | Option | user    system | user    system | user    system |
--------+--------+----------------+----------------+----------------|
SPARC   |  -O4   | 334.50    3.48 | 648.28    5.76 | 281.20    4.65 | 
station |        |                |                |                | 
IPC     |        |                |                |                | 
--------------------------------------------------------------------|
IBM     |  -O    | 141.46    1.30 | 280.20    0.40 | 118.67    0.40 |
RS/6000 |        |                |                |                |
--------|-----------------------------------------------------------|
Cray    |  -O    | 231.25   12.36 | 445.25   20.85 | 195.51   19.29 |
Y-MP    |        |                |                |                |
8/864   |        |                |                |                |
--------+-----------------------------------------------------------+

speed ups:

      IBM  vs. Sparc : 2.31 - 2.37
      Cray vs. Sparc : 1.44 - 1.50
      IBM  vs. Cray  : 1.59 - 1.65  


3. database gbinv.wrp (release 68):

        |Compiler|    col-1.nt    |    myo-2.nt    |   unc-15.nt    |
Machine | Option | user    system | user    system | user    system |
--------+--------+----------------+----------------+----------------|
SPARC   |  -O4   |  90.96    4.03 | 328.60    2.30 | 144.56    2.26 | 
station |        |                |                |                | 
IPC     |        |                |                |                | 
--------|-----------------------------------------------------------|
Pana-   |  -O4   |  83.26    3.18 | 295.91    2.03 | 128.73    2.23 |
sonic   |        |                |                |                |
Solbourne Laptop |                |                |                |
--------|-----------------------------------------------------------|
IBM     |  -O    |  37.07    0.52 | 143.76    0.36 |  61.53    0.22 |
RS/6000 |        |                |                |                |
--------|-----------------------------------------------------------|
Cray    |  -O    |  60.80    2.34 | 222.80    8.65 |  98.45   12.95 |
Y-MP    |        |                |                |                |
8/864   |        |                |                |                |
--------+-----------------------------------------------------------+

3.

Software:   blastp, blastn, blastx, tblastn, blast3 (parallel versions)
Author:     Dr. Stephen Altschul, et. al
            National Center for Biotechnology Information
            National Library of Medicine
            Bethesda, MD 20894-0001                               
Versions:   blastp: 1.1.22MP (9-25-91)
            blastn: 1.1.21MP (9-27-91)
            blastx: 1.1.20MP (9-25-91)
            tblastn: 1.1.19MP (9-19-91)
            blast3: 1.1.18MP (9-25-91)
Language:   C
Purpose:    compare query sequences against databases.          
Changes:    Cray compiler directives added instead of Silicon
            Graphics parallel instructions. goto instructions
            replaced by other constructs, because they prevent
            tasking. No performance optimization.
Date:       10-30-91 - 11-5-91


all times are user times, measured in seconds

The value Concurrent CPUs (Avg.) is not a speedup factor, but
rather an indication of average overlap. It is probably close
to the speedup factor, which may be a little bit higher.


program blastn :
----------------
        query sequence used: unc-15.nt  
        database used: gbpri.wrp (GenBank, release 68)


        sequential program:                      8.09
        parallel program, 1 processor used:      8.37 

        part before parallel region:            <0.3
        part after parallel region:             <0.03

        results obtained by run in parallel queue,
        using the dedcpu command, 4 cpus:

        (Concurrent CPUs * Connect seconds = CPU seconds)
         ---------------   ---------------   -----------
    
                       1 *          0.3277 =      0.3277
                       2 *          0.8394 =      1.6788
                       3 *          0.9866 =      2.9599
                       4 *          0.8549 =      3.4195

        (Concurrent CPUs * Connect seconds = CPU seconds)
              (Avg.)           (total)         (total)
         ---------------      --------------   -----------

                    2.79 *          3.0086 =      8.3859



program tblastn :
----------------
        query sequence used: col-1.aa.g1
        database used: gbinv.wrp (GenBank, release 68)


        sequential program:                     58.73
        parallel program, 1 processor used:     59.35 

        part before parallel region:            <0.2
        part after parallel region:             <0.1

        results obtained by run in parallel queue,
        using the dedcpu command, 4 cpus:

        (Concurrent CPUs * Connect seconds = CPU seconds)
         ---------------   ---------------   -----------

                       1 *          0.2713 =      0.2713
                       2 *          1.7599 =      3.5198
                       3 *          4.3181 =     12.9543
                       4 *         10.7542 =     43.0169

        (Concurrent CPUs * Connect seconds = CPU seconds)
              (Avg.)           (total)         (total)
         ---------------      --------------   -----------

                    3.49 *         17.1036 =     59.7624



program blastp :
----------------
        query sequence used: col-1.aa.g1
        database used: swiss-prot_all.wrp (Swiss-Prot, release 14)


        sequential program:                     46.86
        parallel program, 1 processor used:     47.53 

        part before parallel region:            <0.2
        part after parallel region:             <0.1

        results obtained by run in parallel queue,
        using the dedcpu command, 4 cpus:

        (Concurrent CPUs * Connect seconds = CPU seconds)
         ---------------   ---------------   -----------

                       1 *          0.2604 =      0.2604
                       2 *          4.8022 =      9.6043
                       3 *          2.3360 =      7.0081
                       4 *          7.6686 =     30.6743

        (Concurrent CPUs * Connect seconds = CPU seconds)
              (Avg.)           (total)         (total)
         ---------------      --------------   -----------

                    3.16 *         15.0672 =     47.5471



program blastx :
----------------
        query sequence used: col-1.nt   
        database used: swiss-prot_all.wrp (Swiss-Prot, release 14)


        sequential program:                    224.57
        parallel program, 1 processor used:    226.72 

        part before parallel region:            <0.9    
        part after parallel region:             <0.1      

        results obtained by run in parallel queue,
        using the dedcpu command, 4 cpus:


        (Concurrent CPUs * Connect seconds = CPU seconds)
         ---------------   ---------------   -----------

                       1 *          1.2393 =      1.2393
                       2 *          3.6862 =      7.3725
                       3 *          8.1599 =     24.4798
                       4 *         48.5807 =    194.3226

        (Concurrent CPUs * Connect seconds = CPU seconds)
              (Avg.)           (total)         (total)
         ---------------      --------------   -----------

                    3.69 *         61.6662 =    227.4143




program blast3 :
----------------
        query sequence used: col-1.aa.g1
        database used: swiss-prot_all.wrp (Swiss-Prot, release 14)


        sequential program:                    172.78
        parallel program, 1 processor used:    177.67 

        part before 1. parallel region:          0.24
        part between parallel regions:           0.43
        part after 2. parallel region:           0.22  

        results obtained by run in parallel queue,
        using the dedcpu command, 4 cpus:

        (Concurrent CPUs * Connect seconds = CPU seconds)
         ---------------   ---------------   -----------

                       1 *          1.0258 =      1.0258
                       2 *          5.5568 =     11.1135
                       3 *          6.4635 =     19.3906
                       4 *         36.9139 =    147.6557

        (Concurrent CPUs * Connect seconds = CPU seconds)
             (Avg.)           (total)         (total)
         ---------------      --------------   -----------

                    3.59 *         49.9600 =    179.1856


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

_____________________________________________________________________

Matthew Witten, Ph.D. MED
Director, Applications Research & Development
Associate Director, 
UT System Center For High Performance Computing
Balcones Research Center, 1.154 CMS
10100 Burnet Road, Austin, TX 78758-4497 USA

Phone: (512) 471-2472  FAX: (512) 471-2445  

E-MAIL  MWITTEN at CHPC.UTEXAS.EDU
      or
        MWITTEN at UTHERMES.BITNET

"some intellectuals make their living by creating 
obscurities for the rest of us to puzzle over"
_____________________________________________________________________



More information about the Bioforum mailing list