Alternative database update and syncing

Reinhard Doelz doelz at comp.bioz.unibas.ch
Sat Dec 5 11:08:51 EST 1992


In the past, there was much discussion going on with respect to the 
desired mechanism of feeding updates per news and polling them via 
ftp. Both mechanisms lack the feature of synchronisation, i.e. the only 
way to be sure that you have all the database entries is to ftp the whole 
updates. Networking in the previuos years might have worked this way. 
However, if hundreds of researchers do this, we run into massive bandwith 
problems. 

I have developed a different scheme witch runs on top of the HASSLE 
protocol (Hierarchical Access System for Sequence Libraries in Europe). 
The documentation is not ready yet but the client software will be 
available, as well as the database tools. Usage is curretly only 
anticipated for the EMBL database. 

The idea is that you send a datafile to a server containing all the 
entries as a listing (for simplicity, I sticked to Peter Stoehr's 
internal routines which list entry name, accession number, version 
number, and date of entry) and the server compares this list with its 
own database. Then, a file is created which contains all the entries 
the client misses, and is sent back to the client. If there's a need 
the entries can be decomposed into single files at the client's end. 

The data transmission runs asyncronously on a special socket (port 375), 
and uses on-the-fly compression and encryption. Accounting and security 
are standard. 

Alternative mechanisms include a request to send all new entries since 
a given date, and the full daily update release. 

The software currently runs on VAX/VMS with UCX,TCPWARE or MULTINET, 
and IRIX,SunOS and ULTRIX. It was developed on a Silicon Graphics 
running the CodeVision environment. Thanks go to Lukas Rosenthaler 
for contributing the compression code, and to Basel University and the 
Swiss National Science Foundation for grant support. 

Mail me if you are interested. 

Regards 
Reinhard 

-- 
+----------------------------------+-------------------------------------+
|    Dr. Reinhard Doelz            | RFC     doelz at urz.unibas.ch         |
|      Biocomputing                | DECNET  20579::48130::doelz         |
|Biozentrum der Universitaet       | X25     022846211142036::doelz      |
|   Klingelbergstrasse 70          | FAX     x41 61 261- 6760 or 267- 2078     
|     CH 4056 Basel                | TEL     x41 61 267- 2076 or 2247    |   
+------------- bioftp.unibas.ch is the SWISS EMBnet node ----------------+
               -----------------------------------------



More information about the Embl-db mailing list