IUBio Biosequences .. Software .. Molbio soft .. Network News .. FTP

No subject

Sun Apr 10 21:06:40 EST 2005

aspects of classification by neural networks, including links 
between neural networks and Bayesian statistical classification, 
incremental learning,...
The project includes theoretical work on classification algorithms, 
simulations and benchmarks, especially on realistic industrial
data. Hardware implementation, especially VLSI option, is the 
last objective. 

The set of databases available is to be used for tests and benchmarks 
of machine-learning classification algorithms.
The databases are splitted into two parts: ARTIFICIALly generated
databases, mainly used for preliminary tests, and REAL ones, used for
objective benchmarks and comparisons of methods.

The choice of the databases has been guided by various parameters, such
as availability of published results concerning conventional
classification algorithms, size of the database, number of attributes,
number of classes, overlapping between classes and non-linearities of
the borders,...  Results of PCA and DFA preprocessing of the REAL
databases are also included, together with several measures useful for
the databases characterization (statistics, fractal dimension,

All these databases and their preprocessing are available together
with a postcript technical report describing in details the different
databases ('Databases.ps.Z' - 45 pages - 777781 bytes) and a report
related to the comparative benchmarking studies of various algorithms
('Benchmarks.ps.Z' - 113 pages - 1927571 bytes) well-known by the
Statistical and Neural Network communities (MLP, RCE, LVQ, k_NN, GQC)
or developped in the framework of the Elena project (IRVQ, PLS).

A LaTeX bibfile containing more than 90 entries corresponding to
the Elena partners bibliography related to the project is also
available ('Elena.bib') in the same directory.  

All files are available by anonymous ftp from the following directory:


The databases are splitted into two parts: the 'ARTIFICIAL' ones, being
generated in order to obtain some defined characteristics, and for
which the theoretical Bayes error can be computed,  and the 'REAL'
ones, collected in existing real-world applications.

The ARTIFICIAL databases ('Gaussian', 'Clouds' and 'Concentric') 
were generated according to the following requirements:   
  - heavy intersection of the class distributions,
  - high degree of nonlinearity of the class boundaries,
  - various dimensions of the vectors,
  - already published results on these databases.     
They are restricted to two-class problems, since we believe it yield 
answers to the most essential questions.  
The ARTIFICIAL databases are mainly used for rapid test purposes on newly 
developed algorithms.

The REAL databases ('Satimage', 'Texture', 'Iris' and 'Phoneme') were 
selected according to the following requirements:
  - classical databases in the field of classification (Iris),
  - already published results on these databases (Phoneme, 
      from the ROARS ESPRIT project and 'Satimage' from the STATLOG ESPRIT 
  - various dimensions of the vectors,
  - sufficient number of vectors (to avoid the ``empty space phenomenon'').
  - the 'Texture' database, generated at INPG for the Elena project is 
      interesting for its high number of classes (11). 


				# DETAILS #

The 'Benchmarks' technical report

The 'Benchmarks.ps' Elena report is related to the benchmarking studies of
various classifiers.  Most of the classifiers which were used for the
benchmark comparative studies are are well known by the neural network
and machine learning community.  These are the k-Nearest Neighbour
(k_NN) classifier, selected for its powerful probability density
estimation properties; the Gaussian Quadratic Classifier (GQC), the
most classical statistical parametric simple classification method; the
Learning Vector Quantizer (LVQ), a powerful non-linear iterative
learning algorithm proposed by Kohonen; the Reduced Coulomb Energy
(RCE) algorithm, an incremental Region Of Influence algorithm; the
Inertia Rated Vector Quantizer (IRVQ) and the Piecewise Linear
Separation (PLS) classifiers, developed in the framework of the Elena
The main objectives of the 'Benchmarks.ps' Elena report report are the 
- to provide an overall comprehensive view of the general problem of
    comparative benchmarking studies and to propose a useful common 
    test basis for existing and further classification methods,
- to obtain objective comparisons of the different chosen classifiers on 
    the set of databases described in this report (each classifier being 
    used with its optimal configuration for each particular database),
- to study the possible links between the data structures of the databases 
    viewed by some parameters, and the behavior of the studied classifiers 
    (mainly the evolution of their the optimal configuration parameters).
- to study the links between the preprocessing methods and the 
    classification algorithms from the performances and hardware constraints 
    point of view (especially the computation times and memory requirements).

Databases format

All the databases available are in the following format (after decompression) :

 - All files containing the databases are stored as ASCII files for
    their easy edition and checking. 
 - In a file, each of the n lines is reserved for each vectorial sample
    (instance) and each line consists  of d floating-point numbers (the
    attributes) followed  by the class label (which must be an integer).


 1.51768 12.65 3.56 1.30 73.08 0.61 8.69 0.00 0.14 1
 1.51747 12.84 3.50 1.14 73.27 0.56 8.55 0.00 0.00 0
 1.51775 12.85 3.48 1.23 72.97 0.61 8.56 0.09 0.22 1
 1.51753 12.57 3.47 1.38 73.39 0.60 8.55 0.00 0.06 1
 1.51783 12.69 3.54 1.34 72.95 0.57 8.75 0.00 0.00 3
 1.51567 13.29 3.45 1.21 72.74 0.56 8.57 0.00 0.00 1
 There are NO missing values. 
If you desire to get a database, you MUST do it in ftp the binary mode. 
So if you aren't in this mode, simply type 'binary' at the ftp prompt.

         EXAMPLE: to get the "phoneme" database :

                      cd REAL
                      cd phoneme
                      get phoneme.txt
                      get phoneme.dat.Z
                      get ...
                      cd ...
          After your ftp session, you simply have to type 
                'uncompress phoneme.dat.Z' 
          to get the uncompressed datafile.

 Contents of the 'ARTIFICIAL' directory 

  The databases of this directory contain only the 'ARTIFICIAL'
  classification problems.
  The present 'ARTIFICIAL' databases are only two-class problems, since it 
  yields answers to the most essential questions. 
  For each problem, the confusion matrix corresponding to the theoretical 
  Bayes boundary is provided with the confusion matrix obtained by a k_NN 
  classifier (k chosen to reach the minimum of the total Leave-One-Out error).

  These databases were selected to use for preliminary test and to study the   
  behavior of the implemented algorithms for some particular problems:

  - Overlapping classes: 
     The classifier should have the ability to form a decision boundary 
     that minimizes the amount of misclassification for all of the overlapping

  - Nonlinear separability:
     The classifier should be able to build decision regions that separate
     classes of any shape and size. 

  There is one subdirectory for each database. In this subdirectory, 
  there is :
  - A text file providing detailed information about the related database     

  - The compressed database ('databasename.dat.Z).
    The different patterns of each database are presented in a random order.

  - For bidimensional databases, a postscript file representing the 2-D
    datasets (those files are in eps format).

  For each subdirectory, the directoryname is the same as the name chosen
  for the concerned database.  Here are the directorynames with a brief

  - 'clouds'

    Bidimensional distributions : the class 0 is the sum of three different
    normal distributions while the the class 1 is another normal, overlapping 
    the class 0.
      5000 patterns, 2500 in each class.
    This allows the study of the classifier behavior for heavy intersection 
    of the class distributions and for high degree of nonlinearity of the 
    class boundaries.

  - 'gaussian'

    A set of seven databases corresponding to the same problem, but with 
    dimensionality ranging from 2 to 8.
    This allows the study of the classifier behavior for different 
    dimensionalities of the input vectors, for heavy overlapped
    distributions and for non linear separability.
    Theses databases where already studied by Kohonen in:
      Kohonen, T. and Barna, G. and Chrisley, R., "Statistical Pattern 
      Recognition with Neural Networks: Benchmarking Studies", 
      IEEE Int. Conf. on Neural Networks, SOS Printing, San Diego, 1988.
    In this paper,the performances of three basis types of neural-like 
    networks (Backpropagation network, Boltzmann machine and Learning 
    Vector Quantization) is evaluated and compared to the theoretical limit.

  - 'concentric' 

    Bidimensional uniform concentric circular distributions.
        2500 instances, 1579 in class 1, 921 in class 0. 
    This database may be used to study the linear separability of the 
    classifier when some classes are nested in other without overlapping.

Contents of the 'REAL' directory 

The databases of this directory contain only the real
classification problem sets selected for the Elena benchmarking studies.
There is one subdirectory for each database. In this subdirectory, 
there are:

- a text file giving detailed information about the related database     
- the compressed original database in the Elena format 
  (`databasename.dat.Z'); the different patterns of each database being 
  presented in a random order.
- By the way of a normalization process, each original feature will have 
  the same importance in a subsequent classification process. 
  A typical method is first to center each feature separately and than 
  to reduce it to a unit variance; this process has been applied on all 
  the REAL Elena databases in order to build the ``CR'' databases 
  contained in the ``databasename_CR.dat.Z'' files.

The Principal Components Analysis (PCA) is a very classical method in pattern
recognition [Duda73].  PCA reduces the sample dimension in a linear way
for the best representation in lower dimensions keeping the maximum of
inertia. The best axe for the representation is however not necessary
the best axe for the discrimination. After PCA, features are selected
according to the percentage of initial inertia which is covered by the
different axes and the number of features is determined according to
the percentage of initial inertia to keep for the classification
process. This selection method has been applied on every REAL database
after centering and reduction (thus on the databasename_CR.dat files).
When quasi-linear correlations exists between some initial features,
these redundant dimensions are removed by PCA and this preprocessing is
then recommended. In this case, before a PCA, the determinant of the
data covariance matrix is near zero; this database is thus badly
conditioned for all process which use this information (the quadratic
classifier for example).

The following files, related to PCA are also available for the REAL databases:
- ``databasename_PCA.dat.Z'', the projection of the ``CR'' database on its 
   principal components (sorted in a decreasing order of the related 
   inertia percentage),
- ``databasename_corr_circle.ps.Z'', a graphical representation of the 
    correlation between the initial attributes and the two first 
    principal components,
- ``databasename_proj_PCA.ps.Z'', a graphical representation of the 
    projection of the initial database on the two first principal 
-  ``databasename_EV.dat'', a file with the eigenvalues and associated
    inertia percentages

The Discriminant Factorial Analysis (DFA) can be applied to a learning
database where each learning sample belongs to a particular class
[Duda73]. The number of discriminant features selected by DFA is fixed
in function of the number of classes (c) and of the number of input
dimensions (d); this number is equal to the minimum between d and c-1.
In the usual case where d is greater than c, the output dimension is
fixed equal to the number of classes minus one and the discriminant
axes are selected in order to maximize the between-variance and to
minimize the within-variance of the classes. The discrimination power
(ratio of the projected between-variance over the projected
within-variance) is not the same for each discriminant axis: this ratio
decreases for each axis. So for a problem with many classes, this
preprocessing will not be always efficient as the last output features
will not be so discriminant. This analysis uses the information of the
inverse of the global covariance matrix, so the covariance matrix must
be well conditioned (for example, a preliminary PCA must be applied to
remove the linearly correlated dimensions). The DFA preprocessing
method has been applied on the 18 first principal components of the
'satimage_PCA' and 'texture_PCA' databases (thus by keeping only the 18
first attributes of these databases before to apply the DFA
preprocessing) in order to build the 'satimage_DFA.dat.Z' and
'texture_DFA.dat.Z' database files, having respectively 5 and 10
dimensions (the 'satimage' database having 6 classes and 'texture'

  For each subdirectory, the directoryname is the same as the name chosen
for the contained database.  Here are the directorynames with a brief
numerical description of the available databases. 

  - phoneme

    French and Spannish phoneme recognition problem. 
  The aim is to distinguish between nasal (AN, IN, ON) and oral 
  (A, I, O, E, E') vowels.

      5404 patterns, 5 attributes (the normalized amplitudes of the five 
     first harmonics), 2 classes.

     This database was in use in the European ESPRIT 5516 project ROARS.
  The aim of this project is the development and the implementation of a
  REAL time analytical system for French and Spannish phoneme

  - texture

    The aim is to distinguish between 11 different textures (Grass lawn, 
  Pressed calf leather, Handmade paper, Raffia looped to a high pile, Cotton 
  canvas, ...), each pattern (pixel) being characterised by 40 attributes 
  built by the estimation of fourth order modified moments in four orientations:
  0, 45, 90 and 135 degrees.

    5500 patterns, 11 classes of 500 instances (each class refers to a type 
    of texture in the Brodatz album).

    The original source of this database is:
  P. Brodatz "Textures: A Photographic Album for Artists and Designers",
  Dover Publications, Inc., New York, 1966.
    This database was generated by the Laboratory of Image Processing 
  and Pattern Recognition (INPG-LTIRF Grenoble, France) in the development 
  of the Esprit project ELENA No. 6891 and the Esprit working group ATHOS
  No. 6620. 

  - satimage (*)

    Classification of the multi-spectral values of an image of the Landsat
  satellite. Each line contains the pixel values in four spectral bands 
  of each of the 9 pixels in a 3x3 neighbourhood and a number indicating 
  the classification label of the central pixel (corresponding to the type 
  of soil: red soil, cotton crop, grey soil, ...).
  The aim is to predict this classification, given the multi-spectral     

     6435 instances, 36 attributes (4 spectral bands x 9 pixels in  
    neighbourhood), 6 classes. 

    This  database was in use in the European StatLog project, which
  involves comparing the performances of machine learning,
  statistical, and neural network algorithms on data sets from REAL-world
  industrial areas including medicine, finance, image analysis, and
  engineering design:

    D. Michie, D.J. Spiegelhalter, and C.C. Taylor, editors.
    Machine learning, Neural and Statistical Classification.
    Ellis Horwood Series In Artificial Intelligence,
    England, 1994.


  - iris (*)

   This is perhaps the best known database to be found in the pattern
   recognition literature.  Fisher's paper is a classic in the field
   and is referenced frequently to this day.  (See Duda & Hart, for
   example.)  The data set contains 3 classes of 50 instances each,
   where each class refers to a type of iris plant.  One class is
   linearly separable from the other 2; the latter are NOT linearly
   separable from each other.
   4 attributes (sepal length, sepal width, petal length and petal width).

 (*) These databases are taken from the ftp anonymous "UCI Repository Of 
     Machine Learning Databases and Domain Theories" 
     (ics.uci.edu: pub/machine-learning-databases):
  Murphy, P. M. and Aha, D. W. (1992). "UCI Repository of machine
  learning databases" [Machine-readable data repository]. Irvine, CA:
  University of California, Department of Information and Computer Science.

 Duda, R.O. and Hart, P.E.,
 Pattern Classification and Scene Analysis,
 John Wiley & Sons, 1973.



  Neural networks are now known as powerful methods for empirical
  data analysis, especially for approximation (identification,
  control, prediction) and classification problems. The ELENA project
  investigates several aspects of classification by neural networks,
  including links between neural networks and Bayesian statistical
  classification, incremental learning (control of the network size
  by adding or removing neurons),...

  URL: http://www.dice.ucl.ac.be/neural-nets/ELENA/ELENA.html

  ELENA is an ESPRIT III Basic Research Action project (No. 6891).
  It involves:
	INPG (Grenoble, F),
	UPC (Barcelona, E), 
	EPFL (Lausanne, CH),
	UCL (Louvain-la-Neuve, B), 
	Thomson-Sintra ASM (Sophia Antipolis, F)
	EERIE (Nimes, F).  

  The coordinator of the project can be
  contacted at: 
      Prof. Christian Jutten, 
      46 av. Flix Viallet, 
      F-38031 Grenoble Cedex, 
      Phone: +33 76 57 45 48, 
      Fax: +33 76 57 47 90, 
      e-mail: chris at tirf.inpg.fr  

A simulation environment (PACKLIB) has been developed in the project;
it is a smart graphical tool allowing fast programming and
interactive analysis. The PACKLIB environment greatly simplifies the
user's task by requiring only to write the basic code of the
algorithms, while the whole graphical input, output and relationship
framework is handled by the environment itself.  PACKLIB is used for
extensive benchmarks in the ELENA project and in other situations
(image processing, control of mobile robots,...). Currently, PACKLIB
is tested by beta users and a demo version available in the public 
  URL: http://www.dice.ucl.ac.be/neural-nets/ELENA/Packlib.html



  VOZ Jean-Luc or Michel Verleysen
  Universite Catholique de Louvain
  DICE - Lab. de Microelectronique
  3, place du Levant
  E_mail : voz at dice.ucl.ac.be
	   verleysen at dice.ucl.ac.be

More information about the Neur-sci mailing list

Send comments to us at biosci-help [At] net.bio.net