blind identification and separation of source signals (abstracts)

Andrew Cichocki cia at kamo.riken.go.jp.
Wed Sep 13 02:31:10 EST 1995



Blind Signal Processing is an emerging area in adaptive signal processing 
and neural networks. 
It was originated in France in the late 80's by Jutten and Herault.
Below please find an advanced program of a special invited session devoted 
to blind separation of sources and their applications at 1995 INTERNATIONAL 
SYMPOSIUM ON NONLINEAR THEORY AND ITS APPLICATIONS , NOLTA'95 in Las Vegas.

Recently we have developed a new class of adaptive learning algorithm for
blind separation of source signals. I am interested especially in 
applications of this approach to image enhancement , filtering and noise 
reduction. Any comments will be highly appreciated.

Dr. Andrzej Cichocki, 
Head of Laboratory for Artificial Brain Systems,
Frontier Research Program RIKEN,
Institute of Physical and Chemical Research,
Hirosawa 2-1, Saitama 351-01,
WAKO-Schi,
JAPAN
E-mail: cia at kamo.riken.go.jp,
FAX (+81) 048 462 4633.
URL: http://zoo.riken.go.jp/bip.html

===========================================================================


                    NOLTA'95,

           1995 INTERNATIONAL SYMPOSIUM ON
         NONLINEAR THEORY AND ITS APPLICATIONS

            Caesars Palace, LAS VEGAS
              Dec. 10 -14, 1995

            Program  for Special Invited Session on

                 "BLIND SEPARATION OF SOURCES

                -Brain Information Processing"

Organizer and chair
 Dr. A. Cichocki
E-mail: cia at kamo.riken.go.jp

Provisional Program:

1. Prof.Christian JUTTEN , Laboratory TIRF, INPG, Grenoble, FRANCE,

"Separation of Sources: Blind or Unsupervised? "

Abstract:
Basically, separation of sources are referred as BLIND methods. However,
adaptive algorithms for source separation only emphasize on UNSUPERVISED
aspects of the learning. In this talk, we propose a selected review of
recent works to show how A PRIORI KNOWLEDGES on the sources or on the
mixtures can simplify the algorithms and improve performance.


2. Prof. Jean-Francois  CARDOSO , Ecole Nationale Superieure des 
Telecomunications, Telecom Paris, FRANCE

 "The Invariant Approach to Source Separation"

Abstract:
The `invariant approach' to source separation is based on the recognition
that the unknown parameter in a source mixture is the mixing matrix, hence
it belongs to a multiplicative group. In this contribution, we show that
this simple fact can be exploited to build source separation algorithms
behaving uniformly well in the mixing matrix.  This is achieved if two
sufficient conditions are met.
 + First, contrast functions (or estimating equations) used to identify the
mixture should be designed in such a way that source separation is achieved
when they are optimized (or solved) **without constraints** (such as
normalization, etc). Examples of such contrast functions will be given,
some of them being simple variants of classic contrast functions. This
requirement is sufficient to guarantee uniform performance of the resulting
batch algorithms.
 + Second, in the case of adaptive algorithm, uniform performance has a
more extensive meaning: not only the residual error but also the
convergence are important.  Again, the multiplicative nature of the
parameter calls for a special form of the learning rule, namely it suggests
a `multiplicative update'.  This approach results in adaptive source
separation algorithms and enjoying uniform performance: convergence speed,
residual error, stable points, etc... do not depend on the mixing matrix.
In addition these algorithms show a very simple (and parallelizable)
structure.
  The paper includes analytical results based on asymptotic performance
analysis that quantify the behavior of both batch and adaptive
equivariant source separators. In particular, these results allow to
determine, given the source distribution, the optimal nonlinearities to
be used in the learning rule.



3. Dr. Jie ZHU, Prof. Xi-Ren CAO, and prof. Ruey-Wen LIU,
 The Hong Kong University of Science and Technology Kowloon, Hong Kong, 
The University of Notre Dame, Notre Dame, NI 46556, U.S.A.
                     
"Blind Source Separation Based on Output Independence - Theory and 
Implementation"

Abstract:
 The paper presents some recent results on the theory and implementation 
techniques of blind source separation. the approach is based on     
independence property of the outputs of a filter. 
In the theory part, we identify and  study two major issues in the blind 
source separation problem: separability   and separation principles. We 
show that separability is an intrinsic property  of the measured signals 
and can be described by the concept of $m$-row   decomposability introduced 
in this paper, and that the separation principles  can be developed by 
using the structure characterization theory of random  variables. In 
particular, we show that these principles can be derived   concisely and 
intuitively by applying the Darmois-Skitovich theorem, which   is 
well-known in statistical inference theory and psychology. 

  In the implementation part, we show that if at most one of the source 
  signals has a zero third (or fourth) order cumulant, then these signals
  can be separated by a filter whose parameters can be determined by a
  system of nonlinear equations using only third (or fourth) order      
  cumulants of the measured signals. This results covers some previous
  results as special cases.


4. Dr. Jie HUANG, Prof. Noboru OHNISHI and Dr. Noboru Sugie ;
 Bio-Mimetic Control Research Center , The Institute of Physical and  
 Chemical Research (RIKEN), Nagoya, JAPAN

 "Sound Separation Based on Perceptual Grouping of Sound Segments"

Abstract :
We would like to propose a sound separation method,
which combines spatial cues (source direction)
and structural cues (continuity and harmony).
Sound separation is important in various scientific fields.
There are mainly two different approaches to achieve this goal.
One is based on blind estimation of inverse transfer functions
from multiple sources to multiple receivers (microphones).
The other is based on grouping sound segments in time-frequency domain.
Our approach is based on the sound segments grouping.
However, we use multiple microphones to obtain the spatial information.
This approach is strongly inspired by the precedence effect
and the cocktail party effect of human auditory system.
The precedence effect suggests to us the way of coping with
echoes in reverberant environment.
The cocktail party effect suggests the use of spatial cues
for sound separation.
Psychological factors of auditory stream integration and segregation,
such as continuity and harmony, are used as structural cues.
It is realized by a continuity enhancement filter and a harmonic
 histogram to supplement the spatial segments grouping.
The use of this method with real human speeches
recorded in an anechoic chamber and a normal room was demonstrated.
The experiments have shown that the method was effective to
separate sounds in reverberant environments.



5. Dr. Kiyotoshi MATSUOKA and Dr. Mitsuru KAWAMOTO, Department of Control 
Engineering, Kyushu Institute of Technology,
1-1,Tobata, Kitakyushu, 804 Japan

 "Blind Signal Separation Based on a Mutual Information Criterion"


Abstract:
    This paper deals with the problem of the so-called blind separation
of sources.  The problem is to recover a set of source signals from their
linear mixtures observed by the same number of sensors, in the absence of
any particular information about the transfer function that couples the
sources and the sensors.  The only a priori knowledge is, basically, the
fact that the source signals are statistically mutually independent. Such
a task arises in noise canceling of sound signals, image enhancement,
medical measurement, etc.
        If the observed signals are stationary, Gaussian, white ones, then
blind separation is essentially impossible.  Conversely, blind separation
can be realized by exploiting  some information on nonstationary,
non-Gaussian, or nonwhite characteristics of the observed signals, if any. 
Most of the conventional methods stipulate that the source signals are
non-Gaussian, and use some high-order moments or cumulants.  However, it is
sometimes difficult to accurately estimate non-Gaussian statistics because
random signals in practice are usually not so far from Gaussian.       
  
In this paper we propose an approach that utilizes only
second-order moments of the observed signals.  We consider two cases:
(i)  the source signals are nonstationary; 
(ii) the source signals have some temporal correlations, i.e., they are
nonwhite signals.
        To realize signal separation we consider a recovering filter which
takes in the observed signals as input and provides an estimate of the
source signals as output.   The parameters of the filter are determined
such that all the outputs of the filter be mutually statistically
independent.  As a criterion of the statistical independence we adopt the
well-known mutual information between the outputs.  The adaptation rule for
the filter's parameters is derived from the steepest descent minimization
of the criterion function. 
        A remarkable feature of our approach is that it is able to treat
time-convolutive mixtures of a general number of (stationary) source
signals.  In contrast, most of the conventional studies on blind separation
only consider the static mixing of the source signals.  Namely, any delay
in the mixing process is not taken into account.  So, those methods are
useless for many of the important applications of blind separation, e.g.,
separation of sound signals.  Although there are some studies that deal
with convolutive mixtures involving some delay, all of them consider only
the case of two sources (2 x 2 channels) and do not seem extendible to the
case of more than two sources.  In our approach, also for convolutive
mixtures, the adaptation rule is easily obtained by defining the
information criterion in the frequency domain. 

       
6.   Dr. Eric MOREAU, and Prof. Odile MACCHI  Laboratoire des Signaux 
et Systems, CNRS-ESE, FRANCE

   "Adaptive Unsupervised Separation of Discrete Sources"

ABSTRACT: We consider the unsupervised source separation problem where
observations are captured at the output of an unknown linear mixture of 
random signals called sources.
The sources are assumed discrete, zero-mean and statistically independent.
In this paper we consider the problem with a prewhitening stage. The a 
priori knowledge that sources are discrete with known level, is used in 
order to improve performances.
A novel contrast which combines two parts, is proved. The first part forces
statistical independence of the outputs while the second one forces the 
outputs to have the known distribution. Then a stochastic gradient adaptive 
algorithm is proposed.
Its performance is illustrated thanks to computer simulations that clearly 
show that the novel contrast achieves much better performance.

  
7. Dr. Adel BELOUCHRANI, and Prof. Jean-Francois CARDOSO , 
 Telecom Paris, CNRS URA 820, GdR TdSI   46 rue Barrault, 75634 Paris Cedex 
13, FRANCE          

   "Maximum Likelihood Source Separation  by the Expectation-Maximization        
    Technique: Deterministic and Stochastic Implementation"

Abstract:
This paper deals with the  source separation problem which consists 
in the separation of a mixture of independent  sources without 
a priori knowledge about the mixing matrix. When the source distributions
are known in advance, this problem can be solved  via the maximum 
likelihood (ML) approach by maximizing the data likelihood function using 
(i) the Expectation-Maximization (EM) algorithm and (ii) a stochastic 
version of it, the SEM, wich is efficiently implemented by resorting to 
Metropolis sampler. Two important features of our algorithm are  that 

(a) the covariance of the additive noise can be estimated as a regular 
parameter, 

(b) in the case of discrete sources, it is possible to separate   more 
sources than sensors. 

The effectiveness of this method is illustrated by numerical simulations.

8. Prof. L. TONG and  Dr. X. CHEN, University of Connecticut, USA.

   "Blind Separation of Dynamically Mixed Multiple Sources
    and its Applications in CDMA Systems"


Abstract:
In this paper, we consider the problem of separating dynamically mixed 
multiple sources. Specifically, we address the problem of recovering the 
sources of an multiple-input multiple-output system.  Two issues will be 
addressed: 

(i) Source Blind Separability; 

(ii) Blind Signal Separation Algorithms. 

Applications of the proposed approach to code-division multiple-access 
schemes in wireless communication are presented.


9. Prof. Shun-ichi AMARI, Prof. Andrzej CICHOCKI and Dr. Howard Hua YANG,
Frontier Research Program RIKEN (Institute of Physical and Chemical 
Research), Wako-shi, JAPAN

 "Multi-layer Neural Networks with Local Learning Rules for Blind
   Separation of Sources"

Abstract:
  In this paper we will propose  multi-layer neural network models 
(feedforward and recurrent) with novel, local, adaptive, unsupervised 
learning rules which enable  not only to separate on-line independent 
sources but also determine the number of active  sources. 
In other words, we assume that the number of sources and their waveforms 
are completely unknown. Moreover, the separation problem can be very 
ill-conditioned and/or badly scaled. In fact the performance of the learning 
algorithm is independent of scaling factors and a condition number of the 
mixing matrix.
Universal (flexible) computer simulation program will be presented which 
enable comparison of validity and performance of various  recently 
developed adaptive on-line learning algorithms.


10.. Dr.L. De Lathauwer and Dr.P. Comon;

E.E. Dept. - ESAT - SISTA, K.U.Leuven, BELGIUM,	CNRS - I3S, Sophia 
Antipolis, 
Valbonne, FRANCE	

"Higher - Order Power Method"

Abstract
The scientific boom in the field of higher-order statistics
involves an increasing need for numerical tools in multilinear 
algebra: higher-order moments and cumulants of multivariate
stochastic processes are higher-order tensors.
We consider the problem of generalizing the computation of the
best rank-R approximation of a given matrix to the computation
of the best rank-(R1,R2,...,RN) approximation of an Nth-order
tensor. We mainly focus on the best rank-1 approximation of 
third-order tensors.
It is shown that this problem leads in a very natural way to
a higher-order equivalent of the well-known power method for
the computation of the eigendecomposition of matrices.
It can be proved that each power iteration step decreases the 
least-squares error between the initial tensor and the lower-rank
estimate. In the tensor case several stationary points might
exist, each with a different domain of attraction.
Surprisingly, the power iteration for a super-symmetric tensor 
can produce intermediate results that are unsymmetric.
Imposing symmetry on the algorithm does not necessarily
improve the convergence speed; the symmetric power iteration
can even fail to converge.
In the matrix case truncation of the singular value decomposition (SVD)
yields the best rank-R approximation; in the tensor case it can
only be proved that truncation of the higher-order singular
value decomposition (HOSVD) yields a fairly good approximation.
All our simulations show that the HOSVD-guess belongs to the 
attraction region corresponding to the optimal fit.  
---------------------------------------------------------------------------




More information about the Neur-sci mailing list