Neuron Activation Functions, Biological Plausibility
edstrom at hsc.ucalgary.ca
Thu Jun 18 10:42:56 EST 1992
In article <l3vsifINN39b at libyan.cs.utexas.edu> sirosh at cs.utexas.edu (Joseph Sirosh) writes:
>Sigmoidal activation functions have been used in many models of neurons.
>Some models allow this activation function to change, from flatter
>sigmoids to steeper or vice versa. Is this biologically plausible?
The quick answer is yes.
>This question can be elaborated a bit further. A sigmoidal activation
>function approximates a plot of the output firing rate of a neuron( y-axis )
>against the input activation of the neuron( x-axis).
>When a sigmoid becomes steeper, the lower threshold increases, and
>the upper threshold decreases. This would mean that our model neuron
>starts firing at a higher threshold, but reaches its peak firing rate
>(saturation) at a lower input activation. Is there any biological
>evidence that this process happens in real neurons?
Remember, the sigmoid function is a very simplified model and the
threshold in the model shouldn't be taken too literally. The aspect
of the real neuron it captures is the distance between the resting
membrane potential and the threshold potential. Thus, anything that
depolarizes the cell will shift the curve leftward toward the origin
and hyperpolarizing influences will shift it right. Both without
changing the slope. This is observed regularly in real neurons and
results from the presence of modulatory substances, that is,
substances involved in long-term regulatory functions. In electrical
circuit terms think of a bias offset.
The shape of the curve can change for two reasons. One is that
properties of the neuron change so that it can support higher or lower
maximum firing rates. The result of this would be to change the
ceiling of the output side of the function so that a given change in
firing rate would become, for example, a smaller percentage of a new,
larger maximum. This is seen with factors that reduce accomodation or
recurrent inhibition so that for a given input a burst or higher
frequency burst results. Another possibility is that the strength of
coupling between the input and output is changed. The synaptic events
are chemical in nature and their immediate effect is to induce current
in the post synaptic cell. By ohm's law you can see that anything
that affects the whole-cell input impedence will change the magnitude
of the response. An increase in the membrane resistance will allow a
given synaptic current to cause a greater change in voltage.
Try to think of the position and shape of the curve as independent
parameters. You can, of course, have any combination of the above
happen at any time.
>Change in activation function of a neuron can happen in many ways. One
>of them would be migration of ion channels over the surface of the neuron.
>The proportion of ion channels of various types in the axon hillock would
>determine some of the triggering characteristics of the neuron. I have
>come across evidence that ion channels migrate during synapse formation
>at the neuromuscular junction( Kandel, Schwartz, Jessel - Principles of
>Neuroscience ). However, do similar migration of ion channels happen in
>the normal neurons? In particular, is there any evidence that migration
>of ion channels is responsible for phenomena like habituation? Is there
>evidence that the activation function changes when neurons are repeatedly
>stimulated? Say with a tetanus?
Not that I know of. But its certainly a possibility.
John Edstrom | edstrom @ elmer.hsc.ucalgary.ca
RM 2104, HSc Building, Div. Neuroscience
U. Calgary School of Medicine, 3330 Hospital Drive NW
Calgary, Alberta T2N 3Y4
(403) 220 4493 (wk)
More information about the Bioforum