IUBio Biosequences .. Software .. Molbio soft .. Network News .. FTP

Neural Servo: Theory and Source

mervyn mervyn at xs4all.nl
Mon Dec 22 15:02:47 EST 1997


(sorry if this article shows up several times, but my ISP's 
 mailserver appears to have some problems the last few days)

Create new sensory worlds and grow adaptive systems, tomorrow!
 
this is the source code of a neural network program that
simulates a specific type of recurrent neural net: a neural
servo. I developed the program over the last two years
and wrote and published a paper called "Feedback in 
Knowledge-Oriented Neural Networks", discussing its
most appealing features (from an authentic, observer's 
point of view). I believe that, if you understand the content
of the paper, it must be possible for anyone with some
FORTH* programming experience to understand the objects
and their interaction and to extend the library of worlds to be
modelled or controlled by the neural servo network.
 
The paper and more documentation is available at 
my homepage: http://www.xs4all.nl/~mervyn

MOPS (available for Mac and Powermac) and more Forth
documentation is linked at: 
http://www.netaxs.com/~jayfar/mops.html 

 
H a v e  a  N i c e  D a y ,
 
	M e r v y n
 
	Mervyn at xs4all.NL
	Amsterdan- Holland	
 
*the language is actually MOPS, an object oriented Forth.
However, since any Forth is as-good-as object oriented,
translating the thing should not be too much work.

 
( Neural Servo <mervyn at xs4all.nl> **  V5.0 Modulation )
( For MOPS: Object Oriented Forth 24-09-96                   )
 
: Randomize when: fEvent 200 mod 0 DO 10 random drop LOOP ; ( RANDOMIZER )
 
(Tune your Network:)
:class NSNet super{ object }
( Size )
6 value Width    4 value Height
width height * value nNeurons  nneurons dup * value nConnect
nneurons array Potential       nneurons array NextPotential
nconnect array Weight          nconnect array Strength
width array InputLayer         int Novelty
int TrphAgent                  int ModLev   int ModFlag
int Reinforce                  int Nr       int RelivedFlag
:m Measure: at: Potential ;m ( <<< REQUEST: POTENTIAL OF NEURON )
:m Connection?: at: Strength 0> ;m ( <<< REQUEST: CONNNECTION PRESENT? )
:m MeasureNovelty: get: Novelty ;m ( <<< REQUEST: NOVELTY? )
:m SensedBy: Width 0 DO i to: InputLayer LOOP ;m ( >>> IN: INPUT )
:m Fire: +to: NextPotential ;m ( >>> TRIGGER: FIRE AT NEURON )
:m Relive: 0 put: ModFlag 1 put: RelivedFlag ;m ( >>> TRIGGER: ENABLE
REGRESSION )
:m Regress: 4 +: ModLev  ;m ( >>> TRIGGER: REGRESS NETWORK )
:m Enhance: -4 +: ModLev  ;m ( >>> TRIGGER: ENHANCE NETWORK )
:m Modulate: ( >>> TRIGGER: MODULATE NETWORK )
   get: ModLev 12 = IF 1 put: ModFlag THEN
   get: ModFlag 0= IF Regress: self 
   ELSE Enhance: self  get: ModLev 0= IF 0 put: RelivedFlag THEN THEN ;m
:m Relived?: get: RelivedFlag 0= ;m ( REQUEST: UNMODULATED? )
:m ReleaseAgent: 3 put: TrphAgent ;m  ( TRIGGER: RELEASE AGENT )
:m Update: ( TRIGGER: IMPLEMENT CHANGES FOR NEXT TIME STEP )
   nNeurons 0 DO i at: NextPotential dup i to: Potential  
   drop 0  i to: NextPotential LOOP ;m
:m GlobAdjust: ( TRIGGER: REINFORCE NETWORK )
   put: Reinforce nConnect 0 DO i Connection?: self IF get: Reinforce i 
   +to: Strength  i at: Strength 1 < IF 0 i to: strength  0 i to: Weight
THEN THEN LOOP ;m
:m GrowConnect: ( TRIGGER: GROW ONE CONNECTION )
   get: Nr nNeurons * nNeurons random + dup dup 
   at: Strength 0= IF 100 random 90 < IF 11 ELSE -110 THEN   
   swap to: Weight 1 swap to: Strength 
    -1 +: TrphAgent ELSE drop drop THEN  ;m
:m Activate: ( TRIGGER: SIMULATE SINGLE NEURON ACTION & GROWTH )
   dup put: Nr Measure: self  10 > IF 
   nNeurons 0 DO get: Nr nNeurons * i + dup
   at: Strength get: ModLev > IF at: Weight i Fire: self ELSE drop THEN
LOOP
   get: TrphAgent 0> IF 30 random 1 = IF GrowConnect: self THEN THEN THEN
;m 
:m KillNew: ( TRIGGER: KILL ZERO-STRENGTH CONNECTIONS )
   nConnect 0 DO i at: strength 1 = IF 0 i to: strength
   0 i to: weight THEN LOOP ;m
:m Feedback: ( TRIGGER: SIMULATE COMPARATOR FEEDBACK )
   ( If input and Prediction layer mismatch,       )
   ( Activate novelty neurons, Count active novelty layer neurons )
   0 put: Novelty  width 0 DO    
   nNeurons Width - i + at: Potential 10 > IF 11 ELSE 0 THEN
   i at: InputLayer  <> IF 
   1 +: Novelty  11 i ELSE 0 i THEN 
   drop i at: inputlayer
   to: NextPotential LOOP ;m
:m Punish: ( TRIGGER: PREPARE PUNISHING REINFORCEMENT )
   0 put: TrphAgent  -1 GlobAdjust: self ;m
:m Reward: ( TRIGGER: PREPARE REWARDING REINFORCEMENT )
   1 GlobAdjust: self ;m
;class 
NSNet Net
 
:class Sequence super{ object }  
int SeqTime
:m Init: ;m ( >>> TRIGGER: INITIALIZE WORLD) 
:m provide: ( >>> TRIGGER: EXPOSE NETWORK TO 4x 6-BIT PATTERN WORLD )
   ( IN: -   OUT: - Generate pattern sequence, Call for sensing by Net )
   1 +: seqtime  get: seqtime
   dup 11 = IF 1 put: seqtime drop 0 THEN
   dup 1 = IF  11  11  0   0   11   11 
   			sensedby: net  drop ELSE 
   dup 3 = IF  11  11 11   11   0   0  
   		sensedby: net  drop ELSE
   dup 5 = IF  0   11  0   0   11   11 
   		sensedby: net  drop ELSE
       7 = IF  11  11  0   11  11   0  
       	sensedby: net  THEN THEN THEN THEN ;m
;class 
 
:class NoisySequence super{ object }  
int SeqTime int counter
:m Init: 0 put: counter
       ;m ( >>> TRIGGER: INITIALIZE WORLD) 
:m provide: ( >>> TRIGGER: EXPOSE NETWORK TO 4x 6-BIT PATTERN WORLD )
   ( IN: -   OUT: - Generate pattern sequence, Call for sensing by Net )
   1 +: seqtime  get: seqtime
   dup 11 = IF 1 put: seqtime drop 0 THEN
   dup 1 = IF  11  300 random 297 > if 0 else 11 then  0   0   11   11 
   			sensedby: net  drop ELSE 
   dup 3 = IF  11  11 300 random 297 > if 0 else 11 then   300 random 297
> if 0 else 11 then   0   0  
   		sensedby: net  drop ELSE
   dup 5 = IF  0   11  0   0   300 random 298 > if 0 else 11 then   300
random 297 > if 0 else 11 then 
   		sensedby: net  drop ELSE
       7 = IF  300 random 297 > if 0 else 11 then  11  0   11  11   0  
       	sensedby: net  THEN THEN THEN THEN       	
       	;m
;class
 
( Pick a World or... create your own :-)
NoisySequence World
 
:class NSInterface super{ object } ( <<< USER INTERFACE ELEMENTS >>> )
window fvp  window cvp  window avp  window lvp
int Xl  int N  int Pos  int Time  int Score int Count int y1 int y2
nNeurons array X  nNeurons array Y  
:m n>xy: ( IN: neuron Index  OUT: X and Y for related graph location )
   dup put: n  width mod 20 * 50 + ( x ) 130 get: n width / ( y ) 20 * -
;m 
:m LineInit: ( IN: -   OUT: - Initialize window & background )
   test: avp  0 0 move: avp  800 500 size: avp 
   test: lvp  " Performance" name: lvp 20 140 move: lvp  400 120 size: lvp
;m
:m ConInit: ( IN: -   OUT: - Initialize window ) 
   0 nNeurons 1 -  DO i n>xy: self i to: y i to: x -1 +LOOP ;m
:m FieldInit: ( IN: -   OUT: - Initialize window )
   test: fvp " Network" name: fvp 420 140 move: fvp 200 160 size: fvp ;m
:m Refresh: ( IN: -   OUT: Clear score plot screen, Reset running variable
)
   set: lvp  cls  0 put: xl  ;m
:m Check: get: xl ;m
:m Line: put: y1
   set: lvp   Check: self 380 > IF Refresh: self THEN
   Relived?: Net  IF 157 115 moveto get: y1 100 * 76 / 100 swap -  . 
                                      ." % compression              " THEN
   2 +: xl  
   get: xl 100 get: y2 - MoveTo   get: xl 2 + 100 get: y1  - LineTo 
   get: y1 put: y2 ;m
:m Field: ( IN: -   OUT: - active neuron Plot )
   set: fvp cls  nneurons 0 DO i dup at: x swap at: y moveto  i measure:
net 
   10 > IF ." ()" THEN  LOOP ;m 
:m Connectivity: ( IN: -  OUT: connectivity Plot )
   set: fvp  0 put: Count  nConnect 0 DO i Connection?: Net IF
   1 +: Count  i nNeurons / ( source ) dup at: x 6 + swap at: y 4 - moveto
   i nNeurons mod ( target ) dup at: x 6 + swap at: y 4 - lineto THEN LOOP 
   57 154 Moveto get: Count . ." connections      " ;m
:m Build: ( IN: score OUT: - Builds interface, handles events )
   put: Score 
   set: avp get: pos dup 50 > IF 500 * 0 DO i drop LOOP Field: self 
   ELSE drop THEN
   1 +: time get: time 20 = IF 0 put: time        ( seq 20 / mov 14 )
     ?Terminal IF quit THEN            
	get: mouse drop  put: Pos
    get: Score Line: self
    set: avp get: pos 30 > IF Connectivity: self THEN
   THEN ;m
;class
NSInterface Interface ( Quality Check: Passed 28-09-96 16:30 )
 
:class Animated super{ object } ( COORDINATE OBJECT INTERACTION )
 int Score   int LastScore int BestScore   int Time
 :m Init: ( >>> TRIGGER: INITIALIZE ALL OBJECTS: INTERFACE, WORLD, NETWORK
)  
    100 put: Score  100 put: LastScore  101 put: BestScore  randomize
    LineInit: Interface 
    ConInit: Interface
    FieldInit: Interface 
    init: world ;m
 :m Animate: ( THIS IS WHY THINGS HAPPEN ) 
    Init: self
    BEGIN
    Update: Net       Provide: World  
    nNeurons 0 DO i   Activate: Net  LOOP    
    Feedback: Net     MeasureNovelty: Net  +: Score
    get: score        Build: Interface
    1 +: Time  get: Time  20 = IF  0 put: Time  ( Every 14/20 time steps:
)
     Relived?: net IF       
      ( REINFORCEMENT )
      get: LastScore get: Score = IF ReleaseAgent: Net 100 random 80 > IF
KillNew: Net THEN  THEN
      get: Score get: LastScore > IF Punish: Net  Relive: Net   THEN
      get: LastScore get: Score > IF Reward: Net  ReleaseAgent: Net  THEN 
      get: Score put: LastScore 
      ELSE Modulate: Net THEN  0 put: Score
    THEN 
    AGAIN ;m
;class
Animated Servo
Animate: Servo   bye
 
 




More information about the Neur-sci mailing list

Send comments to us at biosci-help [At] net.bio.net