(sorry if this article shows up several times, but my ISP's
mailserver appears to have some problems the last few days)
Create new sensory worlds and grow adaptive systems, tomorrow!
this is the source code of a neural network program that
simulates a specific type of recurrent neural net: a neural
servo. I developed the program over the last two years
and wrote and published a paper called "Feedback in
Knowledge-Oriented Neural Networks", discussing its
most appealing features (from an authentic, observer's
point of view). I believe that, if you understand the content
of the paper, it must be possible for anyone with some
FORTH* programming experience to understand the objects
and their interaction and to extend the library of worlds to be
modelled or controlled by the neural servo network.
The paper and more documentation is available at
my homepage: http://www.xs4all.nl/~mervyn
MOPS (available for Mac and Powermac) and more Forth
documentation is linked at:
http://www.netaxs.com/~jayfar/mops.html
H a v e a N i c e D a y ,
M e r v y n
Mervyn at xs4all.NL
Amsterdan- Holland
*the language is actually MOPS, an object oriented Forth.
However, since any Forth is as-good-as object oriented,
translating the thing should not be too much work.
( Neural Servo <mervyn at xs4all.nl> ** V5.0 Modulation )
( For MOPS: Object Oriented Forth 24-09-96 )
: Randomize when: fEvent 200 mod 0 DO 10 random drop LOOP ; ( RANDOMIZER )
(Tune your Network:)
:class NSNet super{ object }
( Size )
6 value Width 4 value Height
width height * value nNeurons nneurons dup * value nConnect
nneurons array Potential nneurons array NextPotential
nconnect array Weight nconnect array Strength
width array InputLayer int Novelty
int TrphAgent int ModLev int ModFlag
int Reinforce int Nr int RelivedFlag
:m Measure: at: Potential ;m ( <<< REQUEST: POTENTIAL OF NEURON )
:m Connection?: at: Strength 0> ;m ( <<< REQUEST: CONNNECTION PRESENT? )
:m MeasureNovelty: get: Novelty ;m ( <<< REQUEST: NOVELTY? )
:m SensedBy: Width 0 DO i to: InputLayer LOOP ;m ( >>> IN: INPUT )
:m Fire: +to: NextPotential ;m ( >>> TRIGGER: FIRE AT NEURON )
:m Relive: 0 put: ModFlag 1 put: RelivedFlag ;m ( >>> TRIGGER: ENABLE
REGRESSION )
:m Regress: 4 +: ModLev ;m ( >>> TRIGGER: REGRESS NETWORK )
:m Enhance: -4 +: ModLev ;m ( >>> TRIGGER: ENHANCE NETWORK )
:m Modulate: ( >>> TRIGGER: MODULATE NETWORK )
get: ModLev 12 = IF 1 put: ModFlag THEN
get: ModFlag 0= IF Regress: self
ELSE Enhance: self get: ModLev 0= IF 0 put: RelivedFlag THEN THEN ;m
:m Relived?: get: RelivedFlag 0= ;m ( REQUEST: UNMODULATED? )
:m ReleaseAgent: 3 put: TrphAgent ;m ( TRIGGER: RELEASE AGENT )
:m Update: ( TRIGGER: IMPLEMENT CHANGES FOR NEXT TIME STEP )
nNeurons 0 DO i at: NextPotential dup i to: Potential
drop 0 i to: NextPotential LOOP ;m
:m GlobAdjust: ( TRIGGER: REINFORCE NETWORK )
put: Reinforce nConnect 0 DO i Connection?: self IF get: Reinforce i
+to: Strength i at: Strength 1 < IF 0 i to: strength 0 i to: Weight
THEN THEN LOOP ;m
:m GrowConnect: ( TRIGGER: GROW ONE CONNECTION )
get: Nr nNeurons * nNeurons random + dup dup
at: Strength 0= IF 100 random 90 < IF 11 ELSE -110 THEN
swap to: Weight 1 swap to: Strength
-1 +: TrphAgent ELSE drop drop THEN ;m
:m Activate: ( TRIGGER: SIMULATE SINGLE NEURON ACTION & GROWTH )
dup put: Nr Measure: self 10 > IF
nNeurons 0 DO get: Nr nNeurons * i + dup
at: Strength get: ModLev > IF at: Weight i Fire: self ELSE drop THEN
LOOP
get: TrphAgent 0> IF 30 random 1 = IF GrowConnect: self THEN THEN THEN
;m
:m KillNew: ( TRIGGER: KILL ZERO-STRENGTH CONNECTIONS )
nConnect 0 DO i at: strength 1 = IF 0 i to: strength
0 i to: weight THEN LOOP ;m
:m Feedback: ( TRIGGER: SIMULATE COMPARATOR FEEDBACK )
( If input and Prediction layer mismatch, )
( Activate novelty neurons, Count active novelty layer neurons )
0 put: Novelty width 0 DO
nNeurons Width - i + at: Potential 10 > IF 11 ELSE 0 THEN
i at: InputLayer <> IF
1 +: Novelty 11 i ELSE 0 i THEN
drop i at: inputlayer
to: NextPotential LOOP ;m
:m Punish: ( TRIGGER: PREPARE PUNISHING REINFORCEMENT )
0 put: TrphAgent -1 GlobAdjust: self ;m
:m Reward: ( TRIGGER: PREPARE REWARDING REINFORCEMENT )
1 GlobAdjust: self ;m
;class
NSNet Net
:class Sequence super{ object }
int SeqTime
:m Init: ;m ( >>> TRIGGER: INITIALIZE WORLD)
:m provide: ( >>> TRIGGER: EXPOSE NETWORK TO 4x 6-BIT PATTERN WORLD )
( IN: - OUT: - Generate pattern sequence, Call for sensing by Net )
1 +: seqtime get: seqtime
dup 11 = IF 1 put: seqtime drop 0 THEN
dup 1 = IF 11 11 0 0 11 11
sensedby: net drop ELSE
dup 3 = IF 11 11 11 11 0 0
sensedby: net drop ELSE
dup 5 = IF 0 11 0 0 11 11
sensedby: net drop ELSE
7 = IF 11 11 0 11 11 0
sensedby: net THEN THEN THEN THEN ;m
;class
:class NoisySequence super{ object }
int SeqTime int counter
:m Init: 0 put: counter
;m ( >>> TRIGGER: INITIALIZE WORLD)
:m provide: ( >>> TRIGGER: EXPOSE NETWORK TO 4x 6-BIT PATTERN WORLD )
( IN: - OUT: - Generate pattern sequence, Call for sensing by Net )
1 +: seqtime get: seqtime
dup 11 = IF 1 put: seqtime drop 0 THEN
dup 1 = IF 11 300 random 297 > if 0 else 11 then 0 0 11 11
sensedby: net drop ELSE
dup 3 = IF 11 11 300 random 297 > if 0 else 11 then 300 random 297
> if 0 else 11 then 0 0
sensedby: net drop ELSE
dup 5 = IF 0 11 0 0 300 random 298 > if 0 else 11 then 300
random 297 > if 0 else 11 then
sensedby: net drop ELSE
7 = IF 300 random 297 > if 0 else 11 then 11 0 11 11 0
sensedby: net THEN THEN THEN THEN
;m
;class
( Pick a World or... create your own :-)
NoisySequence World
:class NSInterface super{ object } ( <<< USER INTERFACE ELEMENTS >>> )
window fvp window cvp window avp window lvp
int Xl int N int Pos int Time int Score int Count int y1 int y2
nNeurons array X nNeurons array Y
:m n>xy: ( IN: neuron Index OUT: X and Y for related graph location )
dup put: n width mod 20 * 50 + ( x ) 130 get: n width / ( y ) 20 * -
;m
:m LineInit: ( IN: - OUT: - Initialize window & background )
test: avp 0 0 move: avp 800 500 size: avp
test: lvp " Performance" name: lvp 20 140 move: lvp 400 120 size: lvp
;m
:m ConInit: ( IN: - OUT: - Initialize window )
0 nNeurons 1 - DO i n>xy: self i to: y i to: x -1 +LOOP ;m
:m FieldInit: ( IN: - OUT: - Initialize window )
test: fvp " Network" name: fvp 420 140 move: fvp 200 160 size: fvp ;m
:m Refresh: ( IN: - OUT: Clear score plot screen, Reset running variable
)
set: lvp cls 0 put: xl ;m
:m Check: get: xl ;m
:m Line: put: y1
set: lvp Check: self 380 > IF Refresh: self THEN
Relived?: Net IF 157 115 moveto get: y1 100 * 76 / 100 swap - .
." % compression " THEN
2 +: xl
get: xl 100 get: y2 - MoveTo get: xl 2 + 100 get: y1 - LineTo
get: y1 put: y2 ;m
:m Field: ( IN: - OUT: - active neuron Plot )
set: fvp cls nneurons 0 DO i dup at: x swap at: y moveto i measure:
net
10 > IF ." ()" THEN LOOP ;m
:m Connectivity: ( IN: - OUT: connectivity Plot )
set: fvp 0 put: Count nConnect 0 DO i Connection?: Net IF
1 +: Count i nNeurons / ( source ) dup at: x 6 + swap at: y 4 - moveto
i nNeurons mod ( target ) dup at: x 6 + swap at: y 4 - lineto THEN LOOP
57 154 Moveto get: Count . ." connections " ;m
:m Build: ( IN: score OUT: - Builds interface, handles events )
put: Score
set: avp get: pos dup 50 > IF 500 * 0 DO i drop LOOP Field: self
ELSE drop THEN
1 +: time get: time 20 = IF 0 put: time ( seq 20 / mov 14 )
?Terminal IF quit THEN
get: mouse drop put: Pos
get: Score Line: self
set: avp get: pos 30 > IF Connectivity: self THEN
THEN ;m
;class
NSInterface Interface ( Quality Check: Passed 28-09-96 16:30 )
:class Animated super{ object } ( COORDINATE OBJECT INTERACTION )
int Score int LastScore int BestScore int Time
:m Init: ( >>> TRIGGER: INITIALIZE ALL OBJECTS: INTERFACE, WORLD, NETWORK
)
100 put: Score 100 put: LastScore 101 put: BestScore randomize
LineInit: Interface
ConInit: Interface
FieldInit: Interface
init: world ;m
:m Animate: ( THIS IS WHY THINGS HAPPEN )
Init: self
BEGIN
Update: Net Provide: World
nNeurons 0 DO i Activate: Net LOOP
Feedback: Net MeasureNovelty: Net +: Score
get: score Build: Interface
1 +: Time get: Time 20 = IF 0 put: Time ( Every 14/20 time steps:
)
Relived?: net IF
( REINFORCEMENT )
get: LastScore get: Score = IF ReleaseAgent: Net 100 random 80 > IF
KillNew: Net THEN THEN
get: Score get: LastScore > IF Punish: Net Relive: Net THEN
get: LastScore get: Score > IF Reward: Net ReleaseAgent: Net THEN
get: Score put: LastScore
ELSE Modulate: Net THEN 0 put: Score
THEN
AGAIN ;m
;class
Animated Servo
Animate: Servo bye
