Interactive Activation and Competition

Aim:
Interactive activation and competition nets are a class of neural network different from backpropagation networks. They tend to model choice and clustering phenomena in biological neural networks
Reference: Chapter 2 of McClelland, JL and Rumelhart, DE, Explorations in Parallel Distributed Processing, MIT Press, 1989.
Plan:
  • network architecture
  • update rule
  • network dynamics
  • how competition works
  • Jets and Sharks dataset
  • retrieval from partial descriptions
  • graceful degradation
  • generalization


Intro to Interactive Activation and Competition (IAC) Nets


IAC Network Diagram

IAC network diagram

[If you are looking at a black-and-white printout of this diagram, the (red) inhibitory connections are the ones entirely inside pools, and the (blue) excitatory connections are the ones between pools.]


IAC Unit Diagram

IAC unit diagram


IAC Network Activation, Threshold, and Weights


Net Input, and Change of Activation


The Update Rule


Activation Dynamics


Activation Stable Point


How Competition Works


Resonance

1. Grossberg, S., A theory of visual coding, memory, and development, in E.L.J. Leeuwenberg & H.F.J.M. Buffart (eds), Formal theories of visual perception. New York: Wiley, 1978.


Example of Resonance in the Presence of Decay


Retrieval and Generalisation in Human Memory

An IAC system can be used to illustrate the following properties of human memory:


The "Jets and Sharks" Database

NameGangAgeEducationMarital
Status
Occupation
ArtJets40sJunior HighSinglePusher
AlJets30sJunior HighMarriedBurglar
SamJets20sCollegeSingleBookie
ClydeJets40sJunior HighSingleBookie
MikeJets30sJunior HighSingleBookie
JimJets20sJunior HighDivorcedBurglar
GregJets20sHigh SchoolMarriedPusher
JohnJets20sJunior HighMarriedBurglar
DougJets30sHigh SchoolSingleBookie
LanceJets20sJunior HighMarriedBurglar
GeorgeJets20sJunior HighDivorcedBurglar
PeteJets20sHigh SchoolSingleBookie
FredJets20sHigh SchoolSinglePusher
GeneJets20sCollegeSinglePusher
RalphJets30sJunior HighSinglePusher
PhilSharks30sCollegeMarriedPusher
IkeSharks30sJunior HighSingleBookie
NickSharks30sHigh SchoolSinglePusher
DonSharks30sCollegeMarriedBurglar
NedSharks30sCollegeMarriedBookie
KarlSharks40sHigh SchoolMarriedBookie
KenSharks20sHigh SchoolSingleBurglar
EarlSharks40sHigh SchoolMarriedBurglar
RickSharks30sHigh SchoolDivorcedBurglar
OlSharks30sCollegeMarriedPusher
NealSharks30sHigh SchoolSingleBookie
DaveSharks30sHigh SchoolDivorcedPusher

Characteristics of a number of individuals belonging to two gangs, the Jets and the Sharks. (From "Retrieving General and Specific Knowledge From Stored Knowledge of Specifics" by J. L. McClelland, 1981, Proceedings of the Third Annual Conference of the Cognitive Science Society.)


Some Jets+Sharks Connections

Diagram of Jets and Sharks network connnections

After a diagram in the McClelland paper cited above.


PDP++ Software


Retrieving an Individual from their Name


Retrieval from Partial Description


Graceful Degradation


Spontaneous Generalisation


CRICOS Provider Code No. 00098G
Copyright (C) Bill Wilson, 2012, except where another source is acknowledged.