3. Definition
AIS are adaptive systems inspired by
theoretical immunology and observed
immune functions, principles and models,
which are applied to complex problem
domains
(de Castro and Timmis)
4. Some History
• Developed from the field of theoretical
immunology in the mid 1980’s.
– Suggested we ‘might look’ at the IS
• 1990 – Bersini first use of immune algos to
solve problems
• Forrest et al – Computer Security mid
1990’s
• Hunt et al, mid 1990’s – Machine learning
6. Immune Pattern Recognition
• The immune recognition is based on the complementarity
between the binding region of the receptor and a portion of
the antigen called epitope.
• Antibodies present a single type of receptor, antigens
might present several epitopes.
– This means that different antibodies can recognize a single
antigen
7. Immune Responses
Primary Response Secondary Response Cross-Reactive
Response
Antibody Concentration
Lag
Lag
Response Response to
Lag to Ag1 Ag1 + Ag3
...
Response
to Ag1 Response
to Ag2
...
... ...
Antigens Time
Antigen Ag1 Antigen
Ag1, Ag2 Ag1 + Ag3
9. Immune Network Theory
• Idiotypic network (Jerne, 1974)
• B cells co-stimulate each other
– Treat each other a bit like antigens
• Creates an immunological memory
10. Shape Space Formalism
• Repertoire of the Vε ×
V
ε
immune system is Vε
ε
× ×
complete (Perelson, 1989) ×
×
• Extensive regions of Vε
ε
×
complementarity ×
• Some threshold of
recognition
11. Self/Non-Self Recognition
• Immune system needs to be able to
differentiate between self and non-self cells
• Antigenic encounters may result in cell
death, therefore
– Some kind of positive selection
– Some element of negative selection
12. General Framework for AIS
Solution
Immune Algorithms
Affinity Measures
Representation
Application Domain
13. Representation – Shape Space
• Describe the general shape of a molecule
•Describe interactions between molecules
•Degree of binding between molecules
•Complement threshold
14. Define their Interaction
• Define the term Affinity
• Affinity is related to distance
L
– Euclidian D= ∑ ( Abi − Ag i ) 2
i =1
• Other distance measures such as Hamming,
Manhattan etc. etc.
• Affinity Threshold
16. Bone Marrow Models
• Gene libraries are used to create antibodies from
the bone marrow
• Use this idea to generate attribute strings that
represent receptors
• Antibody production through a random
concatenation from gene libraries
17. Negative Selection Algorithms
• Forrest 1994: Idea taken from the negative
selection of T-cells in the thymus
• Applied initially to computer security
• Split into two parts:
– Censoring
– Monitoring
18. Clonal Selection Algorithm (de
Castro & von Zuben, 2001)
Randomly initialise a population (P)
For each pattern in Ag
Determine affinity to each Ab in P
Select n highest affinity from P
Clone and mutate prop. to affinity with Ag
Add new mutants to P
endFor
Select highest affinity Ab in P to form part of M
Replace n number of random new ones
Until stopping criteria
19. Immune Network Models
(Timmis & Neal, 2001)
Initialise the immune network (P)
For each pattern in Ag
Determine affinity to each Ab in P
Calculate network interaction
Allocate resources to the strongest members of P
Remove weakest Ab in P
EndFor
If termination condition met
exit
else
Clone and mutate each Ab in P (based on a given probability)
Integrate new mutants into P based on affinity
Repeat
20. Somatic Hypermutation
• Mutation rate in proportion to affinity
• Very controlled mutation in the natural immune
system
• The greater the antibody affinity the smaller its
mutation rate
• Classic trade-off between exploration and
exploitation
21. How do AIS Compare?
• Basic Components:
– AIS B-cell in shape space (e.g. attribute
strings)
• Stimulation level
– ANN Neuron
• Activation function
– GA chromosome
• fitness
22. Comparing
• Structure (Architecture)
– AIS and GA fixed or variable sized
populations, not connected in population based
AIS
– ANN and AIS
• Do have network based AIS
• ANN typically fixed structure (not always)
• Learning takes place in weights in ANN
23. Comparing
• Memory
– AIS in B-cells
• Network models in connections
– ANN In weights of connections
– GA individual chromosome
24. Comparing
• Adaptation
• Dynamics
• Metadynamics
• Interactions
• Generalisation capabilities
• Etc. many more.
25. Where are they used?
• Dependable systems
• Scheduling
• Robotics
• Security
• Anomaly detection
• Learning systems
27. AIRS: Immune Principles
Employed
• Clonal Selection
• Based initially on immune networks, though
found this did not work
• Somatic hypermutation
– Eventually
• Recognition regions within shape space
• Antibody/antigen binding
28. AIRS: Mapping from IS to AIS
• Antibody Feature Vector
• Recognition Combination of feature
Ball (RB) vector and vector class
• Antigens Training Data
• Immune Memory Memory cells—set of
mutated Artificial RBs
29. Classification
• Stimulation of an ARB is based not only on its
affinity to an antigen but also on its class when
compared to the class of an antigen
• Allocation of resources to the ARBs also takes
into account the ARBs’ classifications when
compared to the class of the antigen
• Memory cell hyper-mutation and replacement is
based primarily on classification and secondarily
on affinity
30. AIRS Algorithm
• Data normalization and initialization
• Memory cell identification and ARB
generation
• Competition for resources in the
development of a candidate memory cell
• Potential introduction of the candidate
memory cell into the set of established
memory cells
42. AIRS: Observations
• ARB Pool formulation was over
complicated
– Crude visualization
– Memory only needs to be maintained in the
Memory Cell Pool
• Mutation Routine
– Difference in Quality
– Some redundancy
43. AIRS: Revisions
• Memory Cell Evolution
– Only Memory Cell Pool has different classes
– ARB Pool only concerned with evolving
memory cells
• Somatic Hypermutation
– Cell’s stimulation value indicates range of
mutation possibilities
– No longer need to mutate class
46. Features of AIRS
• No need to know best architecture to get
good results
• Default settings within a few percent of the
best it can get
• User-adjustable parameters optimize
performance for a given problem set
• Generalization and data reduction
Uniqueness : each individual possesses its own immune system, with its particular vulnerabilities and capabilities; Diversity : there is a large amount of types of elements (cells, molecules, proteins, etc.) that altogether perform the same role of protecting the body from malefic invaders. Additionally, there are different fronts of defense, like innate and adaptive immunity; Disposability ( robustness ): no single component of the natural immune system is essential for its functioning. Cell death is usually balanced by cell production; Autonomy : the immune system does not require outside management or maintenance. It autonomously classifies and eliminates pathogens, and it repairs itself by replacing damaged cells; Multilayered : multiple layers of different mechanisms are combined to provide high overall security, as summarized in Figure 2.5 (Section 2.3); No secure layer : any cell of the human body can be attacked by the immune system, including those of the immune system itself; Recognition of foreigners : the (harmful) molecules that are not native to the body are recognized and eliminated by the immune system; Anomaly detection : the immune system can detect and react to pathogens that the body has never encountered before; Dynamically changing coverage : as the immune system can not maintain a set of cells and molecules large enough to detect all pathogens, it makes a trade-off between space and time. It maintains a circulating pool of lymphocytes that is constantly being changed through cell death, production and reproduction; Distributability : the immune cells, molecules and organs are distributed all over the body and, most importantly, are not subject to any centralized control; Imperfect detection ( noise tolerance ): an absolute recognition of the pathogens is not required, hence the system is flexible; Reinforcement learning and memory : the immune system can “learn” the structures of pathogens. It retains the ability to recognize previously seen pathogens through immune memory, so that future responses to the same pathogens are faster and stronger; and An arms race : the vertebrate immune system replicates cells to deal with replicating pathogens, otherwise the pathogens would quickly overwhelm the immune defenses.
Mention Bersinis' principles
1. Randomly initialize a population of individuals ( P ); 2. For each pattern of S , present it to the population P and determine its affinity with each element of the population P ; 3. Select n 1 highest affinity elements of P and generate copies of these individuals proportionally to their affinity with the antigen. The higher the affinity, the higher the number of copies, and vice-versa; 4. Mutate all these copies with a rate proportional to their affinity with the input pattern: the higher the affinity, the smaller the mutation rate, and vice-versa; 5. Add these mutated individuals to the population P and re-select n 2 of these maturated (optimized) individuals to be kept as the memory M of the system; 6. Replace a number n 3 of individuals with low affinity by (randomly generated) new ones; 7. Repeat Steps 2 to 6 until a certain stopping criterion is met.
Initialise the immune network (P) For each pattern in Ag Determine affinity to each P’ Calculate network interaction Allocate resources to the strongest members of P Remove weakest P EndFor If termination condition met exit else Clone and mutate each P (based on probability a) Integrate new mutants into P based on affinity Repeat
Sparse in AIS literature Not as straight forward as initially suspected
MCmatch is found
New ARBs are generated to be put into the population
The competition for system wide resources The use of mutation for diversification and shape-space exploration The use of an average stimulation threshold as a criterion for determining when to stop training on a given antigen
The competition for system wide resources The use of mutation for diversification and shape-space exploration The use of an average stimulation threshold as a criterion for determining when to stop training on a given antigen
Compare response of MCmatch and MCcandidate to the antigen. Compare the affinity value of MCmatch and MCcandidate to each other
Introduce the just-developed candidate memory cell, MCcandidate , into the set of existing memory cells MC Replace MCmatch The evolved memory cells are available for use for classification.
Iris: 3 way classification problem; 150 data items; 5XCV; avg. 3 times; 4 features Ionosphere: 2-way classification, good & bad radar returns; 34 features ; 200 in training, 151 test set Diabetes: 2-way class, has diabetes or not; 10XCV; 8 features ; 768 instances total Sonar: 2-way class; 13XCV; 60 features ; 16 instances in each test set