SlideShare une entreprise Scribd logo
1  sur  47
Télécharger pour lire hors ligne
Artificial Immune Systems

      Andrew Watkins
Why the Immune System?
• Recognition
    – Anomaly detection
    – Noise tolerance
•   Robustness
•   Feature extraction
•   Diversity
•   Reinforcement learning
•   Memory
•   Distributed
•   Multi-layered
•   Adaptive
Definition

    AIS are adaptive systems inspired by
   theoretical immunology and observed
immune functions, principles and models,
  which are applied to complex problem
                                domains
                 (de Castro and Timmis)
Some History
• Developed from the field of theoretical
  immunology in the mid 1980’s.
  – Suggested we ‘might look’ at the IS
• 1990 – Bersini first use of immune algos to
  solve problems
• Forrest et al – Computer Security mid
  1990’s
• Hunt et al, mid 1990’s – Machine learning
How does it work?
Immune Pattern Recognition




• The immune recognition is based on the complementarity
  between the binding region of the receptor and a portion of
  the antigen called epitope.
• Antibodies present a single type of receptor, antigens
  might present several epitopes.
   – This means that different antibodies can recognize a single
     antigen
Immune Responses
                                   Primary Response               Secondary Response            Cross-Reactive
                                                                                                  Response
Antibody Concentration




                                                                                         Lag
                                                       Lag

                                                                   Response                          Response to
                             Lag                                    to Ag1                            Ag1 + Ag3
                                                                                          ...
                                            Response
                                             to Ag1                           Response
                                                                               to Ag2
                                                        ...
                                                        ...                               ...
                                                       Antigens                                                  Time
                   Antigen Ag1                                                            Antigen
                                                       Ag1, Ag2                          Ag1 + Ag3
Clonal Selection
Immune Network Theory
• Idiotypic network (Jerne, 1974)
• B cells co-stimulate each other
   – Treat each other a bit like antigens
• Creates an immunological memory
Shape Space Formalism
• Repertoire of the                Vε           ×
                                                         V
                                            ε

  immune system is                              Vε
                                                         ε
                              ×                      ×
  complete (Perelson, 1989)                              ×

                                                             ×
• Extensive regions of            Vε
                                        ε
                                                ×
  complementarity                       ×

• Some threshold of
  recognition
Self/Non-Self Recognition
• Immune system needs to be able to
  differentiate between self and non-self cells
• Antigenic encounters may result in cell
  death, therefore
  – Some kind of positive selection
  – Some element of negative selection
General Framework for AIS

                         Solution

                     Immune Algorithms

             Affinity Measures

        Representation

Application Domain
Representation – Shape Space
• Describe the general shape of a molecule




  •Describe interactions between molecules
  •Degree of binding between molecules
  •Complement threshold
Define their Interaction
• Define the term Affinity
• Affinity is related to distance
                           L
   – Euclidian    D=     ∑ ( Abi − Ag i ) 2
                          i =1

   • Other distance measures such as Hamming,
     Manhattan etc. etc.
   • Affinity Threshold
Basic Immune Models and
             Algorithms
•   Bone Marrow Models
•   Negative Selection Algorithms
•   Clonal Selection Algorithm
•   Somatic Hypermutation
•   Immune Network Models
Bone Marrow Models
• Gene libraries are used to create antibodies from
  the bone marrow
• Use this idea to generate attribute strings that
  represent receptors
• Antibody production through a random
  concatenation from gene libraries
Negative Selection Algorithms
• Forrest 1994: Idea taken from the negative
  selection of T-cells in the thymus
• Applied initially to computer security
• Split into two parts:
   – Censoring
   – Monitoring
Clonal Selection Algorithm (de
  Castro & von Zuben, 2001)
Randomly initialise a population (P)
 For each pattern in Ag
     Determine affinity to each Ab in P
     Select n highest affinity from P
        Clone and mutate prop. to affinity with Ag
     Add new mutants to P
   endFor
   Select highest affinity Ab in P to form part of M
   Replace n number of random new ones
Until stopping criteria
Immune Network Models
    (Timmis & Neal, 2001)
Initialise the immune network (P)


For each pattern in Ag
    Determine affinity to each Ab in P
    Calculate network interaction
               Allocate resources to the strongest members of P
              Remove weakest Ab in P
EndFor
    If termination condition met
              exit
    else
              Clone and mutate each Ab in P (based on a given probability)
              Integrate new mutants into P based on affinity
Repeat
Somatic Hypermutation
• Mutation rate in proportion to affinity
• Very controlled mutation in the natural immune
  system
• The greater the antibody affinity the smaller its
  mutation rate
• Classic trade-off between exploration and
  exploitation
How do AIS Compare?
• Basic Components:
  – AIS  B-cell in shape space (e.g. attribute
    strings)
     • Stimulation level
  – ANN  Neuron
     • Activation function
  – GA  chromosome
     • fitness
Comparing
• Structure (Architecture)
  – AIS and GA fixed or variable sized
    populations, not connected in population based
    AIS
  – ANN and AIS
     • Do have network based AIS
     • ANN typically fixed structure (not always)
     • Learning takes place in weights in ANN
Comparing
• Memory
  – AIS  in B-cells
    • Network models in connections
  – ANN  In weights of connections
  – GA  individual chromosome
Comparing
•   Adaptation
•   Dynamics
•   Metadynamics
•   Interactions
•   Generalisation capabilities
•   Etc. many more.
Where are they used?
•   Dependable systems
•   Scheduling
•   Robotics
•   Security
•   Anomaly detection
•   Learning systems
Artificial Immune Recognition
        System (AIRS):

  An Immune-Inspired Supervised
       Learning Algorithm
AIRS: Immune Principles
            Employed
• Clonal Selection
• Based initially on immune networks, though
  found this did not work
• Somatic hypermutation
  – Eventually
• Recognition regions within shape space
• Antibody/antigen binding
AIRS: Mapping from IS to AIS
• Antibody        Feature Vector
• Recognition     Combination of feature
  Ball (RB)       vector and vector class
• Antigens        Training Data
• Immune Memory   Memory cells—set of
                  mutated Artificial RBs
Classification
• Stimulation of an ARB is based not only on its
  affinity to an antigen but also on its class when
  compared to the class of an antigen
• Allocation of resources to the ARBs also takes
  into account the ARBs’ classifications when
  compared to the class of the antigen
• Memory cell hyper-mutation and replacement is
  based primarily on classification and secondarily
  on affinity
AIRS Algorithm
• Data normalization and initialization
• Memory cell identification and ARB
  generation
• Competition for resources in the
  development of a candidate memory cell
• Potential introduction of the candidate
  memory cell into the set of established
  memory cells
Memory Cell Identification
A                Memory Cell Pool




                       ARB Pool
MCmatch Found
A   1           Memory Cell Pool

          MCmatch




                      ARB Pool
ARB Generation
A   1                             Memory Cell Pool

                            MCmatch

        Mutated Offspring
                            2



                                        ARB Pool
Exposure of ARBs to Antigen
 A       1                             Memory Cell Pool

                                 MCmatch

             Mutated Offspring
     3                           2



                                             ARB Pool
Development of a Candidate
      Memory Cell
A       1                             Memory Cell Pool

                                MCmatch

            Mutated Offspring
    3                           2



                                            ARB Pool
Comparison of MCcandidate and
          MCmatch
   A       1                             Memory Cell Pool

                                   MCmatch

                                                              4    A
               Mutated Offspring
       3                           2                        MC candidate




                                               ARB Pool
Memory Cell Introduction
A       1                             Memory Cell Pool

                                MCmatch

                                                           4    A
    3
            Mutated Offspring             5
                                2                        MCcandidate




                                              ARB Pool
Memory Cells and Antigens
Memory Cells and Antigens
AIRS: Performance Evaluation

                         Pima Indians Diabetes
Fisher’s Iris Data Set
                                Data Set


Ionosphere Data Set         Sonar Data Set
Iris                  Ionosphere           Diabetes                        Sonar
1    Grobian       100%    3-NN +       98.7%   Logdisc               77.7%     TAP MFT              92.3%
     (rough)               simplex                                              Bayesian
2    SSV           98.0%   3-NN         96.7%   IncNet                77.6%     Naïve MFT Bayesian   90.4%
3    C-MLP2LN      98.0%   IB3          96.7%   DIPOL92               77.6%     SVM                  90.4%
4    PVM 2 rules   98.0%   MLP + BP     96.0%   Linear Discr. Anal.   77.5%-    Best 2-layer MLP     90.4%
                                                                      77.2%     + BP, 12 hidden
5    PVM 1 rule    97.3%                        SMART                 76.8%     MLP+BP, 12 hidden    84.7%
                           AIRS         94.9
6                          C4.5         94.9%   GTO DT (5xCV)         76.8%     MLP+BP, 24 hidden    84.5%
     AIRS          96.7
7    FuNe-I        96.7%   RIAC         94.6%   ASI                   76.6%     1-NN, Manhatten      84.2%
8    NEFCLASS      96.7%   SVM          93.2%   Fischer discr. anal   76.5%     AIRS                 84.0
9    CART          96.0%   Non-linear   92.0%   MLP+BP                76.4%     MLP+BP, 6            83.5%
                           perceptron                                           hidden
10   FUNN          95.7%   FSM +        92.8%   LVQ                   75.8%     FSM -                83.6%
                           rotation                                             methodology?
11                         1-NN         92.1%   LFC                   75.8%     1-NN Euclidean       82.2%
12                         DB-CART      91.3%   RBF                   75.7%     DB-CART, 10xCV       81.8%
13                         Linear       90.7%   NB                    75.5-     CART, 10xCV          67.9%
                           perceptron                                 73.8%
14                         OC1 DT       89.5%   kNN, k=22, Manh       75.5%
15                         CART         88.9%   MML                   75.5%
…                                               ...
22                                              AIRS                  74.1
23                                              C4.5                  73.0%
                                                11 others reported with lower
                                                scores, including Bayes,
                                                Kohonen, kNN, ID3 …
AIRS: Observations
• ARB Pool formulation was over
  complicated
  – Crude visualization
  – Memory only needs to be maintained in the
    Memory Cell Pool
• Mutation Routine
  – Difference in Quality
  – Some redundancy
AIRS: Revisions
• Memory Cell Evolution
  – Only Memory Cell Pool has different classes
  – ARB Pool only concerned with evolving
    memory cells
• Somatic Hypermutation
  – Cell’s stimulation value indicates range of
    mutation possibilities
  – No longer need to mutate class
Comparisons: Classification
           Accuracy
• Important to maintain accuracy
                AIRS1: Accuracy   AIRS2: Accuracy




   Iris         96.7              96.0



   Ionosphere   94.9              95.6




   Diabetes     74.1              74.2



   Sonar        84.0              84.9




• Why bother?
Comparisons: Data Reduction
• Increase data reduction—increased
  efficiency
               Training Set Size   AIRS1: Memory Cells   AIRS2: Memory Cells



  Iris         120                 42.1 / 65%            30.9 / 74%


  Ionosphere   200                 140.7 / 30%           96.3 / 52%


  Diabetes     691                 470.4 / 32%           273.4 / 60%


  Sonar        192                 144.6 / 25%           177.7 / 7%
Features of AIRS
• No need to know best architecture to get
  good results
• Default settings within a few percent of the
  best it can get
• User-adjustable parameters optimize
  performance for a given problem set
• Generalization and data reduction
More Information
• http://www.cs.ukc.ac.uk/people/rpg/abw5
• http://www.cs.ukc.ac.uk/people/staff/jt6
• http://www.cs.ukc.ac.uk/aisbook

Contenu connexe

Plus de Franco Bressan (20)

Systems se
Systems seSystems se
Systems se
 
Especial novarejo
Especial novarejoEspecial novarejo
Especial novarejo
 
1317856
13178561317856
1317856
 
1316975
13169751316975
1316975
 
1315741
13157411315741
1315741
 
1316244
13162441316244
1316244
 
Logic of expression_intro
Logic of expression_introLogic of expression_intro
Logic of expression_intro
 
29404
2940429404
29404
 
Paty,m 1997d unverslde-cienc
Paty,m 1997d unverslde-ciencPaty,m 1997d unverslde-cienc
Paty,m 1997d unverslde-cienc
 
Fragoso5
Fragoso5Fragoso5
Fragoso5
 
Deleuze luiz
Deleuze luizDeleuze luiz
Deleuze luiz
 
01 02 03_marcelo
01 02 03_marcelo01 02 03_marcelo
01 02 03_marcelo
 
Tnhp daniel garber
Tnhp daniel garberTnhp daniel garber
Tnhp daniel garber
 
18394
1839418394
18394
 
2004 033 33001014010_p9_disc_ofe
2004 033 33001014010_p9_disc_ofe2004 033 33001014010_p9_disc_ofe
2004 033 33001014010_p9_disc_ofe
 
Hz242 a
Hz242 aHz242 a
Hz242 a
 
Sens public f_caprio_substancia_liberdade_e_existencia
Sens public f_caprio_substancia_liberdade_e_existenciaSens public f_caprio_substancia_liberdade_e_existencia
Sens public f_caprio_substancia_liberdade_e_existencia
 
M16yg
M16ygM16yg
M16yg
 
Ed. 1 tjdft_ed._abt
Ed. 1 tjdft_ed._abtEd. 1 tjdft_ed._abt
Ed. 1 tjdft_ed._abt
 
22946884 a-coarse-taxonomy-of-artificial-immune-systems
22946884 a-coarse-taxonomy-of-artificial-immune-systems22946884 a-coarse-taxonomy-of-artificial-immune-systems
22946884 a-coarse-taxonomy-of-artificial-immune-systems
 

Dernier

AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 

Dernier (20)

AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 

Ais machine learning

  • 1. Artificial Immune Systems Andrew Watkins
  • 2. Why the Immune System? • Recognition – Anomaly detection – Noise tolerance • Robustness • Feature extraction • Diversity • Reinforcement learning • Memory • Distributed • Multi-layered • Adaptive
  • 3. Definition AIS are adaptive systems inspired by theoretical immunology and observed immune functions, principles and models, which are applied to complex problem domains (de Castro and Timmis)
  • 4. Some History • Developed from the field of theoretical immunology in the mid 1980’s. – Suggested we ‘might look’ at the IS • 1990 – Bersini first use of immune algos to solve problems • Forrest et al – Computer Security mid 1990’s • Hunt et al, mid 1990’s – Machine learning
  • 5. How does it work?
  • 6. Immune Pattern Recognition • The immune recognition is based on the complementarity between the binding region of the receptor and a portion of the antigen called epitope. • Antibodies present a single type of receptor, antigens might present several epitopes. – This means that different antibodies can recognize a single antigen
  • 7. Immune Responses Primary Response Secondary Response Cross-Reactive Response Antibody Concentration Lag Lag Response Response to Lag to Ag1 Ag1 + Ag3 ... Response to Ag1 Response to Ag2 ... ... ... Antigens Time Antigen Ag1 Antigen Ag1, Ag2 Ag1 + Ag3
  • 9. Immune Network Theory • Idiotypic network (Jerne, 1974) • B cells co-stimulate each other – Treat each other a bit like antigens • Creates an immunological memory
  • 10. Shape Space Formalism • Repertoire of the Vε × V ε immune system is Vε ε × × complete (Perelson, 1989) × × • Extensive regions of Vε ε × complementarity × • Some threshold of recognition
  • 11. Self/Non-Self Recognition • Immune system needs to be able to differentiate between self and non-self cells • Antigenic encounters may result in cell death, therefore – Some kind of positive selection – Some element of negative selection
  • 12. General Framework for AIS Solution Immune Algorithms Affinity Measures Representation Application Domain
  • 13. Representation – Shape Space • Describe the general shape of a molecule •Describe interactions between molecules •Degree of binding between molecules •Complement threshold
  • 14. Define their Interaction • Define the term Affinity • Affinity is related to distance L – Euclidian D= ∑ ( Abi − Ag i ) 2 i =1 • Other distance measures such as Hamming, Manhattan etc. etc. • Affinity Threshold
  • 15. Basic Immune Models and Algorithms • Bone Marrow Models • Negative Selection Algorithms • Clonal Selection Algorithm • Somatic Hypermutation • Immune Network Models
  • 16. Bone Marrow Models • Gene libraries are used to create antibodies from the bone marrow • Use this idea to generate attribute strings that represent receptors • Antibody production through a random concatenation from gene libraries
  • 17. Negative Selection Algorithms • Forrest 1994: Idea taken from the negative selection of T-cells in the thymus • Applied initially to computer security • Split into two parts: – Censoring – Monitoring
  • 18. Clonal Selection Algorithm (de Castro & von Zuben, 2001) Randomly initialise a population (P) For each pattern in Ag Determine affinity to each Ab in P Select n highest affinity from P Clone and mutate prop. to affinity with Ag Add new mutants to P endFor Select highest affinity Ab in P to form part of M Replace n number of random new ones Until stopping criteria
  • 19. Immune Network Models (Timmis & Neal, 2001) Initialise the immune network (P) For each pattern in Ag Determine affinity to each Ab in P Calculate network interaction Allocate resources to the strongest members of P Remove weakest Ab in P EndFor If termination condition met exit else Clone and mutate each Ab in P (based on a given probability) Integrate new mutants into P based on affinity Repeat
  • 20. Somatic Hypermutation • Mutation rate in proportion to affinity • Very controlled mutation in the natural immune system • The greater the antibody affinity the smaller its mutation rate • Classic trade-off between exploration and exploitation
  • 21. How do AIS Compare? • Basic Components: – AIS  B-cell in shape space (e.g. attribute strings) • Stimulation level – ANN  Neuron • Activation function – GA  chromosome • fitness
  • 22. Comparing • Structure (Architecture) – AIS and GA fixed or variable sized populations, not connected in population based AIS – ANN and AIS • Do have network based AIS • ANN typically fixed structure (not always) • Learning takes place in weights in ANN
  • 23. Comparing • Memory – AIS  in B-cells • Network models in connections – ANN  In weights of connections – GA  individual chromosome
  • 24. Comparing • Adaptation • Dynamics • Metadynamics • Interactions • Generalisation capabilities • Etc. many more.
  • 25. Where are they used? • Dependable systems • Scheduling • Robotics • Security • Anomaly detection • Learning systems
  • 26. Artificial Immune Recognition System (AIRS): An Immune-Inspired Supervised Learning Algorithm
  • 27. AIRS: Immune Principles Employed • Clonal Selection • Based initially on immune networks, though found this did not work • Somatic hypermutation – Eventually • Recognition regions within shape space • Antibody/antigen binding
  • 28. AIRS: Mapping from IS to AIS • Antibody Feature Vector • Recognition Combination of feature Ball (RB) vector and vector class • Antigens Training Data • Immune Memory Memory cells—set of mutated Artificial RBs
  • 29. Classification • Stimulation of an ARB is based not only on its affinity to an antigen but also on its class when compared to the class of an antigen • Allocation of resources to the ARBs also takes into account the ARBs’ classifications when compared to the class of the antigen • Memory cell hyper-mutation and replacement is based primarily on classification and secondarily on affinity
  • 30. AIRS Algorithm • Data normalization and initialization • Memory cell identification and ARB generation • Competition for resources in the development of a candidate memory cell • Potential introduction of the candidate memory cell into the set of established memory cells
  • 31. Memory Cell Identification A Memory Cell Pool ARB Pool
  • 32. MCmatch Found A 1 Memory Cell Pool MCmatch ARB Pool
  • 33. ARB Generation A 1 Memory Cell Pool MCmatch Mutated Offspring 2 ARB Pool
  • 34. Exposure of ARBs to Antigen A 1 Memory Cell Pool MCmatch Mutated Offspring 3 2 ARB Pool
  • 35. Development of a Candidate Memory Cell A 1 Memory Cell Pool MCmatch Mutated Offspring 3 2 ARB Pool
  • 36. Comparison of MCcandidate and MCmatch A 1 Memory Cell Pool MCmatch 4 A Mutated Offspring 3 2 MC candidate ARB Pool
  • 37. Memory Cell Introduction A 1 Memory Cell Pool MCmatch 4 A 3 Mutated Offspring 5 2 MCcandidate ARB Pool
  • 38. Memory Cells and Antigens
  • 39. Memory Cells and Antigens
  • 40. AIRS: Performance Evaluation Pima Indians Diabetes Fisher’s Iris Data Set Data Set Ionosphere Data Set Sonar Data Set
  • 41. Iris Ionosphere Diabetes Sonar 1 Grobian 100% 3-NN + 98.7% Logdisc 77.7% TAP MFT 92.3% (rough) simplex Bayesian 2 SSV 98.0% 3-NN 96.7% IncNet 77.6% Naïve MFT Bayesian 90.4% 3 C-MLP2LN 98.0% IB3 96.7% DIPOL92 77.6% SVM 90.4% 4 PVM 2 rules 98.0% MLP + BP 96.0% Linear Discr. Anal. 77.5%- Best 2-layer MLP 90.4% 77.2% + BP, 12 hidden 5 PVM 1 rule 97.3% SMART 76.8% MLP+BP, 12 hidden 84.7% AIRS 94.9 6 C4.5 94.9% GTO DT (5xCV) 76.8% MLP+BP, 24 hidden 84.5% AIRS 96.7 7 FuNe-I 96.7% RIAC 94.6% ASI 76.6% 1-NN, Manhatten 84.2% 8 NEFCLASS 96.7% SVM 93.2% Fischer discr. anal 76.5% AIRS 84.0 9 CART 96.0% Non-linear 92.0% MLP+BP 76.4% MLP+BP, 6 83.5% perceptron hidden 10 FUNN 95.7% FSM + 92.8% LVQ 75.8% FSM - 83.6% rotation methodology? 11 1-NN 92.1% LFC 75.8% 1-NN Euclidean 82.2% 12 DB-CART 91.3% RBF 75.7% DB-CART, 10xCV 81.8% 13 Linear 90.7% NB 75.5- CART, 10xCV 67.9% perceptron 73.8% 14 OC1 DT 89.5% kNN, k=22, Manh 75.5% 15 CART 88.9% MML 75.5% … ... 22 AIRS 74.1 23 C4.5 73.0% 11 others reported with lower scores, including Bayes, Kohonen, kNN, ID3 …
  • 42. AIRS: Observations • ARB Pool formulation was over complicated – Crude visualization – Memory only needs to be maintained in the Memory Cell Pool • Mutation Routine – Difference in Quality – Some redundancy
  • 43. AIRS: Revisions • Memory Cell Evolution – Only Memory Cell Pool has different classes – ARB Pool only concerned with evolving memory cells • Somatic Hypermutation – Cell’s stimulation value indicates range of mutation possibilities – No longer need to mutate class
  • 44. Comparisons: Classification Accuracy • Important to maintain accuracy AIRS1: Accuracy AIRS2: Accuracy Iris 96.7 96.0 Ionosphere 94.9 95.6 Diabetes 74.1 74.2 Sonar 84.0 84.9 • Why bother?
  • 45. Comparisons: Data Reduction • Increase data reduction—increased efficiency Training Set Size AIRS1: Memory Cells AIRS2: Memory Cells Iris 120 42.1 / 65% 30.9 / 74% Ionosphere 200 140.7 / 30% 96.3 / 52% Diabetes 691 470.4 / 32% 273.4 / 60% Sonar 192 144.6 / 25% 177.7 / 7%
  • 46. Features of AIRS • No need to know best architecture to get good results • Default settings within a few percent of the best it can get • User-adjustable parameters optimize performance for a given problem set • Generalization and data reduction
  • 47. More Information • http://www.cs.ukc.ac.uk/people/rpg/abw5 • http://www.cs.ukc.ac.uk/people/staff/jt6 • http://www.cs.ukc.ac.uk/aisbook

Notes de l'éditeur

  1.          Uniqueness : each individual possesses its own immune system, with its particular vulnerabilities and capabilities;          Diversity : there is a large amount of types of elements (cells, molecules, proteins, etc.) that altogether perform the same role of protecting the body from malefic invaders. Additionally, there are different fronts of defense, like innate and adaptive immunity;          Disposability ( robustness ): no single component of the natural immune system is essential for its functioning. Cell death is usually balanced by cell production;          Autonomy : the immune system does not require outside management or maintenance. It autonomously classifies and eliminates pathogens, and it repairs itself by replacing damaged cells;          Multilayered : multiple layers of different mechanisms are combined to provide high overall security, as summarized in Figure 2.5 (Section 2.3);          No secure layer : any cell of the human body can be attacked by the immune system, including those of the immune system itself;          Recognition of foreigners : the (harmful) molecules that are not native to the body are recognized and eliminated by the immune system;          Anomaly detection : the immune system can detect and react to pathogens that the body has never encountered before;          Dynamically changing coverage : as the immune system can not maintain a set of cells and molecules large enough to detect all pathogens, it makes a trade-off between space and time. It maintains a circulating pool of lymphocytes that is constantly being changed through cell death, production and reproduction;          Distributability : the immune cells, molecules and organs are distributed all over the body and, most importantly, are not subject to any centralized control;          Imperfect detection ( noise tolerance ): an absolute recognition of the pathogens is not required, hence the system is flexible;          Reinforcement learning and memory : the immune system can “learn” the structures of pathogens. It retains the ability to recognize previously seen pathogens through immune memory, so that future responses to the same pathogens are faster and stronger; and          An arms race : the vertebrate immune system replicates cells to deal with replicating pathogens, otherwise the pathogens would quickly overwhelm the immune defenses.
  2. Mention Bersinis' principles
  3. 1.      Randomly initialize a population of individuals ( P ); 2.      For each pattern of S , present it to the population P and determine its affinity with each element of the population P ; 3.      Select n 1 highest affinity elements of P and generate copies of these individuals proportionally to their affinity with the antigen. The higher the affinity, the higher the number of copies, and vice-versa; 4.      Mutate all these copies with a rate proportional to their affinity with the input pattern: the higher the affinity, the smaller the mutation rate, and vice-versa; 5.      Add these mutated individuals to the population P and re-select n 2 of these maturated (optimized) individuals to be kept as the memory M of the system; 6.      Replace a number n 3 of individuals with low affinity by (randomly generated) new ones; 7.      Repeat Steps 2 to 6 until a certain stopping criterion is met.
  4. Initialise the immune network (P) For each pattern in Ag Determine affinity to each P’ Calculate network interaction Allocate resources to the strongest members of P Remove weakest P EndFor If termination condition met exit else Clone and mutate each P (based on probability a) Integrate new mutants into P based on affinity Repeat
  5. Sparse in AIS literature Not as straight forward as initially suspected
  6. MCmatch is found
  7. New ARBs are generated to be put into the population
  8. The competition for system wide resources The use of mutation for diversification and shape-space exploration The use of an average stimulation threshold as a criterion for determining when to stop training on a given antigen
  9. The competition for system wide resources The use of mutation for diversification and shape-space exploration The use of an average stimulation threshold as a criterion for determining when to stop training on a given antigen
  10. Compare response of MCmatch and MCcandidate to the antigen. Compare the affinity value of MCmatch and MCcandidate to each other
  11. Introduce the just-developed candidate memory cell, MCcandidate , into the set of existing memory cells MC Replace MCmatch The evolved memory cells are available for use for classification.
  12. Iris: 3 way classification problem; 150 data items; 5XCV; avg. 3 times; 4 features Ionosphere: 2-way classification, good & bad radar returns; 34 features ; 200 in training, 151 test set Diabetes: 2-way class, has diabetes or not; 10XCV; 8 features ; 768 instances total Sonar: 2-way class; 13XCV; 60 features ; 16 instances in each test set