Real-time Learning of Vehicle Suspension Control Laws Talk given in 1997 on applying learning automata so control

1. Real-time Learning of
Vehicle Suspension
Control Laws
M. N. Howell BSc MSc PhD
G. P. Frost BEng
T.J. Gordon BA MA PhD
*Q.H. Wu MSc PhD

2. OUTLINE OF TALK
• Introduction
• Suspension Control Problem
• Learning Control
• Results
• Conclusions

3. SUSPENSION CONTROL
Objective:
Improve Ride Comfort
By
Reduce Vehicle Body Acceleration

4. Types of Suspension
• Passive Suspension
Standard Spring and Damper
• Active
Force Generator Replaces Damper
• Semi-Active
Controllable Dampers

6. Motivation for Learning Control
• Difficult System To Model
• Non-linear
• Interaction with Engine Modes
• Noise Disturbance Input
Measurement Noise
• Uncertainty In System Parameters
• System Identification is an Approximation

7. REINFORCEMENT
LEARNING CONTROL
• Direct Learning
No System Model
• On-Line Learning
No Pre-Training
• No Separate Training Stage Required

8. Pseudo Code
Repeat
Generate Actions
Execute The Actions On The
Environment
Receive Performance Evaluation For This
Action
If "Favourable" Then
Promote Action In Future
Situations
Until Convergence Criterion Satisfied.

9. CONTINUOUS ACTION
REINFORCEMENT LEARNING
AUTOMATA ( CARLA )
• Related To Discrete Learning Automata
• Each CARLA Operates On A Single Action
Variable
• Continuous Parameter Range
• Reward/Inaction Based
• Suitable For Stochastic Systems
• Distributed Control
• Simple Implementation

12. Learning Implementation
Environment
Actions
Performance
Evaluation
Function
β
J
(Vehicle)
CARLA
CARLA
CARLA

17. PASSIVE REFERENCE
STAGE
LEARNING CONTROLLER
TRANSIENT RECOVERY
STAGE
PASSIVE REFERENCE
STAGE
STAGE
LEARNING CONTROLLER
STAGE
( 2 Seconds )
( 8 Seconds )
( 0.1 Seconds )
( 2 Seconds )

18. Learning on Real Roads

19. Conclusions
• Learning Approach Vindicated
• Ride Improvement Achieved
• Minimal System Modelling Required
• Reduced Design Time
• Robust to Noise