Introductory lecture for course on Critical Systems Engin

1. Critical Systems Engineering
Prof Ian Sommerville
Dr John Rooksby
Critical systems engineering, 2012 Slide 1

2. Course aims
• When you have completed this course, you should:
– understand what is meant by a critical system and have
learned about different types of critical systems.
– understand the fundamental concepts of system
dependability and security and know about the key technical
activities – specification, development and assurance - in
critical systems engineering.
– understand that critical systems are usually not simply
technical systems but are socio-technical systems that
include people and processes and are profoundly affected by
organisational politics and policies.
Critical systems engineering, 2012 Slide 2

3. Presentation
• 3 hour slot, one afternoon per week (normally
Thursdays) from 13.30 to 16.30. Short breaks at
14.25 and 15.35.
• Benefits of this approach
– Gives time for coverage of a topic so that you don’t forget
material between lectures
– Provides an opportunity to integrate work on case studies
with the lecture material
– Allows time for class exercises where required
• Problems
– More tiring for students (and lecturer) than separate lecture
slots
Critical systems engineering, 2012 Slide 3

4. Course topics
• Introduction to critical systems (IS), System failure
(JR)
• Requirements engineering, dependability concepts
(IS)
• Human error and reliability (JR)
• Dependability specification (IS)
• Learning from failure (JR)
• Dependability engineering, fault tolerant system
architectures (IS)
Critical systems engineering, 2012 Slide 4

5. Course topics
• Organisations and organisational failure (JR)
• Security engineering (IS)
• Methods of dependability assurance, dependability
cases (IS)
• Critical infrastructure and the internet (JR)
Critical systems engineering, 2012 Slide 5

6. Assessment
• Examination (40%)
– Covering all topics in the course
• Coursework (60%)
– Two pieces of coursework – 1 on the technical and 1 on the
socio-technical aspect of the course. Each will be of equal
weight (30%)
Critical systems engineering, 2012 Slide 6

7. Web site
http://www.cs.st-
andrews.ac.uk/~ifs/Teaching/MScCritSysEng2012/index.
html
Copies of slides are on Slideshare (as well as studres)
and will be linked from the course web site.
Twitter: @StACS5032CritSy
Critical systems engineering, 2012 Slide 7

8. Critical systems
Critical systems engineering, 2012 Slide 8

9. Critical system essentials
Safety
The system should not harm people or
the system’s environment
Reliability Availability
The system must operate without The system must be available to
serious failures deliver services when requested
to
do so
Security
The system must be able to protect itself
and its data from malicious use
Critical systems engineering, 2012 Slide 9

10. Classes of critical system
• Safety-critical systems
– Failure results in loss of life, injury or damage to the environment
e.g. chemical plant protection system;
• Mission-critical systems
– Failure results in failure of some goal-directed activity e.g.
spacecraft navigation system;
• Business-critical systems
– Failure results in high economic losses e.g. customer accounting
system in a bank;
• Infrastructure systems
– Failure results in a loss of infrastructure capability e.g. power
distribution control system, broadband communications, etc.
Critical systems engineering, 2012 Slide 10

11. Critical systems stack
Critical system External systems
X
Operating system and middleware
System hardware
Infrastructure systems
Physical infrastructure
Critical systems engineering, 2012 Slide 11

12. System dependencies
• Independent critical systems
– Infrastructure/hardware is part of the system
– System operation is not dependent on external systems
– Embedded control systems such as those in medical devices
• Critical software systems
– Usually rely on commodity hardware/OS
– System operation is dependent on external infrastructure
provision
– Hospital appointments system
Critical systems engineering, 2012 Slide 12

13. Systems of systems
• A critical system is rarely a single system but is a
network of several software-intensive systems as well
as infrastructure systems
• Systems that support organisational needs (e.g. an
inter-bank payments system) have to be designed to
be robust so that they can cope with failures and
unavailability in the other systems on which they
depend
Critical systems engineering, 2012 Slide 13

14. Systems of systems
• Systems of systems (SoS) are complex socio-
technical systems with
– Different owners and management policies
– Distributed operation
– Heterogeneous hardware and software
• Individual systems may be part of several SoS so
– Conflicting requirements from different uses of the system
– Complex negotations may be required when system changes
are to be made
Critical systems engineering, 2012 Slide 14

15. Socio-technical systems
• Socio-technical systems
include IT systems and the
social and organisational
environment in which these
systems are used
• Key influences are human
behaviour, organisational
processes and
policies, regulations, cultur
e
Critical systems engineering, 2012 Slide 15

16. Socio-technical systems
Social and political environment
Laws, regulations, custom & practice
System Business
users Software-intensive system processes
Organisational policies and culture
Organisational strategies and goals
Critical systems engineering, 2012 Slide 16

17. Regulation
• Regulators are government-appointed bodies whose
job is to ensure that companies and other bodies
conform to national and international laws.
• This normally involves interpreting the law and
government policy and establishing standards and
regulations that must be followed by industry.
• Examples of regulators
– Data protection authority
– Civil Aviation authority
– Bank of England / Financial Services Authority
– Ofgen – electricity and gas regulator
Critical systems engineering, 2012 Slide 17

18. Regulators and critical systems
• Some critical systems may have to be certified by
regulators before they are put into use. This is
particularly true for safety-critical systems.
• This means that the regulators check that the system
is conformant to current regulations and standards.
– This normally involves the system developers producing
evidence (a safety case or a dependability case e.g.) that
demonstrates that the system is dependable.
• Examples of certifiers
– Civil Aviation Authority – aircraft systems
– Medical Devices Directorate – medical devices and
instruments
Critical systems engineering, 2012 Slide 18

19. System criticality
• Primary critical systems
– Systems where system failure leads directly to an incident
that has an associated loss of some kind
– Typically, these are control systems or systems that are
closely associated with a control system
– Example – failure of engine management system in a car
causes engine to cut out while driving
• Secondary critical systems
– Systems whose failure may (but need not) lead to failure in
an associated system that then leads to loss of some kind
– Example – medical information system that maintains
incorrect information about treatment
Critical systems engineering, 2012 Slide 19

20. Critical systems engineering
• Focus is on the use of techniques and methods to
develop dependable and secure systems.
• The costs of critical system failure are so high that
development methods may be used that are not cost-
effective for other types of system.
• An important aim for many critical systems is
certification and the development process has to be
geared to achieving such certification.
• Certification costs can exceed development costs.
Critical systems engineering, 2012 Slide 20

21. Software engineering for critical systems
• Formal methods for systems specification and
analysis.
• Use of specialized tools such as model checkers and
static analyzers.
• Risk-driven approach to system specification and
management.
• Argumentation systems to support the development
of dependability cases.
• Disciplined configuration management of all software
and hardware.
• Detailed process record keeping.
Critical systems engineering, 2012 Slide 21

22. Denver airport baggage system
• System to control baggage
transfer at the (then new)
Denver airport in the USA.
• Example system illustrating
some of the issues and
problems that arise with
complex socio-technical critical
systems.
• This is a business critical system
– the effective functioning of the
airport relies on its baggage
handling system.
Critical systems engineering, 2012 Slide 22

23. System overview
• New baggage handling
system, which was software
controlled, based on
individual baggage carts
rather than conveyor belts.
• Intention was automated handling so that there was no
manual handling of bags from plane to passenger.
• Very complex hardware/software system procured from
several different companies.
• Encountered complex organisational, hardware and software
problems.
Critical systems engineering, 2012 Slide 23

24. “Denver airport saw the future:
It didn’t work”
– Baggage system did not recognise blockages and simply
continued to unload bags
– Bags fell off the carts due to timing problems
– System loaded bags onto carts that were already full
• At the time of the airport opening, only a very limited
version of the system was available.
– This system had a 10% error rate (i.e. 10% of bags were
delivered to the wrong place)
• Airport 18 months late opening
• System abandoned in 2005
Critical systems engineering, 2012 Slide 24

25. Key points
• Economic and human activities are increasingly
dependent on software-intensive systems. These can
be thought of as critical systems.
• For critical systems, the costs of failure are likely to
significantly exceed the costs of system development
and operation.
• Consequently, the dependability and security of the
system are the most important development
considerations.
• Critical systems are often subject to external
regulation.
Critical systems engineering, 2012 Slide 25