Duncan Brown Rolls-Royce Engineering Fellow – Safety Critical Software

1. Trusted to deliver excellence
© 2016 Rolls-Royce plc and/or its subsidiaries
The information in this document is the property of Rolls-Royce plc and/or its subsidiaries and may not be copied or communicated to a third party, or
used for any purpose other than that for which it is supplied without the express written consent of Rolls-Royce plc and/or its subsidiaries.
This information is given in good faith based upon the latest information available to Rolls-Royce plc and/or its subsidiaries, no warranty or representation
is given concerning such information, which must not be taken as establishing any contractual or other commitment binding upon Rolls-Royce plc and/or
its subsidiaries.
An Alternative Approach to DO-178B
Duncan Brown
Rolls-Royce Engineering Fellow – Safety Critical Software

2. What is DO-178?
2
• Guidance (or guidelines) for software in airborne systems
• A standard??
• A set of objectives and activities to achieve them
• A collection of best practices from the late 1970’s for safety
critical software
• A somewhat arbitrary relaxation of these practices by Design
Assurance Level
• A tried and trusted acceptable means of compliance for
airborne software which has saved thousands of lives!

3. Why DO-178C?
• Best practice has moved on from 1980 and often now
includes:-
- Model Based Development
- Formal Methods
- Object Oriented Design
- Extensive support tooling (Static analysis, auto-code
generation, simulation etc.)
• Because of the success of DO-178B the terms of reference
included minimal change to the core
• So the concept of “Supplements” was developed with an
idea that future supplements could be created without
change to the core document
3

4. DO-178C – Rationale, Change and TearsTiers
• A sub-group was created on DO-178C to try to recall the
rationale behind the objectives but this never materialised
• A proposal for a goal based or safety case approach to
the DO-178 objectives
• A proposal by the Formal Methods group to remove the
testing “means” from the core document and to have a
supplement for the traditional approach
• A proposal (IP217) to abstract DO-178 into a process that
iterated around multiple tiers of requirements/design until
code emerged
Note that this concept re-appeared in 2014 in a paper by Mike
Dewalt called “Technology Independent Assurance Method or TIAM”
4

5. Perceived Problems
(As presented by the FAA to GAMA in 2015)
• Product based certification leads to multiple products being
separately scrutinised
• Prescriptive domain specific detailed objectives in standards
such as DO-178C, DO-254 and ARP4754 preclude or make
difficult alternative approaches
• Separation of System, Software and Complex Hardware
disciplines within the authorities and the applicants causes
wasted effort
• No need for four DAL levels, A is very close to B and C to D
5

6. What is GAMA?
• The General Aviation Manufacturers Association
• Founded in 1970 to “foster and advance the general welfare,
safety, interests and activities of general aviation”
• Worldwide committee based organisation with head quarters
in Washington and Brussels
• Scope includes general aviation aircraft (Part 23) and more
recently rotorcraft (Part 29)
6

7. Why GAMA?
7

8. An example review of DO-178C from a “GA”
perspective
• Activities should not be there, standard should be objectives only
• Process standards rather than product standards would be better
• Have one set of objectives for all levels of requirements and design
• Remove Parameter Data Item objectives
• Remove structural coverage objectives
• Remove data and control coupling objectives
• Eliminate the requirement for traceability data
• Remove configuration index documents
• Have QA audit against company standards
8

9. What is Streamlining?
• Harmonization – The FAA and EASA should employ
harmonized approaches to certification
• Move to process based audits where a company can be
shown to have a good, mature process and that it is being
re-used on a number of projects
• Create some domain independent goals that all certifications
must satisfy to allow alternative approaches with appropriate
justification
• Audit for systems, software and complex hardware in
parallel
• Look at revising the number of DALs to two
9

10. The FAA Initiative
– Streamlining Workshop(s)
• October 2015 FAA (Software CSTA Mike Dewalt) sent out
work shop invites to a number of people with a structured
distribution (Countries, industry sectors etc.)
• First workshop (and the only one planned originally) was in
December 2015
• The plan given by Mike Dewalt at the first meeting was to
conclude on the number of DAL levels and a set of less than
10 “meta-objectives” by the close of the meeting and to take
these to an open FAA conference in April 2016
• The idea was to issue an Advisory Circular in the autumn of
2016!
10

11. Mike Dewalt’s Vision
11

12. More FAA workshops….
• At the end of the December meeting it was decided that:-
- No real advantage in reducing the number of DALs
- We needed more time on meta-objectives
• Another meeting was arranged for April 2016
• At the end of the April meeting:-
- A plan for an FAA conference in September 2016 had
been firmed up
- It was decided that we needed another meeting on meta-
objectives (and they probably weren’t really meta-objectives)
• Another meeting was arranged for July 2016
12

13. Final FAA workshop
• In July 2016 we held the last workshop which:-
- Concluded with a set of three “Overarching Properties”
- Prepared material to disseminate the information at the
September conference
• The “open” FAA conference was held in September 2016 with ~225
attendees (Workshop members, cert authorities and technical
representatives from industry such as DERs)
• The meeting conclusions were not clear, however general consensus
that
- The three OPs are logically correct
- They are probably too abstract to be useful without additional
information/training etc.
- They do not help much in solving “variation” across FAA and
EASA (in fact they may make it worse)
13

14. The European Initiative - RESSAC
• The FAA initiative was international however it was decided
that a separate European approach would be sensible
• IRT St. Exupery in Toulouse launched a research project in
early 2016 involving representatives from industries in
Europe
• The original proposal was a two year project to come up with
- A set of “meta-objectives” (or Overarching Properties)
- Criteria for how the evidence against these could be
judged
- A worked case study against these OPs
14

15. RESSAC and AeroSpace and Defence
Industries Association of Europe (ASD)
15

16. Overarching Properties (aka Meta Objectives)
16
Desired System
Behaviour
Defined
Intended
Function
Implementation
Requirements
Capture
Development
IntentNecessity
Correctness

17. Progress
• The FAA workshops have:-
- Made a decision to continue with four DALs
- Refined three Overarching Properties in a standardised
form
17
Overarching Property Statement:
Correctness: The implementation is correct with respect to its defined intended functions, under foreseeable
operating conditions.
Definitions: words / phrases in the Overarching Property description
a. Implementation: Item or collection of items contributing to system realization, for which acceptance or
approval is being sought; item (from ARP 4754A) is a hardware or software element having bounded and
well-defined interfaces.
b. Defined intended functions: The record of the system needs and constraints as expressed by stakeholders.
c. Foreseeable operating conditions: External and internal conditions in which the system is used,
encompassing all known normal and abnormal conditions.
Pre-requisites: which must exist to allow Overarching Property satisfaction to be shown
a. Defined intended functions exists.
b. The implementation of the functions exists.
c. The record of the foreseeable operating conditions exists.
Constraints: on how Overarching Property satisfaction must be demonstrated
a. The process to satisfy this Overarching Property must be defined and conducted as defined.
b. When tiers of decomposition are used, the means of showing correctness among the tiers and to the
defined intended functions must be defined and conducted as defined.
c. The implementation must be correct when functioning as part of the integrated system or in
environment(s) representative of the integrated system.
d. Criteria for evaluating the artifacts are defined and shown to be satisfied individually and collectively.
e. All artifacts required to establish the Overarching Property are under configuration management and
change control.
f. All design and manufacturing data to support consistent replication of the type design and instructions for
continued airworthiness must be established.
Assumptions: which need only be stated, not justified (if any)
None.
Overarching Property Statement:
Intent: The defined intended functions are correct and complete with respect to the desired system behavior.
Definitions: words / phrases in the Overarching Property description
a. Desired system behavior: System needs and constraints expressed by the stakeholders.
b. Defined intended functions: The record of the system needs and constraints as expressed by stakeholders.
c. Failure Condition(s): A condition having an effect on the aircraft and/or its occupants, either direct or
consequential, which is caused or contributed to by one or more failures or errors, considering flight
phase and relevant adverse operational or environmental conditions or external events. (From ARP
4754A)
Pre-requisites: which must exist to allow Overarching Property satisfaction to be shown
a. Defined intended functions exists.
b. Failure conditions are defined for the aircraft systems.
c. Design Assurance Levels (DALs) are assigned using the failure condition classifications.
Constraints: on how Overarching Property satisfaction must be demonstrated
a. The process to satisfy this Overarching Property must be defined and conducted as defined.
b. The defined intended functions must address the failure conditions.
c. Criteria for evaluating the artifacts are defined and shown to be satisfied individually and collectively.
d. All artifacts required to establish the Overarching Property are under configuration management and
change control.
Assumptions: which need only be stated, not justified (if any)
a. Stakeholders have the system knowledge to express the desired system behavior.
b. Performing system safety assessments is not covered by these Overarching Properties.
Overarching Property Statement:
Necessity: All of the implementation is either required by the defined intended functions or is without
unacceptable safety impact.
Definitions: words / phrases in the Overarching Property description
a. Unacceptable Safety Impact: An impact which compromises the system safety assessment.
b. Defined intended functions: The record of the system needs and constraints as expressed by
stakeholders.
c. Implementation: Item or collection of items contributing to system realization, for which acceptance or
approval is being sought; item (from ARP 4754A) is a hardware or software element having bounded
and well-defined interfaces.
Pre-requisites: which must exist to allow Overarching Property satisfaction to be shown
a. Defined intended functions exists.
b. The implementation or a representation of the implementation exists.
c. The system safety assessment exists.
Constraints: on how Overarching Property satisfaction must be demonstrated
a. The process to satisfy this Overarching Property must be defined and conducted as defined.
b. The system safety assessment must address all of the implementation.
c. Criteria for evaluating the artifacts are defined and shown to be satisfied individually and collectively.
d. All artifacts required to establish the Overarching Property are under configuration management and
change control.
Assumptions: which need only be stated, not justified (if any)
a. For a TSOA appliance there may not be a complete system safety assessment for the final installation
at the appliance level.

18. Next Steps
• RESSAC will continue as a research project until mid 2018
and publish the deliverables at that point
• FAA has decided to continue the workshops both virtually
(as a telecom with collaboration on a web site) and as face
to face workshops possibly US and Europe
• The FAA has also re-defined the team working this to allow
all members of RESSAC to be involved as well as a wider
invite to GAMA / AIA / ASD and others
• The aim for the FAA still seems to ultimately be an Advisory
Circular
• It is not yet clear what ASD and EASA might do with the
RESSAC outputs
18

19. Summary
• It is generally accepted that there is still work to do in harmonizing the
certification authorities both internally and globally
• It is felt that existing guidance for compliance to regulations in some
circumstances can incur disproportionate effort
AND
could inhibit or even preclude innovative approaches in systems,
software and complex electronics which could improve safety
• There is broad agreement about three “Overarching Properties” to be
met to comply with regulation
BUT
It is still a challenge to understand how these can be applied
harmoniously! Key to this are clear criteria to judge the approaches
and the evidence.
19

20. Timeline
20
1992 2000 2005 2010 2015 2020

21. Questions
21