DO-254 Basic Training Overview - One Hour - excerpt from AFuzion's 2-day Basic DO-254 Training. See www.afuzion.com for full training course listings. AFuzion: over 11,500 trained in DO-254 & DO-178C onsite in 35 countries - more than all other trainers in the world ... combined. Free DO-254 Whitepapers on https://afuzion.com/do-254-introduction/ Additional DO-254 Class Training information at https://afuzion.com/private-training/avionics-hardware-do-254-training-class/
Summary of AFuzion 2-day DO-254 Private Training:
Avionics hardware world-wide is now commonly required to follow “DO-254” for literally all phases of development: Safety, Requirements, Design, Logic Implementation, V&V, Quality Assurance, etc. DO-254 was partly copied from software’s DO-178B, but there are many differences between hardware and software which must be understood to “truly” implement DO-254. While DO-254 may seem onerous to follow, most planes, helicopters, and many UAV’s flying today must comply with it: First-time users often complain of costs and schedules doubling while trying to comply. But is DO-254 really complex? What are the true meanings of DO-254? How can DO-254 be understood and applied cost-effectively the first time? What are the top mistakes when starting DO-254 projects and how to avoid them? What are the best practices for avionics hardware requirements, design, logic implementation, configuration management, V&V, QA, and certification? All of these topics are explained in this fast-paced Basic DO-254 class. The developer/teacher is the principal founder of two of the world’s largest avionics consulting companies and the principal author of the world’s most popular publications on DO-254: Vance Hilderman has taught over 11,000 avionics engineers and managers worldwide, including FAA and EASA officials, and engineers from 95 of the world’s largest 100 aviation companies: more than all the competitor’s current trainers in the world, combined.
KEY FEATURES:
Understanding DO-254’s basic principles: DO-254 explained for the “real world”: yours
Understanding DO-254’s true intent by understanding the original authors’ goals
Understanding the avionics development ecosystem of Safety, Hardware, Systems and Certification
Understanding DO-254’s five Plans, plus Standards; includes Requirements, Design, Logic, V&V, Quality/Process Assurance, and Configuration Management
Common DO-254 initiation mistakes: from beginner to intermediate quickly
Think like a DO-254 auditor and pass audits the first time
Understanding and applying AMC 20-152A, CAST-27, and EASA-CM-SWCEH-001 for avionics hardware development per ED-80 and DO-254
Basic DO-254 for COTS IP, FPGA’s, and SOC’s; sample VHDL Standards Walkthrough
2. Slide 2
Overview
The following presentation is a 1-hour Excerpt from AFuzion’s 2-day Basic
DO-254 Training course. Over 11,500 engineers trained worldwide by
AFuzion’s trainers in DO-254 and DO-178: more than all other trainers in the
world … COMBINED.
Full Training information available at: http://afuzion.com/training/
Free Avionics development technical whitepapers available at:
http://afuzion.com/avionics-safety-critical-training-whitepapers/
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
3. About Your Instructor
(Today: Vance Hilderman)
BSEE, MBA, MSEE (Hughes Fellow)
Founder of two of the world’s largest avionics
development services companies
Has personally trained over 10,500 persons in DO-XXX;
more than any other instructor in the world
Has successfully contributed to over 200 diverse avionics
projects
Proven Systems, Hardware and Software success with
over 100 different clients
Have worked with 40+ of North America’s largest avionics
companies and 75 of world’s 100 largest aerospace
companies
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
4. Slide 4
First, Software:
RTCA DO178C:
“Software Considerations in Airborne Systems and
Equipment Certification”
Developed 1980 – 1992
via 100+ Industry and Government personnel
Many compromises to satisfy different goals
Not a recipe book or “How To” guide
“Discussion” flow for guidance; able to accommodate
many different development approaches
Lawyers versus Avionics Engineers; who wins?
In practice: The Golden Rule …
History of DO-254 & DO-178C?
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
5. Slide 5
Then, “Hardware”:
RTCA DO-254:
“Design Assurance Guidance for Airborne Electronic
Hardware”
Developed 1996 – 2000
via 100+ Industry and Government personnel
The committee was mostly software people (Thus
similar to DO-178C)
Strong focus on Complex Electronic Hardware (CEH)
devices (with embedded ‘logic’)
Provides design assurance for CEH including
Programmable Logic Devices (PLDs) and Application
Specific Integrated Circuits (ASICs)
Covers ALL Electronic Hardware
History of DO-178C & DO-254?
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
6. Guidance is applicable to, but not limited to:
Line
Replaceable
Units (LRUs)
Circuit Card
Assemblies
(CCAs)
Custom micro-
coded components
such as ASICs,
PLDs, FPGAs,
including any
associated macro
functions.
Integrated
technology
components,
such as
hybrids and
multi-chip
modules
Commercial-
Off-The-Shelf
(COTS)
hardware
components
Slide 6
Scope of DO-254
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
7. AC 20-152 then CAST 27 limited the scope
of the application of the DO-254 guidance:
Only complex devices
such as ASICs and
PLDs.
Application of the
guidance to level D
design assurance is
optional and on a
case by case basis to
Regulator oversight
and or approval.
The AC has also
strengthened the
exemption for
Commercial-Off-The-
Shelf (COTS)
microprocessors
Slide 7
DO-254 Scope
Limited by AC 20-152 & CAST 27
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
10. Slide 10
1. Introduction
2. System Aspects
3. Design Lifecycle
4. Planning Process
5. Design Process
6. Validation & Verification
7. Configuration Mgmt
8. Process Assurance
9. Certification Liaison
10. Lifecycle Data
11. Additional Considerations
A. Modulation based on level
B. Level A & B Specifics
DO-254 Layout
Planning
Correctness
Development
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
11. Planning Process – Occurs first
Development Process – Follows Planning
Integral Processes – Continuous Throughout Project
Slide 11
Three Key Processes
1. Planning
Process
2. Development
Process
3. Integral Processes
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
16. PA does NOT Engineer:
PA does NOT Create or work on Engineering artifacts
PA does NOT Review
PA does NOT Test
PA does NOT Analyze
What then does PA “do”?!?
Again, PA:
1. Approves Project Plans, Standards, Checklists
2. Assesses Engineer’s “Creation” and “Verification”
Therefore PA ensures PROCESSES are followed
Slide 16
PA Does Not “Engineer”
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
17. Slide 17
PHAC: Plan for Hardware Aspects of Certification
PA/QA: Process/Quality Assurance Plan
CM: Configuration Management Plan
DP: Development Plan
V&VP: Verification & Validation Plan
** Plus 4 Standards (Requirements, Design, Coding,Verification)
1.
PHAC
2.
PA/QA
3.
CM
4.
DP
5.
V&VP
Five Key Plans
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
18. Slide 18
Plan for Hardware Aspect of Certification (PHAC)
30 – 40 pages
Overview of System, Architecture, & Hardware, Interfaces
Discusses Safety, Criticality, & Certification
References Other Plans & Artifacts
Help prepared by and Approved by DER
Submitted to and Approved by FAA
PHAC
1.
PHAC
2.
PA/QA
3.
CM
4.
DP
5.
V&VP
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
19. Slide 19
• Process/Quality Assurance Plan
Addresses the Role of PA throughout process
Ensures that all the plans are coordinated and integral part of
process, and are followed
Ensures that Transition Criteria is adhered
Addresses the conformity reviews and inspections
Provides guidance and timelines for audits/reviews by PA
(Including checklists)
1.
PHAC
2.
PA/QA
3.
CM
4.
DP
5.
V&VP
PA/QA Plan
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
20. Slide 20
CM Plan
1.
PHAC
2.
PA/QA
3.
CM
4.
DP
5.
V&VP
Configuration Management Plan
Established identification of data items
Addresses the use of the CM system
Defines criteria for Development CM
Addresses the Problem reporting and change tracking
system
Defines the accessibility and authority for data.
Addresses security, maintainability and repeatability
Addresses the reproduction of image and all data item
generated, and ability to repeat for manufacturing
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
21. Slide 21
Development Plan
Defines the Schedule, Staffing, dependencies,
deliverables, milestones and organizations
Addresses the development environment and tools
Addresses the lifecycle processes (Requirements,
Conceptual Design, Detailed Design,
Logic(Implementation), Synthesis/Integration)
Addresses the CM and QA/PA involvement (including
Transition Criteria) as well as deliverables
1.
PHAC
2.
PA/QA
3.
CM
4.
DP
5.
V&VP
Development Plan
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
22. Slide 22
Validation & Verification Plan
Addresses Reviews and acceptance
Traceability
All aspects of testing and integration
Test environments and tools
Regression and re-verifications
Resources and management
1.
PHAC
2.
PA/QA
3.
CM
4.
DP
5.
V&VP
Validation
Functional
Tests
Simulation
Verification
Validation & Verification Plan
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
23. Slide 23
Criticality Level Pyramid
A
B
C
D
E
• Level A: Catastrophic
• Level B: Hazardous/Severe
• Level C: Major
• Level D: Minor
• Level E: No Effect
Criticality Levels
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
24. Slide 24
“Hardware, whose anomalous behavior, as shown by the system safety assessment
process, would cause or contribute to a failure of system functions
Level A; “resulting in a catastrophic failure condition for the aircraft”
Level B; “resulting in a hazardous/severe-major failure condition for the aircraft”
Level C; “resulting in a major failure condition for the aircraft”
Level D; “resulting in a minor failure condition for the aircraft” – DO-254 Option per AC
20-152
Level E: “with no effect on aircraft operational capability or pilot workload.”
** No further application of DO-254 is required.
Criticality Levels
(Part 25 Failure Rates)
Level E
(NA)
Level D
(>1E-05)
Level C
(<1E-05)
Level B
( <1E-07)
Level A
<1E-09
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
25. Slide 25
Why Does DO-254 Have Different Criticality Levels?
Who were major DO-178C & DO-254 contributors?
What were their major concerns?
Schedule
Cost
Safety, but with reasonableness
Level A
<1E-09
Level B
<1E-07
Level C
<1E-05
Level D
>1E-05
Level E
NA
Why Different Criticality
Levels?
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
26. Typical System Criticality Level
DAL examples of Systems
DO-178 Level Sample of historical Systems
Slide 26
A Flight controls, Engine Controllers, Primary Displays
B FMS, Many Radios and communication systems.
Many navigation Systems
C Back up Displays, back up communication
systems (SATCOM),
D Maintenance systems, Monitoring Systems (Engine
Vibration Monitors, etc.)
E In Flight entertainment Systems (May be Level D),
Video Systems. Coffee makers and galley services.
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
27. (Note: for more DO-254 Cost/Benefit info, request AFuzion’s free whitepaper at
www.Afuzion.com) and defer to Cost Metrics section of this training on Day 2.
Slide 27
Clearly DO-254 Adds Costs
Why Adopt?
1 Earlier Defect Prevention & Fewer In-Field Defects
2 Greater Reusability
3 Improved Visibility & Confidence of Change Impact
4 Improved Visibility by Management, and of Suppliers
5 Quantified & Improved Safety Determination
6 Reduced Cost, Schedule, & Risks
7 Fewer Assumptions = Fewer Defects
8 More Known & Complete Test Coverage
9 More Complete Documentation Easier Maintenance
10 Wider & Deeper Market Penetration
11 Higher Quality
30. DO-254 Level C, Level B, Level A differences
Copyright 2015Slide 30
Per
AC 20-152
DO-254 is
optional for
Level D
Per
AC 20-152
DO-254 is
intended for
complex
coded
components
only
RTCA DO-254
Appendix A
requires
independence for
all verification
activities for
Levels A& B.
31. Copyright 2015Slide 31
DO-254 Level C, Level B, Level A differences
Design Assurance Strategy Differences Per Level
Failure
Classification
Design
Assurance
Level
Design Assurance Strategy Requirement
Catastrophic Level A Invokes Appendix B of DO-254
Hazardous Level B Invokes Appendix B of DO-254
Major Level C Invokes Appendix B of DO-254
OPTIONAL
Minor Level D May opt not to use DO-254 per FAA AC.
Invokes Appendix B of DO-254
OPTIONAL
38. Slide 38
Best Practice:
Write Tests BEFORE Design?
Traditional Engineering Sequence:
DO-178C
9. Review Tests & Results
8. Execute Test Cases
7. Write Test Cases Based on Requirements
6. Review Logic
5. Write Logic
4. Review Design
3. Develop Design
2. Review Requirements
1. Write Requirements
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
39. Slide 39
Best Practice:
Write Tests BEFORE Design!
Recommended “Best Practice” Engineering Sequence:
DO-178C
8. Review Tests & Results
7. Execute Test Cases
2. Write Test Cases Based on Requirements
6. Review Logic
5. Write Logic
4. Review Design
3. Develop Design
2. Review Requirements, by writing Test Cases!
1. Write Requirements
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
40. Slide 40
“The FPGA shall as a goal, calculate aircraft
heading, altitude, and global position to a
sufficient accuracy.”
Is this acceptable -- Why or Why Not?
Requirements Quiz
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
41. Sample Requirement
System Requirement:
Electrical power shall be regulated and include a backup voltage
regulator {XYZ_System_1072}.
Hardware Requirements:
The Voltage Regulation System (VRS) FPGA shall continuously
monitor primary power bus voltage and determine if the
voltage is within a valid operating range (XYZ_VRS_309}.
If the VRS FPGA determines the primary power bus voltage is
either above or below the valid operating range, the backup
voltage regulator shall be assigned to take over the role of
the primary voltage regulator {XYZ_VRS_310}.
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
42. Sample Requirement
More Hardware Requirements:
The valid operating range for the primary power bus
voltage shall be 24 to 28 Volts D.C. {XYZ_VRS_871}
When the backup voltage regulator becomes primary, a
Backup_Voltage_Regulator_Becomes_Primary message
shall be transmitted to the ground station
{XYZ_VRS_872}.
Derived Requirements (“Safety”):
If the backup voltage regulator is not healthy and the
primary power bus voltage is not within the valid
operating range for more than three consecutive
queries, the operator display shall alert via
Invalid_Operating_Condition message (XYZ_VRS_873}.
Slide 42Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
43. Avoid Temptation. Improve
Requirements First.
> Consider:
For the preceding Voltage Regulation System FPGA
hardware equirements:
Ask yourself:
Are the Requirements fully testable?
Are the Requirements deterministic?
Does testing rely upon certain assumptions?
Might two different hardware logic developers
implement differently?
Finally, if you cannot unambiguously determine
expected intent, how could HW developer?
Must improve requirements first: many, many assumptions
unaddressed:
Slide 43Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
44. Improve Requirements First.
Ask Clarifying Questions: Could you write test cases?!?
> Questions to clarify:
1. What happens with over-voltage, e.g. > 28 volts?
2. Is a single instance of over-voltage a trigger
3. Same as above for under-voltage, e.g. < 24 volts.
4. What determines “health”?
5. Do the Alerts have priorities, and what are the specific priorities?
6. How often is voltage monitored?
7. What is the resolution accuracy of the voltage monitoring?
8. Etc, Etc, Etc
Ask yourself:
“ Can two different developers read the same requirements and garner
different assumptions regarding correct implementation? “
Slide 44Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
45. Best Practice:
Conduct Requirements Reviews Often
Review Requirements Early and Often – Consider Lean
and Agile, even if informal for safety-critical aviation:
Conduct more frequent iterative reviews
Review requirements in smaller batches
Increase quality by having more thorough reviews &
defined criteria
Use collaborative techniques during review
Slide 45Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
46. Best Practice:
Review Requirements in Context
Review Requirements with their traces in context.
Slide 46
Checklist
LLR
HLR
Test
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
47. Best Practice:
Requirements Traceability-Related
When creating new requirements, tests cases, or logic:
Establish the required traces immediately, not later
when you need to demonstrate those traces to the
auditor
Make traceability updates mandatory for review of
Requirements, Logic, & Tests
Two-way traceability for:
1. Requirements to Logic Functions
2. Logic Functions to Tests
3. Requirements to Tests
Slide 47
Requirements
Logic Tests
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
48. Requirements are the foundation for the development
Requirements specification is an art even if you know
what you want to build
Requirements review (scrub) is important
Most hardware errors are due to poorly specified
requirements
Slide 48
Requirements Summary
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
49. Slide 49
A Safety-Critical Requirements Quiz:
Answers
1. T/F: On most aviation projects, most requirements pertain to FALSE
Safety.
2. T/F: Hardware requirement development must follow a FALSE
Waterfall.
3. T/F: The best way to assess hardware requirements is via FALSE
test execution.
4. T/F: DO-254 provides clear guidance for FALSE
developing and assessing requirements.
5. T/F: Most hardware product defects are due to bugs FALSE
and manufacturing defects.
6. T/F: When using model-based development, both system and FALSE
hardware requirements must be textual and external to the model.
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
54. Slide 54
Top DO-254 Mistakes
1. Neglecting “Independence”
2. Science projects versus proven technologies
3. Inadequate formal plans and not following them
4. Inadequate level of detail in Requirements
5. Inadequate and non-automated Traceability
6. Excessive logic iterations via inadequate reviews/tools
7. Lack of path coverage capture during functional tests
8. Lack of automated testing = Expen$ive Regression Test
9. Neglecting to eliminate early-stage coding errors
10. Neglecting to prevent unwarranted changes via CM
11. Insufficient PHAC
12. Insufficient Tool Qualification
13. Not taking credit for existing legacy work => “Gap Analysis”
14. Weak DO-254 Checklists & poor Checklist management
Full Details on avoiding above DO-254 Mistakes on Day 2.
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
55. Slide 55
DO-254:
Best Practices 2015
DO-254: Best Practices
For 2017.
(1-Hour Module – See
AFuzion Training Day-2)
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
56. Slide 56
Gap Analysis
Full DO-254 Gap Analysis Info at: http://afuzion.com/gap-
analysis/
Typical “Gaps” for a
CMMI Level 3
Organization
Versus Level B
CERTIFICATION ACTIVITY % “GAP”
PHAC 80%
PA Plan 20 - 30%
HW CM Plan 10 - 20%
HW Development Plan 40 - 50%
HW Verification & Validation Plan 60 - 70%
Safety Assessment 80 - 90%
Requirements Definition 20 – 30%
Design / Top Level Drawings 10 – 15%
Implementation 5 - 10%
Functional Test 5 – 10%
Elemental Analysis (Levels A/B) 90 – 100%
CM Process 10 – 30%
PA Process 50%
Traceability 40-75%
Tool Qualification 100%
Checklists 30 – 50%
Reviews 30 – 50%
Audits 30 – 50%
DER Liaison 100%
GAPS
58. ISSUES Hardware
Considerations
Functionality with no regulatory basis
Search & Rescue
Dedicated communication radios
Coupled flight
Dedicated communications radios
Autoflight customizations
Aerial refueling software
Boom control
Fuel management
Weapons delivery
Terrain following or low-level operations
“Black” or “Silent” communications/navigation
High-performance operations
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
59. ISSUES Hardware
Considerations
Differences for Military DO-254:
Less, but different, emphasis on Safety Analysis
Less redundancy but harsher operational
environments; does Commercial measure up?
Agency approval: generally not FAA/EASA
All documents reviewed by military/customer; not
just PHAC & HAS
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
61. MilitaryIntegrator/Supplier
Considerations
Ensure compliance baseline; reference and use if
appropriate:
Applicable Technical Standard Orders (TSO’s)
Applicable Advisory Circulars (AC’s)
Applicability of ARP-4761 (Safety) and ARP-4754A
(Systems) Guidelines
Applicability of DO-254 (Airborne Hardware)
Applicability of DO-278 (Ground Based Software)
Ensure Military Authority has:
Right to access & audit ALL artifacts.
Ability to attend all SOI’s
Ongoing visibility into supplier Problem Reports/Resolution
Copyright AFuzion Inc, 2004 – 2017: www. Afuzion.com Slide ‹#›
62. Slide 62
Conclusion Q & A
Coming late 2016:
For additional free technical
information:
1. “DO-178C For Engineers & Managers” –
LinkedIn group
2. www.afuzion.com/whitepapers
(free whitepapers for training attendees)
DO-178C
DO-254
DO-278A ISO 26262
AFuzion: When Safety is Critical.TM
63. Slide 63
Conclusion Q & A
2017:
For additional technical
information:
1. “DO-254 For Engineers & Managers” –
LinkedIn group; free.
2. www.afuzion.com/whitepapers
(free whitepapers for training attendees)