Engineering Safety and Security-Related Requirements

Engineering
Safety- and Security-Related
Requirements

Donald Firesmith
Software Engineering Institute
Carnegie Mellon University
Pittsburgh, PA 15213
18 June 2012
Half-Day Tutorial

© 2012 Carnegie Mellon University

Tutorial Goals
To familiarize safety, security, and requirements engineers with the:
• Common challenges they face
• Common foundational concepts underlying each other’s disciplines
• Useful reusable techniques from each other’s disciplines

• Different types of safety- and security-related requirements

To provide safety, security, and requirements engineers with a common
consistent method for engineering these requirements and thereby:
• Improve collaboration
• Engineer better safety- and security-related requirements
• Decrease the incidence of accidents and successful attacks due to poor
safety- and security-related requirements

Engineering Safety- & Security-Related
Requirements 2
Donald Firesmith, 18 June 2012

Intended Audiences
Primary audience:
• Safety engineers
• Security engineers
• Requirements engineers

Secondary audience:
• Project managers (acquisition, development, or sustainment)
• Lead system engineers and system architects
• System, software, and hardware engineers
• Safety and security SMEs and consultants

• Safety and security trainers and academic educators and researchers

• Specialty engineers (e.g., human factors and reliability)
• Any other stakeholders in safety and security

Requirements 3

History
Work began in 2002
Presented as conference tutorials and in journal articles
(see http://donald.firesmith.net/home/publications)
Taught at 5 NASA Centers (2008) and US Army ASSIP Program
Presented to nuclear engineers at:
• System Engineering Challenges International Workshop in Russia
(RuSEC 2010) in Moscow, Russia on 23 September 2010
• Kärnteknik-2011 (Nuclear Technology 2011) Nordic Symposium in
Stockholm Sweden on 8 December 2011
Free Open Source Tool began Fall 2011
http://donald.firesmith.net/home/safety-and-security-analysis-tool
Book to be published Winter 2012
Requirements 4

Where We Are

Related Disciplines ◄
Challenges
Common Example
Free, Open-Source Tool
Requirements Engineering Overview
Safety and Security Engineering Overview
Safety- and Security-related Requirements
Collaborative Defensibility Engineering Method
Conclusion

Requirements 5

Safety Engineering – Traditional Definition
Safety
freedom from accidental harm to people, property, and the environment

Safety Engineering
the process of ensuring that a system is safe to operate

Major Limitations:
• No nontrivial system is free from hazards that can cause accidental harm.
• It is always a matter of degree and risk management.
• Not just harm, but also accidents, vulnerabilities, and hazards
• In spite of best efforts, accidents can (and sometimes do) happen.
• It is important to address not just the prevention of accidental harm (and
accidents, vulnerabilities, hazards, and risks), but also detecting their
existence/occurrence, and reacting properly.

Requirements 6

Security Engineering – Traditional Definitions
Security (of information and services) is often defined in terms of specific
properties, whereby a system is secure when it exhibits these properties in
spite of attack:
• Access Control (including identification, authentication, and authorization)
• Availability
• Confidentiality (including both privacy and anonymity)
• Integrity (including data and software)
• Non-repudiation (of transactions)

Security Engineering
the process of ensuring that a system is sufficiently secure to operate

Major limitations:
• No nontrivial system is free from threats that can cause malicious harm.
• It is always a matter of degree and risk management.
• Not just harm, but also attacks, vulnerabilities, and threats
• In spite of best efforts, attacks can (and typically will) happen.
• It is important to address not just the prevention of malicious harm (and attacks,
vulnerabilities, threats, and risks), but also detecting their existence/occurrence, and
reacting properly.
Requirements 7

Defensibility Engineering – New Definitions
Safety Engineering
the systems engineering discipline concerned with lowering the risk of unintentional
(i.e., accidental) unauthorized harm to defended assets to a level that is acceptable to
the system’s stakeholders by preventing, detecting, and properly reacting to such harm,
mishaps (i.e., accidents and safety incidents), system-internal vulnerabilities, system-
external unintentional abusers, hazards, and safety risks
Security Engineering
the systems engineering discipline concerned with lowering the risk of intentional (i.e.,
malicious) unauthorized harm to defended assets to a level that is acceptable to the
system’s stakeholders by preventing, detecting, and properly reacting to such harm,
civilian misuses (i.e., attacks and security incidents), system-internal vulnerabilities,
system-external intentional civilian abusers, threats, and security risks
Survivability Engineering
the systems engineering discipline concerned with lowering the risk of intentional (i.e.,
malicious) unauthorized harm to defended assets to a level that is acceptable to the
system’s stakeholders by preventing, detecting, and properly reacting to such harm,
military misuses (i.e., attacks and survivability incidents), system-internal vulnerabilities,
system-external intentional military abusers, threats, and survivability risks
Requirements 8

Safety/Security Similarities and Differences
Similarities:
• Systems engineering rather than software engineering
• Lower risks to acceptable levels:
— Never zero – never perfectly safe, secure, or survivable
• Prevention, detection, and reaction:
— Unauthorized harm to defended assets
(people, property, environment, services)
— Abuses (safety mishaps and security misuses)
— System-internal vulnerabilities
— System-external abusers (exploit vulnerabilities to cause harm)
— Dangers (safety hazards and security threats)
— Risks (safety and security)
Differences:
• Unintentional (safety) vs. intentional (security and security)
• Terminology
Requirements 9

Requirements Engineering
Requirements Engineering
the engineering discipline within systems/software engineering concerned with
identifying, analyzing, reusing, specifying, managing, verifying, and validating
goals and requirements (including safety- and security-related requirements)

Safety- and security-related requirements are primarily system-level
requirements.

Requirements 10

Where We Are
Related Disciplines

Challenges ◄
Common Example
Conclusion

Requirements 11

Challenges1
Requirements engineering, safety engineering, and security engineering
have different:
• Communities
• Disciplines with different training, books, journals, and conferences
• Professions with different job titles

• Foundational concepts and terminologies
• Tasks, techniques, and work products

Some of these differences are unnecessary and represent detrimental
redundancies:
• Underlying concepts and terminologies

• Tasks and techniques
• Work products (models and documents)

Requirements 12

Challenges2
Safety and security engineering are:
• Typically treated as secondary specialty engineering disciplines
• Performed separately from, largely independently of, and lagging behind the
primary engineering workflow:
(requirements, architecture, design, implementation, integration, testing,
deployment, sustainment)
Safety and security are too often tested into existing systems (reactively)
rather than adequately built into systems during development (proactively)

Requirements 13

Challenges3
Separation of requirements engineering, safety engineering, and security
engineering causes:
• Poor safety- and security-related requirements that are often:
— Vague, unverifiable, and infeasible capabilities or goals rather than true
requirements
— Potentially inappropriate architectural and design constraints
— Inadequate to drive architecture, design, and implementation
— Specified and managed separately from all other requirements
— Relegated to “second class” “supplementary” specifications
— Often ignored by requirements engineers and architects
— Produced too late to drive architecture and testing
• Safety and security are too often iterated and tested into existing systems
(reactively) – “find and fix” – rather than adequately built into systems during
development (proactively).
Requirements 14

Challenges4
How much safety and security is enough?
• Insufficient defensibility is dangerous.
• Excessive defensibility wastes scarce project resources.
• Current requirements lack practical and precise thresholds.

• Defenses (safeguards and countermeasures) should be commensurate with
risk.
— Goldilocks rule: not too little and not too big.
• Without well-defined thresholds, how can:
— Architects perform engineering trade-offs?
— Testers set test completion criteria for the testing of safety- and security-
related requirements?

Requirements 15

Challenges5
Many mandated security “requirements” are actually constraints such as:
• Security subsystems
• Industry “best practices”

What about safety and security requirements for preventing, detecting, and
reacting to:
• Harm to defended assets?
• Abuses (safety mishaps and security misuses)?
• Vulnerabilities?
• Abusers (unintentional and intentional)?
• Dangers (safety hazards and security threats)?
• Risks (safety and security)?

Requirements 16

Challenges6
Current separate methods for performing requirements, safety, and
security engineering are inefficient and ineffective.
Poor requirements are a primary cause of more than half of all project
failures (defined in terms of):
• Major cost overruns
• Major schedule overruns
• Major functionality not delivered

• Large number of defects delivered
• Delivered systems that are never used

Poor requirements are a major “root cause” of many (or most) accidents
involving software-intensive systems.

Requirements 17

Challenges7
Current state-of-the-practice cries out for improvement:
• Better consistency between safety and security engineering
— More consistent concepts and terminology
— Reuse of techniques across disciplines
— Less unnecessary overlap and avoidance of redundant work
• Better collaboration:
— Between safety and security engineering
— With requirements engineering
• Easier certification and accreditation

• Better safety- and security-related requirements
• Fewer accidents and successful attacks that cause less harm to defended
assets

Requirements 18

Where We Are
Related Disciplines
Challenges

Common Example ◄
Conclusion

Requirements 19

Common Example use throughout Tutorial
Problem: How to enable large numbers of zoo patrons to quickly and
conveniently tour the habitats of a huge new zoo?
Proposed solution: the Zoo Automated Taxi System (ZATS)
• Numerous small family-sized automated taxis:
— Leisurely tours of habitats and fast transport between habitats
— Inexpensive to operate (no driver salaries and benefits)
— Reliable, easy to use, safe, and secure
— Green (electric to minimize air and noise pollution)
• Elevated concrete guideways:
— Separate passengers from animals
— Provide good views
— Avoid collisions of taxis with pedestrians and vehicles in parking lot
• Conveniently located taxi stations
• Operations and maintenance facility in zoo back lots
Requirements 20

Where We Are
Related Disciplines
Challenges
Common Example

Free, Open-Source Tool ◄
Conclusion

Requirements 21

Free and open-source tool
Developed to:
• Support the collaborative defensibility engineering method
• Store the complete analysis results of the ZATS common example

Exists in two forms:
• Empty database for new project use

• Populated database storing the “complete” analysis results of common
example: ZATS
Implemented using Microsoft Access
Free download from:
http://donald.firesmith.net/home/safety-and-security-analysis-tool

Requirements 22

Initial Screen

Requirements 23

Prepare to Analyze Safety and Security Menu

Requirements 24

Analyze Safety and Security Menu

Requirements 25

Where We Are
Related Disciplines
Challenges
Common Example

Requirements Engineering Overview ◄
Conclusion

Requirements 26

Requirements Engineering Tasks
Business Analysis (i.e., Customer, Competitor, Market, Technology, and User Analysis as
well as Stakeholder Identification and Profiling)
Visioning
Requirements Identification (a.k.a., Elicitation)
Requirements Analysis
Requirements Specification
Requirements Management
Requirements Validation

Scope Management (Management)
Change Control (Configuration Management)
Quality Control (Quality Engineering)

Requirements 27

Requirements Engineering Work Products
Business analyses
Stakeholder profiles
Vision statement
• Capabilities and Goals
Concept of Operations (ConOps) or operational concept document
(OCD)
• Mission threads
• Use cases
• Usage scenarios
Requirements repository and published specifications
• Actual requirements
Requirements prototypes
Domain model
Glossary
Requirements 28

Difficulty of Requirements Engineering
“The hardest single part of building a software system is deciding precisely
what to build. No other part of the conceptual work is as difficult as
establishing the detailed technical requirements, including all the
interfaces to people, to machines, and to other software systems. No other
part of the work so cripples the resulting system if done wrong. No other
part is more difficult to rectify later.”

Fred Brooks, No Silver Bullet, IEEE Computer, 1987

Requirements 29

Goals
Goal
an informally documented perceived need of a legitimate stakeholder
Goals are:
• Not requirements.
• Drive the identification and analysis of the requirements.

• Typically ambiguous and/or unrealistic (i.e. impossible to guarantee).

Major problems with safety and security goals:
• Ambiguous
• Stated as absolutes
• Not 100% feasible

• Not verifiable

Typically documented in a vision statement.
Requirements 30

Representative ZATS Goals
ZATS will take passengers on leisurely tours that provide excellent viewing
of the zoo habitats.
ZATS will take passengers rapidly to their desired taxi stations in the zoo
and its parking lot.
ZATS will prevent animals from reaching passengers in taxis and taxi
stations.
ZATS will never injure or kill a passenger.
ZATS will not cause air and noise pollution.
ZATS will be easy and intuitive for all of its passengers to use.
ZATS will allow passengers to use securely use bank cards to pay for
trips.

Requirements 31

Use Case Models and Mission Threads
Usage Scenario
a specific functionally cohesive sequence of interactions between user(s), the
system, and potentially other actors that provides value to a stakeholder
Use Case
a general way to perform a function
a functionally cohesive class of usage scenarios
Use Case Path (a.k.a., flow and course)
an equivalence set of usage scenarios that follow the same course through a
use case
Mission Thread
an end-to-end sequence of use case paths that together achieves one of the
system’s missions

Requirements 32

Use Case Models
Use case paths:
• Can be either:
— Normal (Sunny Day or Happy Path)
— Exceptional (Rainy Day)
• Should have their own preconditions, triggers, and postconditions
• Are often documented with text, sequence diagrams, or activity diagrams

Use cases, use case paths, and usage scenarios:
• Typically documented in a ConOps or operational concept document (OCD)
• Drive the identification and analysis of the [primarily functional] requirements

• Often include potential architectural and design information

Use cases are far more than merely use case diagrams.

Requirements 33

Characteristics of Good Requirements

All Types of Requirements Active Voice
Correct Configuration Controlled
Consistent (internally and with other requirements)
Cohesive (individual)
Differentiated from Non-requirements
Complete Externally observable
Concise Grammatically Correct and no Typos
Feasible Managed (in requirements repository)
Mandatory (necessary) Prioritized (for scheduling implementation)
Properly Specified
Normal and Exceptional
Rationalized
Relevant Scheduled
Unique Stakeholder-centric
Unambiguous Situation-specific (to mode, state, and/or event)
Validatable Traced (to source and to architecture)
Uniquely identified
Verifiable
Usable by stakeholders
What or how well, not how http://www.jot.fm/issues/issue_2003_07/column7
Requirements 34

Poor Requirements Cause Accidents1
“For the 34 (safety) incidents analyzed, 44% had inadequate
specification as their primary cause.”
Health and Safety Executive (HSE), Out of Control: Why Control Systems
Go Wrong and How to Prevent Failure (2nd Edition), 1995

“Almost all accidents related to software components in the past 20
years can be traced to flaws in the requirements specifications, such as
unhandled cases.”
Grady Lee, Jeffry Howard, and Patrick Anderson, “Safety-Critical
Requirements Specification and Analysis using SpecTRM”, Safeware
Engineering, 2002

Requirements 35

“Erroneous specification is a major source of defects and subsequent
failure of safety-critical systems. Many failures occur in systems using
software that is perfect, it is just not the software that is needed
because the specification is defective.”
John C. McKnight, “Software Challenges in Aviation Systems,”
Proceedings of the 21st International Conference on Computer
Safety, Reliability and Security, 2002
“Software-related accidents almost always are due to
misunderstandings about what the software should do.”
Kathryn Anne Weiss, “An Analysis of Causation in Aerospace
Accidents,” Proceedings of the 2001 Digital Avionics Systems
Conference, updated 7 September 2004

Requirements 36

“Software-related accidents are usually caused by flawed requirements.
Incomplete or wrong assumptions about the operation of the controlled
system can cause software related accidents, as can incomplete or wrong
assumptions about the required operation of the computer. Frequently,
omitted requirements leave unhandled controlled-system states and
environmental conditions.”
Nancy G. Leveson, Safeware: System Safety and Computers, 2003

“Software-related accidents are almost all caused by flawed
requirements:
• Incomplete or wrong assumptions about the operation of the controlled
system or required operation of the computer
• Unhandled controlled-system states and environmental conditions.”
Nancy G. Leveson, A new Approach to Ensuring Safety in Software and
Human Intensive Systems, July 2009

Requirements 37

The Many Types of Requirements
Business Industry Laws and Natural Organizational
Rules Standard Regulations Laws Policies
Innovative
Rules
Requirements
Positive (shall) Documentation Entity Data *
Requirements Requirements Requirements Requirements

Stakeholder Facility Hardware Object
Negative
(shall not)
Requirements
Derived People (Role) Procedure Material
(Technical) Requirements Requirements Requirements
Requirements Requirements
Software System /
Development Requirements Subsystem
Environment Requirements
Requirements Architecture
Process Constraints
Manufacturing Product
(Method)
Requirements Requirements Primary Mission Design
Requirements
Requirements Constraints
Operational
Requirements Supporting Implementation
Training
Requirements
“Pure” Integration
Maintenance Constraints
Requirements
Non-Functional Requirements Manufacturing
Sustainment

Retirement Functional Entity (Data *) Interface Quality Deployment
Requirements Requirements Requirements Requirements Requirements Constraints

Requirements 38

Product Requirements
A product requirement is a requirement for a product (e.g., system,
subsystem, software application, or component).
• A functional requirement is a product requirement than specifies a
mandatory function (i.e., behavior) of the product.
• A data requirement is a product requirement that specifies mandatory [types
of] data that must be manipulated by the product.
• An interface requirement is a product requirement that specifies a mandatory
interface with (or within) the product.
• A quality requirement is a product requirement that specifies a mandatory
amount of a type of product quality (characteristic or attribute).
• A constraint is a property of the product (e.g., architecture or design decision)
that would ordinarily not be a requirement but which is being mandated as if it
were a normal requirement

Requirements 39

Quality Models

Architectural
Components
define the
meaning of the
quality of Quality
Systems
Models

define the meaning
of a specific type of
quality of are measure
measured quality
along
Quality along
Quality
Quality Quality
Measurement Measurement
Characteristics Attributes
Scales Methods

are measured using

Developmental Operational
Quality Quality
Characteristics Characteristics

Requirements 40

Quality Characteristics (Operational)

Quality Characteristics

Developmental Operational
Quality Characteristics Quality Characteristics

Configurability Efficiency Functionality Interoperability Serviceability Usability

Compliance Dependability Environmental Habitability Operability Transportability
Compatibility

Robustness Defensibility Performance Soundness

Safety Survivability Availability Correctness Predictability

Security Capacity Reliability

Stability

Requirements 41

Performance Attributes
Jitter

Problem
Latency Prevention
Harm Arrest
Response Time Problem
Detection
Schedualability Harm Mitigation

Problem
Failure Analysis
Throughput Reaction

Failure Recovery
Problem Type Solution Type
Performance Attributes Performance Attributes System Adaptation

measure quality
along
Performance Performance Attributes

are measured
along Quality Quality
Quality Quality
Scales Methods

define the
meaning of the
Quality quality of
Models Systems

Requirements 42

Defensibility Quality Attributes
Occurrence of Unauthorized Harm Harm Arrest

Occurrence of Abuse Harm Mitigation
(Safety Mishap or Security Misuse)
Problem Abuse Analysis
Existence of System-Internal Vulnerability Prevention
Abuse Recovery
Existence of System-External Abuser Problem
Detection System
Existence of Danger (Hazard or Threat) Adaptation
Problem
Existence of Defensibility Risk Reaction Counterattack
(Security and
Survivability)
Problem Type Solution Type
Defensibility Attributes Defensibility Attributes
Safety
measure quality
along
Security Defensibility Defensibility Attributes

Survivability are measured
along Quality Quality
Quality Quality
Scales Methods

define the
meaning of the
Quality quality of
Models Systems

Requirements 43

Different Types of Defensibility Requirements

Problem Type Quality Attributes
Defensibility
Quality Attributes Harm to Defensibility
Abuse Abuser Vulnerability Danger
Valuable Asset Risk

Prevent Prevent Prevent Prevent Prevent Prevent
Prevention Occurrence Occurrence Existence Existence of Existence Existence
Quality Attributes
Solution Type

of Harm of Abuse of Abuser Vulnerability of Danger of Risk
Detect Detect Detect Detect Detect Detect
Detection Occurrence Occurrence Existence Existence of Existence Existence
React to React to React to React to React to React to
Reaction Occurrence Occurrence Existence Existence of Existence Existence

Requirements 44

Components of a Quality Requirement
state stakeholders’
Quality Goals
importance of achieving

specify achievement of define stakeholders’
minimum acceptable levels
of quality for a
Quality Requirements System

are applicable shall
Conditions during or at System-Specific exceed
Quality Thresholds
or Events Quality Criteria

indirectly state directly state are measured are measured
existence of existence of along using

Quality Quality
Quality Quality
Scales Methods

Quality define the meaning of the quality of a
Models

Requirements 45

Example Quality Requirement
Hazard prevention safety requirement:
“Under normal operating conditions, ZATS shall not move when it’s doors
are open at an average rate higher than once every 10,000 trips.”
Component parts:
• Condition:
“Under normal operating conditions”
• Mandatory system-specific quality criterion:
“ZATS shall not move when it’s doors are open”
• Measurement threshold:
“at an average rate higher than once every 10,000 trips.”

Definitions needed to avoid ambiguity:
• Moving – traveling faster than 0.1 cm per second
• Open – open more than 1 cm between doors
• Normal operating conditions – neither during maintenance nor a fire
• Trip – travel with passengers from a starting taxi station to the associated destination
taxi station
Requirements 46

Importance of Measurement Threshold

Measurement threshold is:
• Critical
• Difficult (but not impossible) to determine
• Often left out of quality requirements
• Needed to avoid ambiguity
States how much quality is necessary (sufficient)
Enables architects and architecture evaluators to:
• Determine if architecture is adequate
• Make engineering tradeoffs between competing quality characteristics and
attributes
Enables tester to determine the:
• Test completion criteria
• Number and types of test cases
Requirements 47

Where We Are
Related Disciplines
Challenges
Common Example

Safety and Security Engineering Overview ◄
Conclusion

Requirements 48

Fundamental Safety and Security Concepts
Safety and security as quality characteristics with associated quality
attributes
Stakeholders
Defended assets
Unauthorized harm to defended assets
Abuses (accidents, attacks, and incidents)
Vulnerabilities (system-internal weaknesses or defects)
Abusers (external and internal, malicious and non-malicious)
Dangers (hazards and threats)
Defensibility risks (safety and security)
Goals, policies, and requirements
Defenses (safeguards and counter measures)

Requirements 49

Safety as a Quality Characteristic
Safety is the subclass of defensibility capturing the degree to which:
• The following safety problems:
— Accidental harm to defended assets
— Safety abuses (mishaps such as accidents and safety incidents)
— Safety abusers (people, systems, and the environment)
— Safety vulnerabilities
— Safety dangers (hazards) including the existence (conditions) of non-malicious
abusers who unintentionally exploit system vulnerabilities to accidentally harm
vulnerable defended assets
— Safety risks
• Have safety solutions:
— Prevented (eliminated, mitigated, keep acceptably low)
— Detected
— Reacted to

Requirements 50

Security as a Quality Characteristic
Security is the subclass of defensibility capturing the degree to which:
• The following security problems:
— Malicious harm to defended assets
— Security abuses (misuses such as attacks and security incidents)
— Security abusers (attackers and malware – systems, software, and hardware)
— Security vulnerabilities
— Security dangers (threats) including the existence (conditions) of malicious
abusers who can exploit system vulnerabilities to harm vulnerable defended
assets
— Security risks
• Are security solutions:
— Prevented (eliminated, mitigated, keep acceptably low)
— Detected
— Reacted to

Requirements 51

Where We Are
Three Disciplines
Challenges
Common Example

Safety- and Security-related Requirements ◄
Conclusion

Requirements 52

Types of Safety- and Security-Related Requirements

Too often only a single type of requirements is considered.
Not just:
• Specific types of non-functional requirements (NFRs):
— Safety and security requirements are quality requirements
• Functional, data, and interface requirements with safety or security
ramifications
• Architecture and design constraints

• Safety and security functions/subsystems
• Software requirements
• Constraints on functional requirements

Reason for presentation title:
Safety- and security-related requirements for software-intensive systems

Requirements 53

Types of Requirements
Business Industry Laws and Natural Organizational
Rules Standard Regulations Laws Policies
Innovative
Rules
Requirements
Positive (shall) Documentation Entity Data *

Stakeholder Facility Hardware Object
Negative
(shall not)
Requirements
Derived People (Role) Procedure Material
(Technical) Requirements Requirements Requirements
Software System /
Development Requirements Subsystem
Environment Requirements
Requirements Architecture
Process Constraints
Manufacturing Product
(Method)
Requirements Requirements Primary Mission Design
Requirements
Operational
Requirements Supporting Implementation
Training
Requirements
“Pure” Integration
Maintenance Constraints
Requirements
Non-Functional Requirements Manufacturing
Sustainment

Retirement Functional Entity (Data *) Interface Quality Deployment
Requirements Requirements Requirements Requirements Requirements Constraints

Requirements 54

Different Types of Defensibility Requirements

Problem Type Quality Attributes
Defensibility
Quality Attributes Harm to Defensibility
Abuse Abuser Vulnerability Danger
Valuable Asset Risk

Prevent Prevent Prevent Prevent Prevent Prevent
Prevention Occurrence Occurrence Existence Existence of Existence Existence
Quality Attributes
Solution Type

Detect Detect Detect Detect Detect Detect
Detection Occurrence Occurrence Existence Existence of Existence Existence
React to React to React to React to React to React to
Reaction Occurrence Occurrence Existence Existence of Existence Existence

Requirements 55

Types of Defensibility-Related Requirements – 1

Safety
Security Safety-Significant Safety
Function/Subsystem
Requirements Requirements Constraints
Requirements

Security- Security
Security Security
Significant Function/Subsystem

Defensibility- Defensibility
Defensibility Defensibility
Significant Function/Subsystem

Safety-Related
System Defensibility- Requirements
Requirements Related
Requirements Security-Related
Requirements

Requirements 56

Safety/Security-
Safety/Security Significant Safety/Security

Safety/Security Requirements that
All Requirements Function/Subsystem are neither safety-
Requirements nor security-related

Requirements 57

Safety/Security
Assurance Level (SAL)
Defensibility Quality
Intolerable
Defensibility-Related
Requirements
SAL = 5 Defensibility
Function / Supporting
Critical Subsystem Requirements
Defensibility-Related Requirements
Requirements
Constraints
Constraints
Major
Defensibility-Related Other
Requirements Defensibility-
SAL = 3 Related Functional
Moderate
Defensibility-Related Quality
Requirements Defensibility-Related Requirements
SAL = 2 Requirements
SAL = 1 - 5
System Data
Minor
Requirements
SAL = 1 Interface
Requirements
Defensibility-Independent
Requirements
Constraints
SAL = 0

Requirements 58

1) Safety and Security Requirements
Safety and security requirements are quality requirements.
Quality requirements are product requirements that specify a mandatory
minimum amount of a type of product quality:
• Quality characteristic (generally)
• Quality attributes (specifically)

Safety and security requirements:
• Are typically negative requirements
• Specify what the system shall not cause, enable, or allow to:
— Occur
(e.g., unauthorized harm to defended assets, accidents, attacks)
— Exist
(e.g., vulnerabilities, threats, abusers, hazards, risks)

Requirements 59

Safety and Security Requirements
Quality requirements should be:
• Scalar (how well or how much)
• Based on a quality model defining the specific types of quality and how their
measurement scales
• Stored in requirements repositories and specified in requirements
specifications, NOT just in:
— Secondary specifications
— Safety/security documents
Quality requirements are critically important drivers of the architecture and
testing.

Requirements 60

Example Safety Requirement Templates
When in mode V, the system shall limit the occurrence of accidental harm
of type W to defended assets of type X to an average rate of no more than
Y asset value per Z time duration.
When in mode W, the system shall not cause mishaps of type X with an
average rate of more than Y mishaps per Z trips.
When in mode X, the system shall not cause hazard Y to exist for more
than an average of Z percent of the time.
When in mode X, the system shall not have a residual safety risk level of Y
or above.
When in mode X, the system shall detect accidents of type Y an average
of at least Z percent of the time.
Upon detecting an accident of type W when in mode X, the system shall
react by performing functions Y an average of at least Z percent of the
time.
Requirements 61

Example Security Requirement Templates
When in mode V, the system shall limit the occurrence of malicious harm
of type W to defended assets of type X to an average rate of less than Y
asset value per Z time duration.

When in mode W, the system shall prevent the first successful attacks of
type X for a minimum of Z time duration.

When in mode X, the system shall not have security vulnerability Y for
more than an average of Z percent of the time.

When in mode X, the system shall not have a security risk level of Y.

When in mode X, the system shall detect misuses of type Y an average of
at least Z percent of the time.

Upon detecting a misuse of type W when in mode X, the system shall
react by performing functions Y an average of at least Z percent of the
time.
Requirements 62

2) Defensibility-Significant Requirements – 1

Defensibility-significant requirement

a requirement with significant safety or security ramifications

Are identified based on safety or security (e.g., hazard or threat) analysis

Can be any kind of product requirements:

• Functional, data, interface, quality, and constraints

• Pure safety and security requirements

• Safety and security function/subsystem requirements

• Safety and security constraints

Requirements 63

Defensibility-Significant Requirements – 2
Safety/Security-
Upper 3 circles Safety/Security Significant Safety/Security
Horizontal lines
Not all of the
safety/security
function/subsystem
requirements


Requirements 64

Safety/Security Assurance Levels (SALs)
Safety/Security
Assurance Level (SAL)
Defensibility Quality
Intolerable
Requirements
Function / Supporting
Critical Subsystem Requirements
Defensibility-Related Requirements
Requirements
Constraints
Constraints
Major
Defensibility-Related Other
Requirements Defensibility-
SAL = 3 Related Functional
Moderate
Defensibility-Related Quality
Requirements Defensibility-Related Requirements
SAL = 2 Requirements
SAL = 1 - 5
System Data
Minor
Requirements
SAL = 1 Interface
Requirements
Defensibility-Independent
Requirements
Constraints
SAL = 0

Requirements 65

Example Safety-Significant Requirements
Firing missiles from military aircraft requirements:
• When to arm missiles
• When not to arm missiles (e.g., detecting weight-on-wheels)
• Controlling doors before and after firing missiles

Chemical plant requirements:
• Mixing and heating toxic chemicals
• Controlling exothermic reactions
• Detecting and controlling temperature, pressure, and flow-rate

ZATS requirements:
• Taxi starting, accelerating, cruising, decelerating, and stopping

• Opening and closing doors (taxi, taxi station safety wall, taxi station elevator)

Requirements 66

Example Security-Significant Requirements
Access control requirements:
• Identification, authentication, and authorization
Accountability (e.g., non-repudiation requirements):
• Creation, storage, and transmission of financial transactions
Availability (under attack) requirements:
• Services subject to denial-of-services attacks
Confidentiality requirements:
• Storage and transmission of sensitive information
• Confidential intellectual property in the software or its documentation
Integrity requirements:
• Storage and transmission of sensitive data
• Software that might get Infected by malware

Requirements 67

3) Defensibility Function/Subsystem Rqmts – 1
Defensibility function/subsystem requirements are requirements for
functions or subfunctions that exist strictly to improve defensibility (as
opposed to support the primary mission requirements).
• Safety function/subsystem requirements are requirements for safety
functions or subsystems.
• Security function/subsystem requirements are requirements for security
functions or subsystems.

Requirements 68

Defensibility Function/Subsystem Rqmts – 2
Safety/Security-
Bottom circle Safety/Security Significant Safety/Security
Vertical lines
Overlaps other 3
types
Not all of the
safety/security
function/subsystem
requirements are
safety/security
significant


Requirements 69

Example Safety Functions/Subsystems
Automobile functions or subsystems added strictly for safety:
• Adaptive Cruise Control (ACC) • Forward Collision Warning Systems
• Adaptive Headlights • Intelligent Parking Assist Systems (IPAS)
• Airbag Systems • Lane Departure Warning Systems
• Anti-lock Braking Systems (ABS) • On-Board Diagnostics (OBD)
• Automatic Braking • On-Board Prognostics (OBP)
• Automatic Crash Response • Reverse Backup Sensors
Systems • Seat Belt Systems
• Automotive Night Vision Systems • Tire Pressure Monitoring Systems
• Backup Cameras • Traction Control Systems (TCS)
• Backup Collision Intervention
Systems
• Electronic Brakeforce
Distribution (EBD)
• Electronic Stability Control (ESC)
• Emergency Brake Assist (EBA)

Requirements 70

Example Security Functions/Subsystems – 1
Automobile functions or subsystems added strictly for security:
• Car Alarm Systems
• Door Locks
• Ignition Key Systems
• Stolen Vehicle Recovery Systems

Requirements 71

Example Security Functions/Subsystems – 2
Functions or subsystems strictly added for security:
• Access control
• Antivirus / antispyware / antispam / antiphishing subsystems
• Encryption/decryption subsystem

• Firewalls
• Intrusion detection subsystem (IDS) / intrusion prevention subsystem (IPS)

All requirements for such functions/subsystems are security-related.
Look in the Common Criteria (ISO/IEC 15408) for many generic reusable
security function requirements.

Requirements 72

Example Safety Function/Subsystem Rqmts
Except when the weapons bay doors are open or have been open within
the previous 90 seconds, the weapons bay cooling subsystem shall
maintain the temperature of the air in the weapons bay at or below X° C.
The Fire Detection and Suppression Subsystem (FDSS) shall detect
smoke above X ppm in the weapons bay within 2 seconds at least 99.9%
of the time.
The FDSS shall detect temperatures above X° C in the weapons bay
within 2 seconds at least 99% of the time.
Upon detection of smoke or excess temperature, the FDSS shall begin fire
suppression within 1 second at least 99.9% of the time.

Requirements 73

Example Security Function/Subsystem Rqmts
Access Control Function:
• The Access Control Function shall require at least 99.99% of users to identify
themselves before enabling them to perform the following actions: …
• The Access Control Function shall require at least 99.99% of users to
successfully authenticate their claimed identity before enabling them to
perform the following actions: …
• The Access Control Function shall authorize the system administrators to
configure the maximum number of unsuccessful authentication attempts
between the range of 1 and X.
• The Access Control Function shall perform the following actions when the
maximum number of unsuccessful authentication attempts has been
exceeded: …

Requirements 74

4) Defensibility Constraints – 1
A constraint is any engineering decision that has been chosen to be
mandated as a requirement. For example:
• Architecture constraints
• Design constraints
• Implementation constraints
(e.g., coding standards, safe language subset, and nonhazardous materials)
• Integration constraints
• Deployment or configuration constraints

A safety constraint is any constraint primarily intended to ensure a
minimum level of safety (e.g., a mandated safeguard).
A security constraint is any constraint primarily intended to ensure a
minimum level of security (e.g., a mandated countermeasure).
Safety and security standards often mandate industry best practices as
constraints.
Requirements 75

Defensibility Constraints – 2
Safety/Security-
Right circle Safety/Security Significant Safety/Security
Diagonal lines
Overlaps only 2 of
the other types
All of the defensibility
constraints are
safety- or security-
significant


Requirements 76

Example Safety Constraints

The system shall use hardware interlocks to prevent users from physically
coming into contact with moving parts.
The system shall not have a single point of failure that can cause an
accident unless the associated risk is acceptable to authoritative
stakeholders.
The system shall use a safe subset of C++.
The system shall not contain any of the hazardous materials in Table X in
concentrations in excess of those listed in the table:
• Biologically Hazardous Materials (e.g., infectious agents or toxins)
• Chemically Hazardous Materials (e.g., carcinogens, corrosives, heptatoxins,
irritants, mutagens, nephratoxins, neurotoxins, poisons, or teratogens)
• Physically Hazardous Materials (e.g., combustible chemicals, compressed
gases, explosives, flammable chemicals, oxidizers, or pyrophorics)

Requirements 77

Example Security Constraints

The system shall user use IDs and passwords for identification and
authentication.
The system shall incorporate COTS firewalls to protect servers.
The system shall incorporate the latest version of Norton Antivirus for
malware detection and removal.
The system shall use public key encryption to protect confidential
information.
The system shall use digital signatures to provide nonrepudiation.

Requirements 78

Where We Are
Related Disciplines
Challenges
Common Example

Collaborative Defensibility Engineering Method ◄
Conclusion

Requirements 79

Desired Method Properties
Help meet challenges listed at start of tutorial
Promote close collaboration among safety, security, and requirements
teams
Better integrate safety and security methods:
• Based on common foundational concepts and terminology
• Reuse of techniques and work products
• Based on defensibility (safety and security) analysis

Better integrate safety and security engineering with requirements
engineering:
• Clearly defined role and team responsibilities
• Early input to requirements engineering
• Develop all types of safety- and security-related requirements

• Ensure these requirements have the proper characteristics
Requirements 80

Overall Defensibility Engineering Method

Defensibility
Policy
Monitoring Event Handling
Development

Defensibility
Defensibility
Program
Planning
Analysis

Defense
Compliance Certification and
Determination Assessment Accreditation

Requirements 81

Engineering Safety and Security-Related Requirements

Engineering Safety and Security-Related Requirements

Recommended

Recommended

More Related Content

Similar to Engineering Safety and Security-Related Requirements

Similar to Engineering Safety and Security-Related Requirements (20)

More from Donald Firesmith

More from Donald Firesmith (11)

Recently uploaded

Recently uploaded (20)

Engineering Safety and Security-Related Requirements