1. CESE
Verifying Architectural Design Rules
Of the Flight Software Product Line
Fraunhofer CESE: NASA GSFC:
Dharmalingam Ganesan David McComas
Mikael Lindvall Maureen Bartholomew
Chris Ackermann
1

2. CESE
What is the CFS?
• The Core Flight Software System (CFS) is a
mission-independent, platform-independent,
Flight Software (FSW) environment integrating a
reusable core flight executive (cFE)
• The CFS is a product-line developed by the
NASA Goddard Space Flight Center (GSFC)
2

3. CESE
Motivation for cFE/CFS
• Requirements
– The Requirements for Command and Data Handling
(C&DH) Flight Software are very similar from Flight
Project to Flight Project
– The Requirements for Guidance Navigation and
Control (GNC) Flight Software are also be quite
similar from Flight Project to Flight Project
• So, let’s not “re-invent the wheel” in each project
– cFE responds to this by allowing FSW developers
and testers to concentrate on the uniqueness of a
project
3

4. CESE
cFE/CFS Heritage
SMEX-
Lite
SAMPEX SWAS TRACE WIRE Triana Swift BAT
(launched (launched
8/92) (launched (launched 2/99) (waiting for (12/04)
12/98) 3/98) launch)
XTE (launched
TRMM (launched
12/95)
11/97) JWST
ISIM
(2011)
IceSat GLAS
(01/03) Core FSW Executive
MAP (launched
Future Spacecraft
06/01)
and Instruments
SDO (2007) LRO (2009)
ST-5 (5/06) 4

5. CESE
Business Goals and Arch. Principles
The CFS Business Goals Supporting the Goals
Reduce time to deploy
Layered architecture
high-quality software
Reduce project schedule and Reusable core modules with
cost uncertainty standard API’s
Directly facilitate formalized
Plug and play modules
software reuse
Publish-and-subscribe style
Enable collaboration across
organizations
Standard Middleware/ Software bus
Simplify sustaining engineering
and maintenance Run-time module registration
and integration
Platform for advanced concepts
and prototyping
OS and Hardware Abstraction
Common standards and tools
across NASA
5

6. CESE
Problem/Challenge
• How do we make sure that the implementation
follows all documented architectural styles (e.g.,
Layers), patterns, interface usage rules, etc?
• Otherwise, architecture remains as a hypothesis
• Technical goals can be compromised
– testability, buildability, (subtle) performance, run-time
adaptability, evolvability, etc.,
• Also, business goals can be comprised because
architecture is an important enabler
• cFE/CFS undergoes rigorous code review – are
architecture rules already checked in review?
6

7. CESE
Specified Structure
…
7

8. CESE
Example cFE/CFS Context Diagram
Mass
EDAC Storage
House- Memory Memory Memory Self GN&C Instrument System
keeping Checksum
Scrubber Manager Dwell Test Applications Manager
(4)
Software Data
Storage
Scheduler
File
Stored Manager
Commanding
Local
Storage
Inter-task Message Router (SW Bus) CFDP File
Health & Transfer
Safety
Manager
1553 Bus Telemetry Command Software Time Executive Event Table
Limit Services Services Services Services
Checker Support Output Ingest Bus
Commands
Comm Cards Transponders
cFE core App Summit Chip
Real-time Telemetry (UDP)
CFS Applications File downlink
8
Mission Apps (CFDP)

9. CESE
Approach for Verifying Rules
Bridge
abstraction gap
Mapping - sample
clonefinder
grep
ifnames
RPA
SAVE
9

10. CESE
Category of Derived (static) Rules
• Dependency-restriction rules
– Fraunhofer SAVE , Relation Partition Algebra (RPA)
• Decomposition-restriction rules
– RPA tool, grep
• Redundancy (clones) rules
– Clone finder tool
• Conditional preprocessor usage rules
– ifnames tool
 Let us see sample results (details in the paper)
10

11. CESE
Why dependency-restriction rules?– a
comment from cFE/CFS docs
• Developers are discouraged from using
the OS_QueueCreate, OS_QueueGet and
OS_QueuePut functions. These functions
are a lower level duplication of the
Software Bus Services pipes. Their usage
limit the visibility into data messages being
passed between Applications and they
would also impose a requirement that two
applications must reside on the same
processor. 11

12. CESE
Extracted Dependency from Code
App and Core are developed
almost by the same developers
This implemented view is consistent with the design guideline:
Cfe-app should use Cfe-core, but not vice-versa because
Core should be independently testable/buildable without apps
12
12

13. CESE
Extract Dependencies among Apps
Design Rule
No two applications are allowed to interact directly, and should
instead use the software bus to communicate
It should be possible to start, stop, restart, and remove
13
applications independently for improved fault tolerance.
13

14. CESE
OS Abstraction Layer (OSAL)
• Real Time Operating System APIs
– Tasks, Queues, Semaphores, Interrupts, etc.,
• File System APIs
– Abstracts the file systems that may be present
on a system
– Simulate multiple embedded file systems on a
desktop computer for testing
• Hardware APIs
– Port and memory based I/O access in order to
provide a common way of accessing
14
hardware resources

15. CESE
Why OSAL should be used? – comments
from cFE/CFS docs
• The developer must make the effort to identify
the proper methods through the cFE and OS API
to satisfy their software requirements and not be
tempted to take a “short-cut” and accomplish
their goal with a direct hardware or operating
system software interface.
• …attempting to write to EEPROM using the
standard C function memcpy will fail. Using
OS_MemCpy will succeed because the
EEPROM will be configured for writing before
the copy is performed. 15

16. CESE
OSAL By-Pass
• The CFS guide states that all communications to the OS
and hardware should go through the OSAL for portability
and testability reasons
• The analysis shows that there a few places where the
OS is directly used (i.e., standard C functions)
CFE_ES_StartApplications call memset
CFE_FS_Decompress call memset
FS_gz_inflate_codes call memcpy
main call printf
OS_Memset, OS_Memcpy, and OS_printf
Should have been called 16

17. CESE
Common Look-and-Feel
• A common directory structure template
has been defined
– Where to place header files
– Where to place configuration files
– What data to make public (and private)
• Template for interaction with software bus
and other core components
• Template for a module decomposition
• Template for a task decomposition
17

18. CESE
Common Look-and-feel violations
If modules have different look-and-feel
they are difficult to inspect, test,
understand.
Applications designed to interface with
the cFE should follow standard templates.
QQ Limit Checker (LC)
In fact, found a subtle runtime error
QQ_AppMain LC_AppMain
in a module that deviates from the
QQ template. QQ_AppInit LC_AppInit 18
- Additional performance problems

19. CESE
Module Redundancy (Clone) Analysis
• The source code of the cFE/CFS is offered
to some of its customers
• Presence of clones do not give good
impression about the cFE/CFS design
excellence
• Also, cloning is not a recommended way
to implement product lines
– Hard to evolve multiple cloned versions
19

20. CESE
Module Redundancy Analysis
• Clone finder tool detected a lot of “false”
positives (mainly due to template code)
• However, there are some true clones
within the OS abstraction layer (OSAL)
• There are no clones between Core and
App layers (developed by the same team!)
Important to
understand the context
before criticizing the
team!
20

21. CESE
Conditional Preprocessor Analysis
• Minimizing complexity is one of the
cFE/CFS design goals
• Design decisions were made to use the
preprocessor to the “minimum”
• 80% of files don’t have conditional
preprocessor statements!
Checks variation
point lower and
upper bounds
21

22. CESE
Variability Implementation Strategy
• Missions can plug-in and plug-out
applications
• Abstract APIs with alternative
implementations
• OS and hardware variants are taken care
by the OSAL
• Build scripts select and compile the
selected OS type implementation files
• Header files with the same name is used
22
to manage certain variants

23. CESE
Conclusion and Outlook
• The cFE/CFS team performs rigorous
design and code reviews thus not many
issues remain!
• Nevertheless, some issues remain
undetected
• Detected issues were added into the
discrepancy list and were addressed
• Imagine the number of architectural
violations in hardly reviewed systems!
23

24. CESE
Conclusion and Outlook
• This analysis helped in establishing the
chain of relationships among business
goals, architecture, and source code
• Important to verify whether the
implementation is indeed consistent with
the specified architecture
– Tools help a lot in performing and repeating
• Otherwise, testability, buildability,
performance, etc., can be compromised
• On-going work: run-time behavioral rules 24

25. CESE
Questions
• Dharmalingam Ganesan
– dganesan@fc-md.umd.edu
• Mikael Lindvall
– mlindvall@fc-md.umd.edu
25