This presentation points out gaps in what we teach in systems engineering using a simulation of the ARRL Sweepstakes contest as an example or case study in developing part of a simulation.

1. Systems engineering: It’s an
enabler
Presentation at Orbotech 7 March 2011
Dr Joseph Kasser, CEng, FIET, CM, CMALT
1

2. Assumptions
 You know something about systems
engineering
 Talking about philosophy to make you think about
the way you apply it
 You do not know about amateur radio
 The application domain in the case study
 You understand enough English to
understand this talk
 Example of importance of stating assumptions
 Facilitates communications
2

3. Topics
 Systems engineering camps
 Gaps in what we teach
 Case study extract
 Pointing out some of the gaps
 Lessons learnt from the case study
 Systems engineering: an enabler
3

4. What is systems engineering?
4

5. The systems engineering camps
 The process camp
 Functional perspective
 The discipline camp
 Structural perspective
 The problem camp
 Operational perspective
 The activity camp
 Functional/operational perspective with an
objective definition 5

6. Parable of blind men and elephant*
“… And so these men of Indostan
Disputed loud and long,
Each in his own opinion
Exceeding stiff and strong,
Though each was partly in the right,
And all were in the wrong!
MORAL.
So oft in theologic wars,
The disputants, I ween,
Rail on in utter ignorance
Of what each other mean,
And prate about an Elephant
Not one of them has seen!”
* Yen, D. H., The Blind Men and the Elephant, 2008,
http://www.noogenesis.com/pineapple/blind_men_elephant.html, last accessed 26 October 2010 6

7. A matter of perspective.
Functional
Functional and
operational
Process
Take
over the
world
Operational
Perspectives are
Problem
incomplete and
there are gaps
7

8. Topics
 Systems engineering camps
 Gaps in what we teach
 Case study extract
 Pointing out some of the gaps
 Lessons learnt from the case study
 Systems engineering: an enabler
8

9. Amateur radio - context
 World-wide hobby
 Licensed by governments
 National societies
 American Radio Relay League (ARRL)
 Radio Society of Great Britain (RSGB)
 Singapore Amateur Radio Transmitting Society (SARTS)
 Experimenting and applying
 Emergency communications, terrestrial, satellite, hardware,
software, business, etc.
 Platform for developing systems engineers*
* Kasser, J. E., "How synergy between amateur radio, systems and other engineering
can raise the technical quotient of a nation", proceedings of the 4th Asia-Pacific
Conference on Systems Engineering (APCOSE 2010), Keelung, Taiwan, 2010.
9
http://www.youtube.com/watch?v=nd0tTYgJWEo

10. Big picture - context
 Simulated emergency message traffic
handling
 Contest format
 Exchange messages simulating traffic into
and out of simulated disaster zone
 World-wide contests
 Everybody contacts everybody
 Local and regional contests
 US Sweepstakes, Australia, BERU, etc.
 Target area contests
 Everybody tries to contact stations in a given area
 US State QSO parties, ARRL DX, SAS, WAE, etc.
 Go back in time to 1977/8 10

11. ARRL Sweepstakes contest
[1977]
 Contact (work) as many other stations as
possible within 48 hour period
 Weekends in November
 Exchange simulated emergency message
 Use different frequency bands with different
propagation characteristics
 Score = number of contacts * multiplier
 Multiplier is number of ARRL Sections contacted
 Section only counts once irrespective of frequency band
11

12. Big picture - lifecycle
A1
12

13. Column A: a systems engineering
approach to problem solving*
3
1. Plan the work (Tasks 2-8)
2. Define the problem 1 2 5 6 7 8
3. Conceive solution options 4
4. Identify ideal solution evaluation
criteria  Recursive or self similar
5. Perform trade off to find the
optimum solution  Applies to each box
6. Select the preferred option  Fits in one column of the
7. Formulate strategies and plans to HKMF
implement the preferred option
8. Milestone review to obtain  Fits across several columns
authorisation to proceed to of the HKMF
implementation phase
* Tasks 2-7 from Hitchins, D. K., Systems Engineering. A 21st Century Systems Methodology, John Wiley & Sons 13
Ltd., Chichester, England, 2007., Figure 6.2

14. Focusing in on the problem on
Reliance
 technology
Problem had been recognized
Technology
at least 12 years earlier
 The time wasenable
may 1978
Understand the factors

orinprohibit
involved the ARRL
solution
sweepstakes contest well
enough to enable an operator
in Silver Spring, MD to contact
Problem all the Sections (multipliers)
statement is given the constraints of low
radiated power
different
14
Key words in red

15. Exploring solutions to determine
real need
1. Read about factors 2. Create contest simulation
 Identify the factors  Identify the factors
 Identify relationships  Identify relationships
between factors between factors
 Develop understanding  Develop understanding
 Create simulation
 Run simulation
 Try approaches and
see what happens
 Develop better
understanding 15

16. Radio Propagation
Sky wave
(refraction altitude dependent on frequency and time (solar effect)
Ground wave Interference
My QTH
Distant Section
16

17. ARRL Sections in 1977
Note: Numbers are assigned for this project not by the ARRL 17

18. The pertinent factors
 The Sections
 75, spread out across CONUS, and non-CONUS
 Probability of propagation to Section
 Radio frequency band
 Time of day
 Probability of someone in Section when wanted
 Population distribution
 Probability of interference from other stations
 Transmitted power
 Receiver front end characteristics
18

19. Statement of the problem
 Develop a simulation that is:
1. Realistic enough to enable an operator in
Silver Spring, MD to contact all the
Sections given the constraints of low
radiated power if the operator has
developed an understanding of the
pertinent factors involved in contacting all
the Sections. Realistic enough to enable
successful completion of
2. Fun to play. mission at the appropriate
time 19

20. Feasibility study: Constraints
(implementation domain knowledge)
 INTEL Hardware platform
 Given constraint
 8080 microprocessor (8 bits)
 Assume software written in BASIC
 Memory
 32K RAM
 Interpreter occupies 16K Bytes
 Need some space for Stack and run time
variables
 Leaves about 14K Bytes for program 20

21. Feasibility study: Risk management
(implementation domain knowledge)
 Do some software code sizeWe often decouple
estimation
 Table of Stations callsign array is hardware and
largest data element
(need to know there are 2707 maximum from published scores)
software development

 Conclusion – feasible but with implementation risk

(Except in embedded
RAM Memory may be insufficient
 Risk mitigation possibilities systems)
 Use spaghetti code to cut down on memory use
 Document source code accordingly, REM statements are discarded by
interpreter
 Use IBM 360 if code won’t fit in Intelec MD8/80
 TPM: RAM usage – track during software development
21
Domain knowledge is software programming and PC platform

22. 1977 Published scores*
22
* QST, May 1978, page 68

23. What the operator does
Contact
Start Exchange data Store data
someone
Time out
(0-n minutes)
No Yes
Contest
over ?*
End
* Contest over := (24 hrs of operation [elapsed time]) or (end time GMT) 23

24. Functional flow diagrams
 Start documenting and expanding ideas
 Intuitive
 Identify sequences
 Functions can be simple or complex
 Identify dependencies between functions
 May not provide best grouping
 Are a tool to provide a view
 May not be best tool to create relationships
between functions
24

25. Alternate Function flow chart –
Call CQ (OS1.1) functional
perspective
Send Worked B4
Yes
Call CQ Reply? Yes
Duplicate?
Yes
Enough?
Send Message
Yes
Receive message Complete? Store data Say ‘Bye’
Request repeat
25

26. Function flow chart – Call CQ
(OS1.1) functional perspective
Send Worked B4
Yes
Call CQ Reply? Duplicate?
Yes
No No
Had
Enough Exchange data (QSO)
No ?
Yes
Store data Say ‘Bye’
26

27. Exchange data (QSO) (1)
functional perspective
Send Message1 Receive message2
Yes
Complete?
Request repeat
Note 1, sending station may also request you to resend message
Note 2, message may be incomplete due to interference
27

28. Exchange data (QSO) (2)
functional perspective
Receive message2
Send Message1
Complete?
Yes
No
Request repeat
Note 1, sending station may also request you to resend message
Note 2, message may be incomplete due to interference
28

29. Partial functional N2 chart
F_CQ o o o3
o F_RX o o1 o2 o o4
o F_CK o o o
o F_B4 o
o F_TXM o o o
o o F_RXM o o
o F_LOG o
o F_QSY o
o o o F_QRV
Outputs – horizontal squares, Inputs – vertical squares 29

30. F_QSO
Partial functional N2 chart
Single input, candidate
for aggregation
F_CQ o o o3
o F_RX o o1 o2 o
o F_CK o o o May need
a new
o F_B4 o interface
function
o F_TXM o o o here
o o F_RXM o o
o F_LOG o
o F_QSY o
Candidate for
o o aggregation with F_QSO o F_QRV
outputs – horizontal squares, inputs – vertical squares
30

31. How do you determine system
functions?
 Top down?
 Bottom up?
 Middle out?
 All of the above
 Tendency to flowchart functions
 N2 chart is better way
 Both approaches is best way
 Checks and balances 31

32. Do we need requirements?
 Size of project
 Very small
 Concept of operations
 Well understood
 Likelihood of changes during SDLC
 Very low
 Number of people/organizations
involved
 1
32

33. Requirements analysis -1:
Requirement for number of stations
taking part
 Published scores showed maximum number of
contacts was 2707 for top scorer
 Requirement
600. The system shall contain at least 2707 stations.
 Can set number at 3000
 (TPM: watch RAM usage, and drop to 2710 if necessary; use
parameter for value)
Critical:
 Feasibility analysis (maximum) customer
 3,000/24=125 contacts/hour = 2/minute involvement
 Reasonable (application domain knowledge)
33

34. Requirements analysis -2:
Requirement for number of stations in
each Section
 Published scores in QST list participation by Section
 Two separate parts (weekends) to contest
 Phone (SSB) and Morse code (CW)
 Set requirements for stations in each Section based on
1. SSB participation
2. CW participation
3. Combination participation in the two parts
 Assumption
 Ratio of number of stations not submitting entries is about the
same as for those submitting entries
3
1 2 5 6 7 8
4
34

35. Requirements analysis -2:
Number of stations in each
Section
3
Selection criterion
1 2 5 6 7 8
 4
 At least one station in each Section?
 If none of the options contains at least one station, then
a station may need to be inserted – see next slide for
options.
Criteria CW SSB Combined
At least one in each No No Yes
Section
Missing Sections WY,NWT CZ None
35

36. Dealing with WY, NWT and CZ
(1 participant)
3
 Options 1 2
4
5 6 7 8
1. Set value at 1 (published results)
 For all iterations of simulation
2. Set value at either 0, 1 or 2
 At start of each iteration of simulation
 Selection criteria
 It’s a simulation, numbers are small, should
have minimum effect
 Makes game more interesting (and real?)
 Uncertain if a ‘clean sweep’ can be achieved
36

37. 630. Requirements for number of
stations in Canada1
631 The system shall contain 7 stations in MAR2.
632 The system shall contain 8 stations in QB.
633 The system shall contain 16 stations in ONT.
634 The system shall contain 9 stations in MAN.
635 The system shall contain 3 stations in SK.
636 The system shall contain 7 stations in AB.
637 The system shall contain 8 stations in BC.
638 The system shall contain 0, 1 or 2 stations in NWT.
Notes 1Numbers are determined by distribution expect for NWT
37
2 See Section TBD for abbreviations

38. Looking back at the problem
statement
 Number of stations in each Section that is needed for the
understanding?
 Exact or relative?
 Let the number of stations in each Section vary by a
small percentage each time the simulation is run
 Make simulation game more interesting
 Makes for a different situation
 Valid
 distribution was assumed based on 1 set of entries
 there was an assumption on the distribution in slide 64
 Put ± tolerances on the stations in each Section
38

39. 630. Requirements for number of
stations in Canada1
631 The system shall contain between 6 and 8 stations in MAR.
632 The system shall contain between 7 and 9 stations in QB.
633 The system shall contain between 14 and 18 stations in ONT.
634 The system shall contain between 8 and 10 stations in MAN.
635 The system shall contain between 1 and 4 stations in SK.
636 The system shall contain between 6 and 8 stations in AB.
637 The system shall contain between 7 and 9 stations in BC.
638 The system shall contain 0, 1 or 2 stations in NWT.
1. Tolerances from application domain knowledge
39

40. Alternate approach to setting
requirements for number of stations
in Canada
 Use probabilistic approach and calculate for each iteration
of simulation
 Section calculation approach (F_Section)
 Test feasibility of requirements
 Calculate percentage of entries in each Section in contest from
published results
 Whole contest or by call area (easier to manage)
 Write software module to set up distribution of Sections based on
calculated percentages (± tolerance, allowing for a 0 value)
 Exercise module several times and compare results of model to
published data from results to validate module
 Write requirements to allow for both design approaches
40

41. Partial published results
CW SSB TOTAL % CANADA % CONTEST
MAR 4 3 7 11.86 0.26
QB 6 2 8 13.56 0.30
ONT 10 6 16 27.12 0.59
MAN 6 3 9 15.25 0.33
SK 2 1 3 5.08 0.11
AB 2 5 7 11.86 0.26
BC 4 4 8 13.56 0.30
NWT 0 1 1 1.69 0.04
SUM 34 25 59 100.00 2.19
41

42. 630. Requirements for number of
stations in Canada1
631 0.26±p% of the stations in the contest shall be in MAR.
632 0.30±p% of the stations in the contest shall be in QB.
633 0.59±p% of the stations in the contest shall be in ONT.
634 0.33±p%of the stations in the contest shall be in MAN.
635 0.11±p% of the stations in the contest shall be in SK.
636 0.26±p% of the stations in the contest shall be in AB.
637 0.30±p% of the stations in the contest shall be in BC.
638 0.04±p% of the stations in the contest shall be in NWT.
p% TBD
42

43. Functions and requirements
 Function
 Contact stations in Canada
 Requirements
 Specify the number of stations in each Section in
Canada
 Requirements
 are one way to quantify functions
 And ..
43

44. Requirements analysis 3:
Platform dependency
101.1 The simulation shall be written in Microsoft
BASIC Version 2.0.
101.2 When executing, the simulation shall be
contained within 12K Bytes of RAM.
OR
101. The simulation shall execute on an INTEL Intelec
8/80 Microprocessor Development System equipped
with 32 K Bytes RAM.
[That is the maximum RAM for that architecture] 44

45. Tools and techniques used in
requirements analysis (summary)
 Looked up data in application and execution domains
 Used domain data to compute values for requirements
 Developed models of probabilistic functions
 Design and realized partial elements of solution system
 Exercised models
 Tested validity of models as part of breadboarding process
 Used problem solving process a number of times
 How the wording of requirements affects the design
45

46. Big picture - lifecycle
46

47. Issues addressed in this phase
 Writing the code (construction)
 Propagation model
 Section distribution model
47

48. Propagation model
Time Frequency Band (M)  For W1/VE1 call area*
(Hrs) 10 15 20 40 80  Group of sections
0000 0 0 0 100 100  Limited to 10-80 M
0400 0 10 50 100 100  Time is EDT or Local
0800 100 0 100 100 0  Four hour time of day
1200
blocks
100 0 100 100 0
 Number indicates
1600 50 10 50 100 50
probability of
2000 0 0 0 100 100 propagation
48
* From fig 5-10 in Kasser J., Software For Amateur Radio, TAB Books, 1984, page 83

49. Propagation model
 Options
1. Logic using IF … THEN statements
2. Table driven approach
3
1 2 5 6 7 8
4
49

50. Logic approach
2230 IF B=B4 OR B=B5 AND H<12 OR B=B3 AND H>=20
AND RND(1)>.5 THEN 2810
2240 IF B=B3 AND (H<20 AND H>=12 OR RND(1)>.5 AND H>=8)
THEN 2810
2250 IF B=B2 AND (H>=20 OR H>=8 AND H<12) AND
RND(1)<0.1 THEN 2810
2260 IF B=B1 AND (H>=12 AND Q=2 AND H<20 OR H>=20
AND RND(1)>.5) THEN 2810
2270 RETURN: REM with Y5 as 0
2810 Y5=1:RETURN
B = Band
B1 … B5 amateur bands
H time of day [block]
50

51. Table driven approach
 Use an array Contact(S, B, H)
 Can be the Table of stations
 Section, Band, Time of day block
 Code
 During initialization
 Store Table values in Contact(S, B, H)
 At run time
 Set up value of S, B and H
 Index into array by value in S, B and H and determine if contact is
possible (C1 = 1)
 Y5 = Contact(S, B, H)
 If Y5 = 0 THEN C1 = 0 ELSE
 If Y5 = 1 THEN C1 = 0 ELSE
 C1 = (RND(1) < Y5) : REM Returns Logical 0 or 1
51

52. Characteristics
 Logic approach  Table array
 Does not require  Requires knowledge of
knowledge of arrays arrays
 Simple set up  Complicated set up
 Many lines of code for  Few lines of code at
execution time execution time
 Flexible probabilities  Flexible probabilities
 By changing values of  By changing data
variables embedded in loaded into table
the code without programming
 Bonus: Location can be
changed to any Section
 By changing data
loaded into table
 without programming
52

53. Selection Criteria
 Development time
 Array approach has learning curve
 Schedule issue
3
1 2 5 6 7 8
4
53

54. Decision
3
 Use logic approach 1 2 5 6 7 8
4
 Lack of domain knowledge
 Schedule driven
 Risk of non-completion of software
 Wrong decision from a ‘systems’ perspective
 Hindsight
 Completion of simulation was secondary objective
 Understanding of the situation was primary
objective
 Table-based software is very powerful and flexible
 Used later in other software 54

55. Topics
 Systems engineering camps
 Gaps in what we teach
 Case study extract
 Pointing out some of the gaps
 Lessons learnt from the case study
 Systems engineering: an enabler
55

56. Lessons learned
 Simulations should be realistic enough to enable successful
completion of mission at the appropriate time
 The same problem solving process is used in all phases of the
system development lifecycle
 Successful systems engineering needs knowledge and experience
in/of
 Systems engineering, application domain, implementation domain
 The customer/user needs to be involved in the development
 Application domain knowledge
 The need to focus on what is important
 In M&S it is “the understanding”
 Simulations don’t provide answers, they [should] provide understanding
56

57. More lessons learned
 Functional flow diagrams may not be best tool to create
relationships between functions
 N2 charts are powerful and versatile tools
 Incorrect aggregation leads to aggravation
 We probably don’t need as many system level
requirements as we think we do
 The wording of requirements affects the design
 Recursiveness and self-similarity of problem solving
process
 Technology influences design decisions
 There is knowledge in the development team that is not
delivered with the solution system
 Work should be, and can be, fun
57

58. Topics
 Systems engineering camps
 Gaps in what we teach
 Case study extract
 Pointing out some of the gaps
 Lessons learnt from the case study
 Systems engineering: an enabler
58

59. Proposed Maturity Model for measuring
competencies of engineer-leaders
Type I Type II Type III Type IV Type V
Knowledge
Declarative Procedural Conditional Conditional Conditional
Systems engineering
Declarative Declarative Conditional Conditional Conditional
Domain (problem solution)
Cognitive characteristics
System Thinking
Declarative Procedural Conditional Conditional Conditional
Descriptive
No No Procedural No Conditional
Prescriptive
Confused fact Perpetual Pragmatic Pragmatic Strategic re-
Critical Thinking finder analyser performer performer visioner
Individual traits (sample)
Communications Yes Yes Yes Yes Yes
Management No Yes Yes Yes Yes
Leadership No No Yes Yes Yes
59

60. Holistic thinking the problem
 Critical thinking
 Can’t expect systems engineers to have
knowledge in all domains
 Systems thinking
 Use continuum perspective/lateral thinking
 Reverse position of knowledge
 Key questions
 What else is used across all domains?
 Is there a discipline which is used in all domains?
 Answer
 Mathematics
60

61. Systems engineering is an
enabler
 Systems engineering is an enabler in “the
making things happen” function
 In different disciplines in different domains
 Applying systems thinking in problem solving
 Activities that deals with parts and their
interactions as a whole
 Similar to mathematics
61

62. An enabler
 “[systems engineering] is a philosophy and a way of
life”
 Hitchins, D. K., "Systems Engineering…In Search of the
Elusive Optimum", proceedings of Fourth Annual Symposium
of the INCOSE-UK, 1998.
 Systems engineering is the art and science of
creating tangible solutions to complex problems and
issues… (Hitchins)
 Application of holistic thinking in the workplace
 Product (application domain)
 Process (implementation domain)
62

63. Engineer-leaders
 Are those people who apply holistic thinking in the
workplace to:
 transform puzzling, troubling and uncertain situations into clearly
articulated problems;
 Identify optimal conceptual solutions;
 Realize those solutions within the constraints of the situation
 They perform such functions defined as design, test,
integration, systems engineering, and project management
 They need different knowledge and skills pertaining to the
domain and the area of the HKMF in which they are working
 They have various job titles (roles)
63

64. What is systems engineering?
64

65. A matter of perspective
Functional
Functional and
operational
Process
Take
over the
world Holistic
Operational
Enabler
Problem
65

66. Summary
 Systems engineering camps
 Gaps in what we teach
 Case study extract
 Pointing out some of the gaps
 Lessons learnt from the case study
 Systems engineering: an enabler
66

67. Questions or comments?
67