The Rugged Way in the Cloud--Building Reliability and Security into Software Talk given at TRISC and INNOTECH 2011

1. The Rugged Way in the
Cloud–Building Reliability
and Security Into Software
James Wickett
james.wickett@owasp.org
1

2. 2

3. @wickett
• Operations and Security for software
delivered on the cloud
• National Instruments, R&D
• Certs: CISSP, GSEC, GCFW, CCSK
• Tags: OWASP, Cloud, DevOps, Ruby
• Blogger at theagileadmin.com
• I do stuff for LASCON (http://lascon.org)
• Twitter: @wickett
3

4. Cloud @ NI
We built a DevOps team to rapidly deliver
new SaaS products and product functionality
using cloud hosting and services (IaaS, PaaS,
SaaS) as the platform and operations, using
model driven automation, as a key
differentiating element.
With this approach we have delivered
multiple major products to market quickly
with a very small staffing and financial outlay.
4

5. National Instruments
• 30 years old; 5000+ employees
around the world, half in Austin,
mostly engineers; $873M in
2010
• Hardware and software for data
acquisition, embedded design,
instrument control, and test
• LabVIEW is our graphical
dataflow programming language
used by scientists and engineers
in many fields
5

6. From toys to black holes
6

7. NI’s Cloud Products
• LabVIEW Web UI Builder
• FPGA Compile Cloud
• more to come...
7

8. ni.com/uibuilder
8

9. 9

10. 10

11. FPGA Compile Cloud
• LabVIEW FPGA compiles take hours and
consume extensive system resources;
compilers are getting larger and more
complex
• Implemented on Amazon - EC2,
Java/Linux,C#/.NET/Windows,
and LabVIEW FPGA
• Also an on premise product,
the “Compile Farm”
11

12. Using the FPGA
Compile Cloud
12

13. Building
Rugged
In
13

14. Am I healthy?
14

15. Am I healthy?
• Latest and greatest research
• Justification to insurance companies
• Measurement and testing as available
• Point in time snapshot
15

16. Am I secure?
16

17. Am I secure?
• Latest and greatest vulnerabilities
• Justification of budget for tools
• Measurement and testing as available
• Point in time snapshot
17

18. People, Process, Tech
18

19. It’s not our problem anymore
19

20. If you want to build a ship, don't
drum up people together to collect
wood and don't assign them tasks
and work, but rather teach them to
long for the endless immensity of
the sea
- Antoine Jean-Baptiste Marie Roger de Saint Exupéry
20

21. Twitter Survey
What is one word that you
would use to describe ‘IT
Security’ people?
21

22. unicorns
paranoid prepared
Tenacious
HAWT!
smart masochistic
demented jaded smart
sisyphean
omnium-gatherum
facebored
passionate
weird
drunk compassionate
22

23. Us vs. Them
• Security professionals often degrade
developers
• Developers don’t get security people
• There is interest across the isle, but often
ruined by negative language
23

24. Why do you see the speck that is in your
brotherʼs eye, but do not notice the log that is in
your own eye?
- Jesus
24

25. Adverse conditions
need Rugged solutions
25

26. Adversity fueled
innovation
• NASA in Space
• Military hard drives
• ATMs in Europe
26

27. Chip and PIN ATM
27

28. The Internets is Mean
• Latency
• Distribution
• Anonymity
• Varied protocols
• People
28

29. Systems are complex
• “How Complex Systems Fail”
• Failure at multiple layers
• Synonyms in other industries
• Defense in Depth
29

30. Software needs to
meet adversity
30

31. Intro to Rugged
by analogy
31

32. Current Software
32

33. Rugged Software
33

34. Current Software
34

35. Rugged Software
35

36. Current Software
36

37. Rugged Software
37

38. Current Software
38

39. Rugged Software
39

40. Current Software
40

41. Rugged Software
41

42. Current Software
42

43. Rugged Software
43

44. 44

45. Rugged Software
Manifesto
45

46. I am rugged... and more importantly,
my code is rugged.
46

47. I recognize that software has become
a foundation of our modern world.
47

48. I recognize the awesome
responsibility that comes with this
foundational role.
48

49. I recognize that my code will be used
in ways I cannot anticipate, in ways it
was not designed, and for longer
than it was ever intended.
49

50. I recognize that my code will be
attacked by talented and persistent
adversaries who threaten our
physical, economic, and national
security.
50

51. I recognize these things - and I
choose to be rugged.
51

52. I am rugged because I refuse to be a
source of vulnerability or weakness.
52

53. I am rugged because I assure my
code will support its mission.
53

54. I am rugged because my code can
face these challenges and persist in
spite of them.
54

55. I am rugged, not because it is easy,
but because it is necessary... and I
am up for the challenge.
55

56. Rugged-ities
• Availability
• Survivability
• Defensibility
• Security
• Longevity
• Portability
56

57. Security vs. Rugged
• Absence of • Verification of
Events quality
• Cost • Benefit
• Negative • Positive
• FUD • Known values
• Toxic • Affirming
57

58. Rugged Survival Guide
• Defensible Infrastructure
• Operational Discipline
• Situational Awareness
• Countermeasures
On YouTube: “PCI Zombies”
58

59. Security as a Feature
• SaaF is possible, but hard for most products
• Tough to measure
• Hiding among other features
59

60. Rugged as a Feature
• RaaF addresses to customer felt needs
• Values that people covet
• Buyers want it
60

61. Qualities of Rugged
Software
• Availability - Speed and performance
• Longevity, Long-standing, persistent - Time
• Scalable, Portable
• Maintainable and Defensible - Topology Map
• Resilient in the face of failures
• Reliable - Time, Load
61

62. Measuring Ruggedness
• Physical: Heat, Cold, Friction, Time, Quantity
of use, Type of use
• Software: Concurrency, Transactions, Speed,
Serial Load, Input handling, Entropy, Lines of
Code
62

63. Measuring Frameworks
• Measured by lack of incidents and
quantifying risk and vulns
• OWASP / CVE tracking
• Common Vuln Scoring System (CVSS)
• Mitre Common Weakness Enumeration
(CWE)
• Common Weakness Scoring System
(CWSS)
63

64. Supply and ______
64

65. Marketing Possibilities
• Positive: Rugged Rating System
• 3rd party verification of Ruggedness
• Self Attestation
• Negative: warning signs
• Buyers Bill of Rights
65

66. Measuring Rugged
66

67. 3rd Party Warnings
67

68. Self Attestation
68

69. Implicit vs. Explicit
69

70. Explicit Requirements
• Customers Demand
• 20% Use Cases
• Most Vocal
• Failure results in loss of customers but not
all customers
70

71. Implicit Requirements
• Customers Assume
• 80% of use cases
• Unsaid and Unspoken
• Most basic and expected features
• Failure results in a loss of most customers
71

72. Is Security Explicit
or Implicit?
72

73. Is Rugged Explicit
or Implicit?
73

74. 74

75. Rugged
Implementations
75

76. build a
rugged
team
76

77. People and Process
• Sit near the developers... DevOpsSec
• Track security flaws or bugs in the same bug
tracking system
• Train to automate
• Involve team with vendors
• Measurement over time and clear communication
77

78. OPSEC Framework
• Know your system and people
• Make security better in small steps
• Add layers of security without
overcompensating
• Use a weekly, iteration-based approach to
security
78

79. 79

80. Programmable
Infrastructure
Environment
80

81. Configuration
Management
• Infrastructure as Code (IaC)
• Model driven deployment
• Version control everything
• PIE (Programmable Infrastructure
Enviroment)
• Know Your Environment if you want to
make it defensible
81

82. What is PIE?
• a a framework to define, provision,
monitor, and control cloud-based systems
• written in Java, uses SSH as transport,
currently supports Amazon AWS (Linux
and Windows)
• takes an XML-based model from source
control and creates a full running system
• to define, provision, monitor, and control
cloud-based systems
82

83. PIE ingredients
• model driven automation
• infrastructure as code
• DevOps
• dynamic scaling
• agility
• security in the model
83

84. 84

85. The Model
• XML descriptions of the system as ‘specs’
• system (top level)
• environment (instance of a system)
• role (“tier” within a system)
• image (specific base box config)
• service (specific software or application)
• commands (for various levels)
• templates (files to be parsed)
85

86. 86

87. 87

88. The Registry
• uses Apache Zookeeper
(part of Hadoop project)
• the registry contains information
about the running system
• specific addressing scheme:
• /fcc/test1/external-services/2/tomcat
• [/<system>/<environment>/<role>/<instance>/<service>]
pie registry.register /fcc/test1/external-services/2
pie registry.bind /fcc/test1
pie registry.list /fcc/test1
88

89. Control
• create, terminate, start, stop instances using
the AWS API
• enforce scaling policy
• execute remote commands
pie control.create /fcc/test1/external-services/2
pie control.stop /fcc/test1/external-services/2
pie control.enforce /fcc/test1
pie control.remote.service.restart /fcc/test1/external-services/2/external-tomcat
pie control.remote.execute /fcc/test1/external-services/2 –i exe[0]=“ls –l /etc/
init.d”
89

90. Provisioning
• deploy services and apps
• two-phase for fast deploys
• update config files and parse templates
pie provision.deploy.stage /fcc/test1/external-services/2 –i pack[0]=lvdotcom-auth
pie provision.deploy.run /fcc/test1/external-services/2 –i pack[0]=lvdotcom-auth
pie provision.remote.updateConfig /fcc/test1
90

91. Monitoring
• integrated with third party SaaS monitoring
provider Cloudkick
• systems register with Cloudkick as they
come online and immediately have
appropriate monitors applied based on tags
set from the model
91

92. 92

93. Logging
• logging in the cloud using splunk
• logging agents are deployed in the model
and they are given the config from registry
and the model as they come online
93

94. Rugged Results
• repeatable – no manual errors
• reviewable – model in source control
• rapid – bring up, install, configure, and test
dozens of systems in a morning
• resilient – automated reconfiguration to
swap servers (throw away infrastructure)
• rugged by design
94

95. build
the new
DMZ
95

96. What’s a DMZ?
• Demilitarized Zone
• Physical and logical divisions between assets
• Military history
• Control what goes in and what goes out
96

97. Control your
environment
• Make every service a DMZ
• Cloud environment
• 3-tier web architecture
• Allow automated provisioning
97

98. Traditional 3-Tier Web Architecture
Firewall
Web
Web
Web
DMZ 1
Firewall
Middle Tier Middle Tier
DMZ 2
Firewall
DB LDAP
DMZ 3
98

99. Rugged Architecture
firewall firewall firewall
Web Web Web DMZ x3
firewall firewall
Middle Tier Middle Tier
DMZ x2
firewall firewall
DB LDAP
DMZ x3
99

100. firewall firewall firewall
Web Web Web
Repeatable
firewall firewall
Verifiable
Middle Tier Middle Tier Prod/Dev/Test Matching
firewall firewall
Controlled
DB LDAP Automated
firewall firewall firewall firewall firewall firewall
Web Web Web Web Web Web
firewall firewall firewall firewall
Middle Tier Middle Tier Middle Tier Middle Tier
firewall firewall firewall firewall
DB LDAP DB LDAP
100

101. firewall firewall firewall firewall firewall firewall firewall firewall firewall
Web Web Web Web Web Web Web Web Web
firewall firewall firewall firewall firewall firewall
Middle Tier Middle Tier Middle Tier Middle Tier Middle Tier Middle Tier
firewall firewall firewall firewall firewall firewall
DB LDAP DB LDAP DB LDAP
firewall firewall firewall firewall firewall firewall firewall firewall firewall
Web Web Web Web Web Web Web Web Web
firewall firewall firewall firewall firewall firewall
Middle Tier Middle Tier Middle Tier Middle Tier Middle Tier Middle Tier
firewall firewall firewall firewall firewall firewall
DB LDAP DB LDAP DB LDAP
firewall firewall firewall firewall firewall firewall firewall firewall firewall
Web Web Web Web Web Web Web Web Web
firewall firewall firewall firewall firewall firewall
Middle Tier Middle Tier Middle Tier Middle Tier Middle Tier Middle Tier
firewall firewall firewall firewall firewall firewall
DB LDAP DB LDAP DB LDAP
101

102. Rugged 3-Tier
Architecture Benefits
• Control
• Config Management
• Reproducible and Automated
• Data can’t traverse environments
accidentally
• Dev and Test Tier accurate
102

103. OWASP Secure Coding
Quick Reference Guide
• Checklist format that can be added to into
your sprints
• Helps development team find common
security flaws
• Topics include: Input Validation, Output
Encoding, Auth, Session Management,
Memory Management, ...
• http://bit.ly/OWASPQuickRef
103

104. Rugged Next Steps
• Use Rugged language
• Know your systems
• Automate, track results, repeat
• Begin weekly OPSEC in your org
• Attend LASCON (http://lascon.org)
104

105. Rugged Resources
105

106. h"ps://groups.google.com/a/owasp.org/group/rugged-­‐so4ware
106

107. Recommended Reading
107