Presented at the ISACA Meeting in Sydney, Australia on 17 November 2010

1. Implementing OpenSAMM and BSIMM
Christian Heinrich
ISACA - Sydney, Australia
17 November 2010
1
Further information is available from:
• http://bsimm.com/
• http://www.opensamm.org/

2. Microsoft SDL
2
Further information is available from http://www.microsoft.com/
security/sdl/

3. Microsoft SDL
3
Further information is available from http://www.microsoft.com/
security/sdl/getstarted/assess.aspx

4. Software Assurance Maturity Model (SAMM)
4
Assessment Scores
0
Implicit starting point with the Practice unfulfilled
1
Initial understanding and ad hoc provision of the Practice
2
Increase efficiency and/or effectiveness of the Practice
3
Comprehensive mastery of the Practice at scale
+ indicates that some activities of the higher level are present

5. OpenSAMM
Open Software Assurance Maturity Model (SAMM)
• An OWASP Project (Funded by Fortify)
Releases
• Draft (Beta) - August 2008
• Final (1.0) - March 2009
5

6. BSIMM
Building Security In Maturity Model
• Forked from OpenSAMM (Beta) Draft
• Developed by Fortify and Cigital
Releases
• BSIMM1 - January 2009
• BSIMM1.5 - November 2009
• BSIMM2 - May 2010
6

7. BSIMM - Sample Size - USA
Total of Nine (9) with Two (2) Unnamed out of 25 Most Advanced SSI
7
Financial Services (the other two remain anonymous):
• The Depository Trust and Clearing Corporation (DTCC)
• Wells Fargo
Independent Software Vendors:
• Adobe
• Microsoft
• Qualcomm - Vendor for the Eudora e-mail client
Technology Firms:
• Google
• EMC
Quoted from p2 or p5 (PDF Page Numbering) of BSIMM v1.5

8. BSIMM - Sample Size - Europe
Total of Nine (9) with Five (5) Unnamed out of 56 SSI
8
The bottom row of logo is two companies i.e. total of Five.
Financial Services
• Standard Life
• SWIFT
Media and Telecommunications
• Nokia
• Thomson Reuters
• Telecom Italia
Quoted from BSIMM v1.5 p51 or p54 (PDF Page Numbering)

9. BSIMM2 - Sample Size
Total of 30
9
Vague in terms of who is unnamed from the BSIMM (Europe and USA) - may have been able to reverse the
prior unnamed from BSIMM (Europe and USA)
Financial Services
• Bank of America
• Capital One
• SallieMae
Independent Software Vendors
• VMWare
• Intel
• Intuit
• Symantec
Quoted from p4 or p7 (PDF Page Numbering) of BSIMM2

10. Licensing
Both are Creative Commons (Attribution and Share Alike).
Data for BSIMM is COMMERCIAL-IN-CONFIDENCE
• Rumoured that VMWare is “VirtualWare” Case Study within OpenSAMM
10

11. Approach - OpenSAMM
Integrates with the existing internal development organisational structure.
• Must be reasonably mature development culture lacking secure SDL
11

12. Approach - BSIMM
BSIMM dictates the creation of a “new” Software Security Group (SSG)
Executive Representation and Endorsement of Software Security Initiative (SSI)
• Bill Gates (Microsoft) “Trustworthy Computing” Memo in Jan 2002
Scenarios:
• Large and political development team vs smaller existing security group
• Receipt of Outsourced Development
12
Further information on the Memo from Bill Gates is available from http://
www.wired.com/techbiz/media/news/2002/01/49826

13. Implementation - OpenSAMM - Lightweight
13
Further information is available from p21 of p96 (PDF Numbering) of
OpenSAMM v1.0

14. Implementation - OpenSAMM - Detailed
14
Further information is available from p21 of p96 (PDF Numbering) of
OpenSAMM v1.0

15. Implementation - OpenSAMM - Detailed
15
Further information is available from p21 of p96 (PDF Numbering) of
OpenSAMM v1.0

16. Implementation - OpenSAMM - Detailed
16
!"#$%&#
!
"#$%!&#""#'!()#*+,"$!-%!%.,!&#/,!+,0,+
!1-2'%-2'!$#3%4-),!42%.!)5/2",'%-)6!
$,&5)2%6!*,$%7()-&%2&,$!2'!(+-&,
!8,%!*-$,+2',!3#)!$,&5)2%6!9'#47
.#4!-"#':!%,&.'2&-+!$%-33
!;'-*+,!<5-+2%-%20,!$,&5)2%6!&.,&9$!
3#)!*-$,+2',!$,&5)2%6!9'#4+,/:,
#$''"##()"&!*'#
!=>?@!/,0,+#(",'%!$%-33!*)2,3,/!#'!
$,&5)2%6!2$$5,$!42%.2'!(-$%!A!6,-)
!=B>@!$,'2#)!/,0,+#(",'%C
-)&.2%,&%!!$%-33!*)2,3,/!#'!$,&5)2%6!
2$$5,$!42%.2'!(-$%!A!6,-)
!D-5'&.!%,&.'2&-+!:52/-'&,!42%.2'!
'+#&#
!E)-2'2':!&#5)$,!*52+/#5%!#)!+2&,'$,
!F':#2':!"-2'%,'-'&,!#3!
%,&.'2&-+!:52/-'&,
,"!#+--"%
!G,0,+#(,)$!HA7I!/-6$C6)J
!K)&.2%,&%$!HA7I!/-6$C6)J
!"%.&"/(%"0"%#
!L#+2&6!M!N#"(+2-'&,!7!I
!8,&5)2%6!O,<52),",'%$!7!A
!8,&5),!K)&.2%,&%5),!7!A
Further information is available from p21 of p96 (PDF Numbering) of
OpenSAMM v1.0

17. Implementation - OpenSAMM - Roadmap
Examples provided for:
• Independent Software Vendors
• Online Service Providers
• Financial Services Organisations
• Government Organisations
17
Further information is available from p27-p31 of p96 (PDF Numbering) of
OpenSAMM v1.0

18. Implementation - OpenSAMM - Scorecard
18
Further information is available from p26 of p96 (PDF Numbering) of
OpenSAMM v1.0

19. Implementation - BSIMM - Skeleton
Consider all objectives from BSIMM and apply as applicable
19
Quoted from BSIMM 1.5 p3
Unify into “Buckets”:
Frequency of activities across all nine (9) organisations.
Creating maturity levels from the “Buckets”.
This was performed independently and then merged and created “BSIMM Skeleton”
Quoted from BSIMM v1.5 p35/p38 (PDF Numbering) “The BSIMM skeleton provides a way to view the maturity
model at a glance and is useful when assessing a software security program. The skeleton includes one
page per practice organized by three levels. Each activity is associated with an objective. More complete
descriptions of the activities, examples, and term definition can be found in the main document”

20. Implementation - BSIMM
20

21. BSIMM - Activities - Global
21
Yellow - 8 out of 9 USA
Yellow/Blue - More common to USA
Blue - 8 out of 9 Europe
Table quoted from p53 or p56 (PDF Page Numbering) of BSIMM v1.5
SM is “Strategy and Metrics”
CP is “Compliance and Policy”
T is “Training”
AM is “Attack Models”
SFD is “Security Features and Design”
SR is “Standards and Requirements”
AA is “Architecture Analysis”
CR is “Code Review”
ST is “Security Testing”
PT is “Penetration Testing”
SE is “Software Environment”
CMWM is “Configuration Management and Vulnerability Management”

22. BSIMM2 - Activities
22
Fifteen (15) core activities are highlighted in yellow
Quoted from p50 or p53 (PDF Page Numbering) from BSIMM2
SM is “Strategy and Metrics”
CP is “Compliance and Policy”
T is “Training”
AM is “Attack Models”
SFD is “Security Features and Design”
SR is “Standards and Requirements”
AA is “Architecture Analysis”
CR is “Code Review”
ST is “Security Testing”
PT is “Penetration Testing”
SE is “Software Environment”
CMWM is “Configuration Management and Vulnerability Management”

23. Ten Core Activities Everybody Does
Objective Activity
build support throughout organization create evangelism role/internal marketing
meet regulatory needs or customer demand with
a unified approach
create policy
promote culture of security throughout the organization provide awareness training
see yourself in the problem create/use material specific to company history
create proactive security guidance around security features build/publish security features (authentication,
role management, key management, audit/log,
crypto, protocols)
build internal capability on security architecture have SSG lead review efforts
drive efficiency/consistency with automation use automated tools along with manual review
use encapsulated attacker perspective integrate black box security tools into the QA
process (including protocol fuzzing)
demonstrate that your organization’s code needs help too use external pen testers to find problems
provide a solid host/network foundation for software ensure host/network security basics in place
[SM1.2]
[CP1.3]
[T1.1]
[T2.2]
[SFD1.1]
[AA1.3]
[ST2.1]
[PT1.1]
[SE1.2]
[CR2.1]
BSIMM - Top Ten - USA
23
“3 out of 12 Practices are not implemented i.e.
• “Attack Models”
• “Standards and Requirements”
• “Configuration and Vulnerability Management”
Quoted from BSIMM v1.5 p47/p50 (PDF Page Numbering)
Within the “Governance” Domain:
• SM is “Strategy and Metrics” Practice
• CP is “Compliance and Policy” Practice
Within the “Intelligence” Domain:
• SFP is “Security Features and Design” Practice
Within the “SDL Touchpoints” Domain:
• AA is “Architectural Analysis” Practice
• CR is “Code Review” Practice
• ST is “Security Testing” Practice
Within the “Deployment” Domain:
• PT is “Penetration Testing” Practice
• SE is “Software Environment” Practice

24. Three Core Activities that Most Organizations Do
Objective Activity
understand the organization’s history collect and publish attack stories
meet demand for security features create security standards
use ops data to change dev behavior identify software bugs found in ops monitoring and
feed back to dev
[AM1.4]
[SR1.1]
[CMVM1.2]
BSIMM - Top 3 Uncommon- USA
24
Recommended as future activities to be performed.
Within the “Intelligence” Domain:
• AM is “Attack Models” Practice
• SR is “Standards and Requirements” Practice
Within the “Deployment” Domain:
• CMVM is “Configuration Management Vulnerability Management” Practice
Table above quoted from BSIMM v1.5 p47/p50 (PDF Page Numbering)

25. BSIMM - SSI Duration - Global
0
3.75
7.50
11.25
15.00
USA (Jan 2009) Europe (Nov 2009)
Oldest
Average
Newest 25
USA - 1 (year) 1/2 (6 months), 5 (years) 1/3 (4 months), 10 (years)
USA Data Quoted from BSIMM v1.5 pp2 and 3 (same as PDF Page Number)
European - 1 1/2 (6 months), 6 2/3 (8 months), 14 (years)
European Data Quoted from BSIMM v1.5 p51 or p54 (PDF Page Numbering)
BSIMM2 - 1/4 (3 months), 4 5/12 (5 Months), 14 Years - September 2009
Quoted from BSIMM2 p4 or p7 (PDF Page Numbering)

26. BSIMM - Resourcing - Global
USA - Jan 2009
Developer Satellite SSG
Median 5000 20 20
Average 7550 79 41
Largest 30000 300 100
Smallest 450 0 12
Europe - Nov 2009
Developer Satellite SSG
Median 5000 0 11.5
Average 4664 29 16
Largest 12000 140 50
Smallest 400 0 1
26
Colours used in table signify Pink -> Average, Blue -> Less and Purple ->
More
Europe has a significant lower number of resources within their SSG
compared to the USA. Yet their (European) SSI has been executing for a
longer duration.
“Satellite” are professionals outside of the SSG who have an interest in
software security” as per the definition quoted from p6 or p9 (PDF Page
Numbering) of BSIMM v1.5

27. BSIMM2 - Resourcing - Global
May 2010
Developer Satellite SSG
Median 3000 11 13
Average 5061 39.7 21.9
Largest 30000 300 100
Smallest 40 0 0.5
27
Major differences from BSIMM are highlighted in green
Quoted from BSIMM2 p4 or p7 (PDF Page Numbering)

28. BSIMM - Global
28
“The largest deltas appear in the Training and Security Testing practices.
There are three practices where the European companies show evidence of more activity: Compliance and Policy, Penetration Testing, and Software Environment.
When it comes to Strategy and Metrics, the averages are exactly the same.
In general, this reflects a European situation that is more process and compliance driven (including privacy compliance) and more driven to measurement.
However, the Europeans tend to carry out fewer assurance activities (for example, reviewing source code to look for bugs) and instead focus more energy getting a handle on the problem and meeting compliance criteria
through penetration testing.”
Graph quoted from BSIMM v1.5 p52/p55 (PDF Page Numbering)
SM is “Strategy and Metrics”
CP is “Compliance and Policy”
T is “Training”
AM is “Attack Models”
SFD is “Security Features and Design”
SR is “Standards and Requirements”
AA is “Architecture Analysis”
CR is “Code Review”
ST is “Security Testing”
PT is “Penetration Testing”
SE is “Software Environment”
CMWM is “Configuration Management and Vulnerability Management”

29. BSIMM2
29
Quoted from p9 or p12 (PDF Page Numbering) from BSIMM2
SM is “Strategy and Metrics”
CP is “Compliance and Policy”
T is “Training”
AM is “Attack Models”
SFD is “Security Features and Design”
SR is “Standards and Requirements”
AA is “Architecture Analysis”
CR is “Code Review”
ST is “Security Testing”
PT is “Penetration Testing”
SE is “Software Environment”
CMWM is “Configuration Management and Vulnerability Management”

30. BSIMM2
30
Quoted from p9 or p12 (PDF Page Numbering) from BSIMM2
SM is “Strategy and Metrics”
CP is “Compliance and Policy”
T is “Training”
AM is “Attack Models”
SFD is “Security Features and Design”
SR is “Standards and Requirements”
AA is “Architecture Analysis”
CR is “Code Review”
ST is “Security Testing”
PT is “Penetration Testing”
SE is “Software Environment”
CMWM is “Configuration Management and Vulnerability Management”

31. BSIMM2
31
Refer back to BSIMM Top 30 on prior slide to compare Financial Services.
Twelve (12) Financial Services vs Seven (7) ISV
ISV is “Independent Software Vendors” - Include Adobe, Microsoft
Quoted from p10 or p13 (PDF Page Numbering) from BSIMM2
SM is “Strategy and Metrics”
CP is “Compliance and Policy”
T is “Training”
AM is “Attack Models”
SFD is “Security Features and Design”
SR is “Standards and Requirements”
AA is “Architecture Analysis”
CR is “Code Review”
ST is “Security Testing”
PT is “Penetration Testing”
SE is “Software Environment”
CMWM is “Configuration Management and Vulnerability Management”

32. BSIMM2
32
Nominalised but Dataset has not been released.
Quoted from p10 or p13 (PDF Page Numbering) from BSIMM2

33. Thanks Sandra, Carmen and David
christian.heinrich@cmlh.id.au
Slides are Published on
• http://www.slideshare.net/cmlh
Slides can be downloaded from
• http://github.com/cmlh/
In Closing
33