This document discusses using formal specification techniques like UML, Alloy, and Prolog to discover design flaws related to application and website security. It provides examples of how these techniques can be used to find issues with input validation, data flow, and access control by modeling the system and its requirements. Formal modeling allows exploring complex behaviors and edge cases that may lead to security vulnerabilities. The document advocates applying a "cocktail" of different modeling approaches to more thoroughly test for flaws during design.

1. Application and Website Security –
Designer Edition:
Using Formal Specification Techniques To Discover
Design Flaws
Daniel Owens
IT Security Professional

2. INTRODUCTION

3. Introduction
• Systems generally come in two forms – Single
Level Security Systems (SLSS) and Multilevel
Security Systems (MLSS)
– In an SLSS, the system processes information of the
same sensitivity or privilege level, ignoring the user’s
need-to-know and clearance, but still enforce access
controls
– In an MLSS, the system must take into account
sensitivity levels, privileges, need-to-know, clearance
levels, and enforce access controls
• This work is generalized enough to apply to both

4. Previous Works
• Many previous works have looked for specific
flaws in specific systems
– The works were never intended to be general, simply
to provide analysis of a specific flaw in a specific
system
– This means that much of the works available are
difficult to apply to current or more general problems
• Additionally, most works ignore that functional
and security flaws stem from the same issues

5. Common Root Causes
• Most flaws only become problems after the flaw or
data forged by the flaw has successfully passed
through multiple checkpoints
– Any of the checkpoints could have prevented the flaw
– There are instances, such as faulty subsystems upon which
the current software relies
• Even these can be tested for, but that becomes a never-ending and
very costly thing to do, so we assume that those we have no
choice but to rely on have been heavily tested
• There is no such thing as a strictly ‘security’ or strictly
functionality flaw

6. Common Root Causes (cont)
• In 1996, Alph1 [5] clearly showed how buffer overflows can
be used to escalate privileges and execute arbitrary code
on the victim
– Buffer overflows cause visible functionality problems too – most
notably, the software being overflowed into and the software
doing the overflow will likely crash, have inconsistent data, or
operate in an unintended fashion
• More recently, Watchfire [32]made public an exploitation
technique to use dangling pointers to execute arbitrary
code on the victim
– Dangling pointers may cause visible functionality problems too –
again, most notably, the software could crash, have inconsistent
data, or operate in an unintended fashion

7. Common Root Causes (cont)
• Even weak passwords, often regarded as strictly a
security problem can create undesirable functionality
flaws
– The Morris worm exploited weak passwords and caused a
substantial degradation of services
– Conficker, which exploited a Windows bug and weak passwords,
and many other worms rely on bugs and weak passwords
• Even the most seemingly benign flaws, like off-by-one
errors can cause disastrous security and functionality
flaws
– Such flaws can be exploited to execute arbitrary code, but also
can be triggered by mere use and cause the system to crash or
have degraded services [13][15][19]

8. INPUT VALIDATION

9. Input Validation
• Input validation flaws arise when input (data egressing the
process or thread from a source outside of the process or
thread) is not properly verified and sanitized prior to use
– These include come buffer overflows, format string flaws,
integer overflows, SQL injection, LDAP injection, SMTP injection,
code injection, command injection, Cross-Site Scripting, and
many other problems
• Proper and complete design documentation can help stop
these
• The flaws can be discovered using a variety of different
techniques, but no single technique can stop them all, only
a ‘cocktail’

10. Finding Input Validation Flaws With
UML
• Many input validation flaws can be found
using UML diagrams and use case scenarios,
or can be protected against using well-crafted
OCL statements that are translated into code
during implementation
– This includes detecting or preventing certain
instances of injection attacks (SQL, XSS, command,
code, LDAP, SMTP, JSON, etc) and overflows
(buffer, off-by-one, etc)

11. UML Input Validation Example
• There is the potential that query could allow SQL injection
(DBHandler performs the Database handling), or that the user
could enter otherwise incorrectly formatted information
– OCL and supporting documentation comes into effect here
– context GUI
inv: query.matchesRegEx(“a-zA-Z0-9 ”) and query.size()<40
• Note that this requires an OCL extension be written that accepts regular
expressions
• This also forces the design team to clearly specify all data

12. Sidenote
• Regular expressions that dictate data content should be
whitelists, not blacklists
– Whitelists dictate what is allowed, blacklists dictate what is not
• All input should have OCL constraints that dictate the
allowable content (a whitelist) and the maximum (and
minimum in certain cases) size
– This helps to stop overflows and injection, as well as providing
carefully crafted and examined whitelists that conform to the
requirements, while providing input validation
• Of course if keywords and characters with special meaning
are in the whitelist, the input will have to be properly
escaped in the code – hard to convey in UML diagrams, but
pre conditions and invariants can be used to do this with
more OCL extensions

13. DATA FLOW

14. Data Flow
• Data flow flaws arise when data take paths that are
untrusted or can present various problems
– Many of these issues concern integrity and confidentiality
– The data may be modified while it traverses the less
trusted/untrusted path such that it causes problems – note
that such data should be examined prior to use
• Examples include traversing a hostile environment,
traversing faulty systems, and experiencing the data
loss that is normal in any environment

15. Data Flow (cont)
• It is extremely important to know data flow in
environments where there are levels of trust or in
MLSSs
– Most kernels implement a limited form of hierarchical
protection domains, or ‘protection rings’
• This prevents problems in outer rings from impacting the inner
ring (both functionality and security benefits)
• The military and systems requiring high fault tolerance benefit
• Limiting access points by having ‘gates’ can help
prevent or limit the scale of compromises, as well as
contain damage should a fault occur

16. Finding Data Flow Flaws With UML
• Certain data flow flaws can be exposed by
UML diagrams, but mostly in Use Case
diagrams
– This involves ensuring that data re-entering the
system has not be tampered with or is not
malicious (in some cases, both should be checked)
• This is similar to input validation, since the
data reentering the system is assumed to be
data from an external source

17. Use Case Data Flow Example
Use Case Number 1
Use Case Name Initial Data Distribution
Overview Follows the distribution of initial data in the distributed system
Type Primary
Actors Master (P0) and Slave Processors (P1-PN)
Properties
Performance O(n)
Security Data crosses system’s boundaries prior to use
Preconditions Other
All slaves are up and waiting for the master to communicate
Flow Main Flow:
1. Master prepares each matrix segment (one per slave)
2. Master sends each matrix segment to the responsible slave
3. Master enters waiting state pending responses
4. Slaves receive matrix segment
5. Slaves transmit go-ahead to master

18. Use Case Data Flow Example (cont.)
• What happens if the slaves never receive the matrix segments sent to
them?
– The master will wait forever
– This should be handled as an Alternate Flow in the Use Case
• What happens if some of the data in the matrix segment is lost or
modified?
– At best the result will be inaccurate
– At worst the slave may become compromised or crash, resulting in the master
waiting forever for the slave to respond
– This should be handled as subflows – the master should hash the matrix segment
and send the hash along with the segment
• The hash should also be encrypted or signed if the environment is hostile
• What if someone injects a slave response (affirmative or negative)?
– Perhaps random network chatter
– Perhaps a malefactor
– This should be handled as a subflow (if it is a concern) – the master and slave
should use encrypted channels or traffic
• All environments outside the software (the network and even the
system’s bus) should be assumed buggy/malicious in most cases

19. UML Data Flow Example
• Data flowing between the two packages (VideoLibrarySystem and Users)
should be protected
– This is easiest by providing ‘chokepoints’ – normally making a single class on each
side (frontend, backend, middleware, etc) that handles interprocess/internetwork
communication and protecting all methods in that class
• Classification should actually be placed in each package
– This not only serves to decouple the interface and backend, but also helps to serve
as another sanity check and reduce the likelihood of a fault in one package
effecting the other package

20. Finding Data Flow Flaws With Models
• UML cannot be easily used to find many of the more
complex data flow flaws
– Alloy, ORC, and SMV, as well as similar modeling languages
can find many of the more complex data flow flaws
• Examples include verifying that algorithms effectively protect
against or detect data tampering, spoofing, denial of service (to
the extent possible), and information disclosure
• More than anything, modeling can help to verify the
adequacy of controls, but (as a consequence) also
detect when the data flow could potentially expose the
system to threats

21. Alloy Data Flow Example
/** Program to verify enforcement of the x86 protection rings **/
abstract sig Ring{}
one sigRingZero, RingOne, RingTwo, RingThreeextends Ring{}
one sig Policy{order:Ring->Ring}
{order=(RingThree->RingTwo)+(RingTwo->RingOne)+(RingOne->RingZero)+(RingZero->Ring)+(RingOne->Ring)+
(RingTwo->RingTwo)+(RingTwo->RingThree)+(RingThree->RingThree)}
sig Program{onRing: one Ring}
sig State{programs: setProgram,connections: programs->programs}
{ -- Note that we are allowing programs to connect to themselves (this is common)
noprog: Program | prognot in programs -- Ensures all Programs are in our state
no progA, progB: Program | progA->progBin connections and progA.onRing->progB.onRingnot in Policy.order-- Enforce Policy.order
}
assert verifyDataFlow{all state: State | not getAnInvalidDataFlow[state]}
predgetAnInvalidDataFlow[state: State]{ -- Attempts to find an invalid data flow (one that violates Policy.order
some progA, progB: Program | progA->progBinstate.connectionsand progA.onRing->progB.onRingnot in Policy.order
}
• Models can be used to verify control of data flow
• They can also show that algorithms to protect or contain
damage are accurate
• Boolean logic models can even check fault tolerance, like
examining mechanisms that automatically adjust data
flow to route around faulty paths
– This ensures that the algorithm works

22. ACCESS CONTROL

23. Access Control
• Access control flaws occur when access to a
resource is improperly granted or denied
– Can exist under numerous circumstances:
• Race conditions, improperly assigning access control
restrictions, improperly protecting resources, shared
resources, etc.
• A tremendous amount of work has been done
concerning access control violations and
modeling
– The works tend to pick on specific access control
violations, like time-to-check-time-to-use and
breaking Bell-La Padula model

24. Finding Access Control Flaws With
UML
• Many access control flaws can be exposed by UML
diagrams
– Sequence diagrams can expose race conditions, some
instances of starvation and deadlock, and time-to-check-
time-to-use
– Use cases and design diagrams can expose some instances
of improperly protected resources and instances where
shared resources can lead to access control flaws
• Note that UML will expose these flaws in the design
and may not expose flaws in algorithms or internal
working mechanisms – for that models must be used

25. Use Case Access Control Example
• In this example of a basic medical record system, there are four actors –
the Doctor, Nurse, Records Office, and Front Desk
– The Front Desk can create a record (person) and delete a record
– The Records Office can only view records
– Doctors can view and modify records
– Nurses can view and modify records
• There are multiple potential access control flaws
– Deletion is often as good as modification
– Modification can include deletion (erase all of the data in the record is the same as
deleting the record and creating a new one)
• In fact, the two can replace most of each other’s functionality and capability, with regard to the
destructive force/results

26. UML Race Condition Example
• In this example, a parallel Branch and Bound Knapsack solver written in
Java spawns multiple threads that then attempt to solve the knapsack
problem
– Each of the thread (ParallelKnapsackWorker) works for a period of time, say 5
seconds and then the thread notifies ParallelBranchAndBoundthe best knapsack
found thusfar
• If the thread has finished, it posts a final notification and is joined
– The threads sleep immediately after sending notification so that
ParallelBranchAndBound can then grab the current data from the thread (to
prevent a race condition where the data is changing as the parent retrieves it)
– The parent then calls notify() to wake the thread
• This can, itself, create race conditions, deadlock, and starvation because Java doesn’t allow the
program to specify which thread to wake

27. UML Race Condition Example (cont)
• Note that we now have two threads notifying
ParallelBranchAndBound at the same time
– This means that one of the notifies is either going to go unheard
(image on the left) or…
– …We have a very serious race condition where what we were trying to
protect from (ParallelBranchAndBoundgrabbing data while it is being
updated) occurs and worse – we have the complete loss of data as one
thread wakes another instead of ParallelBranchAndBound, who then
performs another notify() – potentially accidentally waking another thread
and causing a complete loss of control

28. UML Access Control Example
• Without any OCL to prevent it, a Student can rent a scientific video
– something that the requirements clearly stated should not be
possible
– If implemented as the diagram dictates, the system does not meet the
required access control capabilities… OCL should be included

29. UML Access Control Violations
• Carefully examine privileges
– Fields should always be private unless they are marked ‘final’
– Methods should be private unless there is good reason
• In most cases only constructors, getters, and setters should be anything other
than private
– Getters and setters should be limited and only available when required
• Carefully examine sequence diagrams for potential starvation,
deadlock, and race conditions
• Examine OCL to ensure that it prevents instances where
access to certain resources is inadvertently granted by the
diagram were the OCL incorrect or missing
– Common examples include bypassing the login to gain access to
protected data and using input validation flaws to gain access

30. Finding Access Control Flaws With
Models
• UML can find many of the access control flaws, but
may also create some
– As in the example where the lack of OCL allowed a
violation of the requirements and a breech
• Modeling, however, can help to discover complex or
covert access control flaws
– Models trivially discover covert channels and side channels
– Models also can be used to define the rules that will
enforce intended controls
• They can also prove that intended controls fail to achieve the
desired effect
– Models have even been used to discover previously
unknown access control flaws

31. Prolog Access Control Example
/** Excerpt from [14] -- a model to ensure database subtransactions are atomic, do not conflict, and can be rolledback/recovered from **/
haveto(Node1, Action1, Node2, Action2) :-
catch(not(maynot(Node1, Action1, Node2, Action2)), _, true).
cdefs(a, A, B) :-
cdefs(t, A, B),
haveto(A, abort, B, abort).
cdefs(b, A, B) :-
checkorder(A, begin, B, _).
cdefs(ba, A, B) :-
cannot(A, commit, B, _),
checkorder(A, abort, B, _).
cdefs(bc, A, B) :-
cannot(A, abort, B, _),
checkorder(A, commit, B, _).
cdefs(c, A, B) :-
checkorder(A, commit, B, commit).
cdefs(ex, A, B) :-
haveto(A, commit, B, abort).
• The full model identified several new subtransaction conflicts as
well as the previously unknown state of ‘ignored dependencies’
that were easy to discover using a logic model
– Conflicts include instances where race conditions are present, the
database cannot rollback/recover if something fails, access control
mechanisms are violated, or two subtransactions are attempting to
perform conflicting operations like committing and strongly committing
the same link
• Using Prolog, similar flaws are trivial to detect and correct
– Alloy can also be used to find certain access control flaws – Prolog is
generally better, though

32. ENCRYPTION

33. Encryption
• Cryptographic flaws exist in all known crypto schemes
and their implementations
– Some flaws are far more egregious than others
– Flaws includepoor randomness, poor entropy collection,
poor keys, weak S-boxes, key exchange flaws, key sizes,
and reversibility of the algorithm
• Evil oracles have always been the Achilles’ heel and allowed the
reversing of the entire MD5 family (MD5, SHA, etc)
• Modeling and diagramming have uncovered previously
undiscovered flaws in common software
– Web-browsers have been shown to be trivial to hijack
sessions from because of the way they handle SSL

34. Finding Encryption Flaws With UML
• UML can be used to find some common cryptographic
flaws
– Most of the flaws are found in state and sequence
diagrams
– Encryption flaws found in state diagrams are usually just
for that ‘oracle’
• If a state diagram shows a flaw in OpenSSL’s handling of SSL, a
similar flaw may or may not exist in other implementations of SSL
– Flaws discovered in sequence diagrams are likely flaws in
the actual algorithm
• These flaws, of course, exclude mathematical errors,
other than to point out that the particular
implementation relies on a library that provides weak
entropy or randomness (e.g. rand)

35. UML Encryption Flaw Example (cont)
• Note the Trusted Third Party
– What if this trust were undue?
• The entire algorithm could be broken
• SSH suffers this problem as well
• What if the attacker spoofed the client,
rather than the server?
– The client never posts a certificate, so can
be trivially spoofed
• SSHv1 suffered this flaw; SSHv2 and
TLSv1/TLSv1.1 normally ‘authenticate’ the
client (weakly)
• What if the attacker downgraded the
cipher spec?
– This would allow the attacker to more
easily retrieve the session key
• This is common and normally done by
spoofing the client
• In many cases the client or server
downgrades as a matter of course
– Compatibility being the major reason
– The result is a much weaker algorithm

36. Finding Encryption Flaws With Models
• UML can only find encryption and cryptographic flaws
that are ‘surface level’
– Flaws in S-boxes, the underlying mathematical
assumptions, etc, are invisible in any UML diagram
• Modeling, however, can be used to discover many of
the flaws that UML and OCL are unable to find
– Models easily evaluate truth statements, such as those
about trust and who knows what when
• As well as who trusts whom and when
– Models can also help determine why levels of trust are
what they are
– Most importantly, models can verify the math behind the
algorithm

37. Prolog Encryption Flaw Example
/** Excerpt from a model to ensure the proper sharing of cryptographic keys**/
trust(Node1, Level1, Node2, Level2) :-
catch(not(violatesTrust(Node1,Level1, Node2, Level22)), _, true).
keyExchange(Key, A, B) :-
trust(A,_,B,high).
keyExchange(Key, A, B, C):-
trust(A,high,C,high),
trust(B,high,C,high).
sessionKeyExchange(Key,A,B):-
trust(A,high,B,high),
keyExchange(Key,A,B).
• The full model explains how most systems perform key exchange
– Or more correctly how most systems should perform the key exchange
• Models like this can help prevent a system from performing a poor
key exchange or one that leaves a system vulnerable to hijacking
before, during, or after the key exchange
• Using Prolog, logic statements like those governing trust and
security decisions can prevent the mistakes of SSHv1 and SSL
– The mathematics could be proven in Prolog, but there are much better
tools for that…

38. Coq Encryption Flaw Example
/** Excerpt from a model to prove mutual exclusion of a cryptographic algorithm**/
Lemma correct:forallmemo:Memo,Cycle memo->Oracle->CorrectTrace.
refine
(cofix choice : forallmemo:Memo,Cycle memo->Oracle->CorrectTrace :=
fun (memo:Memo)(cycle:Cycle memo)(oracle:Oracle)=>
SCons _ _ memo _ (choice (proc (hd oracle) memo) _ (tl oracle))).
• Numerous languages exist to help prove the correctness of cryptographic
algorithms
– Many of them rely on Prolog to provide the knowledge base and logic
• Coq and PVS are commonly used to prove the underlying mathematical
theorems
– In the example above, Coq is proving the lemma that illustrates a key property of
mutual exclusivity
– Failing this lemma means that the algorithm fails to provide mutual exclusion
• By using a mixture of Prolog-based languages and theorem solvers, most
common encryption flaws can be eliminated or reduced
– Alloy generally fails to provide the capabilities and is much more difficult in this
realm than Prolog-based solutions

39. IN EXIGO

40. Conclusion
• Formal Specification Techniques can be used to detect design flaws
– In some cases, after-the-fact models and diagrams have detected
flaws in implementations
• These are more costly and difficult to remedy than had they been caught in
the design phase
• In most instances, the flaws can be caught through both modeling
and to some degree with diagrams
– Many of the flaws can be seen through sequence diagrams and use
cases
– Other flaws can be caught with class diagrams
• Prolog is a powerful language that enables most modeling
languages to discover some of the more complex flaws
– This is particularly true with flaws that require changing states or
heavy logic and a powerful knowledge base

41. Acknowledgements
• This work relied heavily on previous works of
mine for examples, to include the modeling
examples provided

42. References
[1] Alan Shaffer, Mikhail Auguston, Cynthia Irvine, and Timothy Levin, “A Security Domain Model to Assess Software for Exploitable Covert
Channels,” Proceedings of the ACM SIGPLAN Third Workshop on Programming Languages and Analysis for Security, ACM Press, 2008.
[2] Alan Shaffer, Mikhail Auguston, Cynthia Irvine, and Timothy Levin, “A Security Domain Model for Implementing Trusted Subject
Behaviors,” Unpublished.
*3+ Alan Shaffer, “A Security Domain Model for Static Analysis and Verification of Software Programs,” Proceedings of the 20th International
Conference on Software Engineering and Knowledge Engineering, Redwood City, California, 2008, pp. 673-678.
[4] Alan Shaffer, Mikhail Auguston, Cynthia Irvine, and Timothy Levin, “Toward a Security Domain Model for Static Analysis and Verification of
Information Systems,” Proceedings of the 7th OOPSLA Workshop on Domain-Specific Modeling, 2007.
*5+ Aleph1, “Smashing The Stack For Fun And Profit,” Phrack, 7(49), 1996.
[6] AliakseiTsitovich, “Detection of Security Vulnerabilities Using Guided Model Checking,” Lecture Notes in Computer Science, Springer-
Verlag, Berlin, 2008, pp. 822-823.
[7] Andrei Lapets, “Formal representation and reasoning approaches in modeling cryptographic protocols”, Unpublished.
*8+ Catherine Meadows, “Formal Verification of Cryptographic Protocols: A Survey,” Lecture Notes in Computer Science, Springer, 1995, pp.
135-150.
*9+ Chad R Dougherty, “Vulnerability Note VU#395412: Apache mod_rewrite contains off-by-one error in ldap scheme handling,” US-CERT, US
Department of Homeland Security.
[10] C.R. Ramakrishnan and R. Sekar, “Model-based analysis of configuration vulnerabilities,” In Journal of Computer Science, IOS Press, 10(1-
2), 2002, pp. 189-209.
[11] Daniel Jackson, Software Abstractions: Logic, Language, and Analysis. The MIT Press, Cambridge, Mass, 2006.
[12] Daniel Owens. Application and Website Security - Fundamental Edition, National Aeronautics and Space Administration, 2009.
[13] Daniel Owens. Application and Website Security – Developer Edition, National Aeronautics and Space Administration, 2010.
[14] Daniel Owens. Automated analysis of dependencies in advanced transaction models, SystemSecurities, 2006.
[15] Daniel Owens. Certification Test & Evaluation Report for the Common Link Integration Processing (CLIP) Release 1.2.1.26, Certification
Test and Evaluation Report, Booz Allen Hamilton, 2008.
[16] Daniel Owens. Common Link Integration Processing (CLIP) Integrity, Whitepaper, Booz Allen Hamilton, 2008.
[17] Daniel Owens. Integrating Software Security Into The Software Development Lifecycle, SystemSecurities, 2008.
[18] Daniel Owens. IT Security Threats: Improper or Inadequate Change Management, National Aeronautics and Space Administration, 2009.

43. References (cont)
[19] Daniel Owens. Fault Tolerance In Sensitive Embedded and Real-Time Information Technology Systems, SystemSecurities, 2009.
[20] Jim Arlow and IlaNeustadt, UML 2 and the Unified Process: Practical Object-Oriented Analysis and Design, 2nded, Addison-Wesley Professional, 2005.
[21] Junfeng Yang, Paul Twohey, Dawson Engler, Madanlal, Musuvathi. “Using model checking to find serious file system errors,” USENIX Symposium on
Operating Systems Design and Implementation, 2004.
[22] Michael Howard and D. LeBlanc, Writing Secure Code. 2nded, Microsoft Press, 2002.
[23] Mikhail Auguston, Alan Shaffer, “Security Domain Model and Implementation Modeling Language Reference Manual, version 2.0,” Unpublished, May
2008.
[24] MusabAlTurki and Jose Meseguer, “Reduction Semantics and Formal Analysis of Orc Programs,” Electronic Notes in Theoretical Computer Science, 200
(3), 2008, pp. 25-41.
[25] O.H. Alhazmi, Y.K. Malaiya, I. Ray, “Measuring, analyzing and predicting security vulnerabilities in software systems,” Computes & Security, 26 (3), 2007,
pp. 219-228.
[26] R. Ritchey and P. Ammann, “Using model checking to analyze network vulnerabilities,” In Proceedings of the IEEE Symposium on Security and Privacy,
2001, pp. 156-165.
[27] Shuo Chen, Jose Meseguer, Ralf Sasse, Helen J. Wang, Yi-Min Wang, “A Systematic Approach to Uncover Security Flaws in GUI Logic,” in IEEE
Symposium on Security and Privacy, Oakland, California, 2007.
[28] Shuo Chen, ZbigniewKalbarczyk, Jun Xu, Ravinshankar K. Iyer, “A Data-Driven Finite State Machine Model for Analyzing Security Vulnerabilities,” IEEE
International Conference on Dependable Systems and Networks, 2003, pp. 605-614.
[29] Shuo Chen, Ziqing Mao, Yi-Min Wang, Ming Zhang, “Pretty-Bad-Proxy: An Overlooked Adversary in Browsers’ HTTPS Deployments,” 2009 30th IEEE
Symposium on Security and Privacy, 2009, pp. 347-359.
[30] SMV. SMV: A Symbolic Model Checker. http://www.cs.cmu.edu/~modelcheck/.
[31] Stuart McClure, Joel Scambray, and George Kurtz, Hacking Exposed: Network Security Secrets and Solutions. 6thed, McGraw-Hill Osborne Media, 2009.
[32] Watchfire, “Dangling Pointer: Smashing the Pointer for Fun and Profit,” Blackhat Convention, 2007.
*33+ Y. Shin and L. Williams, “An Empirical Model to Predict Security Vulnerabilities using Code Complexity Metrics,” Proceedings of the 2nd International
Symposium on Emperical Software Engineering and Measurement, Association for Computing Machinery, New York, 2008, pp. 315-317.