Code coverage in theory and in practice form the do178 b perspective
•Télécharger en tant que PPT, PDF•
1 j'aime•1,033 vues
A lecture from Wind River Safety & Security Seminar that was held as part of the International Aerospace Conference, Jerusalem 2011
Recommandé
Contenu connexe
Tendances
Tendances (19)
En vedette
En vedette (20)
Similaire à Code coverage in theory and in practice form the do178 b perspective
Similaire à Code coverage in theory and in practice form the do178 b perspective (20)
Dernier
Dernier (20)
Code coverage in theory and in practice form the do178 b perspective
- 1. Code Coverage in Theory and in practice form the DO178B perspective Daniel Liezrowice – Engineering Software Lab (The Israeli Center for Static Code Analysis ) Presentation is Supported by Parasoft Makers of Automated DO178B Certified Analysis Tools & A Wind River Certified partner
- 2. Testing of Code Coverage is essential part of Safety-Critical Software Certification
- 19. Code Coverage Tools Visualization of code not covered in the code editor Visualization of coverage rate achieved Warnings generated by the tool to point to code not covered Coverage rate achieved broken down by packages and classes
- 29. Code Coverage: Brief Summary Method coverage Statement coverage Branch coverage Condition coverage Path coverage
- 30. Condition/Decision Coverage Condition/Decision Coverage is a hybrid metric composed by the union of condition coverage and decision coverage .
- 31. Modified Condition/Decision Coverage DO178B Level A Every point of entry and exit in the program has been invoked at least once, every condition in a decision has taken all possible outcomes at least once, every decision in the program has taken all possible outcomes at least once, and each condition in a decision has been shown to independently affect that decisions outcome. A condition is shown to independently affect a decisions outcome by varying just that condition while holding fixed all other possible conditions This metric is specified for safety critical aviation software by RCTA/DO-178B
- 32. How It is Done? - Instrumentation 1 void foo() 2 { 3 found= false ; 4 for (i=0;(i<100) && ( ! found );i++) 5 { 6 if (i==50) break ; 7 if (i==20) found= true ; 8 if (i==30) found= true ; 9 } 10 printf("foo"); 11 }
- 33. How It is Done? - Automatic Instrumentation. Instrumentation for statement coverage 1 char inst[5]; 2 void foo() 3 { 4 found= false ; 5 for (i=0;(i<100) && (! found);i++) 6 { 7 if (i==50 ) { inst[0]=1 ; break ;} 8 if (i==20 ) { inst[1]=1 ;found= true ;} 9 if (i==30 ) { inst[2]=1 ;found= true ;} 10 inst[3]=1 ; } 11 printf("foo"); 12 inst[4]=1 ; }
- 34. How It is Done? - Instrumentation Inserting the full instrumentation code for the condition coverage in this example will produce the following code: 1 char inst[15] ; 2 void foo() 3 { 4 found= false ; 5 for (i=0;((i<100)? inst[0]=1 :inst[1]=1,0) && ((! found)? inst[2]=1 :inst[3]=1,0);i++) 6 { 7 if ((i==50? inst[4]=1 : inst[5]=1 ,0) ) { inst[6]=1 ; break ;} 8 if ((i==20? inst[7]=1 : inst[8]=1 ,0) ) { inst[9]=1 ;found= true ;} 9 if ((i==30? inst[10]=1 : inst[11]=1 ,0) ) { inst[12]=1 ;found= true ;} 10 inst[13]=1 ; } 11 printf("foo"); 12 inst[14]=1 ; } Full code coverage instrumentation at condition level
- 37. Thank You Daniel Liezrowice – Engineering Software Lab (The Israeli Center for Static Code Analysis & Dynamic testing) [email_address] 09-8855803 www.eswlab.com
Notes de l'éditeur
- Statement Coverage only requires one test case (condition evaluates to true) to satisfy this code Decision/Condition Coverage requires two test cases (condition evaluates to true and false) to satisfy this code
- Decision/Condition Coverage only requires two test cases, one to execute the if branch, one to execute the else branch MCDC requires 4 test cases Must make “A OR B” both true and false, while simultaneously making the entire statement true and false.
- White box testing is a structural technique, which is based on the structure of the test item. This structure is evaluated and measurements are taken from it. These assess characteristics such as The control flow of the item under test Data flows into, out of and within the item under test Which part of the source that have been covered by a particular set of tests In order to take these measurements, it is necessary to enhance the code with additional statements that record key events during execution. This is called Instrumentation As instrumentation may influence the behaviour of the test object, all tests should be repeated without coverage measurement, to exclude the possibility of side effects.
- Code coverage as acceptance criteria: Statement about expected coverage in relation to the maximal possible coverage in terms of a certain coverage measurement. E.g. 90 % statement coverage means: 90% of all statements in the source code are covered by the tests executed. Empirical studies have shown that a incredibly difficult to achieve coverage rate must be reached to improve error detection noticeably (90 – 95 % or higher). This implies that punctually using coverage measurement to visualize areas of the code that are not covered with tests is superior to acceptance criteria using coverage measurement. Code Coverage as acceptance criteria may motivate developers to concentrate on developing test code to increase coverage (e.g. testing getter and setters) instead of to find errors in the code. It makes sense to concentrate on critical and complex parts of the code, where most bugs are expected to be. But those parts are often also hard to test and code coverage is increased only to a negligible degree by these tests.
- A good code coverage measurement tool is not only capable of measuring code coverage and check whether some acceptance criteria in terms of code coverage are met. A good code coverage tool should also provide means to visualize achieved coverage in order to help developers identifying source code in need of additional tests. A good example is the reasonable priced Clover, a coverage measurement and visualization tool for Java available from www.cenqua.com. It integrates nicely with the Eclipse IDE and will be used in the labs for this module.
- Method coverage doesn’t really help a lot. There is a need for more elaborate code coverage models.
- While statement coverage certainly is a weak coverage measurement, it can make sense to use it, if the code itself is not complex and does not contain many branches.
- Condition coverage reports the true or false outcome of each boolean sub-expression, separated by logical-and and logical-or if they occur. Condition coverage measures the sub-expressions independently of each other. Multiple condition coverage reports whether every possible combination of boolean sub-expressions occurs. Condition/Decision Coverage is a hybrid measure composed by the union of condition coverage and decision coverage.
- following red path, warning seems justified but red path is infeasible given error is from Flexelint; Orion reports no errors
- Method coverage focuses on called methods. Statement coverage focuses on called statements. Branch coverage focuses on decisions based on results of boolean expressions (true, false). Condition coverage focuses on decisions based each condition of an boolean expression. Path coverage focuses on each possible unique path through a method. Test coverage focuses on finding and elimination errors in the code.