This is the presentation I gave at the International Symposium on Software Testing and Analysis in 2010

1. Directed Test Generation for Effective Fault Localization Shay Artzi, Julian Dolby , Frank Tip, Marco Pistoia IBM T.J. Watson Research Center ISSTA – July 14, 2010 – Trento, Italy

6. Enhanced Fault Localization

7. Directed Test Generation for Fault Localization

8. Evaluation

9. Related Work

10. Conclusion

11. Running PHP Example 1 if (isset($_REQUEST['query']) ){ 2 3 switch ($_REQUEST['query']) { 4 case "model": 5 $kf="model_id"; break; 6 case "make": 7 $kf="make_id"; break; 8 } 9 $v =_REQUEST['key']; 10 11 $sql="SELECT * FROM VEHICLES WHERE " . $kf . " = '” . $v . “'”; 12 $result=mysql_query($sql); 13 $n = mysql_numrows($result); 14 15 } else if (isset($_REQUEST['update']) ) { … }

12. Running Example – Success 1 if (isset($_REQUEST['query']) ){ 2 3 switch ($_REQUEST['query']) { 4 case "model": 5 $kf="model_id"; break; 6 case "make": 7 $kf="make_id"; break; 8 } 9 $v =_REQUEST['key']; 10 11 $sql="SELECT * FROM VEHICLES WHERE " . $kf . " = '” . $v . “'”; 12 $result=mysql_query($sql); 13 $n = mysql_numrows($result); 14 15 } else if (isset($_REQUEST['update']) ) { … } query -> 'make' key -> 'Ford' 'query' is set in request $kf set to 'make_id' SELECT * FROM VEHICLES WHERE make_id = 'Ford'; returns 2 result rows $n is assigned 2

13. Running Example – Failure 1 if (isset($_REQUEST['query']) ){ 2 3 switch ($_REQUEST['query']) { 4 case "model": 5 $kf="model_id"; break; 6 case "make": 7 $kf="make_id"; break; 8 } 9 $v =_REQUEST['key']; 10 11 $sql="SELECT * FROM VEHICLES WHERE " . $kf . " = '” . $v . “'”; 12 $result=mysql_query($sql); 13 $n = mysql_numrows($result); 14 15 } else if (isset($_REQUEST['update']) ) { … } 'query' is set in request switch falls through $kf is never assigned SELECT * FROM VEHICLES WHERE = 'Ford'; returns null due to invalid sql mysql_numrows(): supplied argument is not a valid MySQL result resource query -> 'company' key -> 'Ford'

17. But cannot isolate fault precisely statement 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 passing 1 1 1 1 1 1 1 1 failing 1 1 1 1 1 1 suspiciousness .7 .7 0 0 .7 .7 .7 .7

19. Not captured by statistics over statements

21. Refine blame to kind of result

23. Refine blame to specific control flow outcome

24. if statement s become (s, true) or (s, false)

25. switch statement s becomes (s, case number taken )

27. Return of null result on sql error not checked statement 1 (3,2) (3,0) 6 7 9 11 (12,obj) (12,null) 13 passing 1 1 1 1 1 1 1 1 failing 1 1 1 1 1 1 suspiciousness .7 0 1 0 0 .7 .7 0 1 .7

30. Direct choices to aid fault localization

34. Negate a tail:

36. Note second path will yield successful run

37. Example Generated Execution 1 1 if (isset($_REQUEST['query']) ){ 2 3 switch ($_REQUEST['query']) { 4 case "model": 5 $kf="model_id"; break; 6 case "make": 7 $kf="make_id"; break; 8 } 9 $v =_REQUEST['key']; 10 11 $sql="SELECT * FROM VEHICLES WHERE " . $kf . " = '” . $v . “'”; 12 $result=mysql_query($sql); 13 $n = mysql_numrows($result); 14 15 } else if (isset($_REQUEST['update']) ) { … }

38. Example Generated Execution 2 1 if (isset($_REQUEST['query']) ){ 2 3 switch ($_REQUEST['query']) { 4 case "model": 5 $kf="model_id"; break; 6 case "make": 7 $kf="make_id"; break; 8 } 9 $v =_REQUEST['key']; 10 11 $sql="SELECT * FROM VEHICLES WHERE " . $kf . " = '” . $v . “'”; 12 $result=mysql_query($sql); 13 $n = mysql_numrows($result); 14 15 } else if (isset($_REQUEST['update']) ) { … }

39. Example Generated Execution 3 1 if (isset($_REQUEST['query']) ){ 2 3 switch ($_REQUEST['query']) { 4 case "model": 5 $kf="model_id"; break; 6 case "make": 7 $kf="make_id"; break; 8 } 9 $v =_REQUEST['key']; 10 11 $sql="SELECT * FROM VEHICLES WHERE " . $kf . " = '” . $v . “'”; 12 $result=mysql_query($sql); 13 $n = mysql_numrows($result); 14 15 } else if (isset($_REQUEST['update']) ) { … }

45. Localize each fault

48. PCS : Directed test generation using path-constraint similarity

50. measure % of well-localized faults

55. Reduce time and number of tests by > 85%

56. Enhancements needed to fault localization