Software Visualization - Promises & Perils

1. Software VisualizationPromises & Perils Michele Lanza REVEAL @ Faculty of Informatics University of Lugano, Switzerland

2. The use of the crafts of typography, graphic design, animation, and cinematography with modern human- computer interaction and computergraphics technology to facilitate both thehuman understanding and effective use of computer software. John Stasko et al. Software Visualization Programming as a Multimedia Experience MIT University Press, 1998

3. algorithm visualizationsoftware visualization program visualization

4. the visualization of t h e h i g h e r- l e v e l algorithm visualization abstractions which describe softwaresoftware visualization program visualization

5. the visualization of t h e h i g h e r- l e v e l algorithm visualization abstractions which describe softwaresoftware visualization the visualization of the actual program program visualization code or data structures in static or dynamic form

6. program visualization

7. static code/data visualizationprogram visualization dynamic code/data visualization

8. what issoftware? Soft...

9. Software: Theintangible objects inside the computer. where?

10. Programming is a kind of writing. Gerald M. Weinberg Psychology of Computer Programming Dorset House, 1971

11. programming ~ writing

12. programming ~ writingsoftware ~ text

13. Deciduous shrub, glabrous or nearly so, withweak, trailing sub-glaucous, often purple-tinted stems, either decumbent and forminglow bushes 50 - 100 cm high, or climbingover other shrubs, rarely more erectand reaching 2 m. Prickles hooked,all +- equal. L’ﬂets 2-3 pairs, 1-3.5cm, ovate or ovate-elliptic, simply,rarely double serrate, glabrous onboth sides or pubescent on theveins (rarely all over) beneath,rather thin; petiole usuallywith some stalked glands;stipules narrow, auriclesstraight. Flowers 1-6, white3-5 cm diam.; pedicels 2-4 cmwith stalked glands, rarelysmooth; buds short... etc. etc.

14. Deciduous shrub, glabrous or nearly so, withweak, trailing sub-glaucous, often purple-tinted stems, either decumbent and forminglow bushes 50 - 100 cm high, or climbingover other shrubs, rarely more erectand reaching 2 m. Prickles hooked,all +- equal. L’ﬂets 2-3 pairs, 1-3.5cm, ovate or ovate-elliptic, simply,rarely double serrate, glabrous onboth sides or pubescent on theveins (rarely all over) beneath,rather thin; petiole usuallywith some stalked glands;stipules narrow, auriclesstraight. Flowers 1-6, white3-5 cm diam.; pedicels 2-4 cmwith stalked glands, rarelysmooth; buds short... etc. etc.

15. It would be a great mistake to suppose that the dominant role of visualobservation has now been overthrown by more ‘analytical’ methods. John Ziman Reliable Knowledge An Exploration of the Grounds for Belief in Science Cambridge University Press, 1978

16. old habits die hard

17. /***************************************************************************/ if(p<3&y==E)H=y^16; /* shift capt.sqr. H if e.p.*//* micro-Max, */ t=b[H];if(t&k|p<3&!(r&7)!=!t)break; /* capt. own, bad pawn mode *//* A chess program smaller than 2KB (of non-blank source), by H.G. Muller */ i=99*w[t&7]; /* value of capt. piece t *//***************************************************************************/ if(i<0||E-S&&b[E]&&y-E<2&E-y<2)m=I; /* K capt. or bad castling *//* version 3.2 (2000 characters) features: */ if(m>=l)goto C; /* abort on fail high *//* - recursive negamax search */ if(h=d-(y!=z)) /* remaining depth(-recapt.)*//* - quiescence search with recaptures */ {v=p<6?b[x+8]-b[y+8]:0; /* center positional pts. *//* - recapture extensions */ b[G]=b[H]=b[x]=0;b[y]=u&31; /* do move, strip virgin-bit*//* - (internal) iterative deepening */ if(!(G&M)){b[F]=k+6;v+=30;} /* castling: put R & score *//* - best-move-first sorting */ if(p<3) /* pawns: *//* - a hash table storing score and best move */ {v-=9*(((x-2)&M||b[x-2]!=u)+ /* structure, undefended *//* - full FIDE rules (expt minor ptomotion) and move-legality checking */ ((x+2)&M||b[x+2]!=u)-1); /* squares plus bias */ if(y+r+1&S){b[y]|=7;i+=C;} /* promote p to Q, add score*/#define F(I,S,N) for(I=S;I<N;I++) }#define W(A) while(A) v=-D(24-k,-l-(l>e),m>q?-m:-q,-e-v-i, /* recursive eval. of reply */#define K(A,B) *(int*)(T+A+(B&8)+S*(B&7)) J+J(0),Z+J(8)+G-S,F,y,h); /* J,Z: hash keys */#define J(A) K(y+A,b[y])-K(x+A,u)-K(H+A,t) v-=v>e; /* delayed-gain penalty */ if(z==9) /* called as move-legality */#define U 16777224 {if(v!=-I&x==K&y==L) /* checker: if move found */struct _ {int K,V;char X,Y,D;} A[U]; /* hash table, 16M+8 entries*/ {Q=-e-i;O=F;return l;} /* & not in check, signal */ v=m; /* (prevent fail-lows on */int V=112,M=136,S=128,I=8e4,C=799,Q,N,i; /* V=0x70=rank mask, M=0x88 */ } /* K-capt. replies) */ b[G]=k+38;b[F]=b[y]=0;b[x]=u;b[H]=t; /* undo move,G can be dummy */char O,K,L, if(Y&8){m=v;Y&=~8;goto A;} /* best=1st done,redo normal*/w[]={0,1,1,3,-1,3,5,9}, /* relative piece values */ if(v>m){m=v;X=x;Y=y|S&G;} /* update max, mark with S */o[]={-16,-15,-17,0,1,16,0,1,16,15,17,0,14,18,31,33,0, /* step-vector lists */ } /* if non castling */ 7,-1,11,6,8,3,6, /* 1st dir. in o[] per piece*/ t+=p<5; /* fake capt. for nonsliding*/ 6,3,5,7,4,5,3,6}, /* initial piece setup */ if(p<3&6*k+(y&V)==S /* pawn on 3rd/6th, or */b[129], /* board: half of 16x8+dummy*/ ||(u&~24)==36&j==7&& /* virgin K moving sideways,*/T[1035], /* hash translation table */ G&M&&b[G=(x|7)-(r>>1&7)]&32 /* 1st, virgin R in corner G*/ &&!(b[G^1]|b[G^2]) /* 2 empty sqrs. next to R */n[]=".?+nkbrq?*?NKBRQ"; /* piece symbols on printout*/ ){F=y;t--;} /* unfake capt., enable e.p.*/ }W(!t); /* if not capt. continue ray*/D(k,q,l,e,J,Z,E,z,n) /* recursive minimax search, k=moving side, n=depth*/ }}}W((x=x+9&~M)-B); /* next sqr. of board, wrap */int k,q,l,e,J,Z,E,z,n; /* (q,l)=window, e=current eval. score, E=e.p. sqr.*/ C:if(m>I/4|m<-I/4)d=99; /* mate is indep. of depth */{ /* e=score, z=prev.dest; J,Z=hashkeys; return score*/ m=m+I?m:-D(24-k,-I,I,0,J,Z,S,S,1)/2; /* best loses K: (stale)mate*/ int j,r,m,v,d,h,i=9,F,G; if(!a->K|(a->X&M)!=M|a->D<=d) /* if new/better type/depth:*/ char t,p,u,x,y,X,Y,H,B; {a->K=Z;a->V=m;a->D=d;A->K=0; /* store in hash,dummy stays*/ struct _*a=A; a->X=X|8*(m>q)|S*(m<l);a->Y=Y; /* empty, type (limit/exact)*/ /* lookup pos. in hash table*/ } /* encoded in X S,8 bits */ j=(k*E^J)&U-9; /* try 8 consec. locations */ /*if(z==8)printf("%2d ply, %9d searched, %6d by (%2x,%2x)n",d-1,N,m,X,Y&0x77);*/ W((h=A[++j].K)&&h-Z&&--i); /* first empty or match */ } a+=i?j:0; /* dummy A[0] if miss & full*/ if(z&8){K=X;L=Y&~M;} if(a->K) /* hit: pos. is in hash tab */ return m; {d=a->D;v=a->V;X=a->X; /* examine stored data */ } if(d>=n) /* if depth sufficient: */ {if(v>=l|X&S&&v<=q|X&8)return v; /* use if window compatible */ main() d=n-1; /* or use as iter. start */ { }X&=~M;Y=a->Y; /* with best-move hint */ int j,k=8,*p,c[9]; Y=d?Y:0; /* dont try best at d=0 */ }else d=X=Y=0; /* start iter., no best yet */ F(i,0,8) N++; /* node count (for timing) */ {b[i]=(b[i+V]=o[i+24]+40)+8;b[i+16]=18;b[i+96]=9; /* initial board setup*/ W(d++<n|z==8&N<1e7&d<98) /* iterative deepening loop */ F(j,0,8)b[16*j+i+8]=(i-4)*(i-4)+(j-3.5)*(j-3.5); /* center-pts table */ {x=B=X; /* start scan at prev. best */ } /*(in unused half b[])*/ Y|=8&Y>>4; /* request try noncastl. 1st*/ F(i,M,1035)T[i]=random()>>9; m=d>1?-I:e; /* unconsidered:static eval */ do{u=b[x]; /* scan board looking for */ W(1) /* play loop */ if(u&k) /* own piece (inefficient!)*/ {F(i,0,121)printf(" %c",i&8&&(i+=7)?10:n[b[i]&15]); /* print board */ {r=p=u&7; /* p = piece type (set r>0) */ p=c;W((*p++=getchar())>10); /* read input line */ j=o[p+16]; /* first step vector f.piece*/ N=0; W(r=p>2&r<0?-r:-o[++j]) /* loop over directions o[] */ if(*c-10){K=c[0]-16*c[1]+C;L=c[2]-16*c[3]+C;}else /* parse entered move */ {A: /* resume normal after best */ D(k,-I,I,Q,1,1,O,8,0); /* or think up one */ y=x;F=G=S; /* (x,y)=move, (F,G)=castl.R*/ F(i,0,U)A[i].K=0; /* clear hash table */ do{H=y+=r; /* y traverses ray */ if(D(k,-I,I,Q,1,1,O,9,2)==I)k^=24; /* check legality & do*/ if(Y&8)H=y=Y&~M; /* sneak in prev. best move */ } if(y&M)break; /* board edge hit */ }

18. free your mind

19. Software Visualization The use of the crafts of typography, graphic design, animation, and cinematography with modern human- computer interaction and computergraphics technology to facilitate both thehuman understanding and effective use of computer software. Lanza, 2010

20. (software)visualization myths

21. a picture is wortha thousand words

22. n ga picture is worth ro words wa thousand

23. UML is softwarevisualization

24. gUML is software n r o wvisualization

25. visual programming issoftware visualization

26. n gvisual programming is r o wsoftware visualization

27. static visualizationsoftware visualization dynamic visualization

28. ng ro ngw static visualization ro wsoftware visualization ng o dynamic wr visualization

29. about software ofsoftware software visualization

30. aboutsoftware software of software information visualization visualization

31. William Playfair 1759 - 1823

32. Jacques BertinSemiology of Graphics,1967

33. visualization ~ story

34. It was the twenty-eight of November. Animmense confused mass of men, horses,vehicles besieged the narrow entrances tothe bridges and began to flow over them.

35. It was the twenty-eight of November. Animmense confused mass of men, horses,vehicles besieged the narrow entrances tothe bridges and began to flow over them.Those in front, pushed by the weight of those behind werecrushed, trampled on, or forced into the ice-filled water of theBerezina.The confusion was so great that when Napoleon himself wishedto cross, it was necessary to use force to clear a passage.Some there were who, determined to pass at all costs, cut ahorrible way for themselves with their swords. Others openedan even crueler road for their carriages, driving them pitilesslythrough the helpless crowd, crushing men and women, in theirodious greed sacrificing their companions in misery.

36. seeing ~ understanding

37. We acquire more information through vision than throughall the other senses combined. Colin WareInformation Visualization Perception for Design Morgan Kaufmann, 2004

38. Sensation Perception (Physical Process) (Cognitive Process)Stimulus Sensory Organ Perceptual Organ Brain

39. The EyeA camera with a maximumresolution of 625 dpi in the fovealzone of the retina.

40. The EyeA camera with a maximumresolution of 625 dpi in the fovealzone of the retina.The retina is coated with rods (fordim light) and cones (for brightlight and colors).

41. The EyeA camera with a maximumresolution of 625 dpi in the fovealzone of the retina.The retina is coated with rods (fordim light) and cones (for brightlight and colors).Three types of cones, specializedon the blue, green, and red rangeof the color spectrum.

42. Breaking NewsWe do not see with the eyes

43. we see withthe brain

44. iconicmemory

45. short-termmemory

46. long-termmemory

47. Pre-attentiveprocessing

48. Category Form Color Spatial Motion Motion Orientation Hue 2D position Flicker Line length Intensity Direction Line width SizeAttribute Shape Curvature Added marks Enclosure

49. Orientation

50. Line Length

51. Line Width

52. 2D Position / Proximity

53. Enclosure

54. Shape / Similarity

55. Connectedness

56. Categorical vs Quantitative Encoding of Pre-attentive Attributes

57. Type Attribute Quantitative? Orientation No Line Length Yes Line Width Partially Size Partially Form Shape No Curvature No Added Marks No Enclosure No Hue No Color Intensity PartiallyPosition 2D Position Yes

58. about software ofsoftware software visualization

59. Stephen Few Edward TufteShow Me the Numbers The Visual Display of Quantitative InformationAnalytics Press, 2004 Morgan Kaufmann, 2004

60. about software ofsoftware visualization software

61. Computer graphics ✔

62. Computer graphics ✔Gestalt principles ✔

63. simple..but useless

64. Computer graphics ✔Gestalt principles ✔Semantics ✘

65. InterludeMetrics

66. Interlude IISoftwareAnalysis

67. WCRE 1999, TSE 2003

68. visualization ~ interaction

69. visualization ~ interactioninteraction ~ tool

70. visualization ~ interactioninteraction ~ tooltool ~ cool

71. CodeCrawler (1998 - 2005)

72. X-Ray Eclipse Plug-in xray.inf.usi.ch

73. the class blueprint

74. Initialize Interface Internal Accessor Attribute

75. Initialize Interface Internal Accessor Attribute

76. RegularInitialize Interface Internal Accessor Attribute Overriding Extending Abstract Constant Delegating Setter Getter OOPSLA 2001, TSE 2005

77. RegularOverridingExtending Abstract ConstantDelegating Setter Getter

80. OK! But, is this useful?

81. the unbearablelightness of beingvisual

82. Solution 1: Anecdotal Evidence

83. Solution 2: Pattern Language

84. Software Systems as Cities

85. City Metaphor VISSOFT 2007

86. City Metaphor class buildingpackage district VISSOFT 2007

87. City Metaphor class buildingpackage district VISSOFT 2007

88. City Metaphor class buildingpackage district nesting level color VISSOFT 2007

89. City Metaphor number of methods (NOM) height number of attributes (NOA) base size number of lines of code (LOC) color class buildingpackage district nesting level color VISSOFT 2007

90. Program Comprehension ArgoUML LOC 136,325 ICPC 2007

91. Program Comprehension

92. FacadeMDRImplNOA 3 skyscraperNOM 349LOC 3,413

93. CPPParserNOA 85 ofﬁce buildingNOM 204LOC 9,111

94. JavaTokenTypesNOA 173 parking lotNOM 0LOC 0

95. house PropPanelEvent NOA 2 NOM 3 LOC 37

96. Program Comprehension ICPC 2007

97. Design Quality Assessmentdisharmony mapArgoUML classes brain class 8 god class 30 god + brain 6 data class 17 unaffected 1,715 SoftVis 2008

98. System Evolution Analysistime traveling WCRE 2008

99. System Evolution Analysis time ArgoUMLtime traveling 8 major releases 6 years WCRE 2008

100. System Evolution Analysis time ArgoUMLtime traveling 8 major releases 6 years WCRE 2008

101. http://codecity.inf.usi.ch implemented in Smalltalk ICSE 2008 tool demo

102. Is ituseful ?

103. A Controlled Experiment

104. Design

105. technical report 2010State of the art?

106. Design desiderata 1 Avoid comparing using a technique against not using it. 2 Involve participants from the industry. 3 Provide a not-so-short tutorial of the experimental tool to the participants. 4 Avoid, whenever possible, giving the tutorial right before the experiment. 5 Use the tutorial to cover both the research behind the approach and the tool. 6 Find a set of relevant tasks. 7 Choose real object systems that are relevant for the tasks. 8 Include more than one object system in the design. 9 Provide the same data to all participants.10 Limit the amount of time allowed for solving each task.11 Provide all the details needed to make the experiment replicable.12 Report results on individual tasks.13 Include tasks on which the expected result is not always to the advantage of the tool being evaluated.14 Take into account the possible wide range of experience level of the participants.

107. Design desiderata 1 Avoid comparing using a technique against not using it. 2 Involve participants from the industry. 3 Provide a not-so-short tutorial of the experimental tool to the participants. 4 Avoid, whenever possible, giving the tutorial right before the experiment. 5 Use the tutorial to cover both the research behind the approach and the tool. 6 Find a set of relevant tasks. 7 Choose real object systems that are relevant for the tasks. 8 Include more than one object system in the design. 9 Provide the same data to all participants.10 Limit the amount of time allowed for solving each task.11 Provide all the details needed to make the experiment replicable.12 Report results on individual tasks.13 Include tasks on which the expected result is not always to the advantage of the tool being evaluated.14 Take into account the possible wide range of experience level of the participants.

108. Finding a baseline1. program comprehension2. design quality assessment3. system evolution analysis

111. Finding a baseline1. program comprehension2. design quality assessment

112. Tasks

113. Tasks program comprehension 6A1 Identity the convention used in the system to organize unit tests.A2.1& What is the spread of term T in the name of the classes, their attributes andA2.2 methods?A3 Evaluate the change impact of class C, in terms of intensity and dispersion.A4.1 Find the three classes with the highest number of methods. Find the three classes with the highest average number of lines of code perA4.2 method.

114. Tasks program comprehension 6A1 Identity the convention used in the system to organize unit tests.A2.1& What is the spread of term T in the name of the classes, their attributes andA2.2 methods?A3 Evaluate the change impact of class C, in terms of intensity and dispersion.A4.1 Find the three classes with the highest number of methods. Find the three classes with the highest average number of lines of code perA4.2 method.B1.1 Identify the package with the highest percentage of god classes.B1.2 Identify the god class with the largest number of methods. Identify the dominant (affecting the highest number of classes) class-levelB2.1 design problem.B2.2 Write an overview of the class-level design problems in the system. design quality assessment 4

115. Tasks program comprehension 6 5A1 Identity the convention used in the system to organize unit tests.A2.1& What is the spread of term T in the name of the classes, their attributes andA2.2 methods?A3 Evaluate the change impact of class C, in terms of intensity and dispersion.A4.1 Find the three classes with the highest number of methods. Find the three classes with the highest average number of lines of code perA4.2 method.B1.1 Identify the package with the highest percentage of god classes.B1.2 Identify the god class with the largest number of methods. Identify the dominant (affecting the highest number of classes) class-levelB2.1 design problem.B2.2 Write an overview of the class-level design problems in the system. design quality assessment 4

116. Tasks quantitative 9 8A1 Identity the convention used in the system to organize unit tests.A2.1& What is the spread of term T in the name of the classes, their attributes andA2.2 methods?A3 Evaluate the change impact of class C, in terms of intensity and dispersion.A4.1 Find the three classes with the highest number of methods. Find the three classes with the highest average number of lines of code perA4.2 method.B1.1 Identify the package with the highest percentage of god classes.B1.2 Identify the god class with the largest number of methods. Identify the dominant (affecting the highest number of classes) class-levelB2.1 design problem.B2.2 Write an overview of the class-level design problems in the system. qualitative 1

117. Main research questions

118. Main research questions 1 Does the use of CodeCity increase the correctness of the solutions to program comprehension tasks, compared to non-visual exploration tools, regardless of the object system size?

119. Main research questions 1 Does the use of CodeCity increase the correctness of the solutions to program comprehension tasks, compared to non-visual exploration tools, regardless of the object system size? 2 Does the use of CodeCity reduce the time needed to solve program comprehension tasks, compared to non-visual exploration tools, regardless of the object system size?

120. Variables of the experiment

121. Variables of the experiment correctnessdependent completion time CodeCity tool Eclipse + Excelindependent medium object system size large beginner experience level advancedcontrolled academia background industry

124. Variables of the experiment correctnessdependent FindBugs completion time 1,320 classes 93,310 LOC CodeCity tool Eclipse + Excelindependent medium object system size large beginner experience level advanced Azureuscontrolled academia 4,656 classes background industry 454,387 LOC

127. The experiment’s design between-subjects randomized-block

128. The experiment’s design between-subjects randomized-block CodeCity T1 large Tool T2 medium Size Ecl+Excl T3 large T4 medium

129. The experiment’s designbackground academia industryexperience beginner advanced beginner advanced B1 B2 B3 B4 between-subjects randomized-block CodeCity T1 large Tool T2 medium Size Ecl+Excl T3 large T4 medium

130. The experiment’s designbackground academia industryexperience beginner advanced beginner advanced B1 B2 B3 B4 between-subjects randomized-block CodeCity T1 large Tool T2 medium Size Ecl+Excl T3 large T4 medium

131. Execution

132. Experimental runs

133. Experimental runs day 1 timetraining session(1 hour)

134. Experimental runs day 1 timetraining session(1 hour) e1experiment session(2 hours) c1

135. Experimental runs day 1 day 2 timetraining session(1 hour) e1 e2experiment session(2 hours) c1 c2

136. Experimental runs day 1 day 2 day 3 timetraining session(1 hour) e1 e2 e3experiment session(2 hours) c1 c2

137. Experimental runs day 1 day 2 day 3 day 4 timetraining session(1 hour) e1 e2 e3experiment session(2 hours) c1 c2 c4

138. Testing the waters 2009 November December 18 24 25 2 9Lugano 1 3 1 1 1 1 1

139. Timeline of the experiment 2009 2010 November December January February ... April 18 24 25 2 9 21 28 5 8 14 28 18 22 24 25 14Lugano 1 3 1 1 1 1 1 1 1 1 3 Bologna 2 1 6 1 1 1 1 Antwerp 5 6Bern 4 1 6

140. Timeline of the experiment 2009 2010 November December January February ... April 18 24 25 2 9 21 28 5 8 14 28 18 22 24 25 14Lugano 1 3 1 1 1 1 1 1 1 1 3 remote sessions Bologna 2 1 6 1 1 1 1 Antwerp 5 6 remote sessionBern 4 1 6

141. Treatments and subjects academia industry beginner advanced advanced large 2 2 6 10 CodeCity medium 3 2 7 12 large 2 3 3 8 Ecl+Excl medium 2 5 4 11 9 12 20 41

142. Collecting raw data

143. Collecting raw data solution

144. Collecting raw data completion time

145. Controlling time

146. Controlling timecommon time

147. Controlling timecommon timeinfo on subjectsName (Task): Remaining time

148. Assessing correctness 1 T2: Findbugs, analyzed with CodeCity A3: Impact Analysis B1.2 A1 Multiple locations. The god class containing the largest number of methods in the system is There are 40/41 [0.5pts] classes class MainFrame [0.8pts] Dispersed. [1pt] defined in the following 3 packages [1/6pts for each]: defined in package edu.umd.cs.findbugs.gui2 [0.1pts] which contains 119 [0.1pts] methods. • edu.umd.cs.findbugs A2.1 • edu.umd.cs.findbugs.bcel B2.1 Localized [0.5pts] in package edu.umd.cs.findbugs.detect [0.5pts]. • edu.umd.cs.findbugs.detect The dominant class-level design problem is DataClass [0.5pts] A2.2 A4.1 which affects a number of 67 [0.5pts] classes. Dispersed The 3 classes with the highest number of methods are [ 1 pts each correctly placed and 1 pts each misplaced]: 3 6 in the following (max. 5) packages [0.2pts for each]: 1. class AbstractFrameModelingVisitor • edu.umd.cs.findbugs defined in package edu.umd.cs.findbugs.ba contains 195 methods; • edu.umd.cs.findbugs.anttask 2. class MainFrame • edu.umd.cs.findbugs.ba defined in package edu.umd.cs.findbugs.gui2 contains 119 methods; • edu.umd.cs.findbugs.ba.deref 3. class BugInstance • edu.umd.cs.findbugs.ba.jsr305 defined in package edu.umd.cs.findbugs • edu.umd.cs.findbugs.ba.npe contains 118 methods or • edu.umd.cs.findbugs.ba.vna class TypeFrameModelingVisitor defined in package edu.umd.cs.findbugs.ba.type • edu.umd.cs.findbugs.bcel contains 118 methods. • edu.umd.cs.findbugs.classfile A4.2 • edu.umd.cs.findbugs.classfile.analysis The 3 classes with the highest average number of lines of code per method are [ 1 pts each correctly placed and 1 pts each • edu.umd.cs.findbugs.classfile.engine 3 6 misplaced]: • edu.umd.cs.findbugs.classfile.impl 1. class DefaultNullnessAnnotations • edu.umd.cs.findbugs.cloud defined in package edu.umd.cs.findbugs.ba has an average of 124 lines of code per method; • edu.umd.cs.findbugs.cloud.db 2. class DBCloud.PopulateBugs • edu.umd.cs.findbugs.detect defined in package edu.umd.cs.findbugs.cloud.db has an average of 114.5 lines of code per method; • edu.umd.cs.findbugs.gui 3. class BytecodeScanner • edu.umd.cs.findbugs.gui2 defined in package edu.umd.cs.findbugs.ba • edu.umd.cs.findbugs.jaif has an average of 80.75 lines of code per method. • edu.umd.cs.findbugs.model B1.1 • edu.umd.cs.findbugs.visitclass oracles The package with the highest percentage of god classes in the system is • edu.umd.cs.findbugs.workflow edu.umd.cs.findbugs.ba.deref [0.8pts] which contains 1 [0.1pts] god classes out of a total of 3 [0.1pts] classes.

149. Assessing correctnessblinding 1 T2: Findbugs, analyzed with CodeCity A3: Impact Analysis B1.2 A1 Multiple locations. The god class containing the largest number of methods in the system is There are 40/41 [0.5pts] classes class MainFrame [0.8pts] Dispersed. [1pt] defined in the following 3 packages [1/6pts for each]: defined in package edu.umd.cs.findbugs.gui2 [0.1pts] which contains 119 [0.1pts] methods. • edu.umd.cs.findbugs A2.1 • edu.umd.cs.findbugs.bcel B2.1 Localized [0.5pts] in package edu.umd.cs.findbugs.detect [0.5pts]. • edu.umd.cs.findbugs.detect The dominant class-level design problem is DataClass [0.5pts] A2.2 A4.1 which affects a number of 67 [0.5pts] classes. Dispersed The 3 classes with the highest number of methods are [ 1 pts each correctly placed and 1 pts each misplaced]: 3 6 in the following (max. 5) packages [0.2pts for each]: 1. class AbstractFrameModelingVisitor • edu.umd.cs.findbugs defined in package edu.umd.cs.findbugs.ba contains 195 methods; • edu.umd.cs.findbugs.anttask 2. class MainFrame • edu.umd.cs.findbugs.ba defined in package edu.umd.cs.findbugs.gui2 contains 119 methods; • edu.umd.cs.findbugs.ba.deref 3. class BugInstance • edu.umd.cs.findbugs.ba.jsr305 defined in package edu.umd.cs.findbugs • edu.umd.cs.findbugs.ba.npe contains 118 methods or • edu.umd.cs.findbugs.ba.vna class TypeFrameModelingVisitor defined in package edu.umd.cs.findbugs.ba.type • edu.umd.cs.findbugs.bcel contains 118 methods. • edu.umd.cs.findbugs.classfile A4.2 • edu.umd.cs.findbugs.classfile.analysis The 3 classes with the highest average number of lines of code per method are [ 1 pts each correctly placed and 1 pts each • edu.umd.cs.findbugs.classfile.engine 3 6 misplaced]: • edu.umd.cs.findbugs.classfile.impl 1. class DefaultNullnessAnnotations • edu.umd.cs.findbugs.cloud defined in package edu.umd.cs.findbugs.ba has an average of 124 lines of code per method; • edu.umd.cs.findbugs.cloud.db 2. class DBCloud.PopulateBugs • edu.umd.cs.findbugs.detect defined in package edu.umd.cs.findbugs.cloud.db has an average of 114.5 lines of code per method; • edu.umd.cs.findbugs.gui 3. class BytecodeScanner • edu.umd.cs.findbugs.gui2 defined in package edu.umd.cs.findbugs.ba • edu.umd.cs.findbugs.jaif has an average of 80.75 lines of code per method. • edu.umd.cs.findbugs.model B1.1 • edu.umd.cs.findbugs.visitclass oracles The package with the highest percentage of god classes in the system is • edu.umd.cs.findbugs.workflow edu.umd.cs.findbugs.ba.deref [0.8pts] which contains 1 [0.1pts] god classes out of a total of 3 [0.1pts] classes.

150. Results

151. Statistical test two-way analysis of variance (ANOVA) 95% conﬁdence interval

152. Correctness Ecl+Excl CodeCityDoes the use of CodeCity increase the 8correctness of the solutions to programcomprehension tasks, compared to non- 7visual exploration tools, regardless of the 6object system size? 5 4 3 2 1 0 medium large

153. Correctness Ecl+Excl CodeCityDoes the use of CodeCity increase the 8correctness of the solutions to programcomprehension tasks, compared to non- 7visual exploration tools, regardless of the 6object system size? 5 24.26% 4 3 2 more correct with CodeCity 1 large effect size (d=0.89) 0 medium large

154. Correctness Ecl+Excl CodeCityDoes the use of CodeCity increase the 8correctness of the solutions to programcomprehension tasks, compared to non- 7visual exploration tools, regardless of theobject system size? 6 24.26% 5 4 more correct with CodeCity 3 large effect size (d=0.89) 2 medium large

155. Completion time Ecl+Excl CodeCityDoes the use of CodeCity reduce the time 60needed to solve program comprehensiontasks, compared to non-visual exploration 50tools, regardless of the object system size? 40 30 20 10 0 medium large

156. Completion time Ecl+Excl CodeCityDoes the use of CodeCity reduce the time 60needed to solve program comprehensiontasks, compared to non-visual exploration 50tools, regardless of the object system size? 40 12.01% 30 20 less time with CodeCity 10 moderate effect size (d=0.63) 0 medium large

157. Completion time Ecl+Excl CodeCityDoes the use of CodeCity reduce the time 60needed to solve program comprehensiontasks, compared to non-visual explorationtools, regardless of the object system size? 50 12.01% 40 30 less time with CodeCity moderate effect size (d=0.63) 20 medium large

158. after the ﬁrst roundCodeCityvsEcl+Excl+24% correctness-12% completion time