If you create software that is planned for continuous enhancements over several years then you’ll need a sustainable strategy for code quality.
Code coverage by automated tests is one important metric, and its value should equal 100 per cent at all times.
This talk will show why this is so and how it can be achieved in real world software (with examples in Java)
By Andreas Czakaj, mensemedia Gesellschaft für Neue Medien mbH
This presentation was held at the W-JAX 2017 conference (https://jax.de/core-java-jvm-languages/100-code-coverage-in-real-world-software/)
2. Once upon a time
*** *** Shipment Tracking
Only a few
deployments slots per year
per deployment -> $$$
Re-deployment?
wait for next slot
pay more $$$
-> Quality or die!
4. Why automated testing?
It‘s 2017, so a part of me wants to say…
Srsly?
Pixabay, License: CC0 Creative Commons
5. Why automated testing? (1/4)
OK… we want automated testing in order to…
Protect against (regression) bugs
Keep confidence & control
Manage knowledge
Ensure long term adaptability
Work faster
Build upon working parts
Q = 1..n qi
6. Systems Engineering: Software Lifecycle (2/4)
”Be nice to future-you“ – unknown
People working on software after…
… 1 year? -> new people
… 3 years? -> new > original
… 6 years? -> 0 originals anymore
Future “generations“ of team members must be prepared for…
… changes, enhancements etc.
… patches, updates, upgrades, replacements etc.
1 2 5 †*
11. Testability & Code Design (4/4)
TDD promotes these ideas:
If it‘s hard to test…
…it likely has design issues
So don‘t try to find more and
more clever ways to test…
… instead change the code to
make it more testable
-> Testable is the new clever
Test Driven Development & Design
13. Why Code Coverage?
We want to find out what we don‘t know.
Use it as a checklist
Find blind spots
Find dead, unused code
static double divide(double a, double b) {
if (b == 0) {
}
throw new DivisionByZeroException();
}
return a / b;
15. Why 100% Code Coverage? (1/6)
Tests help a lot…
… for managing the code
the tests cover
… for code they don‘t cover
test offer no help
That‘s like feeling bigly about
having the engine running…
… when you should have all 4 of them running
Pixabay, License: CC0 Creative Commons
16. Why 100% Code Coverage? (2/6)
Maybe you had a reason not to
cover some code… at that time…
Will the future team still
know that reason?
And, after the usual and
inevitable drifts in the code base…
…will the reason still
be valid in the future?
”U Turn“ by André-Pierre du Plessis,
License: CC BY 2.0
17. Why 100% Code Coverage? (3/6)
Often, bug clump in some areas
As the saying goes: “90% of problems
are caused by 10% of the code“
90% Code Coverage means you don‘t
cover 10%
What if that‘s THOSE 10%?
No? How can you be sure w/out tests?
18. Why 100% Code Coverage? (3/6)
OK, that was number magic…
…let‘s put it a different way:
Which code will likely
cause more problems:
highly tested code?
untested, potentially even
untestable code?
Edvard Munch, “The Scream”
19. 100% Code Coverage to prevent decay (4/6)
”It wasn’t me. It was that way already.“ – my children
Prevent decay by
“Broken Windows“:
Some code sub-standard…
-> more code will follow
… and developers can blame it
on the darned legacy code
Instead, have a
“No code left behind“ strategy
“Broken windows, Northampton State Hospital”
by Karan Jain, License: CC BY-SA 2.0
20. 100% Code Coverage as the Team’s Rule (5/6)
Use the power of metrics
Rules that everyone needs to stick to
… objectively measured by a 3rd party
… let the metric “reward“ the team
when at 100%
… use the metric to
reduce complexity to…
“Complexity kills. It sucks the life out of developers […]“ – Ray Ozzie
Source: Geek & Poke
25. 25
There is no reason why you
SHOULDN‘T WANT 100% Code Coverage.
(There may be reasons why you
CAN‘T get to 100%, though…)
26. Code Coverage is a fallacy just a tool.lielie fallacy
It depends on what you use it for…
… and how you interpret its output.
27. How to trick Code Coverage
“Ohh, I‘m a liar. Yeah.“ – Henry Rollins
When you run a Test Case
it will always cover some code
E.g. assertNotNull(new MyClass())
-> x% Code Coverage
-> 0 value
@Test
void test_init() {
assertNotNull(
new MyClass());
}
28. Just because Code Coverage can be faked
does not mean it is irrelevant.
29. Code Coverage != Test Coverage
Code Coverage means…
… code is covered by tests
… but you still can have bugs
Code Coverage does NOT mean…
… all scenarios are covered
-> that would be Test Coverage
static double divide(double a,
double b) {
return a / b;
}
@Test
void test_divide() {
double actual = divide(6, 2);
assertEquals(3, actual, .1);
}
// no validations,
// weak precision check,
// division by zero is possible!
// ...
Understand the metric
30. Where Code Coverage cannot be measured
“I don't think anybody tests enough of anything.“ – James Gosling
Views
JSP, JSTL, JSF and others that
allow for markup + conditional code
Configuration
Did we test ALL configuration entries?
How can we spot obsolete configs?
“Game over”by porpom, License CC BY 2.0,
<c:if test="${temp > 18 && temp < 27}">
<p>Let‘s take a walk.</p>
</c:if>
32. Alien Code
Generally: full coverage can be impossible
if you cannot change the production code
3rd party code
pasted “alien“ code
generated alien code
e.g. XSD to Java, UML to code exports,
RAD tools, scaffolding scripts etc
-> consider moving it out (JAR, src, …)
Pixabay, License: CC0 Creative Commons
33. Legacy Code
(Your own, untested) legacy code…
static calls
global calls
global state
wrong dependencies
…
…is like alien code to new team members
https://www.amazon.de/dp/0131177052/
34. Conscious Trade Offs
Sometimes you find yourself saying:
“I don‘t really know…
… what I want yet.“
… how to do it yet.“
-> You need a Proof of Concept (PoC)
But TDD and 100% Code Coverage
are hard to do when you don‘t know what to assert
However, professionals should create PoCs only as throw-away code
Pixabay, License: CC0 Creative Commons
35. Lack of Knowledge
Things you don‘t know how to test (yet)
Sometimes you need to
master the technology first
It may take some time & effort
to find solutions for testing
-> Don‘t write testing off for good,
keep looking for solutions
+ look beyond “The Common Practice™”
Image: “Digging the Drain”by
International Disaster Volunteers, License: CC BY 2.0
38. TDD – Test Driven/First Development
Start @ 100%, Stay @ 100%
Image by user “Jgu7“
Get familiar with
Test Driven Development
… and the Test First approach
39. TDD DON‘Ts
Start @ 100%, Stay @ 100%
“Hexagonal Architecture”a.k.a.
“Ports and Adapters”by Alistair Cockburn,
Don‘t start on the “outside“,
i.e. do NOT start with:
Controllers / Commands
Views
Data Access Objects / Queries
File System Accessors
…
40. Also, it‘s a psych thing
If you start on the outside you‘ll
have slow progress
feel the need for non-Unit tests
(i.e. slow tests)
feel the need to mock stuff
have a Coverage below 100%
have less fun (always in debt)
Sigmund Freud, Wikipedia
41. Also, it‘s a psych thing
You might make bad decisions
You might put more into the
class, e.g. the Controller,
than you should…
… to kind of make the effort
spent “pay off“
-> poor design and wrong
dependencies are possible
Pixabay, License: CC0 Creative Commons
42. The Dependency Rule
Inspired by “The Clean Architecture”
by Uncle Bob Martin
Enterprise
Domain
specific
Controllers
“source code
dependencies
can only point
inwards”
Actros, MPC,
Enterp. Rules
Trucker Profile,
App Rules
DI Container, OSGi
SysDate
DAOs
FS
SQL DB
NoSQL DB
SOAP/REST
SQL, Queries
43. Ports & Adapters / Dependency Inversion
“But the Use Cases and Domain objects
need data from the outer layers!”
“How can I read from and write
to the outer layer without depending on it?”
-> Dependency INVERSION
-> “Ports”
(simplified: outer layers behind interfaces)
45. The “No Logic“ argument
”Tests are for logic. My code does not contain logic “
You might not see any logic…
…but, usually, it‘s there…
…it‘s probably just
in the wrong place
BIG
Controller
BIG
DAO
tiny
Service
“Weight Training Crossfit Fitness Models”
by ThoroughlyReviewed.com, License: CC BY 2.0
46. Identify the Domain
“The Clean Architecture”by Uncle Bob Martin
… yes, it exists
Start at the core: the Domain
… also, it’s the code that’s the
easiest to test!
you’ll start at 100% coverage
… then stay at 100%
… while progressing outwards
“source code dependencies can only point inwards”
48. Hard to test? -> keep it out
Pixabay, License: CC0 Creative Commons
Keep your Domain, Rules and
APIs clean, i.e. keep
hard to test stuff out
e.g. 3rd party frameworks
& libraries
… (especially) if they
interfere with testability
49. Trust the test:
If (your own) code is hard to test
then it likely has a design problem.
51. Reality Check
Our experience shows:
100% Code Coverage is
feasible & relevant
However, you might need to
code and design differently…
… but that will eventually
lead you to Clean Code and
Clean Architecture
“Big_Data_Higgs”by KamiPhuc, License: CC BY 2.0
52. Useful Links
Ports & Adapters
Dependency Rule
Clean Architecture (new Book)
DAO / Repository Pattern
3 Rules of TDD
Bowling Game Kata
“The more testing you do…”
55. #HappyHacking!
Andreas Czakaj
Managing Director IT / CTO
mensemedia Gesellschaft für Neue Medien mbH
Neumannstr. 10
40235 Düsseldorf, Germany
Email: andreas.czakaj@mensemedia.net
Twitter: @AndreasCzakaj
Blog: https://blog.acnebs.com/