81. Dev-Ops An engineer with a pager is an engineer passionate about code quality.
82. Summary Unit Tests Global Tests Monitoring Does my code do what I intended it to do? Is my code likely to break? Is my code actually working right now?
The price of clean code is eternal vigilance. Everyone wants to work with clean code, but no one wants to be the enforcer. In this session we'll look how at KaChing integrates style checkers and static analysis tools into the build process to keep errors out without getting in the way of developers. Many of the tools discussed are specific to Java or Scala, but the techniques are generally applicable. Room 8338 1:15pm Sunday
If you can't produce working code, you aren't going to be working for very long as an engineer. But whether you produce good code or just working code depends almost as much as the environment as your own abilities. The business guy says “hey, I need a chart that shows X”. It's easy to see if he has a chart, and that it looks like it's showing X. When I say “solves tomorrows problem” I don't mean over-engineering. I believe YAGNI. I mean it's extensible, understandable.
The programmer isn't sufficiently motivated to go beyond working code. Something external needs to push him that way. Nobody wants to be the asshole. The computer doesn't mind being the asshole.
No distinction between Unit, Functional and Integration tests. These are all tests that your code does what you intended it to do. Style and static analysis test that your code does it in the right way. Is the code likely to have unforeseen bugs, is the code going to be hard to understand to the next guy. Automated monitoring keeps testing your code after it goes out to production. If it throws an exception, that needs to get back to the engineer. This can happen automatically.
KaChing lets you invest with professional money managers who normally only take on wealthy clients. If you know investing, we simplify separately managed accounts and provide a transparent marketplace. If not, personal mutual fund. This is a different tolerance for errors than petswearingfunnyhats.com Continuous deployment means the automation is all that stands between your editor window and the customer. Compiler technologies are our secret sauce for testing the untestable.
I'm here today to talk about automation. Automation is only part of why we are successful. Some key items that I'm not addressing are the quality of engineers, our culture of test-worship, and real buy-in from the engineers that everything that can be automated, should be automated. We also use Fix-it days when we go back and try to fix a problem that has crept into our code. For example, we may have a fix-it day soon to eliminate all our outstanding PMD warnings.
I'm going to assume you know what junit is. It might surprise you that we don't just test Java with junit, but also Scala. No on has gotten around to setting up something like specs. We run it in Eclipse Jmock is such a powerful too you owe it to yourself to learn how to use it, unless you already have a favorite mock system. If a unit test fails, you cannot deploy. You can't even check-in unless it's #rollback or #buildfix
Java BigDecimal says 1.0 and 1.00 are different. We usually consider them the same. Same sort of thing for XML, JSON
First test ensures that Guice can find bindings for everything that gets injected. The second test ensures that the queries follow rules about not performing logic in constructors that's needed for Pascal's instantiator framework.
We'd really like something more expressive than package or public. Dependency test and Visibility test using the @VisibleForTesting annotation let us express the difference between “this is public, anyone can use it” vs. “this is public because I need to access it from another package”.
This is minor, but I just wanted to include it as a point of expressing conventions as code.
This isn't as ready to go out of the box as PMD or FindBugs, but you can use it without much effort. DeclarativeTestRunner lets you set up a bunch of rules that will be matched in turn against a set of instances, and any failures will be reported. Iteration goes the wrong way if you try to express this as individual tests.
Show precommit tests. More or less a collection of all the tests that have a tendency to fail. People run the tests for the class they are working on, but don't want to spend the 3 minutes or so to run the full test suite.
Considering moving this to part of the main build. Some question about whether this will mean we need to run the tool before checkin, since we can't really know whether it will find a problem.
We
This isn't some crazy bureaucratic stuff. This comes from someone who really gets test driven development. We reject it though.
Engineers don't need to be watched like prisoners. They just need a jolt “hey are you sure you mean to do that”.
We discussed this for QueriesAreTestedTest. If I annotate the class, then I know if I go in there an modify it, it's not tested so be careful. But if you annotate the class, then I can't easily go to a centralized place to find all the exceptions. Easy to see what “rules” need to be worked on. Java leans towards in-place, which I guess is a good argument against it.
Engineers don't need incentives from above when they have personal incentives to make sure the code is good. It's all about aligning interests. Try to internalize all the relevant competing interests. That said, I'm not on the ops rotation yet, but I believe someone just obligated me to be up at 6:30am when the markets open in NY.
Queries are the high level building blocks. David recently added a test that they are tested (very loosely – a test must instantiate it). Still hundreds of exceptions, but new code generally covered. If it doesn't need to be tested, add an exception. Checkin a new class without a new test without #notestneeded results in email. No further action needed, but you should either add a test or reply explaining why not “covered by existing tests” Coverage very useful at NexTag adding tests after the fact. Pretending coverage was a useful metric led to gaming at another company. Test that just instantiates the class, worse than nothing because you think it's covered.
I'm not being strict about the meanings here, but as an example, one test starts up two servers which talk to each other over FIX, a brokerage protocol. Slow and fragile. We don't run it as part of build. Initial validation and testing new functionality. Rely on lower level unit tests to check regressions.
This might get into the politics of things. No one likes to tell people their code stinks. I mean, maybe it isn't how you would write it, but you can't point to anything really wrong about it. Because of automation, these lose a lot of their value. “Whoops, you can't use BigDecimal.divide like that” That's already caught
I somewhat disagree with this. I think demanding comments on “what the hell is this object” is entirely reasonable. Enforcing a rule of “public has a comment or @SelfExplanatory annotation” is reasonable.
Engineers like finishing things. I'm done, I'll send it to QA, few days later, QA sends it back. Mentally, these are new tasks. I already finished it. And now I get to accomplish more by doing these tasks QA has brought to me.
Arguments on both sides. We recently turned on “no commits while the build is broken” to put pressure on people to fix it. Concerned that the build queue will slow down pushing things to production.
Hazy. I'm mentioning it
We do continuous deployment, so we don't really want to stage things for QA to test, but we need an environment for more experimental things, and to see whether some approach will work.
I've seen people act like it's heresy to check in your IDE settings. This might be the case for a big distributed open source project. It's not any more effort to save the project specific code formatting settings than it is to throw up a page on a wiki saying 2 spaces, no tabs.
I'd love to hear your feedback on this presentation or on technologies you have found useful for testing. If you found this interesting, you should check out our blog where you'll find lots more on a similar theme. If you are interested in an alternative to mutual funds, I suggest you check out kaching.com.
![Working Code vs. Good Code ,[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)