Friday, 29 May 2009

Code Coverage of Junit in Ant without instrumentation

Until today we were using Corbertura to do our code coverage report. This required a two phase build process in which the first phase involved compilation, instrumentation and testing, the second phase being compilation and packaging.

Arum DataEye was taking 11 minutes 31 seconds to build and this felt a little too long, even though we're building and testing 26 OSGi bundles and two pure ActionScript libraries. This also violate's Fowler's keep the build fast rule.

I've been using the Eclemma plugin in Eclipse to give me coverage reports on the fly. The integration is simple but effective, always the best way. It occurred to me this morning that this plugin must be using something to do the coverage and that's when I discovered Emma.

So I decided to set about coverting our Ant based build to use Emma instead of Corbertura. I hit a snag - you still need to do instrumentation to run it with the JUnit task ... or at least it appears you do.

Emma provides a number of Ant tasks, most relevant to this discussion is the emmajava task which essentially just replaces the java task setting up Emma support automatically. However, this wasn't going to be enough to get on-the-fly instrumentation running with JUnit.

To cut a long story short, you simply have to trick the Junit ant task in to running the Emma command line runner. This is done with the following Ant snippet:

The important thing here is that the Junit task is forked. We then fool Junit in to running the Emma runner for each fork and voila, no need to instrument.

As a result, we were able to reduce our build process to a single phase of compilation, test, and packaging reducing the total build time Arum DataEye to 4 minutes 33 seconds! A massive saving of nearly 7 minutes!

This (fairly old now) post will get you a good chunk of the way and of course there's the Emma user guide.


Andy Latham said...

Dude, that's awesome. Cobetura drives me nuts.. I'll give that a shot!

Ben Vitale said...

I wonder if you've had any luck excluding your test code from coverage using this on-the-fly approach?

My problem is that our test code often lives in the same package as the code under test. The test code is merely in a different JAR file. I suppose I could use *Test* or *TestCase* filter strings, just doesn't seem like 100% exclusion, since I've got test utility classes that don't necessarily match that pattern..

Chris Brind said...

Ben - it never even occurred to me to do that to be honest. I just automatically exclude them from the results in my head anyway. :)