Cover Your Nose When You Test
Posted by Tim Freund Wed, 24 Sep 2008 06:06:00 GMT
My mom always reminded me to cover my nose when I sneezed. Now I take it a step further, and I cover my nose when I test.
Nose and the nosetest command are used to run unit tests for Pylons and TurboGears 2 applications, as well as a multitude of other Python applications and libraries. Although nose is great at running tests and reporting back issues, it doesn't natively show developers what isn't being tested. For that, we need a code coverage tool.
Just because nose doesn't handle code coverage reports natively doesn't mean this will be a difficult task. Ned Batchelder's coverage package provides exactly those reports, and nose ships with a plugin to enable it. To install coverage and invoke the reports, you could do something like this:
$ easy_install coverage
$ nosetests --with-coverage
Name Stmts Exec Cover Missing
--------------------------------------------------------------------------
_strptime 228 149 65% 23, 80, 84-89, 155, 169-170, 175, 189, 237, 280-294, 303-304, 306, 314-323, 329, 332, 353-360, 366, 368, 374-388, 393-427, 431-432, 443-446
encodings.ascii 19 0 0% 9-42
ez_setup 103 11 10% 53-62, 80-104, 117-151, 156-190, 197-222, 226-229
fixture 10 9 90% 38
fixture.base 124 33 26% 10-19, 25-28, 48-49, 56, 60, 64, 98, 103-104, 122-217
fixture.dataset 301 225 74% 41, 51-52, 55-59, 65, 76-77, 80, 98-110, 141, 144, 225-229, 232-239, 243-248, 277, 285-289, 323-325, 447-450, 456-457, 461, 468, 479, 483, 529-530, 560-564, 571-574, 580, 631, 720-722, 725-737, 740-744, 752-753
... lots of lines cut to keep you from going blind ...
zope.interface.ro 22 22 100%
zope.interface.verify 45 29 64% 46, 51, 56-61, 66, 70, 75-79, 88, 93, 104, 107, 109, 111
--------------------------------------------------------------------------
TOTAL 17257 7646 44%
----------------------------------------------------------------------
Good Gravy, man! That's a big coverage report!?
As cool as it is to know that we can get coverage reports for the entire Pylons/TG2 stack, how about we focus for just a minute on only our project. That's the problem with kids today, no focus.
$ nosetests --with-coverage --cover-package=YOUR_PACKAGE_NAME
OK, that's better. Now the report only shows the coverage for code inside of our package. But, seriously, do we really need to type those arguments every time?
Of course not. Sitting there in the root of your Pylons or TurboGears 2 application is a file named setup.cfg. If I were a betting man, I'd say you've never opened it. Ever. Let's knock the dust off and take a look at it in any decent text editor. We're looking for a section named [nosetests], and since you've never changed the file, it is probably at line 8 and looks just like this:
[nosetests]
with-pylons=test.ini
We can add any additional options for nose in this section. Now is your chance to spring into action. Add the following lines to
with-coverage=true
cover-package=YOUR_PACKAGE_NAME
Save setup.cfg and run your tests. You will see a code coverage report at the end of the test run, and the last two columns will be the most interesting. They show the percentage of the code that was covered and the lines of code that were not covered, respectively. Depending on the outcome of your coverage report, you may be feeling rather smug right now. Stop it, we're not Rails developers, and there is still work to do.
On the other hand, if your code coverage report leaves you feeling a little ashamed of just how much code is uncovered, don't despair. Just knowing what code coverage is and caring about the results of a coverage report already puts you into a minority of all programmers. Pick a block of untested code and write a test. All of the sudden, your numbers are better and you'll start to leave those fly-by-the-seat-of-their-pants programmers in the dust.
Remember that this setup.cfg trick will work with any python application or library that is testable with nose. We happen to be focusing on Pylons and TurboGears because that's what I've been busy using lately. How do you use nose to help you write better code?
