real world programming

code coverage

Posted by kevinup on October 20, 2008

I was recently talking to another developer about code coverage and it made me re-think about code coverage.

I had asked him what his code coverage was on this project, and he replied it was probably pretty bad, but that coverage didn’t matter. The quality of the tests mattered. His argument was that if you had good tests that is all you need. 

At first I agreed, I love high quality tests. I like the idea of having 2 high quality tests over 10 medium to poor quality tests. Depending on the quality of your developers that isn’t always possible to have high quality tests.  I’d rather take over a project with 70% coverage with average quality tests than one with <20% coverage with high quality tests. A high coverage number gives you a lot more confidence about your source code, and any potential changes.

I recently was on a project where we were adding items to a shopping cart. There were around 150 tests built around adding items to the shopping cart, and it broke all the time. The team was around 10 developers, and the issue was that developers didn’t have the time to run all 150 tests, which could take up to 5 minutes. So they would check in and move on without running the tests. (Yeah I know, not a good practice)

I went through and identified 3 tests which captured most of the base functionality. I asked all the developers to at least run those tests before checking code in. Coverage-wise they were probably only hitting 60% of the shopping cart logic. But because they were higher quality tests, and could be run in <30 seconds, we were able to have the best of both worlds.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: