I found a bug today. Quite a biggie too. Not something that will happen everyday but nevertheless it will happen.
We are currently working on a software development kit for a range of temperature sensors we sell. We have a unit test suite with around 80 or so tests give or take. The tests cover what you’d probably expect. Read the sensor, make sure the values coming back are what you know should be coming back.
We created a mock sensor so we know precisely what should be coming back.
I was trawling through the code writing some more tests and it occured to me that I couldn’t see anything in the code that handled what would happen if the sensor’s owner randomly changed the default temperature units on the sensor. Now, that isn’t going to happen very much but it will happen and the code as it is will get it wrong.
So, I am going to have to write a test to cover this scenario. It will fail…and then I can fix it 
But the point is, no amount of unit testing + code coverage analysis would have caught this scenario. The code didn’t handle the scenario, therefore there is no code to cover so to speak.
In the end, it is still down to the designer/developer to properly analyse the problem and create a correct solution taking into account all of the boundary cases.
{ 2 comments… read them below or add one }
Hi,
Just found your blog via Chrisg.com. Some interesting posts you have on here!
Would the situation you describe above have been modelled as a Use Case and have a system level test for it?
@Steve
It was all my fault I am afraid. It was a good old fashioned analysis bug and no amount of unit testing or functional tests will ever find those