Bugs. Bugs. Bugs.

Keith Pitty and Charles Miller have some very useful thoughts on software testing and finding bugs.
Automated software tests taking care of possible regressions are extremly important, especially if you intend to release often. However, automated tests are not an after-thought, neither unit-tests nor acceptance-tests. If you change the fundamental algorithmn, you are not finished as long as the unit tests don’t reflect the new/different boundary conditions. If you add a new feature, you’re not done as long as you’ve added a new acceptance test covering the feature. This requires a lot of discipline on behalf of the developers. Plus it requires patience on behalf of management, because changing the algorithmn/implementing the feature will take a little bit longer than expected. Last but not least, having automated tests provides a great starting point for external testers, ensuring they don’t complain about those nasty regressions.
Having external software testers is vital to find the bugs in areas/paths not vovered by automated tests (most likely the majority of bugs). Developers test their code to see if it works the way they think it should work. They’re testing with knowledge of the code written a couple of minutes ago. They want to see it work, not fail – after all, who want’s to see his/her work fails? External testers expect the code to deliver the results they expect. Plus, they want to see the code fail, which, BTW, create a significant tension between the two camps. Make sure to remind everyone involved that both teams are working together to deliver a great product.
Keith makes a good point about over-reliance on automated tests. However, note that you got to have a comprehensive suite of automated tests first, covering all the regressions you want to avoid by all means. Then start worrying about over-reliance on automated tests. But not the other way round.

Comments are closed.