Jul 4, 2006

Testing Test Readiness

Qualifying a build for test is not easy always. A build that is not “ready” for test, pushed into the test environment can eat a considerable amount of your testing effort thus upsetting the test estimates as well as schedules.

There of course cannot be a universally applicable way to test the test readiness of a build before it gets deployed in the test environment. But, carefully planned smoke tests prior to acceptance of a build for testing can considerably save the time and effort that goes into testing. In a perfect world, developers unit test their code before they deliver the build. But unfortunately we all live in a world that is far less than perfect. We do need to test; we also need to test whether the build is ready for the test. Probably this is one of the most important instances where the testers “refuse to test”.

It therefore becomes important that the selection of tests for smoke testing (the test readiness test) should be appropriate and well representative. There can be a core test set and a supplementary test set based on bug history, testers error guessing capability, prior experience with this or similar applications etc. Some of these tests may retire during successive cycles.

It is a good idea to automate the core test set to reduce the cycle time. You may even want to share the smoke test suite with the developers so that they know what you are looking for. It is always a good idea to brainstorm before arriving at the core smoke test set.

I found some interesting thoughts on smoke testing here...