Arriving at a decision on the target devices and OS to be included in testing is a difficult task. Many times people resort to industry data on phone sales as a reference for selecting the devices that they need to test. While it is a good source of information, one cannot rely entirely on such information nor can they select the latest devices or devices from the largest players alone.
I do not know how many of us know what a Kin is – it ‘was’ a smart phone that Microsoft invented. Reportedly, it was in the market for a little over a month only. Like one reporter observed, many fruit juices have more shelf life than that. When Microsoft builds a mobile, will any one ignore it? Surely will not. If it lived a little longer, it would have inflated the testing budget by adding one more device and resources to test it, before it dies its natural death.
Deciding on the device combination is one of the key success factors for a good testing strategy for mobile devices. With 75% of the market share, Android has a place in all test combinations, followed by iOS with 15% share. However, a deep dive into the target market and its behaviour is required before the selections can be made.
The best way to short list the devices is to segment the target audience, and then depending on the risk appetite, eliminate the devices from the segments to arrive at a combination of Must, Good and Nice to have. This combination will include devices operating systems and their combination (there can be devices that need to be tested on more than one operating system).
Another challenge with the test combination is that it is dynamic. Every release, the test team will have to look at the combinations and revise the matrix to reflect the latest changes to the combination.
I test. Therefore I am.
