Testing on Android – Simplified
How many device models to consider while TESTING?
Are you a mobile application test engineer who often comes up with this question?
No worries, let us simplify this using a template.
How many models one must test depends on the application. It is better to analyze and choose an optimum solution that works best within your budget and satisfies your customers’ needs.
1. User Feedback: Check your user’s pulse! Read app reviews if your app is already on Google play. Not just your app, but your competitors’ app reviews too.
2. The Application: If you were to test a gaming application, you would have to test it on as many devices as possible; ranging from smaller form factors to the largest tablets available.
3. APIs: If the application uses device centric APIs, then test it on as many device models as possible from the different manufacturers.
4. Understand the application layout: It will be good to understand how Android handle’s layouts. Gestures or animations are also supported according to different Android API levels. Take a layout tour while testing the application and test it on all major versions of Android.
5. Data handling: Understand how the application stores data. Can the app be installed on SD card and not just internal storage? If so, test it by installing it on SD card as well as on Internal storage.
6. Hardware Integration: Does the app depend on inherent hardware components like Camera, Compass, GPS, Accelerometer, Gyroscope, Barometer etc…? If so, test it on devices that does not support these sensors first and then later on devices with sensors.
7. App Integration: Does the app interact with native party apps like gallery, access SD cards, access phone contacts, dialer, voice dialer, keyboard, access media? Or does it interact with third party apps or services? In case of native apps, testing it on as many devices as possible is beneficial, since most of them are tweaked by manufacturers to suit their needs. For example: The way manufacturers handle contact access are different for different devices.
8. Telephony, data network vs WiFi only devices: Have you ever wondered why an app installs easily on the phone but does not install on tablets and vice versa? Android applications can be controlled to install only on certain type of devices based on whether the telephony services are available or not. It is always a good idea to discuss with the product manager about the supported device factors and tests based on them.
9. Native binaries:Does the application use NDK binaries? Does the game require binaries based on GPU? Ask your developers if you are not aware of anything that might impact testing across CPU architectures. It is always a good idea to test on x86 based devices apart from ARM.
10. AQuA Guidelines: Read the latest AQuA Testing Criteria for ideas before testing an Android app. AQuA provides in-depth insights on testing Mobile applications across devices.
To conclude, the template is a starting point to help you decide on the device stack you must create before testing. We do encourage you to think of ideas that could help you come up with a your own custom template. Do feel free to leave your comments below.
-Sundaresan Krishnaswami
Don’t do any of this manually. Write automated tests!