Don’t just fix the bugs. Fix whatever permitted the bugs in the first place.
Summary
This article is a compendium of useful tips & tricks for testing mobile apps.
General
- Basics
- Selecting test devices
- Testing biometrics
- Testing Facebook analytics
- Testing valid IBAN numbers
Android
- Changing the ROM on Pixel devices
- Devices supported by Google
- Don't keep activities
- Enabling Developer options
- Getting nice device logs
- How to download a beta version of the app on Android
- How to fake your location on Android
- Installing an app from a foreign Google Play
- Setting up an Android emulator
iOS
- Enabling iOS dev settings
- Installing an app from a foreign App store
- Jailbreaking iOS11 with Uncover
- Jailbreaking iOS12 with Uncover
- How to remove jailbreak on iOS
- Testing production builds on iOS
- How to install iPhone or iPad apps on M1 Mac
General
Basics
Have a look at an one of our blog posts on testing mobile apps.
Selecting test devices
You cannot test your app on all possible devices. That is why it is important to intelligently decide on a set of devices on which you will do you your testing.
The two major mobile operating systems are:
- Android
- iOS
Some parameters to keep in mind:
- The minimum supported OS version (varies from project to project)
- The current OS version
- What the intended audience uses
- Screen size and resolution
- Chipset
For instance, if you had time to test a Croatian mBanking Android app on a mobile device, it would make sense to have:
- A current large flagship Samsung or Huawei device
- A smaller mid-range device which is two or more years old
- An old device running the minimum supported OS version
The device usage percentages of our apps can be retrieved from the Google Dev Console or iTunes Connect. Contact your team lead to get that info.
The usage of particular devices in general can be found on web sites such as StatCounter.
Device-wise, make sure to use:
- Smartphones
- Tablets (if the project supports it!)
Browsers that will almost always fit the criteria in a mobile environment are:
- Chrome (Android)
- Safari (iOS)
Another important parameter in testing is orientation:
- Portrait
- Landscape
Testing biometrics
When you test Android and iOS apps which allow the user to login with TouchID (iOS) or fingerprint (Android), you should check the following cases when the fingerprint login is enabled in the app:
- WHEN a new fingerprint is added to the currently saved fingerprint(s) in the phone's settings, THEN a "Fingerprint settings were changed" message should appear when logging into the app to reactivate the fingerprint login
- WHEN one of multiple saved fingerprints is deleted, THEN a "Fingerprint settings were changed" message should appear when logging into the app to reactivate the fingerprint login
- WHEN all fingerprints are deleted, THEN a "Fingerprint disabled" message should appear when logging in and the fingerprint login toggle should be disabled in the app settings
When testing iOS apps which allow the user to log in with FaceID, you should check the following cases:
- WHEN none FaceID scan is saved in the phone's settings, THEN the FaceID toggle in the app settings should be turned OFF and disabled
- WHEN a FaceID scan is added and the FaceID permission is given for the app, THEN the FaceID toggle in the app settings should be enabled
- WHEN a FaceID scan is added and the FaceID permission isn't given for the app, THEN the FaceID toggle in the app settings should be turned OFF
Testing Facebook analytics
Ensure Facebook SDK is implemented in the application, and that those analytics are set up correctly.
Find FacebookID of the account which you will use for testing. There are two ways of finding this ID, you either login to FB account and click on your profile name in the top left. Your ID will be visible in URL in the browser.
Facebook Analytics Admin needs to add this account to Facebook analytics. Account must be verified; otherwise, it cannot be added.
Go to developers.facebook.com
Click My apps in top right corner and select app you want to test from the dropdown menu. From the dashboard, click on View analytics. On the left side, click on Activity and then on Events
By default, Facebook tracks standard events such as App install and App launch. You can find events that are tracked out of the box by default and list of standard events here.
How to test?
See implemented platforms, check which event is implemented and its requirements. Then try to trigger that event, and if everything is done correctly, the event should show up on the event screen mentioned above.
Facebook analytics dashboard will not show triggered events instantly.
Testing valid IBAN numbers
Check out the list of valid IBAN numbers if you test your application's IBAN validation and make payments through the app using (foreign) IBAN numbers.
Android
Changing the ROM on Pixel devices
- Use a Windows PC, it's easier
- Visit the page https://toolaio.tk/
- Click the Download tab and download the .zip file
- Extract the .zip file and install the app
- Grab you Pixel device
- Ensure the device has Developer options turned ON (if not, click the Build number in the phone's Settings 5 times in a row)
- Open the Developer settings and turn ON OEM update and USB debugging
- Connect the Pixel device with your PC
- The dialog "Trust device?" will appear, click "Allow"
- Open the "TOOL ALL IN ONE" app on your PC
- Ensure the Pixel device is listed
- Click the "Unlock" button under the "Bootloader" section. This step will reboot the device
- After device is turned ON, it could be that you'll need to setup everything again
- Open the "TOOL ALL IN ONE" again, and click the "Flash Factory Image" under the "Other options" section
- Click the "Download Stock Rom" button and find the image you need (it will bring the page based on connected device. In our case, the list of stock Android versions (OEM) will appear)
- Download the zip and without unzipping it, open the "TOOL ALL IN ONE" again
- While you're on the "Generic Fastboot Rom Flasher" dialog, click the "[...}" button
- Find the location where the zip is stored and click select
- Click "Flash Rom"
- After this step, the desired Androd version will be installed on a device
- As a final step, you should lock the bootloader by clicking the "Lock" button in the app
Devices supported by Google
Find them here.
If a device is not on this list, we usually cannot guarantee it is fully supported either.
Don't keep activities
A good practice when testing Android apps is to thoroughly circumnavigate the app with the "Don't keep activities" option enabled.
Users will usually not have this enabled, but it might point to issues that happen when the operating system decides to kill your activity, which it might do as part of its automatic memory management.
Enabling Developer options
You can unlock the Developer options on any Android smartphone or tablet by locating the Build number in your Settings menu and tapping it multiple times.
The exact location of the aforementioned build number may differ depending on your phone’s manufacturer.
Getting nice device logs
- You need to install pidcat:
brew install pidcat
- You need a debug build
- Device has to be recognized (open Terminal, input
adb devices
and verify that the device is listed) - You can start pidcat with
pidcat package.name
- You can find the package name by searching every installed app with
adb shell 'pm list packages'
(or with the Package name application).
How to download a beta version of the app on Android
- Login to G+ community for beta testing with your @infinum.com account
- Force close Google Play Store and delete the cache and data for it
- Run TunnelBear or some other VPN and connect through the country that you need to work on desired app.
- Run Google Play Store, accept the T&C and skip the payment info
- Find the app and install it (log in, activate it, etc.)
- Accept that you are a beta tester for the app (the best way is to open https://play.google.com/apps/testing/package.name.of.theapp/)
- Wait for a little while for the change to propagate to Google Play (mostly for a few minutes)
- Go to Google Play page for the app and update it
How to fake your location on Android
On Android you can use Lockito or FakeGPS.
As always, iOS is a bit more complicated and deserves a special guide.
Installing an app from a foreign Google Play
In 1Password we have stored Google accounts for different countries:
- Germany
- USA
- Austria
- Netherlands
- Slovenia
Steps:
- First, make sure that you have installed some VPN client on your test device
- Next, go to Settings —> (Users &) Accounts and delete all Google accounts
- Then go to Settings —> Apps —> Google Play Store —> Storage —> Clear data - basically, delete cache for Play store
- Open your VPN client and connect to an appropriate country
Login with appropriate Google account:
- Settings —> Users & accounts
- Agree with T&C and accept back up
Open Google Play Store
- Find your app & click Install
- When Complete account setup appears, click on Continue then Skip
After you’ve installed the app, delete the appropriate Google account & log in with our test account
That's it, enjoy testing your app!
Setting up an Android emulator
We usually do not test apps on emulators, but sometimes they can come in handy.
To get an Android emulator, do the following:
- Download Android Studio: https://developer.android.com/studio/index.html
- Install it and during installation make sure "Android Virtual Device" is selected
- Run Android Studio and do not import any settings
- Choose the Standard setup - the Emulator and SDK Tools will be downloaded automatically
- Start a new Android Studio project and use the default names if you wish
- Use the default form factors and minimum SDK settings (you won't be needing that)
- Use the default activity and activity name (you won't be needing that)
- Once Android Studio finishes loading the project, click on "Install missing platform and sync project" or "Install Build Tools" until Android Studio stops reporting error message at the bottom
- Go to View > Toolbar to get the toolbar
- On the toolbar click on the AVD Manager icon
- Click on Create Virtual Device...
- Select a device, e.g. Nexus 5X
- Click on Next
- Download a system image, e.g. API level 25
- Select a system image you just downloaded and click on Finish
- Click on the play icon to start the emulator
- Drag and drop the APK to the emulator
iOS
Enabling iOS dev settings
- Open Xcode and connect your phone with an USB cable
- Open Settings on your iOS device
- Scroll down and tap ON "Developer"
Installing an app from a foreign App store
In 1Password we have stored Apple accounts for different countries:
- Germany
- USA
Steps:
- First, launch App store & tap on Apple ID (your avatar in the top right corner) & Sign out
- Then, open Settings, tap on General --> Language & Region --> Region & select the appropriate country
- Open the App store & once again click on your avatar in the top right corner & sign in with the aprropriate Apple ID account
- Type in the app you are looking for
- Click on Get & enjoy testing your app
- Don't forget to repeat step #1 & sign in with our regulard test account
Jailbreaking iOS11 with Uncover
iOS jailbreaking is the privilege escalation of an Apple device for the purpose of removing software restrictions imposed by Apple on iOS, tvOS and watchOS operating systems.
It means giving apps admin (root) level access, which in term allows installation of other apps, tweaks and themes not on the App Store.
It supports the following iOS 11 versions: iOS 11, iOS 11.0.1, iOS 11.0.2, iOS 11.0.3, iOS 11.1, iOS 11.1.1, iOS 11.1.2, iOS 11.2, iOS 11.2.1, iOS 11.2.5, iOS 11.2.6, iOS 11.3, iOS 11.3.1 and iOS 11.4 beta 3.
Requirements:
- Make sure that you take a complete backup of your iPhone, iPad or iPod touch using iTunes
- Ensure that your device has enough battery level for the jailbreak process to complete
- Download the latest version of Uncover jailbreak IPA from Pwn20wnd’s Github page and Cydia Impactor from here on your Mac or PC. Cydia Impactor is available for macOS, Windows, Linux (32-bit and 64-bit)
Steps:
Install/Sideload Unc0ver IPA using Cydia Impactor
Connect your iPhone, iPad or iPod touch to the computer with the Lightning cable.
Launch Cydia Impactor on your computer. It will detect your iOS device. Drag the Uncover jailbreak IPA file you had downloaded earlier on to its UI. Enter the Apple ID and password for your Apple Developer account when prompted, and wait for Cydia Impactor to sideload the signed app on your iOS device.
Trust Developer Profile
Once Uncover jailbreak IPA is successfully sideloaded. Launch the Settings app, and navigate to Settings > General > Profile(s) & Device Management (in some iOS versions it may just be General > Device Management).
Tap on the entry with your Apple ID under Developer app, then tap on Trust. Tap on the Trust button. The status will change to Delete app.
Note: This step is not required if you’ve used an Apple Developer account.
Enable Airplane Mode
Disable Siri
Reboot iOS device
After your device has rebooted, ensure that Airplane mode is still enabled, and also ensure Wi-Fi is disabled if it hasn’t got disabled while enabling Airplane mode.
Run Jailbreak Process
Launch the Uncover Jailbreak app from the Home screen, and tap the blue Jailbreak button to start the jailbreak process.
Wait for Jailbreak to complete
Cydia should also be installed on your Home screen. You should be able to launch it and install the jailbreak apps and tweaks. If you don’t see Cydia on the Home screen, then reboot your iOS device and launch Uncover Jailbreak and keep trying until it works.
You can also disable Airplane mode and re-enable Siri after your iPhone, iPad or iPod touch has been successfully jailbroken.
The complete guide can be found here.
Jailbreaking iOS12 with Uncover
There are many reasons to jailbreak iOS 12 such as to install iFile, which gives access to the file system, tweaks like Message Customizer that lets you customize each aspect of the Messages app, ability to lock apps and folders using Touch ID and lots more. You don’t even need access to a computer for the entire process.
You can download the latest version of Uncover jailbreak from Github hosted by pwn20wnd.
You can download the latest version of Chimera jailbreak from here hosted by Electra team.
Requirements
- Delete any iOS 12 OTA update file from Settings -> Storage and reboot your device before attempting to jailbreak it. If an OTA file is present, your device will end up in a respring or reboot loop.
- Uncover jailbreak has complete support for Cydia and Substrate.
- Make sure to create a backup of all important data on your device before proceeding with the jailbreaking steps.
Steps:
Download Latest Uncover beta
Go to ignition.fun on your iPhone or iPad running iOS 12.4. Use the search bar to find Unc0ver. Tap on Get followed by Install. Wait for the app to download and install on your iOS device.
Trust Certificate
Head over to Settings-> General -> Device Management. Tap the developer name and trust the certificate. You will not be able to launch the Uncover jailbreak app on your iPhone/iPad without this.
Jailbreak iOS 12.4 using Uncover
Open Uncover, tap the Jailbreak button and wait for the app to do its job. Your iPhone or iPad will respring during the process after which you should see the Cydia icon on your home screen.
If the app ends up freezing, wait for a few minutes. In case that does not work, reboot your iPhone/iPad and then repeat the above steps.
Check out the whole guide here.
How to remove jailbreak on iOS
Sometimes you have done your testing on a jailbroken device and you want to return your device to how it was. In this case, factory reset won't help.
You can easily remove jailbreak from your iOS device if jailbreaking was done with Cydia and Uncover.
Steps:
- Open Uncover app on your jailbroken iPhone.
- Tap on Settings gear icon in the top-right corner.
- Search for Restore RootFS toggle button and enable it.
- Return back from settings and tap on Restore RootFS button on Home screen.
- If you encounter any message dialogs, just follow the steps from the dialog while the removal of jailbreak is in progress. Also, don't allow your device to lock its screen.
- Wait for the removal of jailbreak to complete.
- Device will restart and Cydia will no longer be installed on the device - jailbreak is now completed.
- You can now also uninstall the Uncover app.
Note: Maybe removing jailbreak won't work from first try and you will get some error dialogs. If this happens, follow the steps from the error dialog (e.g. Reboot device and try again).
For reference, you can take a look at this Youtube video.
Testing production builds on iOS
The file that actually gets uploaded to iTunes Connect cannot be installed on a development device since it first has to be signed by Apple.
This means that "Production" builds on Tryoutapps are not identical to what actually ends up on the App Store, only those that you get via Testflight are.
That's why you should make sure to always independently verify Testflight builds before they are published to users.
How to install iPhone or iPad apps on M1 Mac
New Apple desktop devices with M1 chips have a neat ability to install any iPhone or iPad apps on the M1 Macs. This is possible due to the common architecture shared by the two operating systems. The only way to install apps that are made for iPhone or iPad is to download them directly through the AppStore!
No porting required!
iPhone and iPad apps on the App Store are automatically available on the Mac App Store on Apple silicon Macs, there is no need to modify the app. The same frameworks that your apps use on iPhone and iPad are available and tuned just for Mac, taking advantage of the same shared architecture.
Here’s how you can download an app to install on an M1 mac:
- Open the Mac App Store and click your profile from the bottom-left of the page.
- Click ‘iPhone & iPad Apps’ tab situated under Account.
- Select the app from the list and click the download button.
Once the app downloads, you can access it from the Launchpad or the Applications folder.
Some of the apps that you see in the Mac App Store are labeled with a warning that says "Not Verified for macOS," these apps are not optimized for use on a Mac.
Mac App Store availability
By default, apps are published automatically on the Mac App Store. App availability can be managed at any time in App Store Connect.
Circumventing the App Store or "sideloading" apps
Apps could be downloaded and installed via a third-party app (sideloaded) but Apple has disabled this feature on M1 Macs running macOS Big Sur 11.1 or newer. Currently, the only way to install the iPhone or iPad app on the M1 machine is to download it through the App Store.