Test the protected, signed build that users will install, in three layers, and let the combined result decide the release. Functional tests prove the app still works. Performance tests compare the protected build with its unprotected baseline on the same devices. Security-behaviour tests confirm that each enabled control responds as configured on prepared devices and that your backend receives the signal. Plan for one tension from the start: test automation looks like the tooling runtime protection watches for. Otukan supports this for Android and iOS apps, including Flutter and React Native, and each platform is tested on its own.
Test layers
Each layer answers a different question, and no layer can stand in for another.
| Layer | Question it answers | Typical tests | Build under test |
|---|---|---|---|
| Functional | Does every feature still work? | Regression suite, sign-in, payments, deep links, push, in-app purchase | Protected and signed, test profile |
| Performance | Is the protected build acceptable to users? | Start-up, flow latency, memory, crash and ANR or hang rates | Baseline and protected, same commit |
| Security behaviour | Does each control respond as configured? | Prepared devices and tools, one control at a time | Protected and signed, release profile |
Functional tests
Run the full regression suite against the protected artefact. Pay attention to features that rely on inspecting the app itself or its environment: third-party SDKs, WebViews, app extensions on iOS, and the Dart or JavaScript layer in Flutter and React Native apps. Hybrid apps are covered layer by layer on Ionic and Cordova app security.
Performance tests
Compare like with like: baseline and protected builds from one commit, signed the same way, on the same physical devices. The full method is on mobile app protection performance.
Security-behaviour tests
For each enabled control, write down the expected response per flow and the expected backend signal, then trigger it.
| Condition to prepare | Expected in the app | Expected at the backend |
|---|---|---|
| Rooted Android device or jailbroken iPhone | The response set for that flow | A signal with the next sensitive request |
| Debugger attached to the app | The response set for that flow | A signal with the next sensitive request |
| Hooking tool injected | The response set for that flow | A signal, and the server’s decision logged |
| App running in an emulator or simulator | The response set for that flow | A signal marking the environment |
| Copy re-signed with a different key | The response set for that flow | A signal identifying the modified copy |
Android and iOS each get their own table, because the conditions and the tools used to create them differ.
Test environments
Automation and instrumentation
On Android, instrumented tests run inside the app’s process. On iOS, XCUITest drives the app from a separate runner process. Either way, the automation looks like outside control to a runtime check. Run most automated suites on internal builds with a test profile, and keep a smaller release-profile suite on clean devices.
Test-device exemptions
Exemptions let lab devices run automation without triggering responses. Keep them narrow and visible:
- scope each exemption to internal test builds and to named lab devices
- keep exemptions in the internal test profile only, never in the release profile
- add a pipeline check that refuses a release build whose profile is not the approved one
- review the exemption list when devices or team members change
Device sources
Physical devices you own give the cleanest results. Cloud device farms are useful for breadth, but some provide modified or instrumented devices, so confirm how their devices are set up before counting a result from them. Emulators and simulators suit functional checks only. Choosing test devices and setting release criteria are covered on security validation.
Release gates
Acceptance checklist
- the build under test is the protected, signed artefact, with the approved release profile
- the functional suite passed on every platform in the release
- performance stays within the criteria your team set, per platform
- each enabled control produced the expected response and backend signal
- no test exemption is present in the release profile
- results are stored with the build’s protection profile version and logs
Rollback
Stores do not let you replace a published binary. Halt the staged rollout on Google Play or pause the phased release on the App Store, then ship a corrected build with a higher version number.
Incident ownership
| Situation | First owner | Otukan’s role |
|---|---|---|
| A control blocks legitimate users | Release owner, with security | Reads the reports with you and adjusts the profile |
| Protected build crashes on some devices | Mobile team | Reproduces on matching devices with you |
| A signal is missing at the backend | Backend team | Checks the signal path with your engineers |
For how testing fits the release pipeline, return to mobile DevSecOps.