When we started building mobile testing into CodeReviewer, the obvious move was emulators. They’re easy to spin up, easy to reset and easy to run in the cloud. We decided against them, and the reason is simple: your users don’t run your app on an emulator.
The bugs that only real phones show
Anyone who has shipped an Android app has a collection of bugs that never showed up in development:
- Manufacturer skins. Some manufacturers change system dialogs, add their own battery managers, or kill background processes aggressively. An app that behaves perfectly on stock Android can lose its session, miss a notification or fail to relaunch on a popular mid-range phone.
- Real performance. An emulator on a fast workstation is often quicker than the budget phones many of your users own. Cold starts, list scrolling and image-heavy screens behave differently on real hardware.
- Keyboards and screen sizes. A soft keyboard that covers a submit button, or a layout that clips on a narrow screen, is easy to miss when every test runs at one comfortable resolution.
- Permissions and system prompts. Location, camera and notification prompts look and behave differently across Android versions and manufacturers. Flows that pass through them need testing where they actually appear.
Emulators are useful during development. As the final check before release, they give you false confidence.
Why most teams don’t test on real phones
The usual answer is a device farm: rent time on phones in someone else’s data center. It works, but it’s expensive, and the devices are shared and reset between sessions. It’s hard to keep an app installed, signed in and in a realistic state.
The alternative is your own phones on a desk. Traditionally that meant a laptop tethered over USB, developer tooling installed, and often a rooted phone for full automation. That set-up is fragile and hard to maintain, and it keeps one engineer permanently in charge of the “test bench”.
We wanted something between the two: your own real phones, with none of the tethering.
How pairing works
A phone joins your CodeReviewer workspace in a few minutes:
- Install the CodeReviewer Agent by scanning a QR code with the phone’s camera. You don’t need an app store or a cable.
- Work through the checklist. The agent checks its own requirements and gives you a Fix button for each: turn on accessibility (this is how it drives tests), allow notifications, and let it run in the background.
- Scan the pairing code. In your workspace, add a device and scan the QR code it shows. Within seconds the phone appears as online.
No root is required. The agent uses Android’s standard accessibility features to read the screen and tap, type and scroll, the same capabilities assistive apps use. The phone connects outward to your workspace, so it can sit on office Wi-Fi behind a firewall with no inbound ports opened.
We recommend dedicated test phones. Because the agent can read the screen and perform taps, which is what makes testing possible, it shouldn’t run on someone’s personal phone.
From upload to unattended replay
With a phone paired, mobile suites work the same way as web suites:
- Upload your build and choose the feature to cover.
- CodeReviewer reads the code, drives the app on the real phone and records a suite of test cases.
- Suites replay on a schedule or on demand, unattended, on the phone you choose.
- When the app changes, broken tests are repaired and re-verified. Flaky ones are benched so they don’t hide real failures.
Each run keeps its evidence: the steps taken, where it failed and what the screen showed, so a failure reaches your developers ready to reproduce.
One honest limitation
Android deliberately protects the Install button on its system installer from automation, as an anti-malware measure. Without root, nothing can tap it for you. In practice this matters once per phone: you tap Install by hand when you first put your app on the device, and every replay after that reuses the installed app unattended. Suites that need a completely fresh install can opt in, knowing that step needs a person.
We’d rather explain that clearly than pretend a phone can be fully automated when it can’t be without root.
Why it matters
Mobile is often the least-tested surface a team has, even though customers use it most. Putting real-device testing on the phones you already own removes the two biggest barriers, cost and set-up friction. When your users report a bug from the field, the phone that can reproduce it is already on your desk and already running your suite.