Lesson 10 of 11Advanced19 min

Testing Mobile Applications

By the end of this lesson

Test across devices, form factors and network conditions.

Testing a mobile app means testing the parts of it you cannot see from your desk. The logic is the easy half: pure functions, resolvers, the state tiers from earlier in this course — all of it tests quickly and reliably with no device involved, and you should have that coverage. The hard half is the environment, and the environment is where your users' problems come from.

The environment is a specific device with a specific amount of memory, on a specific network, at a text size somebody chose, with a camera that takes longer to open than yours does. An emulator or simulator is a good approximation of your code and a poor approximation of that. It is fast, free, scriptable, and excellent for checking layout and flow, which is exactly why it is easy to mistake for sufficient.

The pattern to expect: bugs found on real hardware are the ones users report, and bugs found in an emulator are the ones you would have found anyway. That is not an argument against emulators. It is an argument for spending some of your testing time where the reports come from.

Where an emulator and a real device genuinely diverge. Layout and navigation behave much the same, so those are absent from this list:

	Emulator or simulator	Real device
Performance	Runs on your development machine's processor and memory, so it is frequently faster than any phone you ship to.	A three-year-old mid-range device has a slower processor, less memory and slower storage. Animations drop frames and startup takes noticeably longer.
Memory pressure	Generously provisioned and lightly loaded, so your backgrounded app is rarely reclaimed.	Competing with a mapping app, a browser and a camera. Termination while backgrounded is routine, which is exactly the case the state lesson covered.
Camera	A synthetic image or your laptop webcam. Opens instantly and never fails.	Takes real time to initialise, needs focus and light, and the permission dialog behaves differently the second time.
Notifications	Often deliverable only through a local test mechanism, bypassing the real path.	Travels through the real push service, so delivery delay, batching and lock screen presentation are visible for the first time.
Network	Your office connection, fast and stable, with a wired path to the internet.	A mobile radio that hands over between cells, loses signal in a lift, and sometimes holds a connection open delivering nothing.
Battery and heat	No battery to drain and no thermal limit to reach, so nothing tells you a screen is expensive.	Power use is measurable, power-saving mode changes behaviour, and a warm device is throttled by the system.
Touch	A mouse click at a precise pixel, with hover available whether you intended it or not.	A fingertip covering several millimetres, one-handed, sometimes gloved, on a screen with a fingerprint smear across it.

A device set worth testing on, and why each entry earns its place. This is a small list on purpose — a matrix nobody has time to run is worse than three devices somebody actually uses:

The oldest and lowest-specification device you support. This is where slow startup, dropped frames and background termination show up, and it is where your users find the problems you did not
A small screen. Layouts designed on a large handset clip, wrap awkwardly, or push the primary action out of thumb reach when the screen gets shorter
A large screen or a tablet, if you claim to support one. A layout stretched to fill it often looks unfinished rather than broken, which is its own kind of bug
One device from each platform, at the oldest operating system version you support. Platform behaviour around permissions, notifications and background work changes between versions, and old versions stay in use for years
Any device at the largest text setting you support. Treat this as a device in its own right, because it breaks different things from a small screen
A device with a display cutout and a gesture-navigation bar, since safe area mistakes are invisible on hardware without them
A device in power-saving mode, which restricts background work and can stop the behaviour you were relying on

A seam that lets every screen be tested under a named network profile

TypeScript

export type NetworkProfile = "fast" | "slow" | "lossy" | "offline";

export interface Transport {
  send(path: string): Promise<string>;
}

export class OfflineError extends Error {}

const PROFILES: Record<NetworkProfile, { delayMs: number; failureRate: number }> = {
  fast: { delayMs: 40, failureRate: 0 },
  slow: { delayMs: 4500, failureRate: 0.05 },
  lossy: { delayMs: 1500, failureRate: 0.4 },
  offline: { delayMs: 0, failureRate: 1 },
};

/** Wraps the real transport so a whole flow can be exercised under one profile. */
export function withProfile(
  inner: Transport,
  profile: NetworkProfile,
  random: () => number,
): Transport {
  const { delayMs, failureRate } = PROFILES[profile];

  return {
    async send(path: string): Promise<string> {
      await new Promise((resolve) => setTimeout(resolve, delayMs));
      if (random() < failureRate) {
        throw new OfflineError("Connection lost part-way through");
      }
      return inner.send(path);
    },
  };
}

The shape is the transferable part. Your HTTP client will offer its own interception point, and the names will differ; what matters is that every request in the app passes through one place you can wrap.
That single seam is what makes this worth building. Without it, testing a slow network means changing code per screen, so it gets done for the screen somebody was worried about and nowhere else.
The delay happens before the failure, deliberately. The bug worth finding is the one where the user waits four seconds and then gets an error, or waits forever because nothing cancels. Failing instantly produces a tidy error path that hides both.
random is injected rather than called directly, which turns the lossy profile into something deterministic. A test that fails one run in ten is a test people re-run until it passes, and it stops carrying information.
OfflineError is a distinct type rather than a message string, so the app can tell a missing connection from a server rejection, as the first lesson in this course argued.
The honest limitation: this exercises your code paths, not the platform's. It will not reproduce a radio handover, a stalled connection that neither completes nor fails, a slow name lookup, or the operating system dropping your socket when the app backgrounds. Use it for coverage in tests and throttle a real device for the rest.

A manual pass before a release, on real hardware. Half an hour, in this order, finds more than an afternoon of emulator work:

Run the main flow on the oldest device you support
Cold start it, sign in, open a visit, write a note, attach a photo. Watch the timings rather than only the outcomes: a four-second launch and a camera that takes two seconds to appear are both findings, and neither shows on your desk.
Turn the text size to its largest supported setting
Walk the same flow. Look for clipped labels, truncated buttons, overlapping rows and anything inside a fixed height. This is the cheapest accessibility check on mobile and it finds real breakage nearly every time, because it breaks later than it is designed.
Throttle the network, then make it worse
Both platforms offer a way to simulate a slow or lossy connection, and network-level throttling tools do the same job for any device. Load a screen on a slow profile and check that something honest appears within a couple of seconds and that a timeout eventually resolves the screen rather than leaving a spinner.
Cut the connection mid-request
Start a photo upload or a note submission and switch the device to airplane mode while it is in flight. This is the code path nobody exercises and users hit weekly. Check what the user is told, that nothing claims success, and that the queued work is still there when connectivity returns.
Force a process kill while backgrounded
Half-write a visit note, background the app, end the process with the platform's development tooling rather than by swiping the app away, then relaunch. Whether the engineer can continue is the whole test. Swiping away is a different event, so use the tool that actually simulates termination.
Interrupt it the way life does
Take an incoming call mid-form. Switch away for five minutes and come back. Revoke the camera or notification permission in device settings while the app is running. Enable power-saving mode. Each one takes seconds and each has its own failure mode.
Install both ways
Test a fresh install and an upgrade over the previous released version, with real stored data from that version. An upgrade path that mishandles an older local store loses the user's queued work, and a fresh install never touches that code.

Common mistake

Testing in conditions your users never have

The default test environment is the newest device the team owns, on office wifi, with a debugger attached, running a debug build, with default text size, signed in already, holding data that was fetched successfully a minute ago. Almost nothing about that describes a working day in the field.

Newest device only. Your app feels quick on it because it is quick on it. The engineer with a three-year-old phone gets slower startup, more dropped frames, and a backgrounded app that is terminated while they take a photo.

Fast, stable network only. Every loading state, timeout, retry and offline path in your app is untested, which means the code you wrote specifically for bad conditions has never run outside a unit test. That is the code that most needs a real trial.

Debugger attached. A debug build behaves differently: timing shifts, some optimisations are off, and the system is far less likely to reclaim your process. A release build on a busy device is the thing to test.

Default accessibility settings only. Your app then works for everyone except the people who changed a setting because they needed to, and the breakage tends to be invisible to you and total for them.

Happy path only. The interesting cases are a permission declined, a session expiring mid-form, a record deleted by somebody else while the user has it open, a duplicate tap on a slow device, and a deep link to something that no longer exists.

Fixing this is not a large programme of work. One old device, one throttled network, one largest-text pass and one forced process kill, run before each release, changes what reaches your users.

Trade-off

Automated UI tests: useful, and more expensive than they look

An automated UI test drives your app the way a user does — tapping, typing, asserting what is on screen. It catches the regression that unit tests cannot see, such as a flow that no longer reaches its final screen, and it is the only automation that tests the app as assembled.

The cost is brittleness, and it is worth being plain about it. These tests are slow, so a suite that covers a lot takes tens of minutes. They depend on timing, so an animation, a slow network or a busy machine turns a pass into a failure. They depend on the layout, so a redesign that changes nothing functionally breaks a batch of them. And they depend on the environment: a permission dialog, a system update prompt or a keyboard appearing at a different moment is enough to fail a run.

The real damage from a flaky suite is not the time spent re-running it. It is that people stop believing failures. Once a red build is assumed to be flakiness, a genuine regression walks straight through, and the suite is then worse than having none because it costs money and provides false assurance.

A proportionate approach: automate a small number of flows that must never break — sign in, open a visit, save a note offline and see it sent — and keep the majority of your coverage in fast tests of logic, resolvers, state transitions and the network layer. Use test identifiers on elements rather than user-visible text so wording changes do not break tests. Wait on conditions rather than on fixed durations. Control the environment by stubbing the network at the seam above, so a test failure means your app changed and not that a server was slow.

And treat flakiness as a defect in the test. Quarantine it, diagnose it, fix or delete it. Wrapping it in automatic retries until it passes converts a real intermittent bug into a green build, which is the outcome you least want.

When to skip it: an app with two screens and a small team gets more value from thorough logic tests and a disciplined manual pass than from a UI suite nobody maintains. Automate the flows whose breakage you would find out about from a support call.

Summary

Emulators test your code well and the environment poorly, so performance, camera, notifications, memory pressure and battery all need real hardware
Test on the oldest low-specification device you support, at a small screen size, and at the largest text setting — that is where users find problems
Simulate slow, lossy and absent connections, including cutting the network mid-request, because those code paths are otherwise never exercised before release
Force a process kill while backgrounded and check the user can continue, and test an upgrade over real data from the previous version
Automated UI tests catch assembled-app regressions and are slow and brittle, so automate a few critical flows, keep logic coverage fast, and treat flakiness as a defect rather than retrying it away

Practice

Attempt each one before opening the solution. Getting it wrong first is how the idea sticks.

Try it yourself

Five minutes of deliberate sabotage

Take an app you are building, or any app with a form, on a real device.

Do four things. Start a submission and switch to airplane mode before it completes. Set the text size to the largest option and revisit the same screen. Background the app mid-form for five minutes with a camera and a browser open, then return. Revoke a permission the app uses in device settings while it is running, then use the feature that needs it.

Write down what the app told you in each case, and what it did with your input.

Show solution

Most apps fail at least two of these, and the failures follow a pattern. The mid-flight disconnection usually produces either a spinner that never resolves or a generic error that does not say whether the submission happened — and not knowing is worse for the user than a clear failure, because their next move is to submit again and risk a duplicate.

The largest text setting usually clips something inside a fixed-height row, for the reason the UI lesson gave: touch targets are checked during design and stay correct, while dynamic type breaks later when somebody adds a tidy fixed height.

The five-minute background is the one that separates careful apps from the rest. If the form is empty on return, the app was holding it in memory, and the fix is the tiered persistence from the state lesson rather than anything to do with this screen.

Revoking a permission mid-session is rarely handled, because the code assumes a permission granted once stays granted. The result is often a crash or a dead button with no explanation. The user needs to be told what is switched off and where to change it.

The reason this exercise is worth your time: none of these needed a test framework, a device lab or a plan. Four deliberate interruptions on real hardware surface the same defects that arrive later as support calls, and you found them in five minutes.

Think about it

Spend a small automation budget

You have room for roughly two days of automated test work on the field app, and a suite that must stay green to be useful.

Decide what you automate and what you leave to a manual pass. The app has sign-in with token refresh, a visit list that works offline, a note form that queues when there is no connection, a photo upload, deep links from notifications, and a settings screen.

Then say what you would do about a test that fails roughly one run in five.

Show solution

A defensible split. Automate as fast logic tests: the deep link resolver, the token refresh rules including the single-flight guard, the queue's ordering and retry behaviour, and the state persistence and restore. These are pure functions or close to it, they run in seconds, and they cover the logic most likely to break silently.

Automate one end-to-end flow, not six: sign in, write a note with the network stubbed offline, restore the connection, and assert the note reached the fake server. That single test covers the app's reason to exist, and it will tell you when the wiring breaks.

Leave to a manual pass anything that depends on real hardware — camera behaviour, notification delivery and lock screen presentation, performance on an old device, the largest text setting, forced process kill. Automating these is possible and it is not two days of work, and the automated version still would not reproduce what the device does.

The settings screen is a reasonable thing to leave uncovered. Weigh what a breakage costs: a broken toggle is noticed and reported, a broken queue loses work an engineer believes is saved. Test budget should follow consequence, not screen count.

For the test failing one run in five: treat it as a defect and stop it blocking anybody, by quarantining it out of the main suite the same day. Then diagnose it, because an intermittent UI failure is frequently telling you about a real race in the app rather than a flaw in the test. Adding automatic retries until it passes is the one response to avoid — it hides both the flaky test and the race underneath it.

Saved in this browser only.