Android Testing

Unit tests in src/test run on your computer's JVM in milliseconds; instrumented tests in src/androidTest are built into a separate APK and run on a device or emulator, where the real framework exists. UI tests are usually instrumented tests that drive the screen. **src/test**: run by testDebugUnitTest, dependencies in testImplementation. The android.jar here is a stub, so framework calls throw. Home of business logic, ViewModels, mappers and repositories over fakes. **src/androidTest**: run by connectedDebugAndroidTest through AndroidJUnitRunner, dependencies in androidTestImplementation. A real Context, resources, SQLite and main Looper. Use it when the device is the subject: real UI, a permission dialog, a Room migration on a real file. Robolectric sits in between: instrumented-style APIs (ActivityScenario, Compose test rules) running in src/test on a simulated framework. Test pyramid: many fast unit tests at the base, fewer integration and Robolectric tests in the middle, a handful of end-to-end UI journeys on top. The higher the level, the slower and flakier the test, so most checks belong at the bottom.

A test double stands in for a real dependency in a test; the kinds differ in how much behaviour they have and whether the test checks how they were called. Prefer a fake for your own stateful collaborators; use a mock only to verify that a call happened. Dummy: fills a parameter and is never used. Stub: returns canned answers (getUser() always returns User(1)), no logic. Fake: a working, simplified implementation: an in-memory repository over a MutableStateFlow, an in-memory Room database. Mock: programmed with expectations; the test verifies the calls (verify { api.send(any()) }). Spy: wraps a real object, records calls and lets you override a few methods. A fake behaves like the real thing (save, then read, and the row is there), so tests assert outcomes instead of call sequences, and one fake is reused by every test. Tests built on mocks tend to break on refactoring that keeps behaviour and stay green when the behaviour breaks. Mocks are still the right tool to verify a side effect at a boundary (an analytics event sent, navigate() called) or to replace a third-party class you cannot implement cheaply.

One behaviour per test: arrange the input, perform one action, and assert on values so the failure message shows expected and actual. Shared setup goes in @Before, which runs before every test; @BeforeClass runs once per class and must be static. Name the behaviour: emptyCartTotalIsZero or a Kotlin backtick name, not testGetTotal. Names with spaces work in src/test; on a device they need API 30+, so instrumented tests stay camelCase. Given / When / Then, with the When on one line. Two actions mean two tests: the first failed assertion stops the test and hides the second half. Assert values, not booleans: assertTrue(total == 50) fails with a bare AssertionError; assertEquals(50, total) prints expected:<50> but was:<100>. JUnit's order is (expected, actual); Truth's assertThat(actual).isEqualTo(expected) cannot be swapped by mistake. No logic in tests: no if, no loops; use a parameterized test for a table of cases. Lifecycle: JUnit 4 creates a new test-class instance per test, and @Before/@After run around each one. @BeforeClass/@AfterClass live in a companion object with @JvmStatic and suit expensive setup such as starting a MockWebServer. Reusable setup and teardown go into a rule (@get:Rule), like a MainDispatcherRule.

TDD (test-driven development) is a loop: write a failing test, write the minimum code that makes it pass, then refactor with the test as a safety net (red, green, refactor). Its main payoff is design: code written test-first ends up with injected dependencies and small units. TDD works best on pure logic (ViewModels, use cases, validation, parsers, mappers). For layouts and animation it rarely pays off; screenshot tests cover that after the fact. Rules worth setting for a team: A test with every bug fix: first a test that reproduces the bug, then the fix, so the bug cannot return silently. Tests run on every pull request, and a red build blocks the merge. A disabled test gets a ticket and an owner. Dependencies are injected (constructor, Hilt), and business logic does not take a Context, so everything below the UI runs on the JVM. Fakes over mocks for your own interfaces, shared through a testFixtures source set or a :core:testing module. **No Thread.sleep, no real network, no real clock**: test dispatchers, MockWebServer or fakes, an injected Clock. Keep tests fast: JVM tests on every commit, device tests on merge.

In src/test the android.jar contains only method signatures, and every body throws RuntimeException("Method ... not mocked"), so the first framework call (Log.d, Uri.parse, TextUtils) fails. The cleanest fix is to keep framework calls out of the logic you test. Move the framework to the edge: hide Log, Uri or Resources behind an interface the class receives, and pass a no-op or a fake in the test. Robolectric: @RunWith(AndroidJUnit4::class) in src/test loads a real framework, so Log.d and Uri.parse work, at the cost of a few seconds of startup. Mock the static: MockK's mockkStatic(Log::class) with every { Log.d(any(), any()) } returns 0, cleaned up by unmockkAll(). Fine for one stubborn call, not as a habit. **unitTests.isReturnDefaultValues = true** makes every stub return 0, null or false instead of throwing. It silences Log, but Uri.parse(...) now returns null and tests can pass for the wrong reason: a last resort for legacy code.

Use RecyclerViewActions from espresso-contrib: actionOnItemAtPosition<VH>(3, click()) or actionOnItem<VH>(hasDescendant(withText("Order 42")), click()). A plain onView(withText(...)) fails for off-screen rows, because a RecyclerView only creates views for the visible ones. Scroll, then assert: RecyclerViewActions.scrollTo<VH>(matcher) brings a row into view so a normal onView(...).check(...) can see it. Duplicates: two rows with the same text make onView(withText(...)) throw AmbiguousViewMatcherException; matching through the list avoids it. A button inside a row: pass a custom ViewAction to actionOnItemAtPosition that finds the child by id and clicks it. **ListView or Spinner** (an AdapterView) uses onData(...), which matches the adapter's data rather than views. **Compose LazyColumn**: performScrollToNode(hasText(...)), then the usual finders.

UI Automator drives the whole device through the accessibility layer, so it can tap system dialogs, notifications, the launcher and other apps; Espresso only sees your own app's views. Use Espresso inside the app and UI Automator for anything outside it. Typical cases: a runtime permission dialog (if you only need the permission granted, GrantPermissionRule is less brittle than tapping it), the notification shade, a share sheet, pressing Home and coming back. Macrobenchmark journeys are also driven with UI Automator. No automatic synchronisation: it does not know when your app is idle, so you wait explicitly with device.wait(Until.findObject(...), timeout).

MockK is built for Kotlin: it mocks final classes, objects and extension functions out of the box and stubs suspend functions with coEvery and coVerify. relaxed = true makes every unstubbed call return a default (0, "", an empty list, a nested mock) instead of throwing. Strict by default: mockk<OrderRepo>() throws MockKException: no answer found for a call you did not stub. That catches unexpected calls; relaxed = true hides them, so prefer relaxUnitFun = true, which relaxes only Unit functions. **every vs coEvery**: the every { } block is not a suspend lambda, so a suspend call inside it does not compile; use coEvery, coVerify and coAnswers. Global mocks: mockkObject, mockkStatic and mockkConstructor change global state; undo them with unmockkAll() in @After or they leak into other tests. Arguments: slot<Order>() with capture(slot) when the assertion is about what was passed. Mockito today: since Mockito 5 the inline mock maker is the default, so final Kotlin classes mock without extra setup, and mockito-kotlin adds whenever and onBlocking for suspend stubs. MockK reads more naturally in Kotlin; Mockito is common in mixed Java/Kotlin codebases. Neither replaces a fake for a stateful dependency you own: a mock only repeats the answers you scripted.

StandardTestDispatcher queues coroutines until the test advances the scheduler; UnconfinedTestDispatcher starts them eagerly on the calling thread. viewModelScope runs on Dispatchers.Main, which does not exist on the JVM, so tests replace it with Dispatchers.setMain(testDispatcher). Virtual time: both dispatchers share a TestCoroutineScheduler, so delay(30_000) returns instantly. runTest uses the scheduler of a test Main dispatcher if one is set; any extra dispatcher must be built as StandardTestDispatcher(testScheduler) to share the same clock. Driving it: runCurrent() runs what is due now, advanceTimeBy(ms) moves the clock (a task scheduled exactly at the target waits for the next runCurrent()), advanceUntilIdle() runs everything. Standard by default: it lets you see intermediate states, like the Loading the user sees first. Unconfined suits collectors launched in backgroundScope and tests that only check the final value. If Unconfined is the only thing that makes a test pass, check that it asserts after the work has run. **Without setMain** the test fails with IllegalStateException: Module with the Main dispatcher had failed to initialize. Wrap setMain/resetMain in a MainDispatcherRule. **Hard-coded Dispatchers.IO** escapes the virtual clock; inject dispatchers.

toList() collects until the flow completes, and a StateFlow never completes, so the test suspends until runTest gives up (after 60 seconds by default). Collect a hot flow with Turbine's test { } or in backgroundScope, and assert one emission at a time. The expected list is wrong too: a StateFlow replays its current value, so the first element is the initial state, and it keeps only the latest value, so start collecting before search() or intermediate states can be conflated away. Turbine: flow.test { } collects in a background coroutine. awaitItem() suspends for the next value, expectNoEvents() asserts silence, cancelAndIgnoreRemainingEvents() ends early. Unconsumed events fail the test, and a missing one fails after a timeout (3 seconds by default) instead of hanging. Without Turbine: launch vm.state.toList(list) in backgroundScope with UnconfinedTestDispatcher(testScheduler); backgroundScope is cancelled when the test ends. toList() is fine for cold flows that complete, such as flowOf(...).

withContext(Dispatchers.IO) moves the work to a real thread pool the test scheduler does not control, so advanceUntilIdle() can return before the result is back and the assertion races the IO thread. Inject the dispatcher and pass a test one in the test. Why it races: advanceUntilIdle() runs only tasks queued on the TestCoroutineScheduler. When the launch reaches withContext(Dispatchers.IO), that queue is empty, so the call returns; the continuation is posted back to Main later from the IO thread. Whether that happens before assertTrue depends on thread timing: green on a laptop, red on a busy CI runner. Virtual time is lost too: a delay() inside the IO block waits in real time, and runTest does not wait for viewModelScope, because it is not a child of the test. The fix: take a CoroutineDispatcher in the constructor (default Dispatchers.IO, or a qualified one from Hilt) and pass StandardTestDispatcher(testScheduler). The IO work now runs on the same scheduler, so advanceUntilIdle() drains it and the test is deterministic. Catch it next time: Detekt's InjectDispatcher rule flags hard-coded Dispatchers.IO/Default in production code; repeat a suspicious test until failure in the run configuration to expose the race locally.

The assertion sits inside collect, so it runs only if an event arrives: when pay() emits nothing, no assertion executes and the test passes vacuously. Collect the events into a list (or use Turbine) and assert after the action, so a missing event fails. Why it stays green: a test fails only when something throws. collect on a SharedFlow never completes; the collector lives in backgroundScope, which runTest cancels silently at the end. "No event" and "the right event" end the same way. The fix: record every emission and assert on the whole list after advanceUntilIdle(). A missing event now fails the assertion (expected a list with OpenReceipt, got an empty list), and a duplicate emission fails too. With Turbine: vm.events.test { vm.pay(); assertEquals(Event.OpenReceipt, awaitItem()) }. awaitItem() fails after its timeout (3 seconds by default) if nothing comes, and unconsumed events fail at the end of test { }. Same trap elsewhere: assertions inside callbacks, observeForever observers or onEach blocks prove nothing when the callback never fires. Catch it next time: watch every new test fail once: break the production line and check the test goes red (the red step of TDD). Mutation testing (PIT) automates that check across the suite.

Build each piece yourself: SavedStateHandle(mapOf(...)) is a public constructor, strings come through an injected interface instead of a Context, and TestLifecycleOwner lets the test move the lifecycle. None of it needs a device. **SavedStateHandle**: SavedStateHandle(mapOf("orderId" to 42L)) seeds navigation arguments or restored state. Read handle["query"] back to prove the ViewModel wrote what must survive. Process death: only the saved-state bundle survives, so build the ViewModel from the same map and assert it comes back correct; ActivityScenario.recreate() keeps the ViewModel and proves nothing here. **Context**: wrap resource lookups in a StringProvider interface. If you truly need resources, use ApplicationProvider.getApplicationContext() under Robolectric; a mocked Context returns null for every call you did not stub. Lifecycle: TestLifecycleOwner from lifecycle-runtime-testing: set currentState = STARTED and DefaultLifecycleObserver callbacks and repeatOnLifecycle blocks run. Without it nothing moves the lifecycle in a unit test.

Robolectric runs the real Android framework code on the JVM and replaces native and system-service parts with "shadows", so tests that need a Context, resources or an Activity run in src/test in seconds. It is not an emulator: it does not prove rendering, OEM behaviour or real system services. What you get: a real Context, resource loading and layout inflation, SharedPreferences, Room over SQLite, and the instrumented-test APIs (ActivityScenario, Espresso, Compose rules) under @RunWith(AndroidJUnit4::class). **A paused main Looper** (the default since 4.4): posted work runs only when you call shadowOf(Looper.getMainLooper()).idle(), which keeps ordering deterministic. No real drawing unless @GraphicsMode(NATIVE) is on, so measured sizes and text layout are not evidence. Shadows are approximations: ShadowConnectivityManager returns what you set; it does not behave like a radio dropping a request. AOSP only: no OEM permission quirks, background limits, real process death or performance numbers. Cost: it downloads an android-all jar per SDK level and starts slower than a plain JVM test; every extra @Config(sdk = [...]) level runs the class again. Rule of thumb: Robolectric for framework-touching logic; a device for anything you would screenshot, measure or blame on the OS.

Before every perform and check, Espresso waits until the main thread's message queue is idle and all registered IdlingResources are idle. Work on other threads (OkHttp, Dispatchers.IO, your own executor) is invisible to it, so register an IdlingResource or replace that work with a fake. Covered for free: posted messages, layout passes, LiveData and flows collected on Dispatchers.Main. Not covered: background threads, WorkManager, a postDelayed far in the future. The symptom is a NoMatchingViewException on a screen that was one frame from correct. **CountingIdlingResource**: increment() before the work, decrement() in finally, registered through IdlingRegistry in @Before and unregistered in @After. Miss the finally on the error path and every later test fails with IdlingResourceTimeoutException after the idle timeout. Endless animations such as an indeterminate ProgressBar keep the queue busy: testOptions.animationsDisabled = true turns off system animations; turn your own looping ones off in tests. **Thread.sleep** makes every run slower and is still flaky. The best option is often no idling resource at all: inject a fake repository or a synchronous executor, so the work is finished before perform() returns.

Compose finders search the merged semantics tree by default, and Button merges its children into one node. A child's testTag does not survive the merge, so no node in that tree carries "price". Use useUnmergedTree = true or assert on the merged button node. Merging: Button, Modifier.clickable and Modifier.semantics(mergeDescendants = true) collapse a subtree into one node whose text is the list of all child texts. That is the node TalkBack reads. Fix 1: onNodeWithTag("price", useUnmergedTree = true) finds the child. Fix 2: tag the button and assert on the whole node: assertTextEquals("Socks", "12.00 €"), or simply onNodeWithText("12.00 €"), which matches the merged node. Debugging: onRoot(useUnmergedTree = true).printToLog("TREE") prints every node with its properties; compare it with the merged tree.

Test the worker's logic directly with TestListenableWorkerBuilder and call doWork(); test the scheduling with WorkManagerTestInitHelper, a SynchronousExecutor and the TestDriver, which lets you declare constraints and delays as met. Logic: TestListenableWorkerBuilder<UploadWorker>(context).setInputData(...).build(), then doWork() and assert the Result (success, retry, failure). No scheduler is involved; setWorkerFactory(...) passes dependencies, for example to a @HiltWorker. Wiring: initializeTestWorkManager(context, config) with a SynchronousExecutor, enqueue the request and check it is ENQUEUED, then getTestDriver(context)!!.setAllConstraintsMet(id). The work runs synchronously, so the state is already SUCCEEDED. setInitialDelayMet and setPeriodDelayMet do the same for delays and periodic work. Where it runs: both need a Context, so run them under Robolectric or as instrumented tests, with the work-testing artifact. Keep the worker thin: let it delegate to an injected class, test that class on the JVM, and use the scheduler harness only to prove the wiring.

A screenshot test renders a screen, compares it pixel by pixel with an approved "golden" image and fails on a difference. The tools differ in what renders the pixels (layoutlib on the JVM, Robolectric's native graphics on the JVM, or a real device), and stability follows from that choice. The options Paparazzi renders with layoutlib, the engine behind Android Studio previews. No emulator, very fast, device settings are data (DeviceConfig, fontScale, locale, night mode). No real framework, though: no interaction before the shot, and text rendering is layoutlib's, not a device's. Roborazzi captures from Robolectric with @GraphicsMode(NATIVE), so you can click into a state with Compose or Espresso APIs first. Slower than Paparazzi, still no emulator; recordRoborazziDebug writes goldens, verifyRoborazziDebug fails the build. Compose Preview Screenshot Testing, the first-party AGP plugin, renders your @Preview functions with no test code: least work if previews are the states you want to pin. On device gives the highest fidelity and the worst stability: the golden depends on the exact system image. What keeps a golden stable Pin the renderer: one library version, one API level, one JVM; record goldens in the same CI container that verifies them, never on a laptop. Pin the inputs: locale, timezone, fontScale, density, night mode, a bundled font, a fixed Clock and seeded data, so "2 minutes ago" never changes. Freeze motion and I/O: no infinite animations or blinking cursor; images from resources or a fake image loader. Keep the tolerance near zero: a threshold large enough to hide anti-aliasing noise also hides real regressions. Goldens catch what semantics assertions cannot: spacing, truncation, contrast, RTL. Review image diffs like code.

Most flakiness comes from four sources: timing (animations, sleeps, unsynchronised background work), real external dependencies (network, clock), state shared between tests, and an unpinned device. Measure failure rates per test first, because the pattern tells you which source it is. Find the cause Record every run per test: name, result, duration, device. One test failing across runs is a race in that feature; many tests failing in one run is the environment; a test that fails only after another one is shared state. Run one suspect alone, many times: filter with -Pandroid.testInstrumentationRunnerArguments.class=.... Never fails alone but fails in the suite means order and state, not timing. Turn on Android Test Orchestrator with clearPackageData: each test gets a fresh process and wiped app data. If the flake disappears, it was shared state. Fix by cause Timing: testOptions.animationsDisabled = true, your own looping animations off in tests, every Thread.sleep replaced by an IdlingResource or Compose waitUntil. External dependencies: MockWebServer or fakes instead of staging, an injected Clock. Shared state: Orchestrator, in-memory databases, a fresh account or seeded data per test. The device: Gradle Managed Devices with a pinned system image (ATD images for headless CI) instead of whatever the runner has. Make failures reproducible A TestWatcher that on failure saves a screenshot, the view or semantics hierarchy and recent logcat. Many "cannot reproduce" flakes turn out to be a system dialog, a lock screen or an ANR. Retries only with a quarantine list and a ticket per entry; an auto-retried test nobody owns stops testing anything.

Line coverage records which lines ran, not which behaviours were checked, so a suite that asserts nothing can reach 80%. Gate on coverage of the lines a change touches, keep the overall number from falling, and use mutation testing to prove the assertions are real. What the number is not A test that calls a method and asserts nothing scores the same as one that pins every case. Coverage only tells you reliably what is untested. On Android it is inflated further: generated code counts unless excluded (Hilt components, *_Factory, serializers, data class methods), and one instrumented tap executes thousands of lines across the stack. Tooling: Kover (org.jetbrains.kotlinx.kover) understands Kotlin's synthetic code better than raw JaCoCo; AGP's switches are enableUnitTestCoverage and enableAndroidTestCoverage per build type. Gates worth having Patch coverage plus a ratchet: require coverage on lines changed in the PR and forbid the module number from falling (koverVerify). Pressure lands where behaviour changes. A regression test with every bug fix: cheap, and impossible to game. Mutation testing where bugs cost money: Pitest flips > to >=, removes a call, negates a condition and reruns the tests; a surviving mutant is a line that runs but is not checked. Too slow for every PR, so run it nightly on pure Kotlin domain modules. The shape of the suite: many JVM tests, a solid middle of Robolectric and Compose tests, a few end-to-end journeys as smoke tests. The honest answer to "80% everywhere": gate hard on modules where a bug costs real money (payments, sync, auth), and cover presentational modules with screenshot tests instead of a percentage.