Irbisa · cheatsheetSeptember 13, 2026

Analytics & Experiments

Event taxonomy, funnels, crash-free rate, ANR, tracing, remote config, A/B tests, sampling

Middle Developer16 itemscompressed for a skim

01
Your event list has grown to checkout_button_clicked, cart_button_clicked and eighteen more like them. What is wrong with that taxonomy?
Easy
Those twenty names are one event with a property, and splitting them across names turns every question about buttons into a twenty-way union.
```
// ❌ One name per button — the vocabulary grows with the UI
analytics.log("checkout_button_clicked")
analytics.log("cart_button_clicked")
analytics.log("profile_save_button_clicked")

// ✅ One event, the varying part in properties
…
```
02
Three new screens shipped without a screen-view event because someone forgot the line in onResume — where should that instrumentation actually live?
Easy
In the navigation layer, registered once: the router already knows every destination and every transition, so tracking there cannot be forgotten by the next person who adds a screen.
```
// One registration in the single Activity — new destinations are tracked for free
@Composable
private fun TrackScreenViews(nav: NavHostController) {
  val analytics = LocalAnalytics.current
  LaunchedEffect(nav) {
    nav.currentBackStackEntryFlow
…
```
03
Product asks which checkout step people drop off at, and your events cannot answer it. What has to be on every event for that funnel to compute?
Medium
A funnel is a join, and the join needs the same identity, the same ordering key and the same flow instance on every step — step names alone give you five unrelated counts.
```
// One flow instance = one id, minted at the entry point, carried to the outcome
class CheckoutFunnel(private val analytics: Analytics, private val clock: Clock) {
  private var checkoutId: String? = null
  private var seq = 0
  private val fired = mutableSetOf<String>()
…
```
04
A trainful of users spends forty minutes in a tunnel and force-quits the app at the end of it. What has to be true for those events to still show up?
Medium
The event has to be on disk before track() returns, and the queue has to survive process death, retry with backoff and flush on background — an in-memory list loses the whole session.
```
@Entity(tableName = "events")
data class QueuedEvent(
  @PrimaryKey val insertId: String,   // minted here, persisted with the row, stable across retries
  val name: String,
  val payload: String,
  val clientTs: Long,
…
```
05
A user's device clock is two days fast and their events land in the future. Which timestamp does the analysis use, and what does the wrong choice do to a retention chart?
Medium
Send both — client time for ordering within a session, server receive time for anything you count or cohort by — and correct the client clock using the skew you already measured on upload.
```
data class Batch(
  val events: List<QueuedEvent>,
  val clientUploadTime: Long          // stamped at upload, not at track
)

@Volatile var skewMs: Long = 0        // positive means this device runs fast
…
```
06
A batch upload times out, the client retries it, and purchases jump six percent overnight. How do you make retries safe when the numbers only need to be roughly right?
Medium
Mint an idempotency key with the event rather than with the request, and let the server dedupe on it — at-least-once delivery plus a stable id is the only combination that is neither lossy nor inflated.
```
// ✅ The id belongs to the event: minted once, persisted with it, reused on every retry
fun log(name: String, props: Map<String, Any?>) = dao.insert(
  QueuedEvent(
    insertId = UUID.randomUUID().toString(),
    name = name,
    payload = json.encodeToString(props),
…
```
07
The dashboard shows 99.9% crash-free sessions and 99.5% crash-free users for the same release. Why do they differ, and which one do you alert on?
Medium
Different denominators: a session counts once, while a user is marked as crashed for the whole reporting window if any single one of their sessions crashed — so the user number is always the harsher of the two.
```
data class SessionRow(val userId: String, val version: String, val crashed: Boolean)

// The two numbers differ only in what you count distinct on
fun crashFreeSessions(rows: List<SessionRow>): Double =
  1.0 - rows.count { it.crashed }.toDouble() / rows.size
…
```
08
Crashlytics says 99.9% crash-free, but the store reviews all say the app freezes. Where do those freezes show up if they never arrive as crashes?
Medium
A freeze is the OS deciding your process is unresponsive, so the report comes from the platform rather than from your crash handler — Play vitals and ApplicationExitInfo on Android, MetricKit and the Xcode Organizer on iOS.
```
// A hang detector: the main looper is healthy only if it can still answer
class MainThreadWatchdog(
  private val thresholdMs: Long = 4_000,
  private val onHang: (Array<StackTraceElement>, Long) -> Unit
) {
  private val main = Handler(Looper.getMainLooper())
…
```
09
A support ticket says an upload silently failed on one phone, and the crash dashboard shows nothing. What do you record so the next one is reproducible?
Medium
Report the caught failure as a non-fatal with a stable exception type, and surround it with breadcrumbs and custom keys — a stack trace without the state that produced it is not reproducible.
```
class UploadFailed(val step: Step, cause: Throwable) : Exception("upload failed", cause)

// WRONG - the order id is in the message, so grouping forks one bug per user
suspend fun uploadBad(order: Order) = try {
  api.upload(order)
} catch (e: IOException) {
…
```

A tap feels slow and ends in a backend span, but the app trace and the server trace are two unrelated traces in two tools. How do you join them?

Medium

Propagate trace context on the wire: the client opens the root span at the tap and injects a traceparent header, so the server's span becomes a child of yours instead of the root of its own trace.

// Root span at the interaction - every child below shares this trace id
fun onCheckoutTap(scope: CoroutineScope) {
  val span = tracer.spanBuilder("checkout_tap").setSpanKind(SpanKind.CLIENT).startSpan()
  span.setAttribute("cart.items", cart.size.toLong())
  // ThreadLocal context does not survive a suspension point: carry it as a context element
  scope.launch(Context.current().with(span).asContextElement()) {
…

11
You flipped a kill switch an hour ago, yet fresh installs still run the broken feature for their entire first session. Why?
Medium
Because until a fetched config has been activated, every read answers from the defaults compiled into the binary — and on a fresh install that window covers the whole first session.
```
// WRONG - the splash screen now waits on the network on every single cold start
suspend fun startupBad() {
  remoteConfig.fetchAndActivate().await()     // up to fetchTimeoutInSeconds, default 60s
  showHome()
}
…
```

A 50/50 experiment reads 60/40 on the dashboard and the lift looks like noise. Where did assignment and exposure go wrong?

Medium

A split that is not the split you configured is a sample ratio mismatch, and on mobile it is almost always the exposure event that is broken rather than the bucketing.

// Deterministic assignment - the warehouse recomputes exactly this from the same inputs
fun variantOf(exp: Experiment, userId: String): String {
  val bucket = Hashing.murmur3_32_fixed()
    .hashString("${exp.key}:$userId", Charsets.UTF_8)
    .asInt().toUInt().toInt() % 10_000
  var cursor = 0
…

Product wants to read the 5% staged rollout as an A/B test against last week's build. What do you tell them?

Hard

A staged rollout has no control group — it compares two different populations in two different weeks, so every difference you see is confounded with who updates early and with whatever else happened that week.

// WRONG - comparing app versions is not an experiment, it is two populations in two weeks
val lift = metrics.forVersion("4.2.0").conversion / metrics.forVersion("4.1.0").conversion

// RIGHT - both arms in one binary, randomised inside the build
@Composable
fun CheckoutEntry(exp: ExperimentClient, flags: FeatureFlags) {
…

The variant lifted checkout conversion 3% but cold start regressed 80 ms. Do you ship it?

Hard

Only after you split the result by device tier, because that 3% was measured on a population that feels 80 ms far less than the users you have the least data about.

// Thresholds are agreed before the first user is bucketed, not after the winner is known
data class Guardrail(val metric: String, val maxRegression: Double, val unit: String)

val guardrails = listOf(
  Guardrail("crash_free_users", 0.1, "pp"),    // enforced by the rollout, not the experiment
  Guardrail("anr_rate", 0.05, "pp"),
…

15
Your analytics bill tripled the month you added an impression event. Which sampling cuts it without quietly breaking the funnel?
Hard
Aggregate on the device first, and if you still have to sample, sample users rather than events — a per-event coin flip keeps step one for one person and step three for another, and every rate you compute from that is wrong.
```
// 1. Aggregate - 500 impressions become one event with everything the metric needs
class ImpressionBatcher(private val analytics: Analytics) {
  private val seen = LinkedHashSet<String>()

  fun onItemVisible(id: String) { seen += id }
…
```
16
A fifth of your users decline ATT and analytics consent, and the DAU chart drops the same day. What can you still measure, and what happens to the events already queued?
Hard
Nothing about your users changed — you lost the right to count a fifth of them, so the job is a measurement plan that keeps "not collected" separate from "did not happen", not a way around the dialog.
```
// Deny by default before any Kotlin of yours runs - the manifest covers the very first launch
// <meta-data android:name="google_analytics_default_allow_analytics_storage"
//            android:value="false" />
// <meta-data android:name="google_analytics_default_allow_ad_user_data" android:value="false" />

class App : Application() {
…
```

Analytics & Experiments

Your event list has grown to `checkout_button_clicked`, `cart_button_clicked` and eighteen more like them. What is wrong with that taxonomy?

Three new screens shipped without a screen-view event because someone forgot the line in `onResume` — where should that instrumentation actually live?

Product asks which checkout step people drop off at, and your events cannot answer it. What has to be on every event for that funnel to compute?

A trainful of users spends forty minutes in a tunnel and force-quits the app at the end of it. What has to be true for those events to still show up?

A user's device clock is two days fast and their events land in the future. Which timestamp does the analysis use, and what does the wrong choice do to a retention chart?

A batch upload times out, the client retries it, and purchases jump six percent overnight. How do you make retries safe when the numbers only need to be roughly right?

The dashboard shows 99.9% crash-free sessions and 99.5% crash-free users for the same release. Why do they differ, and which one do you alert on?

Crashlytics says 99.9% crash-free, but the store reviews all say the app freezes. Where do those freezes show up if they never arrive as crashes?

A support ticket says an upload silently failed on one phone, and the crash dashboard shows nothing. What do you record so the next one is reproducible?

A tap feels slow and ends in a backend span, but the app trace and the server trace are two unrelated traces in two tools. How do you join them?

You flipped a kill switch an hour ago, yet fresh installs still run the broken feature for their entire first session. Why?

A 50/50 experiment reads 60/40 on the dashboard and the lift looks like noise. Where did assignment and exposure go wrong?

Product wants to read the 5% staged rollout as an A/B test against last week's build. What do you tell them?

The variant lifted checkout conversion 3% but cold start regressed 80 ms. Do you ship it?

Your analytics bill tripled the month you added an impression event. Which sampling cuts it without quietly breaking the funnel?

A fifth of your users decline ATT and analytics consent, and the DAU chart drops the same day. What can you still measure, and what happens to the events already queued?

Your event list has grown to checkout_button_clicked, cart_button_clicked and eighteen more like them. What is wrong with that taxonomy?

Three new screens shipped without a screen-view event because someone forgot the line in onResume — where should that instrumentation actually live?

Product asks which checkout step people drop off at, and your events cannot answer it. What has to be on every event for that funnel to compute?

A trainful of users spends forty minutes in a tunnel and force-quits the app at the end of it. What has to be true for those events to still show up?

A user's device clock is two days fast and their events land in the future. Which timestamp does the analysis use, and what does the wrong choice do to a retention chart?

A batch upload times out, the client retries it, and purchases jump six percent overnight. How do you make retries safe when the numbers only need to be roughly right?

The dashboard shows 99.9% crash-free sessions and 99.5% crash-free users for the same release. Why do they differ, and which one do you alert on?

Crashlytics says 99.9% crash-free, but the store reviews all say the app freezes. Where do those freezes show up if they never arrive as crashes?

A support ticket says an upload silently failed on one phone, and the crash dashboard shows nothing. What do you record so the next one is reproducible?

A tap feels slow and ends in a backend span, but the app trace and the server trace are two unrelated traces in two tools. How do you join them?

You flipped a kill switch an hour ago, yet fresh installs still run the broken feature for their entire first session. Why?

A 50/50 experiment reads 60/40 on the dashboard and the lift looks like noise. Where did assignment and exposure go wrong?

Product wants to read the 5% staged rollout as an A/B test against last week's build. What do you tell them?

The variant lifted checkout conversion 3% but cold start regressed 80 ms. Do you ship it?

Your analytics bill tripled the month you added an impression event. Which sampling cuts it without quietly breaking the funnel?

A fifth of your users decline ATT and analytics consent, and the DAU chart drops the same day. What can you still measure, and what happens to the events already queued?

Your event list has grown to `checkout_button_clicked`, `cart_button_clicked` and eighteen more like them. What is wrong with that taxonomy?

Three new screens shipped without a screen-view event because someone forgot the line in `onResume` — where should that instrumentation actually live?