Analytics & Experiments

DAU and MAU count distinct users active in a day or a 30-day window; D7 retention is the share of users who installed on a given day and came back on day seven. All of them need a stable user id, an install event and one "active" event per session. Define "active" once: an app open or session_start, not a background sync or a received push, or silent pushes inflate DAU Install day: first_open (Firebase logs it automatically) fixes each user's cohort; a reinstall should not create a new person Stable id: an anonymous id that survives restarts, linked to the account on login; a new id per launch makes every day look like new users

A feature flag is an on/off gate around code you already shipped, remote config is the server-side store that delivers flag values and other parameters, and an A/B test randomly assigns users to variants and measures the difference. Remote config: key-value delivery with in-app defaults, targeting conditions and a fetch cycle: Firebase Remote Config, LaunchDarkly, Statsig Targeting is not an experiment: enabling a feature for "Android 15 users" changes who sees it, but without random assignment you cannot attribute any change in metrics to it

Those twenty names are one event, button_tapped, with a button_id property. A noun inside an event name is usually a property in disguise, and it turns "which buttons get tapped on this screen" into a twenty-name query. Event name: what happened, as object_action in the past tense: button_tapped, checkout_completed Event property: which one and where: screen, button_id; checkout vs cart belongs here User property: a trait of the person, set once rather than per event: plan, signup_cohort Tool limits: Firebase Analytics allows 500 distinct event names and 25 parameters per event, so a name per button runs out Registry: a checked-in list of events and their properties, verified in CI, keeps the vocabulary closed The opposite mistake: one app_event with a type property breaks every tool that builds funnels by event name

In the navigation layer, registered once: the router already sees every destination change, so a new screen is tracked without anyone remembering a line. Single-activity apps: Firebase's automatic screen_view is keyed on the activity, so in a Compose app every screen reports as MainActivity; log from the NavController instead Name from the route pattern: order/{orderId}, not order/8f2c and never a localized title, or one screen becomes thousands of names Screen views vs actions: a screen view says where the user is; action events carry screen as a property so they can be grouped by it Common mistake: logging in onResume or in an effect without a stable key re-fires on return from background or on recomposition; deduplicate on the back stack entry

Make the event schema the only way to send data: typed events with an allowlist of properties, values limited to enums, ids and numbers, and one scrubbing layer in front of every vendor SDK. No free-form API: hide log(name: String, props: Map<String, Any?>); an untyped map is how an email ends up in an event Describe, do not copy: send search_length and results_count instead of the query, a screen id instead of a title that may contain a name Pseudonymous ids: your own random user id, never an email, a phone number or the advertising ID as a property; Firebase's terms forbid sending PII One scrubber before all vendors: drops undeclared keys and free text, strips query strings from URLs in crash breadcrumbs Review the registry: every new property can change the Play Data safety form, so someone who owns that form reviews the event list

A funnel joins events into one person's single attempt, so every step needs the same user identity, a reliable order and an id for that attempt. Step names alone give you five unrelated counts. Identity: an anonymous id, then the user id after login, linked with an alias or identify call; a mid-flow login is where most funnels split in two Flow instance: a checkout_id minted when the flow starts and carried to every step, so an abandoned attempt and a later success are not merged Ordering: a client timestamp plus a per-session sequence number, since two events can share a millisecond Session id: reset after about 30 minutes idle, so Monday's cart and Friday's payment are not one visit Explicit failure: send checkout_failed with a reason; a missing next step can mean a crash, a closed app or a lost batch, and you cannot tell which

Hide the vendor SDK behind your own Analytics interface, assert on a fake in unit and UI tests, and check the real pipeline by hand in Firebase DebugView before release. Events are a contract with the analysts, so they deserve tests like an API. Fake tracker: records events in a list; a ViewModel test asserts the name and properties after an action Assert the contract: the properties analysts query, and that one action is logged once, not again on rotation or recomposition DebugView: adb shell setprop debug.firebase.analytics.app <package> sends that device's events immediately instead of batching them; .none. turns it off Schema check in CI: compare logged names and properties with the event registry, so a renamed property fails the build, not the dashboard Common mistake: testing through the Firebase SDK itself: it batches, needs a configured FirebaseApp and a device or Robolectric, and gives you nothing to assert on

Each event must be persisted to disk right after track() (on a background writer, never the main thread), and the queue must survive process death, retry with backoff and delete rows only after the server accepts them. An in-memory list loses the whole session on force-quit. Disk first: track() appends to a Room table or an append-only file and returns; it never calls the network inline Batch: upload every ~30 events or ~30 seconds, and schedule a flush when the app goes to background WorkManager for the flush: a unique job with a network constraint and exponential backoff outlives the process; a coroutine started in onStop can be killed mid-upload; after a user Force stop the job is cancelled, so also flush on the next launch Delete only after a 2xx: deleting on send turns every flaky network into missing data Cap the queue: a size limit that drops the oldest rows first, so a device offline for a month does not fill the disk Lost batches are not random noise: loss correlates with bad networks and long sessions, so it skews retention and engagement upward.

Give every event its own id when track() is called, store it with the event, and let the server drop duplicates by that id. Then the client can retry as often as it likes. A timeout is ambiguous: the server may have saved the batch and lost the response. The client cannot tell, so it must resend, and only the server can recognise the repeat. Key per event, not per request: a UUID created once and stored in the queue row; minting it at send time gives every retry a new id Vendors already have the field: $insert_id in Mixpanel, insert_id in Amplitude, messageId in Segment Never hash the contents: two identical taps a second apart are two real events Money is different: key purchases on the order id and deduplicate in the warehouse too; for revenue the source of truth is the backend or Play Billing, not the analytics stream

Send both: client time to order events within a session, server time to correct it. The server measures each device's clock skew at upload and shifts every event by it, so a clock two days fast is cancelled out. Client time: taken at the moment of the action; the only clock that knows when something happened offline Server receive time: cannot be faked, but for a batch that waited in a tunnel it is the upload time, forty minutes late Skew correction: the batch carries client_upload_time; the server computes event_time = client_event_time - (client_upload_time - server_upload_time), as Amplitude does Durations: measure them with SystemClock.elapsedRealtime(); a wall clock corrected by NTP mid-session can give negative durations Retention is "event day minus install day", so uncorrected skew moves users into the wrong cohort cell, and many pipelines drop events dated in the future outright.

They divide by different things: a user counts as crashed for the whole window if any one of their sessions crashed, so crash-free users is almost always the lower number. Alert on crash-free sessions per release; report crash-free users to product. Crash-free sessions: 1 − sessions with a crash ÷ all sessions; a release that adds background sessions inflates the denominator and can hide a regression; a crash loop on a few users hits sessions hard and users barely Crash-free users: 1 − users with a crash ÷ all users; it falls as the window widens, so a 1-day and a 7-day figure are not comparable Make it concrete: at two million monthly users, 99.5% is ten thousand people who saw a crash Alert per version: compare the new version with the one it replaces; the app-wide figure dilutes a bad release behind the healthy installed base It is an upper bound: crashes before the SDK initialises, memory kills and ANRs are not in it

A freeze is the OS deciding the app is unresponsive, so the record comes from the platform, not your crash handler: Play vitals and ApplicationExitInfo on Android. The crash-free figure never includes them. ANR: raised when an input event is not handled within about 5 seconds, or a broadcast receiver or service overruns its timeout; Play Console shows the rate with the main-thread stack Read it yourself: on the next launch, getHistoricalProcessExitReasons (Android 11+) returns REASON_ANR with a trace and REASON_LOW_MEMORY for memory kills Crashlytics: on Android 11+ it collects ANRs from the same API and lists them as a separate event type, outside crash-free Shorter stalls: a 1–4 second freeze never becomes an ANR; JankStats or a main-thread watchdog is how you see it Crash-free counts deaths your code caused; ANRs and memory kills are deaths the OS caused, and the user cannot tell the difference.

Report the caught error as a non-fatal with a distinct exception type, and attach breadcrumbs and custom keys around it. A stack trace without the state that produced it cannot be reproduced. **One recordException: at the layer that knows what the failure means, not in every layer that rethrows, and once per operation rather than per retry Breadcrumbs**: log() the last few actions: screen entered, button tapped, request path and status Custom keys: the state you would ask about on a call: upload step, network type, active flags; Crashlytics keeps up to 64 keys Stable grouping: issues group by exception type and stack, so variable data goes into keys, never into the message, and each failure kind gets its own type Find the report: setUserId with a pseudonymous id lets support look the ticket up; never an email

Until a fetched config is activated, every read returns the defaults compiled into the app. A fresh install has nothing cached, so it runs on those defaults until its first activation, and a kill switch that defaults to "on" keeps the feature on. Fetch is not activate: fetch downloads, activate swaps the values in, fetchAndActivate does both; getBoolean answers from defaults until then How long: until the first fetchAndActivate() completes (seconds), or for the whole session if the app only fetches now and activates on the next start; the 12-hour interval affects installed apps, not the first fetch Fetch interval: the default minimumFetchIntervalInSeconds is 12 hours, so an already installed app may not see the change until its next allowed fetch Real-time updates: addOnConfigUpdateListener pushes changes to a running app, as long as it is open and online Safe defaults: a kill switch's in-app default is the safe branch, off; then a failed or late fetch never turns the feature on Common mistake: blocking the splash screen on fetchAndActivate(): fetchTimeoutInSeconds defaults to 60, so a bad network stalls the cold start

A split that differs from the configured one is a sample ratio mismatch (SRM), and on mobile the most common cause is the exposure event, not the bucketing. Until the ratio is fixed, no metric from the test can be trusted. Deterministic assignment: the server returns the variant, or the client computes hash(experimentKey + userId) % 10000; a client-side random draw cannot be reproduced Bucket by user id: not install id, or one person with a phone and a tablet lands in both arms Exposure where the user sees it: log on the first render of the feature, not when the flag is read at startup, or the sample fills with people who never reached it Log every arm: the classic SRM is a control path that returns early before the exposure line, so only the variant is counted Test the ratio first: a chi-square test on exposed users per arm; p < 0.001 means stop and debug, because the missing users are never a random sample

Propagate trace context over the network: the app opens a root span at the tap and sends its ids in a W3C traceparent header, so the server's span becomes a child in the same trace instead of starting a new one. On the device Root span at the interaction: view model, cache and HTTP calls become child spans with the same trace id Coroutines: OpenTelemetry keeps the current span in a ThreadLocal, which does not follow a coroutine across threads; carry it with asContextElement() Async OkHttp: a hand-written interceptor reads Context.current() on OkHttp's dispatcher thread when Retrofit uses enqueue, finds no span and sends no header; the OpenTelemetry OkHttp instrumentation captures context when the call is created Across the wire Header: traceparent: 00-<trace-id>-<span-id>-01, where the last byte is the sampled flag Standard names: OpenTelemetry semantic conventions such as http.request.method, or backend dashboards will not recognise the spans Sampling The root decides: with a parent-based sampler the server follows the flag in the header, so the client decides whether a slow request is recorded Head sampling cannot see errors: a trace dropped at the root never reaches the collector, so keep 100% of cold starts and checkout at the root (a custom Sampler by span name), and do tail sampling for failures on the backend side or export all client spans to the collector Compare durations, not timestamps: device and server clocks drift

A staged rollout has no control group: it compares different people in different weeks, so every difference mixes the change with who updates early and with whatever else happened that week. Use it to catch crashes, not to measure lift. Early updaters are not random: the first 5% skew toward auto-update, Wi-Fi and newer phones, and they convert better before you change anything Different time window: last week's build was measured on another weekend, another campaign, another price Survivorship: people who dislike the update open the app less and quietly leave the denominator What a real mobile experiment looks like Both arms in one binary: behind a flag, randomised inside the same build, so both share a version and a time window Fix horizon and sample size up front: checking the p-value daily and stopping at the first green inflates false positives Read platforms separately: iOS and Android users are two populations, not two halves of one

Not yet: check the result by device tier first. A net +3% can hide a loss on low-end phones, where 80 ms may become 300; ship if the lift holds there and cold start stays inside the guardrail agreed before launch. A guardrail is a metric that must not regress even when the primary metric wins, with a threshold agreed before launch: crash-free users, ANR rate, cold start p90. Agreed afterwards, it becomes a negotiation the winning variant always wins. The lift already includes the cost: the experiment measured the whole experience, 80 ms included, so +3% is a net number The average hides the shape: +4% on flagships and −1% on the bottom quartile ships a regression to the users with the least headroom Rare events need the rollout: a few hundred thousand users cannot detect a 0.01 pp (one basis point) shift in crash or ANR rate; that guardrail is checked during the staged rollout Startup cost accumulates: five experiments at 80 ms each add 400 ms nobody remembers adding If you ship: file the 80 ms as a tracked regression; otherwise move the variant's new work off the first-frame path and rerun

Aggregate on the device first, and if you still must sample, sample users, not events. A coin flip per event keeps step one for one person and step three for another, so every conversion rate built from it is wrong. Aggregate before sending: one feed_impressions event per screen with a count and item ids replaces hundreds of events Sample by user hash: keep a user when hash(userId) % 100 < 10; all of their events survive, so funnels and retention still join Record the rate: a sample_rate property, multiplied back at query time, or every dashboard silently reads ten times low Never sample the rare, important events: errors, purchases and experiment exposures stay at 100%; by volume they are noise, by meaning they are the pipeline Sampling that quietly biases Dropping on queue overflow: removes the heaviest sessions, exactly your power users Wi-Fi only, or by device model: skews toward new phones and good networks, the users who already look fine A different hash per event type: users fall into different samples for different events, and joins break again

Measure from the consented cohort and compare rates, not counts, plus modelled totals from Consent Mode; hold queued events until the user answers, send only the granted categories, and delete them on refusal. Your users did not change; you lost the right to count a fifth of them. The job is a measurement plan that keeps "not collected" separate from "did not happen", and a queue that never sends what was not granted. What is left Consent denied: stops collection for that purpose; crash reporting often rests on a different legal basis, but that is a decision for legal, not an SDK default Consent Mode: with ANALYTICS_STORAGE denied, Firebase sends no identifiers and Google Analytics models the gap: totals, not user-level funnels Advertising ID: on Android 13+ the AD_ID permission must be declared and the user can reset the ID to zeros, so never key analytics or attribution on it; use your own pseudonymous id Compare rates, not counts: denial skews by region and OS version, so compare conversion inside the consented cohort, and track the consent rate itself from your consent platform The queued events Before the answer: keep events only briefly (in memory, or on disk with a short TTL and no user id), each tagged with the consent category it needs, and do not initialise the vendor SDK On accept: set consent, start the SDK, flush only the granted categories On refusal or withdrawal: delete the queue and rotate the anonymous id; never backfill