AI Features in Mobile Apps
Irbisa · cheatsheetSeptember 13, 2026

AI Features in Mobile Apps

On-device vs server inference, token streaming UI, structured output, latency, cost, privacy

Senior Developer8 itemscompressed for a skim
  1. 01

    Product wants a Summarise button on a long thread. How do you choose between the on-device model and a server call, and what does each platform actually give you?

    Medium

    Availability decides this before quality does: on-device models are free, private and offline, but they exist only on recent hardware, so the server path is the one you must build and the local one is an optimisation you light up when the device says yes.

    // One interface, two backends. The choice is made per call, from a live probe.
interface ThreadSummarizer { suspend fun summarize(text: String): String }

class OnDevice(context: Context) : ThreadSummarizer {
  private val client = Summarization.getClient(
    SummarizerOptions.builder(context)
…
  2. 02

    Your proxy in front of the model vendor works, and now strangers are running their own chatbot through it. What was the proxy missing?

    Medium

    A proxy that forwards whatever body the client sent is an anonymous inference endpoint with your card on file — it has to own the prompt, the model and the budget, not just the secret.

    // WRONG: the client builds the prompt and carries the vendor key.
// Anyone with the APK has the key; anyone with the key has your bill.
val body = ChatBody(
  model = "vendor-large",
  messages = listOf(Message("system", SYSTEM_PROMPT), Message("user", userText)),
)
…
  3. 03

    Your streaming answer stutters as tokens land, and the list keeps jumping while the text grows. What do you change?

    Medium

    One token is not one UI update: buffer the deltas and flush on a frame cadence, append into a single text node instead of a growing list of them, and follow the bottom only while the user is already there.

    // Deltas arrive far faster than a frame. Coalesce once, at the boundary.
val answer: Flow<String> = flow {
  val buf = StringBuilder()
  api.streamAnswer(prompt).collect { delta -> buf.append(delta); emit(buf.toString()) }
}
  .sample(48.milliseconds)        // ~20 flushes/s instead of one per token
…
  4. 04

    The user hits back halfway through a generation. What actually has to happen on the client, and what does the server still need from you?

    Medium

    Cancellation has to reach the socket — closing the response body is what stops the meter — and the partial text already on screen is state, so save it before the scope dies.

    class AnswerViewModel(private val repo: AnswerRepo) : ViewModel() {
  private var job: Job? = null

  fun ask(turnId: String, prompt: String) {
    job?.cancel()                      // a new question kills the old stream, and its cost
    job = viewModelScope.launch {      // dies with the screen, not with the process
…
  5. 05

    You ask the model for JSON and want fields to appear as they arrive. What do you do with a half-written object, and what does an unfinished field look like on screen?

    Medium

    Make the final parse a guarantee and treat every prefix before it as layout information only: schema-constrained decoding, a mirror type whose fields are all nullable for the stream, and no value rendered from a token that has not been closed.

    @Serializable data class Recipe(val title: String, val minutes: Int, val steps: List<String>)

// The streaming mirror: every field nullable, because every field may not exist yet.
@Serializable data class RecipeDraft(
  val title: String? = null, val minutes: Int? = null, val steps: List<String>? = null,
)
…
  6. 06

    The app is killed mid-answer and the user comes back an hour later. What do you restore, what do you re-send, and what must never go twice?

    Medium

    Persist the conversation as durable turns with ids and treat the ViewModel as a projection of them; on return you ask the server about the generation you already started rather than starting it again.

    @Entity
data class TurnEntity(
  @PrimaryKey val id: String,        // minted on the client, reused by every retry
  val conversationId: String,
  val role: String,                  // "user" | "assistant"
  val text: String,
…
  7. 07

    How long may a screen sit there before the first token, and which of the usual tricks for hiding that wait are honest?

    Medium

    Time to first token is the number the feature lives or dies by: about 100 ms feels instant, a second is fine if something acknowledged the tap, and past a few seconds people leave — while total duration barely matters once tokens are moving, because reading is slower than generation.

    // TTFT is a different metric from total duration. Time the first chunk, not the last.
suspend fun ask(prompt: String, sink: (String) -> Unit) {
  val started = SystemClock.elapsedRealtime()
  var firstAt: Long? = null
  var tokens = 0
  try {
…
  8. 08

    The on-device model weighs 1.2 GB and your app download cannot. How do those bytes get onto the phone, and when do you delete them?

    Medium

    Weights are content, not code: they ride the platform's asset-delivery channel, they are versioned with the code that reads them, and they are a cache the system may reclaim — so the feature has to work while they are missing.

    // Play Asset Delivery: the model is an on-demand asset pack, not part of the APK.
class ModelStore(private val packs: AssetPackManager, private val activity: Activity) {

  fun localPath(): String? = packs.getPackLocation(PACK)?.assetsPath()   // null = not here yet

  fun ensure(onReady: (String) -> Unit) {
…