1Product wants a Summarise button on a long thread. How do you choose between the on-device model and a server call, and what does each platform actually give you?Medium
2Your proxy in front of the model vendor works, and now strangers are running their own chatbot through it. What was the proxy missing?Medium
3Your streaming answer stutters as tokens land, and the list keeps jumping while the text grows. What do you change?Medium
4The user hits back halfway through a generation. What actually has to happen on the client, and what does the server still need from you?Medium
5You ask the model for JSON and want fields to appear as they arrive. What do you do with a half-written object, and what does an unfinished field look like on screen?Medium
6The app is killed mid-answer and the user comes back an hour later. What do you restore, what do you re-send, and what must never go twice?Medium
7How long may a screen sit there before the first token, and which of the usual tricks for hiding that wait are honest?Medium
8The on-device model weighs 1.2 GB and your app download cannot. How do those bytes get onto the phone, and when do you delete them?Medium