The most consequential shift in mobile development this decade is not a framework — it's that phones became capable of running useful AI models locally. Both platforms now ship system-level models and APIs for them: Gemini Nano on Android through AI Core and the AI Edge SDK, and Apple's Foundation Models under the Apple Intelligence umbrella on iOS. As someone integrating these into client apps, here is the honest state of what works.
Why On-Device Beats Cloud (When It Does)
Three properties make local inference strategically different from calling a hosted model. Cost: cloud inference bills scale with every user action, while on-device inference is free after the engineering is paid for — at consumer scale this is the difference between a feature that grows margin and one that eats it. Privacy: data that never leaves the device never needs a data-processing story, which simplifies compliance and marketing honestly. Latency and offline: summarising a note in an aeroplane works. These are not incremental benefits; for some products they decide feasibility.
What Small Models Do Well
On-device models are small compared to frontier cloud models, and pretending otherwise produces disappointing features. What they do reliably well: summarisation of moderate-length text, rewriting and tone adjustment, classification and smart tagging, entity extraction, transcription, image description, and constrained question-answering over content you provide. What they do poorly: open-ended reasoning, long-document synthesis, anything requiring current knowledge, and generation where quality is the entire product. The craft is designing features inside the first list.
The Platform Landscape
On Android, Gemini Nano runs through AI Core on supported devices — the system manages the model, so your app doesn't ship gigabytes of weights. Coverage is the catch: system-managed AI is available on flagship and upper-mid devices, so India-focused apps must plan a fallback for the long tail of budget hardware. On iOS, Apple Intelligence exposes on-device foundation models to third-party apps, with the same shape of constraint: newer devices only. For custom needs, runtimes like LiteRT and ONNX Runtime run your own fine-tuned small models on both platforms — more control, more responsibility.
The Hybrid Pattern That Actually Ships
Almost every production feature I build ends up hybrid: on-device first for the common case, cloud fallback for devices without local capability or requests exceeding local quality. The decision logic lives behind a single interface in the codebase, so product behaviour stays consistent while the execution location varies. Budget-wise this caps the cloud bill at the fallback slice — typically 20–40% of requests for an India-market app — instead of 100%.
What It Costs to Build
A single well-scoped on-device AI feature — say, smart summarisation of user content with cloud fallback — adds roughly ₹1–3 lakhs to a build: integration, prompt and output design, fallback logic, and the testing matrix across capable and incapable devices. The testing is the underestimated part: an AI feature has a quality distribution, not a pass/fail state, and evaluating it across devices and content types is real work. Anyone quoting AI features without a testing line item is quoting a demo.
Privacy as a Feature, Not a Footnote
'Your data never leaves your phone' is a marketing sentence users understand instantly — and it's only true if the architecture makes it true. On-device inference lets you say it honestly, which matters more every year as users grow suspicious of AI features that quietly upload everything. For health, journaling, finance, and messaging apps, I'd go as far as saying local-first AI is becoming a category expectation: the first competitor who can make that claim resets what users demand from the rest.
Where to Start
Pick the single feature where intelligence removes the most friction — usually search, summarisation, or data entry — and ship it hybrid. Resist the temptation to sprinkle AI across every screen; one excellent intelligent feature moves retention more than five mediocre ones. If you have an app and a shortlist of AI ideas, I offer a short feasibility review: which ideas run on-device, what the fallback economics look like, and what the realistic build cost is. It's cheaper than discovering the answers mid-project.