The conversation around "AI in mobile apps" has shifted dramatically. A year ago we were discussing how to add a chatbot UI. Today, the serious question is: how do you build Android apps that orchestrate multi-step AI workflows reliably — with retries, state recovery, observability, and graceful degradation?
What "Agentic" Actually Means in Android Context
An agentic AI system doesn't just respond to a single prompt — it breaks a goal into sub-tasks, executes them in sequence or in parallel, uses tool calls (camera, location, calendar, web search), and adapts based on intermediate results. On Android, this means your ViewModel or WorkManager job is now coordinating a dynamic chain of AI operations rather than a static API call.
The Core Problem: Non-Determinism at Scale
LLMs are non-deterministic. A workflow that involves 5 chained model calls has exponentially more failure modes than a single API call. Your architecture must assume that any step can return malformed output, timeout, or produce a semantically incorrect result. Treating LLM calls like database queries will break your app in production within weeks.
Pattern 1: Structured Output Contracts
Every LLM step in your workflow should return a structured JSON schema — not freeform text. Use Gemini's response_mime_type: application/json, OpenAI's structured outputs, or a Kotlin data class + Gson parser with a validation layer. If the model returns output that fails schema validation, you retry with a clarifying prompt before propagating failure.
Pattern 2: Idempotent Step Design
Each agent step should be designed to be safely retried. Store intermediate results in Room database keyed by a deterministic hash of the input. On retry, check the cache first — if valid output exists, skip the LLM call. This makes your workflow both cheaper (fewer tokens) and more resilient.
Pattern 3: Observable Task Graphs with WorkManager
Model your multi-step workflow as a WorkManager chain. Each Worker represents one agent step. Workers can pass typed Data payloads to the next step, WorkManager handles Android lifecycle, and you get built-in retry, backoff, and constraint awareness. You can observe the entire chain's progress through LiveData or Flow exposed from WorkInfo.
On-Device vs Cloud: The Right Split
Use MediaPipe LLM Inference API or Gemini Nano (via AICore) for fast, private, offline-capable steps. Reserve cloud API calls for steps that need larger context windows or tools the device can't run locally. A good heuristic: if the step processes user-private data and doesn't need the internet anyway, run it on-device.
Observability: You Cannot Debug What You Cannot See
Instrument every agent step with structured logging: input token count, output token count, latency, model version, validation pass/fail, retry count. Log these to Firebase Analytics or your own backend. Without this, debugging a production agentic failure is impossible. Build your observability layer before you deploy agentic features.