This article describes a step-by-step tutorial for building a voice-first first-notice-of-loss (FNOL) application. The intent is to let a claimant speak naturally while the system—observed by a live agent—assembles structured claim data in real time, shows the transcript, highlights missing items, classifies and routes the claim, and produces an adjuster-ready Markdown handoff packet.
The implementation uses Google ADK for the workflow graph, Gemini Live for bidirectional audio, Pydantic for data contracts, and deterministic Python policies for required-field checks, document lists, fraud signals and safety escalations.
What we build
Key features of the app:
- Claimant can speak or type.
- Audio responses stream back in real time via Gemini Live.
- Structured claim facts are extracted into Pydantic schemas.
- Claim type and severity are automatically classified.
- Deterministic rules identify missing fields, required documents, fraud/safety flags and route decisions.
- A Markdown-format adjuster handoff packet is constructed during the call.
- The system avoids promising coverage, payment, or liability.
How it works (summary)
For every claimant utterance the back end performs these steps:
- Listen: audio is transcribed; text enters the pipeline.
- Understand: Gemini ingests the full conversation so far and extracts structured facts (name, policy, date, location, what happened, evidence, injuries).
- Classify: Gemini decides claim type and likely severity.
- Apply rules: Python checks for required fields, needed documents, fraud and safety signals.
- Decide routing: the result is routed to one of ready_for_adjuster, needs_docs, special_investigation or emergency_escalation; safety flags override other lanes.
- Build packet: a Markdown handoff packet and the next claimant-facing message are produced.
- Update UI: fields, timeline and packet refresh in the browser and the agent speaks back.
The core principle is split of labor: Gemini handles messy human language; Python enforces repeatable, deterministic decisions.
Prerequisites
The tutorial expects:
- Python installed (recommended 3.12).
- Gemini API key from Google AI Studio.
- A code editor and basic familiarity with Python, FastAPI and async.
- A browser with microphone access.
Source code and project layout
Clone the repository:
git clone https://github.com/Shubhamsaboo/awesome-llm-apps.git
Open the folder:
awesome-llm-apps/voice_ai_agents/insurance_claim_live_agent_team
Files and roles:
- init.py — exports root_agent for ADK CLI tools.
- agent.py — defines the ADK graph and exposes run_claim_workflow.
- schemas.py — Pydantic data contracts (e.g., ClaimNarrative).
- policies.py — deterministic insurance rules.
- examples.py — demo claimant prompts.
- live_demo/ — FastAPI transport and minimal frontend (server.py, index.html, styles.css, app.js).
- requirements.txt, .env.example, README.md
The frontend is static and communicates with three interfaces: /api/message, /api/audio and /ws/live. Any UI can be used as long as it honors those surfaces.
Schemas and deterministic policies
The ClaimNarrative Pydantic model includes fields such as policyholder_name, policy_number, contact_method, date_of_loss, loss_location, loss_description, estimated_loss_usd, injuries_or_safety_concerns, evidence_available and documents_mentioned.
policies.py exposes five public functions that accept structured inputs and return structured outputs: validate_required_claim_fields, apply_coverage_and_evidence_rules, generate_document_checklist, fraud_signal_and_safety_gate and build_claim_intake_packet. TYPE_REQUIRED_DOCS maps required documents per claim type (for example, home_water_damage, auto_collision, etc.). The fraud_signal_and_safety_gate function forces emergency_escalation if injury or unsafe-living mentions appear.
ADK graph and workflow bridge
The workflow is a SequentialAgent composed of seven steps, alternating LLM and Python nodes. LlmAgent nodes specify a Pydantic output_schema to guarantee structured LLM outputs; Python nodes are FunctionNode instances that call the functions in policies.py. The graph is exported as root_agent.
run_claim_workflow is the bridge used by the live demo: it creates an InMemorySessionService, initializes a Runner with the root_agent, and runs the graph asynchronously while feeding the claimant transcript as a genai_types.Content message. The final state is read from session.state and validated with Pydantic before returning.
FastAPI transport and Gemini Live
live_demo/server.py serves the frontend, manages per-claimant sessions, accepts text/audio and WebSocket connections, and forwards work to run_claim_workflow. The small helper _process_with_adk_graph calls run_claim_workflow and—when appropriate—appends the deterministic claimant_next_message from the claim_intake_packet into the session transcript so the UI can show what the agent will say next.
For /ws/live, the graph runs as a background async task (asyncio.create_task), which allows Gemini Live audio to stream continuously while the structured packet updates asynchronously in the background.
Examples and testing
examples.py contains five canned claimant narratives (basement flood, car accident with injuries, stolen laptop without a police report, travel cancellation, plus an intentionally vague claim) helpful for testing in adk web and sanity-checking rules.
Run the app locally
Install dependencies: pip install -r requirements.txt Copy the env file and add API keys: cp .env.example .env (set GOOGLE_API_KEY and Gemini key) Start backend and frontend with:
python -m uvicorn live_demo.server:app --reload --host 127.0.0.1 --port 4177
Open http://127.0.0.1:4177/index.html, click the mic or type a message and watch the right-hand panel populate fields and build the handoff packet.
Conclusion and next steps
The tutorial demonstrates a clear separation of responsibilities: schemas as contracts, deterministic policies for decisions, an ADK graph for orchestration, and FastAPI purely as transport. Suggested extensions include adding new claim types, persisting conversations across restarts, expanding fraud and safety signals, wiring the final packet into a claims system or CRM, and building an eval set to catch regressions. The hybrid LLM-plus-rules approach applies beyond insurance and can be adapted to other domains.
