Nyaya Flow: Voice-First Legal Navigation for India, Built Around a Case, Not a Chat

Most people in India don't start a legal problem with a section number. They start with a sentence like "Mera makan malik deposit wapas nahi de raha", usually said out loud, often in Hindi, Marathi or Hinglish, and usually to someone who can't help. Between that sentence and an actual remedy sit a pile of questions: which law applies, which forum, which portal, which documents, what deadline, and what to do if the first step gets ignored.

Nyaya Flow is my attempt to close that gap. You speak or type your problem in your own language and get back a structured case with an ordered list of next steps, the documents and evidence each step needs, and links to the official portals behind them. I built it for Build in AI for India (Delhi Edition), where it was shortlisted as a finalist on 13 September 2026.

This post covers what is actually in the repo: the voice pipeline, the "zero-AI first" request router, how sources get attached, and the parts that are still hackathon-grade.

The core idea: a case, not a chat

The design decision everything else hangs off is that Nyaya Flow is case-first. A chat transcript is a terrible place to keep the state of a legal dispute. Facts get buried in old messages, the model re-reads everything on every turn, and nothing structured comes out of it that you could track.

So every intake produces a typed CaseAnalysisResult: category, jurisdiction, amounts, urgency, missing information, a reasoning summary, and an actions array. Each action has a purpose, a why, requiredDocuments, evidenceNeeded, sourceIds and a completionCriteria. The app renders a case workspace from that object, and follow-up messages are sent as incremental updates against the current case state (POST /api/ai/update-case) instead of re-sending a growing transcript.

actions: {
  id: string;
  order: number;
  title: string;
  purpose: string;
  why: string;
  priority: 'low' | 'medium' | 'high';
  status: 'pending' | 'in_progress' | 'completed';
  prerequisites: string[];
  requiredDocuments: string[];
  evidenceNeeded: string[];
  deadline?: string;
  officialResource?: string;
  sourceIds: string[];
  completionCriteria: string;
}[];

Language follows the same rule. The case model doesn't depend on language. Switching from English to Hindi calls /api/ai/translate-case, which re-renders only the user-facing text and is explicitly told to keep step IDs, amounts, statute names, portal names and URLs untouched. Your case stays the same case when you change the language you're reading it in.

Architecture

The app is an Expo (SDK 57) / React Native client using expo-router, talking to a small Express + TypeScript backend that holds every API key. The client never calls Gemini or Gnani directly. Cases are cached locally in AsyncStorage and synced to Firestore under users/{uid}/cases when the user is signed in.

1. Voice / text / photo
   expo-audio, image picker
   Raw input from the phone.
2. STT
   Gemini or Gnani.ai v3
   Audio becomes a transcript, server-side.
3. Task classifier
   deterministic
   Decides if an LLM is needed at all.
4. Policy engine
   budgets + bans
   Can veto or downgrade a model call.
5. Gemini
   JSON mode
   Structured case and action steps.
6. Source registry
   12 official sources
   Portals and Acts attached in code.
7. Case workspace
   AsyncStorage + Firestore
   Persisted, translatable, trackable.

Voice: two STT providers behind one interface

Voice is the main way in, because the people who most need this are the least likely to type a long English paragraph. The backend exposes one route, POST /api/ai/transcribe-audio, and the orchestrator picks a provider:

• Gemini (default): audio goes in as inlineData with a short instruction to return only the transcript of someone describing a legal situation in India.
• Gnani.ai STT v3: a dedicated Indic speech provider, which I wanted because accents and code-mixed speech are where general models stumble.

Both providers implement the same VoiceTranscriptionProvider interface and return the same TranscriptionResult, so the rest of the pipeline doesn't care which one ran. The Gnani integration took the most debugging. Its v3 API wants multipart form data with a real audio file, not JSON with base64. So the server decodes the base64 from the phone into a Buffer, wraps it in a Blob, picks a file extension from the MIME type, and posts it with a language code:

const formData = new FormData();
const audioBlob = new Blob([audioBuffer], { type: mimeType });
formData.append('audio_file', audioBlob, `recording.${fileExt}`);
formData.append('language_code', gnaniLanguageCode);
formData.append('format', 'verbatim');

const controller = new AbortController();
const timeoutId = setTimeout(() => controller.abort(), 15000);

The UI's language choice (hi, mr, hinglish, ta, and so on) gets mapped to Gnani's codes (hi-IN, mr-IN, ...), with Hinglish going to hi-IN. Every request has a 15-second abort, and I wrote a cleanErrorMessage helper that turns Gnani's rate-limit, scope and HTTP 522 responses into readable messages that suggest switching to Gemini. During a live demo, "Gnani timed out, switch to Gemini" is far more useful than a raw HTML error page.

Zero-AI first: the deterministic classifier

The most important thing I learned building this is that a lot of legal questions don't need an LLM. "What's the cyber crime helpline?" has one correct answer, and it should never be generated. So before any model is called, TaskClassifier.classifyRequest runs plain string matching over the input:

if (
  text.includes('cyber crime') ||
  text.includes('cybercrime') ||
  text.includes('1930') ||
  text.includes('report cyber') ||
  text.includes('online scam')
) {
  return {
    aiRequired: false,
    intent: 'DIRECT_LOOKUP',
    matchedProviderId: 'src_cybercrime',
    category: 'cyber_crime',
    // ...
  };
}

It sorts input into four intents: DIRECT_LOOKUP (helplines and portals), CASE_MUTATION (things like "mark step done" or "delete case", which are just UI state changes), SIMPLE_EXTRACTION, and COMPLEX_SYNTHESIS. Anything that reads like a real dispute ("deposit", "refund", "withheld", "notice", "is it legal") goes to the model. Direct lookups and mutations return a full case-shaped response with 0 tokens, modelUsed: null, and an empty action list, because an informational question shouldn't produce a fake three-step legal workflow.

After the classifier comes a PolicyEngine, which can block a call before it costs anything. It has a global NO_AI mode, per-model enable switches, a per-session budget for the "strong" tier, and a hard rule that a one-line query can't be routed to the strongest tier. All of this is configurable at runtime through /api/inspector/config.

Where sources come from

This is the part I care most about getting right. When Gemini handles an actionable matter, its prompt asks for strict JSON (responseMimeType: 'application/json'): intent, category, an overview with three fixed sections, missing-information questions with input types, a reasoning summary, and 1 to 4 action steps. The prompt also tells the model not to invent a multi-step workflow when someone is only asking an informational question.

The model does not write the official links. After parsing, the orchestrator calls providerRegistry.findRelevantSources() with the category and keywords. It matches against a hand-curated VERIFIED_PORTAL_REGISTRY of 12 entries: 112, cybercrime.gov.in, RTI Online, e-Daakhil, the National Consumer Helpline, NALSA, eCourts, CPGRAMS, Parivahan eChallan, BNSS/CrPC, the Transfer of Property Act and the Consumer Protection Act 2019. Those sources are attached to each action as officialResource and sourceIds. If the model's JSON doesn't parse, the code falls back to a safe default instead of crashing.

The Pipeline Inspector

Judges reasonably ask "why should I trust this?", so I built a developer screen (app/inspector.tsx) that answers it. Every intake records a trace in telemetryRecorder.ts: input normalization, the classifier's decision, policy evaluation, provider requests (including which STT engine ran and how long it took), and the final model choice with a whyNoModel or whyThisModel explanation. The inspector reads these through GET /api/inspector/traces. You can watch a cyber-fraud question go through with zero model calls and then watch a deposit dispute route to Gemini.

What's honest to say about the current state

The finalist slot was nice, but this is a hackathon build, and I'd rather list its gaps than have someone else find them.

The language detector is also deliberately simple. It counts Devanagari characters, checks a list of Marathi words to tell Marathi from Hindi, and treats Latin-script text as Hinglish if it contains at least two Hinglish grammar words like mera, nahi or wapas. It's fast and testable (it has its own Jest suite), and it's good enough to pick the right STT language code and response language.

What's next

• Point each tier at an appropriately sized model and read real token usage from the response.
• Verify Firebase ID tokens on the server and lock the AI routes down.
• Grow the source registry state by state, starting with consumer commissions, rent authorities and RERA, and move from keyword matching to retrieval over the actual statute text.
• Export the drafted legal notice as a document. The generate-draft and revise-draft endpoints already produce the text; the export flow is what's missing.

The one thing I'd keep exactly as it is: answer known facts with code, use the model only for the parts that need reasoning, and keep the result as a structured case the user can act on.