Samvad — AI Meeting Assistant

Records in-person meetings, transcribes them live, and turns them into assigned, emailed tasks — without anyone taking notes.

PythonFastAPISQLAlchemyAPSchedulerSarvam AIWebSocketRedisPostgreSQLDockerGitHub ActionsRender

The Problem

In-person meetings lose context because nobody takes structured notes. Decisions get made, tasks get assigned verbally, and by the time someone writes up minutes, half the details are gone. I built Samvad to sit in the room, transcribe everything in real time, let you flag important moments by voice, then run an LLM agent after the meeting to extract tasks with assignees and deadlines — and email them automatically.

What I Built

Live, during the meeting: microphone audio (16kHz) streams over WebSocket to Sarvam AI's speech-to-text. I built a regex-based voice command detector — with an LLM fallback for ambiguous phrasing — so you can say things like "start highlighting" to flag key sections in real time.

After the meeting ends: the raw transcript runs through a 3-phase refinement pipeline I designed (rule-based cleanup → LLM grammar fix → a quality gate that reverts to the rule-based version if the LLM corrupts speaker labels or drops content). A ReAct agent then reads the cleaned transcript, fuzzy-matches mentioned names against an employee directory, inserts tasks into the database, and emails assignees via Gmail or SMTP.

Mic (sounddevice) → Sarvam STT (WebSocket) → Voice Command Detector → live highlights
                                                        │
                                              [meeting ends]
                                                        │
                        3-Phase Transcript Refiner → ReAct Agent → Task DB → Email

Key Decisions & Tradeoffs

I used regex + an LLM fallback for voice commands — deterministic phrases resolve instantly; ambiguous speech falls back to an LLM intent call, adding ~2s latency in exchange for handling natural phrasing.
I built a 3-phase refinement pipeline with a quality gate — it explicitly checks whether the LLM's grammar pass corrupted speaker labels or dropped more than half the content, and reverts to the rule-based output if so. I added that complexity specifically to catch LLM hallucination before it reaches a task list.
I picked a single vendor (Sarvam AI) for both STT and the LLM — simpler integration and real Hindi/English code-switching support, at the cost of full vendor lock-in.
I used fuzzy name matching (Dice coefficient) for the employee directory — resilient when STT mishears a name, though short names occasionally produce false positives.

Highlights

Live transcription via Sarvam AI STT streamed over WebSocket with regex+LLM voice command detection for flagging key moments
3-phase transcript refinement pipeline with a quality gate that catches LLM hallucination before it reaches task output
ReAct agent that extracts tasks with assignees and deadlines via fuzzy name matching (Dice coefficient) against an employee directory
Emails tasks to assignees automatically via Gmail or SMTP after each meeting
60% test coverage minimum enforced by CI — with mocked API and email paths

The Problem

What I Built

Mic (sounddevice) → Sarvam STT (WebSocket) → Voice Command Detector → live highlights │ [meeting ends] │ 3-Phase Transcript Refiner → ReAct Agent → Task DB → Email

Key Decisions & Tradeoffs

I used regex + an LLM fallback for voice commands — deterministic phrases resolve instantly; ambiguous speech falls back to an LLM intent call, adding ~2s latency in exchange for handling natural phrasing.

I built a 3-phase refinement pipeline with a quality gate — it explicitly checks whether the LLM's grammar pass corrupted speaker labels or dropped more than half the content, and reverts to the rule-based output if so. I added that complexity specifically to catch LLM hallucination before it reaches a task list.

I picked a single vendor (Sarvam AI) for both STT and the LLM — simpler integration and real Hindi/English code-switching support, at the cost of full vendor lock-in.

I used fuzzy name matching (Dice coefficient) for the employee directory — resilient when STT mishears a name, though short names occasionally produce false positives.

Highlights

Live transcription via Sarvam AI STT streamed over WebSocket with regex+LLM voice command detection for flagging key moments

3-phase transcript refinement pipeline with a quality gate that catches LLM hallucination before it reaches task output

ReAct agent that extracts tasks with assignees and deadlines via fuzzy name matching (Dice coefficient) against an employee directory

Emails tasks to assignees automatically via Gmail or SMTP after each meeting

60% test coverage minimum enforced by CI — with mocked API and email paths