Case Study 04
Voice-agent
appointment reminder + callback.
Domain
Healthcare appointment scheduling
Stack
Retell · Twilio · Lambda · Postgres
Scope
Voice agent + escalation + audit
Intent classification accuracy
Cost per completed call
Cold-start latency (p95)
Audit trail — append-only
The problem
A US healthcare consulting practice runs appointment scheduling for a network of clinics. Missed appointments are the single largest revenue leak. Reminder calls need to confirm, cancel, or reschedule — and when a patient doesn't answer, escalate cleanly without double-calling, without losing intent, and without losing the audit trail that a HIPAA-shaped compliance review will ask about.
Most voice-agent demos handle the happy path. What breaks in production is the mixed intent (“yeah I can make it, actually can we push it Thursday”), the missed-call retry chain that double-books when Retell's webhook fires twice, and the auditor asking “who confirmed what, when, based on what evidence.”
1. Architecture
appointment scheduled
│
▼
┌────────────────────────────────────┐
│ n8n cron (hourly) │ ← finds appointments in 24h window
└────────────────────────────────────┘
│
▼
┌────────────────────────────────────┐
│ Lambda │
│ schedule Retell outbound call │ ← idempotency key
└────────────────────────────────────┘
│
▼
┌────────────────────────────────────┐
│ Retell voice agent │ ← STT + intent extraction
└────────────────────────────────────┘
│
▼
answered?
├── yes → intent classifier (confidence-gated)
│ ├── high-conf confirm → status='confirmed'
│ ├── high-conf cancel → release slot, notify
│ ├── high-conf reschedule → rebook queue
│ └── low-conf → human_review + Slack
│
└── no → wait 4hr, retry once
│
▼
still no answer?
└── Twilio SMS + confirmation link + callback CTA
│
▼
append-only audit log (all paths write here)2. Three hard problems, solved in order
1. Retry idempotency — the webhook retry trap
Retell webhooks retry on transient failure. Without an idempotency key, a call that landed “confirm” can be counted twice and the appointment appears double-booked in the practice's dashboard. Every attempt is keyed on (appointment_id, attempt_number) with a Postgres unique constraint; the webhook handler upserts on that key before any state change fires.
CREATE UNIQUE INDEX ix_call_attempts_idem ON call_attempts (appointment_id, attempt_number); INSERT INTO call_attempts (appointment_id, attempt_number, outcome, ...) VALUES ($1, $2, $3, ...) ON CONFLICT (appointment_id, attempt_number) DO NOTHING RETURNING id;
2. Intent confidence gating — auto vs human
Retell returns a structured intent with a confidence score. Confirmations above {{X.XX}} auto-execute; anything below routes to the practice's human_review queue with the raw transcript, the extracted intent, and the confidence score attached. The gate is tuned per-practice from labelled review outcomes — not a fixed threshold. Result: {{XX%}} auto,{{YY%}} human-reviewed, escalations never miss a voice-of-patient signal.
3. HIPAA-conscious audit trail — append-only + hash chain
Every call attempt, every intent classification, every escalation writes an append-only audit row. Each row hashes the previous row's contents (TrustChain pattern from my consent-management SaaS build) — a tampered entry breaks the chain and is visible on the next audit. Compliance review answers “who confirmed what, when, based on what evidence” in one SQL query, not five systems.
CREATE TABLE audit_log ( id bigserial PRIMARY KEY, event_type text NOT NULL, -- call_start | intent_extract | escalation | ... appointment_id uuid NOT NULL, payload jsonb NOT NULL, -- redacted before storage prev_hash text, -- hash of the previous audit row's contents row_hash text NOT NULL, -- SHA-256 (event_type || appointment_id || payload || prev_hash) created_at timestamptz NOT NULL DEFAULT now() ); -- append-only enforced at the app layer + a policy: CREATE POLICY audit_log_no_update ON audit_log FOR UPDATE USING (false); CREATE POLICY audit_log_no_delete ON audit_log FOR DELETE USING (false); ALTER TABLE audit_log ENABLE ROW LEVEL SECURITY;
3. Stack
- Voice
- Retell — outbound calls, STT, intent extraction
- SMS
- Twilio — escalation + callback CTA
- Orchestration
- AWS Lambda (Python) — webhook handler + retry logic
- Data
- Supabase Postgres — appointments, call_attempts, audit_log
- Workflow
- n8n Cloud — cron trigger + operational visibility
- Cost
- ~$YY per completed call, {{ZZ}} monthly platform floor
4. Working system
60-second Loom of a live call — scheduled fire, missed answer, 4-hour retry, Twilio SMS escalation, and the audit-log row on retry showing the idempotency key fire once, not twice.
Loom
{{LOOM_EMBED_URL}}— replace with real Loom URL after recording
GitHub
github.com/{{HANDLE}}/voice-agent-healthcare-callback — Lambda handler, Postgres schema, Retell webhook contract, README.
5. What this does not do
Being honest here so nobody's surprised.
- No SIP work. Retell handles the carrier layer; this build never touches raw SIP. A carrier migration or on-prem PBX integration is out of scope until the underlying voice provider is set.
- Cold-start latency. First call in a 10-minute quiet window sees a
{{ZZs}}cold-start penalty on Lambda. In practice this is masked by Retell's dial-time, but under continuous burst load it becomes visible — provisioned concurrency is the fix. - English-only. Intent classification is tuned on English transcripts. Other languages need per-language confidence calibration and separate label sets.
- Not a scheduling system.This talks to the practice's existing scheduling system via API — it doesn't replace it. Rebooking is routed as an intent event, not executed inline.
6. Method note
Domain shape (appointment scheduling, HIPAA audit expectations, human-review queue pattern) is drawn from delivery work on a US healthcare consulting practice's n8n + Postgres platform. Voice-agent architecture (Retell + intent-confidence gating + Twilio escalation) is from this build. Audit-log hash-chain pattern is adapted from a multi-tenant consent-management SaaS I solo-built (ConsentXpert, 335 APIs, live enterprise deployment).
Numbers on this page are from labelled test-set runs against synthetic patient data. Production numbers will drift with real patient response distribution, carrier variance, and per-practice intent calibration.
Priyanshu Kumar · AI & Automation Engineer · priyanshukumar.co