Case Study · Call Quality Monitoring · Built with Claude Code
Flatworld Solutions goes from under 5% to 100% call QA coverage on a platform built with Claude Code
Decision Foundry used Claude Code to build QualityAI, a call quality management platform for Flatworld Solutions that replaced manual, sample-based auditing with full coverage across 10 campaigns. In its first month of production, the platform processed 2,500 calls and recovered 100 hours of QA effort.
With Claude Code, Decision Foundry built:
A multi-tenant QA platform that is live across 10 campaigns, with Claude co-authoring 428 of its 520 commits (82%)
Automatic scoring of every call against each client's own rubric, criterion by criterion, with transcript evidence behind each verdict
Dispute, calibration, coaching and SLA workflows that keep human QA specialists in control of the result
QA coverage that went from under 5% to 100% of calls in the first month of production
An estimated 100 hours of manual QA effort recovered in that first month
The Customer
Flatworld Solutions, a BPO (business process outsourcing) company running high-volume inbound and outbound call center operations on behalf of its clients. At any given time its QA teams support 7 to 10 client accounts, each with its own scoring rubric, service standards, and compliance requirements.
The Problem
Every call is expected to be audited against a defined rubric, but with QA teams doing this manually, only a small fraction of calls were ever reviewed. The rest went unaudited, leaving quality gaps, inconsistent feedback, and no reliable way to coach agents or demonstrate compliance. Flatworld Solutions engaged DF to build a platform that could bring structure and scale to the QA process: supporting multiple clients, standardizing how calls are scored, and giving QA teams the workflows they needed to actually do their jobs without the volume becoming a bottleneck.
The Solution
DF built a structured call quality management platform that replaced manual, sample-based auditing with full AI and human coverage across all 10 campaigns.
Unified call queue
Every call, across every client campaign, in one queue with live scoring and sign-off status.
Automatic AI scoring
Every call scored against the client's full rubric automatically, criterion by criterion, with a sentiment timeline and a pass/fail result.
Manual QA review
A structured workspace for QA specialists to score calls manually against the same rubric, synced to the transcript, for the calls that need human judgment.
Dispute and escalation workflow
Agents can dispute a score directly on the criterion in question. Disputes route into a queue with SLA tracking instead of getting lost in email threads.
Agent self-service
Agents see their own call history, scores, and sign-off status in a personal dashboard, without waiting on a QA manager.
Multi-account management
QA teams switch between client accounts and campaigns from a single workspace, so one platform supports all 10 campaigns.
How every call is scored
Recordings arrive from each client's Vicidial dialer through Amazon S3, and the platform routes each file to its campaign automatically. AssemblyAI transcribes the call with speaker labels, and every utterance is numbered and timestamped. The scoring model then reads the full transcript against the campaign's active rubric.
For every rubric attribute, the model returns a YES, NO or N/A verdict, a written justification, and the exact transcript lines that support it. The model gives judgements only and does no arithmetic. A deterministic calculator turns the verdicts into the score: 85 or above passes, anything lower fails, and any fatal attribute fails the call outright.
Because each verdict points to specific lines in the transcript, a QA specialist or an agent can check the reasoning in seconds. That traceability is what makes the dispute workflow practical at 100% coverage.
Decision Foundry built the scoring layer on the Claude API, and Claude Sonnet 5.5 now scores every call in production. The design uses Claude Haiku 4.5 as a lower-cost tier for high-volume campaigns and Claude Opus 5.5 for hard reviews and rubric refinement. Structured outputs guarantee that every rubric attribute comes back in a fixed schema. Prompt caching reuses each campaign's rubric and instructions across calls, and the Message Batches API scores calls that aren't urgent at half the cost.
How We Built It
Decision Foundry built QualityAI, a multi-tenant production platform, with Claude Code.
Claude co-authored 428 of the 520 commits in the repository (82%). The team adopted each new Claude Opus model as it was released, from Claude Opus 4.6 for the initial build through Claude Opus 4.8 and Claude Opus 5. For changes that spanned the whole codebase, the team used the 1M-token context mode.
Claude Code built far more than screens. Most of the platform's governance layer was built with Claude Opus 4.8: the dispute and escalation workflow, AI-to-human calibration, coaching, SLA timers and multi-tenant isolation, including row-level security in PostgreSQL. This is usually the slowest part of building a QA platform, because every client brings its own rubric, users and rules.
For Decision Foundry, Claude Code is what made a build of this scope economical. With Claude co-authoring 82% of the commits, a single team delivered both the scoring pipeline and the full governance layer.
Technology Stack
Claude Code built QualityAI, and its scoring layer is built on the Claude API. The platform runs on AWS.
Layer
Technology
Development
Claude Code with Claude Opus models, from Opus 4.6 to Opus 5
AI scoring
Claude API: Claude Sonnet 5.5 as the scoring model, Claude Haiku 4.5 for high-volume campaigns, Claude Opus 5.5 for hard reviews
Claude API features
Structured outputs, prompt caching, Message Batches API
Transcription and sentiment
AssemblyAI (universal-3-5-pro) with speaker labels, sentiment analysis and content-safety detection
Call recording source
Vicidial dialers exporting to Amazon S3, with manual and bulk upload in the app
Frontend
React 18, TypeScript, Vite, Tailwind CSS, TanStack Query, Recharts
Backend
Python 3.11, FastAPI, SQLAlchemy 2.0, Pydantic v2, Celery and Redis for background jobs
Database
PostgreSQL 16 on Amazon RDS, with row-level security for tenant isolation
Hosting and operations
Docker on Amazon EC2 behind Nginx, GitHub Actions for CI, Sentry for error tracking
See It In Action
One workspace for AI scoring, human review, and disputes.
Every call across every campaign in one queue, with scoring and sign-off status at a glance.
AI analysis scores every call against the rubric automatically, criterion by criterion, and cites the transcript lines behind each verdict.
QA teams review and score calls manually where human judgment is required, against the same rubric.
Disputed scores route into a clear escalation workflow instead of getting lost in email threads.
The Challenges We Faced
Moving from under 5% manual sampling to 100% AI-reviewed coverage raised its own questions. Scoring rubrics differ by client and campaign, so the AI analysis had to apply the correct rubric automatically rather than a single generic model. Agents and QA teams also needed to trust the AI's scores, which meant giving agents a clear way to dispute a result and giving QA leads a fast escalation path instead of a backlog of arguments over email. And because the platform now surfaces every call instead of a small sample, disputes and edge cases had to be resolved quickly enough that 100% coverage did not simply turn into a much larger backlog for the QA team.
The rubric problem was solved with versioning. Each campaign has exactly one active rubric, chosen deterministically: the campaign's own rubric first, then the client's default. Every edit creates a new version, and earlier audits stay tied to the version they were scored on.
Trust was earned through calibration. After each campaign goes live, human QA auditors and the AI score the same set of about 20 calls, and the team compares the results attribute by attribute. Each gap is traced to its cause and fixed as a new rubric version, reviewed with the client's QA lead. The affected calls are then scored again several times to confirm that the fix holds.
On one account, only 6 of 20 calls first landed within 5 points of the human score. Three rubric issues explained the gap, including a courtesy-close rule applied to calls the customer had ended. After the fix, the affected cases scored correctly on 6 of 6 repeat runs, and one disputed call moved from 51.7% to 100%, matching the human review. Across accounts, most early gaps came from the rubric and its edge cases, not from the model's judgement.
The Results
The platform went into production in September 2026.
Early numbers from the first month of production:
10
Campaigns live across 7-10 clients
The platform supports multiple client campaigns from a single, standardized QA workflow.
2,500
Calls processed
Every call in scope was processed through the platform in the first month of production.
203 hrs
Of call audio reviewed
Roughly 12,000 minutes of call audio reviewed through the platform.
<5% → 100%
QA coverage
Coverage moved from under 5% of interactions to 100% of all interactions.
100 hrs
Of manual QA effort recovered
An estimated figure, conservative because it still accounts for calls where scores are under dispute and require human review.
82%
Of commits co-authored by Claude
428 of the 520 commits behind the platform were written with Claude Code.
“Decision Foundry used Claude to build a platform that turned a highly manual QA process into one that evaluates 100% of our calls. We now have far greater visibility into quality across every client campaign, and our QA specialists stay focused on the cases where human judgment matters most.”
Jacob William
CEO, Flatworld Solutions
Get In Touch