Service
AI Voice Evaluation in India
Human evaluation of your speech models: MOS and preference testing for TTS, WER-in-context review for ASR, and native-speaker judgement on naturalness and intelligibility.
- Turnaround
- 1-3 weeks per evaluation round.
- Languages
- 14 Indian languages + Indian English
- Delivery
- Raw per-rater scores
What you get
- Raw per-rater scores
- Aggregated results with confidence intervals
- Error-type analysis
- Recommended fix priorities
Technical specification
Every parameter below is written into the statement of work before recording begins. If your pipeline needs different values, they replace ours rather than being converted after delivery.
| Parameter | Standard |
|---|---|
| TTS | MOS (1-5), MUSHRA, and A/B preference protocols |
| ASR | Error typing: substitution, deletion, insertion, code-switch failure |
| Panel | Native speakers of the target variety, screened and calibrated |
| Sample size | Powered per the effect size you need to detect |
| Reporting | Per-item scores plus aggregate with confidence intervals |
How the work runs
- Protocol design and sample-size calculation
- Panel recruitment and calibration on reference items
- Blind evaluation with attention checks
- Statistical analysis
- Report with per-error-type breakdown
Quality control
Attention checks and reference anchors are embedded so unreliable raters are detected and excluded before analysis.
QA failures are remedied by re-collection, not by editing the delivered files. Repaired audio introduces artefacts that survive into your model.
Speaker and contributor sourcing
Raters are excluded if they contributed to the training data for the model under test.
Consent is captured per participant and mapped to file IDs, so provenance survives an external audit of your training data.
Timeline
1-3 weeks per evaluation round.
Staged delivery is available: first batches ship while later batches are still recording, so training can start early.
The numbers we hold ourselves to
- 100% of delivered files pass automated technical QA for SNR, clipping, duration and silence
- 5-25% of files pass a second native-speaker content review, stratified by city, dialect and transcriber, and escalating to 100% on any batch that fails the agreed threshold
- Accepted yield runs 85-90% for scripted speech, 60-70% for spontaneous, 55-65% for conversational and 50-60% for telephony
- Default cohort quotas: 50/50 gender, with age bands at 30% (18-25), 40% (26-40) and 30% (41-60)
- 48 kHz / 24-bit capture, delivered as 16-bit PCM WAV, with studio sessions held below a -50 dBFS noise floor
- First response within one working day; a scoped, fixed quote within two to three
These are the figures a delivery is measured against, not aspirations. A batch that misses them is re-recorded at our cost rather than repaired.
Commonly used for
- TTS QA
- ASR benchmarking
- Model release gating
Frequently asked
What is the minimum volume for voice evaluation?
Programmes typically start around 50 hours or equivalent units per language. Smaller pilots are accepted when they lead into a larger build, because most of the setup cost is in specification and recruitment rather than recording time.
Can you work to our schema instead of yours?
Yes. Manifest fields, file naming, directory structure and label schema are set by you. Working to your schema from the start avoids a conversion pass that usually loses metadata.
Who owns the delivered data?
You do. Deliverables come with a perpetual, transferable licence and participant consent that covers model training and distribution of the resulting model.
How is pricing structured?
Per delivered hour or per unit, quoted against a written specification. Quotas, recording conditions and QA thresholds all move the price, which is why we quote from a spec rather than from a price list.
Get a quote for ai voice evaluation
Send the specification you already have, or the rough shape of it, and you get a scoped quote with a timeline.