ASR datasets · LLM Evaluation
ASR Training Data for LLM Evaluation
Transcribed speech corpora built to train and evaluate automatic speech recognition, with verbatim transcription, timestamps and per-token language tagging where code-mixing occurs. Applied to llm evaluation, the specification is driven by one thing: rubric scores with confidence intervals.
- Service
- ASR datasets
- Use case
- LLM Evaluation
- Primary metric
- Rubric scores with confidence intervals
Required data profile
- Native-speaker rater panels per language
- Rubric-based scoring with calibration
- Overlapping assignments for agreement
Technical specification
| Parameter | Standard |
|---|---|
| Audio | 16 kHz or 48 kHz PCM WAV |
| Transcription | Verbatim, including disfluencies, false starts and fillers |
| Timestamps | Utterance level by default; word level on request |
| Tagging | Noise, overlap, unintelligible, foreign-language and code-switch tags |
| Normalisation | Raw and normalised text columns delivered separately |
| Split | Train/dev/test splits with no speaker leakage across splits |
Process
- Style guide authored per language, covering numerals, loanwords, script and disfluency rules
- Transcriber calibration round with inter-annotator agreement measurement
- First-pass transcription
- Second-pass native review
- Automated consistency checks against the style guide
- Split generation with speaker-disjoint partitions
Metrics this feeds
- Rubric scores with confidence intervals
- Inter-rater agreement
- Failure-mode distribution
Failure modes to design out
- Raters who are fluent but not native in the variety
- Rubrics written in English and applied to non-English output without localisation
Agreement is measured, not assumed. We report word-level agreement on a held-out sample so you can judge label quality before training on it.
Deliverables
- Audio plus aligned transcripts (JSON/TSV, or your schema)
- Style guide as delivered documentation
- Inter-annotator agreement report
- Speaker-disjoint train/dev/test splits
Frequently asked
Is asr datasets the right service for llm evaluation?
It covers native-speaker rater panels per language. Most llm evaluation programmes combine it with at least one other service; we will say so in the scope rather than selling one line item.
What languages are available?
All 14 languages in the network plus Indian English accent bands.
How is the evaluation set handled?
Collected first, from speakers disjoint from the training cohort, so improvement is measurable.
Scope asr datasets for llm evaluation
Send the metric you need to move.