Efficient transcript fusion
Machine learning experiments combining the best words from several speech-recognition transcripts with a learned, ROVER-inspired method.
I align several possible transcripts and learn which words to keep, using agreement, confidence, context and acoustic evidence. On Whisfusion, the combiner achieved 22.64% word error rate, compared with 28.46% for confidence-based transcript selection and 23.39% for tuned ROVER.
The evaluation used 2,143 held-out recordings across 11 evaluation sets, 15 candidates and three training seeds. Refitted for Drax, the combiner matched tuned ROVER on English and improved it on six-language FLEURS. The benefit depends on the model and data.
Efficient transcript fusion on GitHub