The human decisions inside AI’s new language collection

A recording and its annotations can contain a speaker’s words, a translation and a linguist’s commentary. To a speech model, those distinctions determine what can be learned. To the people whose language is being documented, they help make an archive intelligible and usable.[2][3]

ELARSpeech, a preprint posted on 14 October, assembles 30,202 hours across 675 languages and varieties from the Endangered Languages Archive. More than 90% of the audio is unlabelled. The authors’ main joint speech-recognition training and evaluation cover 233 languages, so the collection’s full breadth should not be read as demonstrated transcription capability.[1]

The source institution has a wider purpose. CLARIN’s description of ELAR identifies preservation, support for depositors and access for language communities alongside access for researchers and the public. Preparing a training set is one possible use of that accumulated work.[3]

Which words belong to the voice?

ELAN, the annotation software used in this kind of documentation, explains why a transcript is more complicated than a text file beside an audio file. Its annotations can occupy separate, related layers: a sentence, an individual word, a gloss, a translation or a comment. They may attach directly to a moment in the recording or refer to another annotation. The software stores these relationships in its EAF format.[2]

That structure matters when preparing examples for an AI. Imagine a short utterance accompanied by both its original wording and an English translation. Pairing the voice with the English layer would teach a different task from recognising the words actually spoken. Flattening the layers without understanding their relationship can silently change the meaning of the training example.[2]

ELARSpeech therefore includes a procedure for selecting transcription layers, with uncertain candidates sent for human review. Its results also resist a simple bigger-is-better story: across 30 comparison languages, separately fine-tuned Whisper models averaged 22.48% character error, versus 30.71% for joint multilingual training. Lower is better; these are averages across languages.[1]

Useful AI can start with a smaller job

A separate 2022 project on Muruwari recordings shows another route from archive to useful tool. Researchers worked with a 136-hour collection, much of it recorded by James “Jimmie” Barker in the 1970s. His grandson Roy Barker, a Muruwari elder and custodian of the collection, was a coauthor. These were restricted materials held at AIATSIS, a different archive.[4]

The proposed workflow identified speech and language, then transcribed English passages. Those passages could help authorised people find nearby Muruwari material. It also preserved a stage for the custodian to review material before other permitted annotators worked on it. The software’s contribution was to make a governed process more manageable.[4]

In a timed study using three recordings of roughly half an hour each and four annotators, assisted segmentation and transcription took about 20% less time overall than manual work. That narrow result does not measure language revival. It measures a practical reduction in the labour of making recordings searchable.[4]

This is an appealing place to look for value: a tool that helps a custodian locate and assess material can be useful before a model becomes fluent in the language. It gives the person responsible for the recordings a better way to work through them.

The archive still has conditions

ELARSpeech’s authors say they consulted ELAR before collection, included Open and User Access materials, and excluded Subscriber Access materials. Their derivative licence does not replace the source collections’ conditions. Provenance and collection-specific terms remain attached to downstream use. The paper’s permission account concerns archive access and collection licences.[1]

There is also a more ordinary access problem. In a 2024 presentation, ELAR staff described how an English-only interface obstructed access for some communities represented in the archive. Their regional-portals work began with Latin America and involved adapting metadata and descriptions for regional languages. That presentation describes the work at the time; it does not establish the present coverage of every portal.[5]

A speech model and an archive interface can consequently expand access in different ways. One may recognise sounds in more languages; the other may let a family find a recording, understand its description and discover whom to ask about it. Both deserve attention. Counting languages in a training collection tells us little about the second kind of progress.[5]

Report an error or suggest a correction