HandsIn brings AI feedback to the moment a learner tries a sign

Watching a sign gives a learner a model to follow. Trying it raises another question: did that movement communicate the intended sign? HandsIn places AI at that second moment.

Data partner CLERC’s 5 October launch announcement says the web platform, at handsinlearning.co, uses sign recognition to give immediate feedback on learners’ signs. Lessons feature Deaf native ASL signers with teaching experience, filmed in HandsIn’s studio, and use contextual clues, recall and repetition.[1]

ASL means American Sign Language: a language with its own grammar, expressed through movements of the hands and face. It is distinct from British Sign Language. The learning target matters because there is no single universal sign language.[3]

The announcement supplies no measured learning gains or detailed evaluation of app corrections. Its stated aim is feedback on repeated attempts at lesson signs.[1]

An example has to move

CLERC says HandsIn’s recognition was trained on its sign-language corpus. The announcement does not identify a particular dataset release or model configuration. A separate public CLERC research release, Épée v0.3, makes the underlying data problem easier to see.[1][2]

That release contains 1,200 clips from six Deaf signers, distributed as motion keypoints and annotations rather than raw video. Its records include the English phrase, a signer identifier and time boundaries around labelled signs. Shared phrase identifiers let researchers compare different people signing the same phrase.[2]

One published record for “Thank you” marks where the sign starts and ends within the clip. The label identifies a stretch of movement. A still frame taken somewhere inside it would leave the surrounding motion unexplained.[2]

Repeating a phrase across signers also makes variation visible. A recognition system must find the relevant pattern through differences in speed, body proportions and signing style. For a learner, a useful comparison concerns whether an attempt expresses the sign; matching a teacher’s appearance frame by frame would be a different task.

The public release is a limited research sample. Its card notes reduced facial landmarks and missing layers of grammatical annotation. It does not establish the app’s correction accuracy or a learner’s overall ASL ability.[2]

The people behind the practice

Native expertise appears in two places in HandsIn’s announced design: the teacher demonstrating a sign and the examples used to train recognition. CLERC says its contributors are paid and consent to the use of their recordings. It plans to record and annotate additional signs as HandsIn adds modules.[1]

That human variation is part of the language. NIDCD describes regional differences in ASL and explains that facial expression and body movement can carry grammatical meaning. A practice tool can help with an attempted sign while lessons and conversation give that sign a wider context.[3]

The promise of the loop is modest enough to be useful: see a skilled signer, make an attempt, receive feedback and try again while the example is fresh. Each repetition can become preparation for the thing a learner ultimately wants to do—understand another person and be understood.

Report an error or suggest a correction