Fingerspelling with Machine Learning

Hello Monday

for

American Society for Deaf Children

Kategori :

Type :

andet

Vinder af Digital Value Proposition

SØLV:
Med stor løsningselegance udnytter Fingerspelling cutting edge tech til at imødekomme en kæmpe udfordring. Det er smukt, engagerende og en sølvpris værdig.

Vinder af Digital Design

GULD:
I Fingerspelling er designet skåret helt ind til benet men har stadig karakter. Både onboarding og tegnsprogstræningen er let og legende i sin interaktion og de få virkemidler der er, er velvalgt for at understøtte den sublime brugeroplevelse.

Vinder af Innovation

SØLV:
Tjenesten løser et problem på en innovativ måde. Det er ikke nyt at lære tegnesprog online, men måden er ny. Løsningen rummer høj æstetisk og brugervenlighed. En løsning der viser evnen til at eksekvere ideen på et højt kvalitativt niveau.

Vinder af The Big Idea

SØLV:
Et eksempel på topklasse gamification indenfor uddannelse. Og en oplevelse der simplet udveksler din tid med både viden og empati.

Vinder af Creative Use Of Technology

BRONZE:
This brilliant simple solution, uses machine learning to teach people a new language, and a language that isn’t easily available to learn. The ease of use and gamification provides a safe space and an accessible entry point for learning while being highly scalable.

Baggrund

Every year 2 - 3 out of every 1,000 children born in the US are deaf or hard-of-hearing. 90% of these children are born to hearing parents, and in many cases, their child is the first deaf person these parents have ever encountered.

Without being introduced to sign language at an early stage, a deaf child may miss out on learning language. This can lead to language delay or deprivation, which has long-term negative impacts on a child’s life. That is why it is so important that parents of deaf children have the opportunity to learn American Sign Language (ASL) as soon as possible.

To address language deprivation and help bridge the communication barrier between the deaf and the hearing, we partnered with the American Society for Deaf Children to create Fingerspelling.xyz.

Fingerspelling uses AI and Machine Vision to teach you the ASL Alphabet.

Løsning

Fingerspelling.xyz is a browser-based app that uses a webcam and machine learning to analyze your hand shapes so you can learn to sign the ASL alphabet correctly.

To start, you are given a series of words and the computer shows you a 3D model of how your hand should be positioned for each letter. When you sign the word, the camera tracks your hand movements and gives you feedback so you can make corrections as needed. This helps you quickly develop your fingerspelling skills and move to the next level of the program.

The fingerspelling game is a great way to introduce the basics of ASL in a fun and playful way. Instead of having to read or watch videos about fingerspelling, we offer an online teaching tool that guides you step by step in how to master fingerspelling - hands on! The game leverages advanced hand recognition technology, matched with machine learning, to give you real time feedback via the webcam for each sign and word you spell correctly. The game is designed for desktop, primarily to be used by parents for deaf children, but kids will also find it a fun way to practice and improve their fingerspelling technique.

Fingerspelling is an essential part of ASL, the primary language of the Deaf community. It is often used for proper nouns or to sign a word you don’t know the sign for.

Resultat

The site was launched in late April 2021 - and within 10 days 150.000 correct hand-signs have been made. Totalling for the last 10 months 2.4 million correct hand signs. And continues to be used as a learning tool.
Engagement: Over 50% of those who started a level - completed it.
To complete a level takes around 2 - 4 minutes.

The site has been featured across a variety of news and blogs - including Engadget, Mashable and Yahoo. And we have helped bridge the gap between the hearing and the deaf.

American Society for Deaf Children

Cheri Dowling

Executive Director

Cheri Dowling

Executive Director

Hello Monday

Anders Sønderby Jessen

CTO

Rikke Agersnap

Project Manager

Patrycja Holm

Designer

Anders Mikkelsen

Product Director

Samarbejdspartnere


Watch Video