[Slling-l] AI + machine learning to teach SLs on the web

[Mark Mandel]

*Machine learning to make sign language more accessible*
<
https://blog.google/outreach-initiatives/accessibility/ml-making-sign-language-more-accessible/amp/
>

*Kemal El Moujahid*
* Director, Product Management *
* Published Dec 01, 2021*

Google has spent over twenty years helping to make information accessible
and useful in more than 150 languages. And our work is definitely not done,
because the internet changes so quickly. About 15% of searches we see are
entirely new every day. And when it comes to other types of information
beyond words, in many ways, technology hasn’t even begun to scratch the
surface of what’s possible. Take one example: sign language.

The task is daunting. There are as many sign languages as there are spoken
languages around the world. That’s why, when we began exploring how we
could better support sign language, we started small by researching and
experimenting with what machine learning models could recognize. We spoke
with members of the Deaf community, as well as linguistic experts, working
closely with our partners at The Nippon Foundation, The Chinese University
of Hong Kong and Kwansei Gakuin University. We began combining several ML
models to recognize sign language as a sum of its parts — going beyond just
hands to include body gestures and facial expressions.

*...*

Advances in AI and ML now allow us to reliably detect hands, body poses and
facial expressions using any camera inside a laptop or mobile phone.
SignTown uses the MediaPipe Holistic model
<https://google.github.io/mediapipe/solutions/holistic> to identify
keypoints from raw video frames, which we then feed into a classifier model
to determine which sign is the closest match. This all runs inside of the
user's browser, powered by Tensorflow.js <https://www.tensorflow.org/js>.


*Click link for full article*

Mark Mandel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/slling-l/attachments/20211203/73c91771/attachment.htm>