[Sw-l] Question about AI recognition of a sign in a video

[Amit Moryossef]

Here is a video explaining how this works https://youtu.be/HrFoEtLyCYs
Please let me know if you need manual captioning or if YouTube is OK.

Here is the colab notebook
https://colab.research.google.com/drive/10QBoGvwgpEt8RLL7662LbGW7pfMQkAx2?usp=sharing

(I see you are using PHP, you can call the API from that, no need for
python)

This isn't the best, but definitely going to improve in the future, and I
think it is worth implementing.

Best of luck!
Amit

On Tue, Nov 26, 2024 at 8:36 PM Adam Frost <icemandeaf at gmail.com> wrote:

> Yes, this does seem something that would work for me.
>
> Can you explain how I would use the API? I have a list of all the current
> videos that I have at https://whatsthatsign.com/directory.php.
>
> The project is continually growing, so I would be having new
> signs/words/data being added all the time. What would be the steps that I
> would need to do to include those with the API as well?
>
> [image: namesign.png]
> Adam
>
> On Nov 26, 2024, at 10:56 AM, Amit Moryossef <amitmoryossef at gmail.com>
> wrote:
>
> Recent work from our lab (https://arxiv.org/pdf/2407.01264) can embed
> and retrieve any single sign video in any sign language.
> You could even retrieve it against spoken language text.
>
> For your use case, you can use our API, send all of your videos and
> receive vectors.
> Then, when a new video is recorded, you can embed it using the API,
> and check for similarity with the existing vectors.
>
> Amit
>
> Amit
>
> On Tue, Nov 26, 2024 at 6:16 PM Adam Frost <icemandeaf at gmail.com> wrote:
>
>
> I recall in the past that there maybe have been someone or maybe a group
> on the list who was doing some work where a user could turn on their camera
> and then sign something in a sign language, and then there would be a
> result telling what sign was made in that video. I just can’t remember who
> it was.
>
> I have a couple of projects that would actually benefit from this sort of
> recognition ability. One to be able to give results of a list of signs from
> a dictionary of videos that closely match the inputed sign. And another
> that would be able to tell the user that they are signing a prompted sign
> correctly or not.
>
> If this is something that you are currently able to do, please contact me
>
>
> Adam
>
> _______________________________________________
> Sw-l mailing list
> Sw-l at listserv.linguistlist.org
> https://listserv.linguistlist.org/cgi-bin/mailman/listinfo/sw-l
>
> _______________________________________________
> Sw-l mailing list
> Sw-l at listserv.linguistlist.org
> https://listserv.linguistlist.org/cgi-bin/mailman/listinfo/sw-l
>
>
> _______________________________________________
> Sw-l mailing list
> Sw-l at listserv.linguistlist.org
> https://listserv.linguistlist.org/cgi-bin/mailman/listinfo/sw-l
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/sw-l/attachments/20241126/4f59484e/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: namesign.png
Type: image/png
Size: 440 bytes
Desc: not available
URL: <http://listserv.linguistlist.org/pipermail/sw-l/attachments/20241126/4f59484e/attachment.png>