[Sw-l] Question about AI recognition of a sign in a video

[Adam Frost]

Thanks for this information. It is a lot for me to grasp, but I am willing to work through it. 

I’ll probably send you a few private messages for more explanation and help. ;-)


Adam

> On Nov 26, 2024, at 12:44 PM, Amit Moryossef <amitmoryossef at gmail.com> wrote:
> 
> Here is a video explaining how this works https://youtu.be/HrFoEtLyCYs
> Please let me know if you need manual captioning or if YouTube is OK.
> 
> Here is the colab notebook https://colab.research.google.com/drive/10QBoGvwgpEt8RLL7662LbGW7pfMQkAx2?usp=sharing 
> (I see you are using PHP, you can call the API from that, no need for python)
> 
> This isn't the best, but definitely going to improve in the future, and I think it is worth implementing.
> 
> Best of luck!
> Amit
> 
> On Tue, Nov 26, 2024 at 8:36 PM Adam Frost <icemandeaf at gmail.com <mailto:icemandeaf at gmail.com>> wrote:
>> Yes, this does seem something that would work for me.
>> 
>> Can you explain how I would use the API? I have a list of all the current videos that I have at https://whatsthatsign.com/directory.php.
>> 
>> The project is continually growing, so I would be having new signs/words/data being added all the time. What would be the steps that I would need to do to include those with the API as well?
>> 
>> <namesign.png>
>> Adam
>> 
>>> On Nov 26, 2024, at 10:56 AM, Amit Moryossef <amitmoryossef at gmail.com <mailto:amitmoryossef at gmail.com>> wrote:
>>> 
>>> Recent work from our lab (https://arxiv.org/pdf/2407.01264) can embed
>>> and retrieve any single sign video in any sign language.
>>> You could even retrieve it against spoken language text.
>>> 
>>> For your use case, you can use our API, send all of your videos and
>>> receive vectors.
>>> Then, when a new video is recorded, you can embed it using the API,
>>> and check for similarity with the existing vectors.
>>> 
>>> Amit
>>> 
>>> Amit
>>> 
>>> On Tue, Nov 26, 2024 at 6:16 PM Adam Frost <icemandeaf at gmail.com <mailto:icemandeaf at gmail.com>> wrote:
>>>> 
>>>> I recall in the past that there maybe have been someone or maybe a group on the list who was doing some work where a user could turn on their camera and then sign something in a sign language, and then there would be a result telling what sign was made in that video. I just can’t remember who it was.
>>>> 
>>>> I have a couple of projects that would actually benefit from this sort of recognition ability. One to be able to give results of a list of signs from a dictionary of videos that closely match the inputed sign. And another that would be able to tell the user that they are signing a prompted sign correctly or not.
>>>> 
>>>> If this is something that you are currently able to do, please contact me
>>>> 
>>>> 
>>>> Adam
>>>> 
>>>> _______________________________________________
>>>> Sw-l mailing list
>>>> Sw-l at listserv.linguistlist.org <mailto:Sw-l at listserv.linguistlist.org>
>>>> https://listserv.linguistlist.org/cgi-bin/mailman/listinfo/sw-l
>>> _______________________________________________
>>> Sw-l mailing list
>>> Sw-l at listserv.linguistlist.org <mailto:Sw-l at listserv.linguistlist.org>
>>> https://listserv.linguistlist.org/cgi-bin/mailman/listinfo/sw-l
>> 
>> _______________________________________________
>> Sw-l mailing list
>> Sw-l at listserv.linguistlist.org <mailto:Sw-l at listserv.linguistlist.org>
>> https://listserv.linguistlist.org/cgi-bin/mailman/listinfo/sw-l
> _______________________________________________
> Sw-l mailing list
> Sw-l at listserv.linguistlist.org
> https://listserv.linguistlist.org/cgi-bin/mailman/listinfo/sw-l

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/sw-l/attachments/20241126/a2ff41dd/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: namesign.png
Type: image/png
Size: 440 bytes
Desc: not available
URL: <http://listserv.linguistlist.org/pipermail/sw-l/attachments/20241126/a2ff41dd/attachment-0001.png>