[Sw-l] Question about AI recognition of a sign in a video

[Honza Fikejs]

Hi Adam and Amit,

I am also interested in this field. I have tested one app that uses some
apple tools, but it is still not clear to me how it works.
I will send you some questions privately.

Honza

On Tue, Nov 26, 2024 at 10:41 PM Adam Frost <icemandeaf at gmail.com> wrote:

> Thanks for this information. It is a lot for me to grasp, but I am willing
> to work through it.
>
> I’ll probably send you a few private messages for more explanation and
> help. ;-)
>
> [image: namesign.png]
> Adam
>
> On Nov 26, 2024, at 12:44 PM, Amit Moryossef <amitmoryossef at gmail.com>
> wrote:
>
> Here is a video explaining how this works https://youtu.be/HrFoEtLyCYs
> Please let me know if you need manual captioning or if YouTube is OK.
>
> Here is the colab notebook
> https://colab.research.google.com/drive/10QBoGvwgpEt8RLL7662LbGW7pfMQkAx2?usp=sharing
>
> (I see you are using PHP, you can call the API from that, no need for
> python)
>
> This isn't the best, but definitely going to improve in the future, and I
> think it is worth implementing.
>
> Best of luck!
> Amit
>
> On Tue, Nov 26, 2024 at 8:36 PM Adam Frost <icemandeaf at gmail.com> wrote:
>
>> Yes, this does seem something that would work for me.
>>
>> Can you explain how I would use the API? I have a list of all the current
>> videos that I have at https://whatsthatsign.com/directory.php.
>>
>> The project is continually growing, so I would be having new
>> signs/words/data being added all the time. What would be the steps that I
>> would need to do to include those with the API as well?
>>
>> <namesign.png>
>> Adam
>>
>> On Nov 26, 2024, at 10:56 AM, Amit Moryossef <amitmoryossef at gmail.com>
>> wrote:
>>
>> Recent work from our lab (https://arxiv.org/pdf/2407.01264) can embed
>> and retrieve any single sign video in any sign language.
>> You could even retrieve it against spoken language text.
>>
>> For your use case, you can use our API, send all of your videos and
>> receive vectors.
>> Then, when a new video is recorded, you can embed it using the API,
>> and check for similarity with the existing vectors.
>>
>> Amit
>>
>> Amit
>>
>> On Tue, Nov 26, 2024 at 6:16 PM Adam Frost <icemandeaf at gmail.com> wrote:
>>
>>
>> I recall in the past that there maybe have been someone or maybe a group
>> on the list who was doing some work where a user could turn on their camera
>> and then sign something in a sign language, and then there would be a
>> result telling what sign was made in that video. I just can’t remember who
>> it was.
>>
>> I have a couple of projects that would actually benefit from this sort of
>> recognition ability. One to be able to give results of a list of signs from
>> a dictionary of videos that closely match the inputed sign. And another
>> that would be able to tell the user that they are signing a prompted sign
>> correctly or not.
>>
>> If this is something that you are currently able to do, please contact me
>>
>>
>> Adam
>>
>> _______________________________________________
>> Sw-l mailing list
>> Sw-l at listserv.linguistlist.org
>> https://listserv.linguistlist.org/cgi-bin/mailman/listinfo/sw-l
>>
>> _______________________________________________
>> Sw-l mailing list
>> Sw-l at listserv.linguistlist.org
>> https://listserv.linguistlist.org/cgi-bin/mailman/listinfo/sw-l
>>
>>
>> _______________________________________________
>> Sw-l mailing list
>> Sw-l at listserv.linguistlist.org
>> https://listserv.linguistlist.org/cgi-bin/mailman/listinfo/sw-l
>>
> _______________________________________________
> Sw-l mailing list
> Sw-l at listserv.linguistlist.org
> https://listserv.linguistlist.org/cgi-bin/mailman/listinfo/sw-l
>
>
> _______________________________________________
> Sw-l mailing list
> Sw-l at listserv.linguistlist.org
> https://listserv.linguistlist.org/cgi-bin/mailman/listinfo/sw-l
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/sw-l/attachments/20241127/0dc9f65b/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: namesign.png
Type: image/png
Size: 440 bytes
Desc: not available
URL: <http://listserv.linguistlist.org/pipermail/sw-l/attachments/20241127/0dc9f65b/attachment.png>