[Sw-l] SignTube Program for SW captions on videos

[Jonathan Duncan]

Hey John,
    I put your message through ChatGTP-4. Just in case any of this helps 
with your project.

Your project sounds very interesting, and you've done a lot of work on 
it already. SWML is a Sport Movement Standard that is used to describe 
and analyze sports movements. Even though it isn't directly related to 
your use case (streaming Humanoid information), the principles used in 
its design could be helpful to consider.

In your Python script, you're running into a problem with repeated TCP 
socket connections in a loop. The error you're getting usually suggests 
that the other end of the socket (in your case, the server) is 
prematurely closing the connection.

There are a couple of potential issues and solutions:

1. Server Side: Ensure your server script handles multiple connections 
efficiently. Perhaps the server limits the number of open connections at 
a time and drops others.

2. Client Side: TCP connections often have a short time delay (a few 
seconds) before fully closing, to ensure that all data gets through. 
Rapidly opening and closing connections could exhaust your available 
"ephemeral ports" and cause connection issues. If possible, try opening 
one persistent connection and sending all your data down this.

3. Data Send: Always ensure data is sent completely. Using `sendall()` 
instead of `send()` is a good practice as it continues to send data from 
the string until either all data has been sent or an error occurs.

Regarding streaming in Python, many people use Python libraries like 
Flask or Django to create streams, so you may find these helpful. 
Popular libraries for socket programming include `socketio`, 
`websockets`, `aiohttp` etc.

Also, for networking issues, you may want to consider using "Wireshark" 
or any other network protocol analyzer tool. This will help you know 
what's happening at the network layer.

As for the future language for your project, JavaScript could be a good 
choice. It is widely used for both front-end and back-end development 
(Node.js), and has good support for sockets and streaming data (via 
libraries like `socket.io`).

Keep in mind, moving away from Python, you'll need to handle connecting 
to MediaPipe in a different way. MediaPipe provides precompiled 
libraries and wrappers for multiple languages including Python, C++, and 
Java. For languages other than these, consider running MediaPipe in a 
Docker container and connect to it with your preferred language.

Finally, do remember that networking and sockets can be tricky. Issues 
can stem from firewall rules, operating system settings, hardware 
limitations and much more. Debug systematically, change one thing at a 
time, and you'll nail down the problem. You're definitely on the right 
path - keep going!

On 8/14/2023 4:22 PM, John Carlson wrote:
> Subject:
> Re: [Sw-l] SignTube Program for SW captions on videos
> From:
> John Carlson <yottzumm at gmail.com>
> Date:
> 8/14/2023, 4:22 PM
>
> To:
> Ronnie Fagundes de Brito <ronniefbrito at gmail.com>
> CC:
> Jonathan Duncan <jonathandouglasduncan at gmail.com>, SignWriting List 
> <sw-l at listserv.linguistlist.org>
>
>
> I'd like to hear more about SWML as I am creating a new streaming 
> format (which is currently unnamed, but a previous version had the 
> extension .rg for route graph, before that I was using serialized 
> Java) for recording and processing Humanoid information from 
> MediaPipe. I'm having trouble with networking using Python, so I 
> haven't done much streaming yet, apparently default sockets on python 
> require many connections, so I might not be able to 
> do streaming unless I find a better socket library. I've tried turning 
> off my firewall, added exclusions to antivirus, and used a virtual 
> environment that was excluded from antivirus.  I get like a page or 
> two of data in the terminal on the server log, and I get: 
> "ConnectionAbortedError: [WinError 10053] An established connection 
> was aborted by the software in your host machine."  I'm on Windows 10.
>
> My format approach is to do something like BVH (Biovision Hierarchy), 
> but a graph instead of a hierarchy (with From/To links for bones 
> between two joints), but I think I will be able to remove things from 
> the "skeleton graph," which is an important feature of my streaming 
> format.   I'm streaming with python that generates data from Google 
> MediaPipe's Holistic.  I haven't moved on from this to the MediaPipe's 
> new tasks structure, but I might if I can get x,y,z coordinates, even 
> if I can't see the skeleton on the video.  At least I'll have 
> video (with opencv).  But I'm moving away from Python, probably to 
> JavaScript.  I don't see Java being taken seriously by MediaPipe. 
> Android/Kotlin, yes--I guess I could learn Kotlin.
>
> I would like to prove a client/server approach so I can separate the 
> skeleton geometry stream generation from the rest of the system.  I 
> realize that saving the stream to a file is an important first step, 
> but if I can't get networking working on my local system, I might use 
> bash piping to go from one tool to the next, or save to a file 
> (yuck!). Using a pipe is more private, but fewer features than networking.
>
> Here's a Python program loop that has multiple connections (couldn't 
> do one), that actually *doesn't* work, Apparently, one's data needs to 
> be extremely short. It varies between the last string being "CHICKEN," 
> "CROW," and "EOF."  I guess it's time to do the stackoverflow thing, 
> if not Copilot.
>
> import socket
>
>
> sock = None
>
> def msgSend(sock, data):
>     if data:
>         sock.send((data).encode())
>
> def socketCreate():
>     HOST, PORT = "localhost", 3000
>     sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM);
>     sock.connect((HOST, PORT))
>     return sock
>
>
> # Create a socket (SOCK_STREAM means a TCP socket)
> if __name__ == '__main__':
>     for i in range(1,2001):
>         print(f"Connection {i}:");
>         sock = socketCreate()
>         msgSend(sock, "DUCK\n")
>         msgSend(sock, "GOOSE\n")
>         msgSend(sock, "RAVEN\n")
>         msgSend(sock, "EAGLE\n")
>         msgSend(sock, "CHICKEN\n")
>         msgSend(sock, "CROW\n")
>         sock.sendall("EOF\n".encode())
>         sock.shutdown(socket.SHUT_RDWR)
>         sock.close()
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/sw-l/attachments/20230815/3f88cf0a/attachment.htm>