"signing avatars" and other machine interpretation projects

[Christian Vogler]

On Thu, Aug 30, 2001 at 04:26:09PM -0400, Angus B. Grieve-Smith wrote:
> On Wed, 29 Aug 2001, Karlin, Ben wrote:
> > I don't think that the signing avatar projects [...] are based on
> > "Sign is Easy."  From what I have seen they are early attempts to push
> > technology in all directions.  They marry machine translation, gesture
> > and speech recognition, video and graphics processing (polygamous
> > marriage, here) in an attempt to meet a need.

>         A few of them do; [...] There are also some that focus on a
>         particular subcomponent, > like synthesis or recognition,
>         and wisely avoid an attempt at full > translation.

>         I have to say that these are only a few.  Most of the projects are
> developed by computer scientists with no understanding of linguistics and
> little or no knowledge of any sign language.  Simply put, there is NO
> machine translation going on for most of these projects.  Even the
> ViSiCAST and CRL projects do not explicitly acknowledge that any form of
> machine translation is neccessary (and hence avoid any acknowledgment of
> the current limits in machine translation), but only make vague references
> to "artificial intelligence" and "natural language processing."

As one of the computer science people who is working on one of the
subcomponents (recognition), I felt that I have to pitch in
here. Personally, I am working on sign language recognition from the
point of view how best to model the language and all the interesting
linguistic properties that come with it.

Most of the research, however, that is being done in this area is
aimed at pushing the technological envelope, as Ben Karlin correctly
points out. I do not see anything wrong with this per se, but I have
to admit that it sometimes bugs me when people start making unfounded
claims about their systems that stem from a lack of understanding of
some of the real difficulties in building a truly usable system. :-)

Most of the time, those people choose sign language recognition as a
topic to work on simply because we know a lot more about signed
languages than about gestures, or other kind of human full-body
movements. So, it is easier to obtain some kind of ground truth for
the research experiments. It makes good technical sense, since this
constitutes a way to simplify an already extremely difficult research
problem, such as recognition, or synthesis.

It seems to me that some of the bad feelings here on this list rise
out of a misunderstanding what the needs are that these researchers
are trying to meet. More often than not, those kinds of research
projects are funded by the army, navy, or air force to push the
envelope in surveillance, training people, and so on. The goals of
these projects are therefore very different from the goals of a
project that is aimed squarely at improving the lives of the
deaf. Choosing sign language is then just a convenient way to
constrain the research problem, and perhaps to pick up some additional
funds from a disabilities research grant somewhere along the way.

I hate to admit it, but the tough competition in computer science for
research grants has led to a pervasive culture of exaggerating claims
what "our" systems can and cannot do, and what they are useful
for. "Helping" the deaf by way of a sign language project looks good
on anyone's CV and in the news media, no matter how much they actually
care or know about the deaf and sign language. So, these people simply
list it as an application of their research and let the world know
about how "close" they are to building something that would actually
be useful.

It is not particularly ethical, but unfortunately this is currently
the reality in computer science, especially in the fields related to
artificial intelligence. This culture of exaggerating claims to gain
an advantage in the competition for grants is deeply wrong, IMHO, but
until it changes, I cannot fault individual researchers too much for
what they do. Most postdoctoral fellows and assistant professors are
in no position to go against the flow, thanks to the demands that
their respective departments place on them.

- Christian