Face glyphs and Software that Transcribes Videos into SignWriting

Mon Sep 1 13:38:47 UTC 2014

Dear Valerie,
thank you very much fro your deep comments, that are very interesting as
usual.
Of course, we do not pretend to transfer on paper any kind of feeling. I
agree that who signs does  also including a kind of "feeling prosody" in
the expression, and this is somehow similar to non-formal writing in verbal
languages too, where this is rendered by a special use of punctuation and
also by a choise of lexicon. However, as for transcription, I adopt as a
reference the transcription from a spoken discourse.

In this case,  in most situations the transcriber should be as neutral as
possible to avoid to risk to change the intended essence of the message
from the speaker, to interleave with one's own opinions and feelings,etc.
The latter would produce an "augmented" message, which is not necessarily
the same as the original.

I'm neither a linguist nor an expert in SLs, as I repeatedly said, but i
think that this caveat should also hold for transcription from SL . This
suggests that Receptive writing should be more appropriate for
transcription (reproduction of the message from another person). As a
matter of fact, while we speak about "creative writing", we never speak
about "creative transcription".

Maybe the problem is a technical one on the meaning of "transcription".

My "practical" interpretation (as opposed to linguistic) is the following.
By writing, one is  directly producing the written representation of a
concept, an idea, a thought, a situation, an information, a rule, etc., and
this is (more or less) what SW is used for.
With the transcript I think we can distinguish more cases:
1) to reproduce in written form or copy a text without any alteration, and this
is again writing in a broader sense
  2) move from one writing system to another (usually possible, for example
from Latin to Cyrillic); in a broader sense, this encompasses translation;
it is worth noticing that translation in litarature (poems, etc.) is so
hard just because one must attempt to preserve the original message, even
when a contamination with one's own feelings may result in a smarter work;
the idea is that of official documents, where translation must exactly
preserve the message;
3) extract the relevant part of the "speech" (not written) by another and
put it in writing while keeping as much meaning as possible and without
altering it (eg. Shorthand).

We are definitely addressing case 3).

Given this, the automatic handling of this activity can only use Receptive.
Otherwise, we should have a "human-in-the-loop" revising the video AND the
transcription together to pass from Receptive to Expressive ... too much
work I suppose, most of all if we aim at creating large corpora from
existing videos. Better to do it directly in a manual way.
On the other hand, techniques borrowed from studies about affective
computing in HCI (Human-Computer Interaction) with expression and emotion
recognition may help to improve also the automatic result.

Actually, we are just starting working about this, and therefore
(unfortunately) there is no software to mention yet ...

What do you think?

Maria

2014-08-31 17:37 GMT+02:00 Valerie Sutton <signwriting at mac.com>:

> SignWriting List
> August 31, 2014
>
> Hello Maria, Fabrizio and Stefan -
> I believe you are asking for corresponding photos or drawings that could
> be placed side by side with each Facial Expression symbol in SignWriting?
>
> There have been others who ask for this too, and we have considered doing
> this, but the problem is that the human face is very different from person
> to person, and I personally cannot even do all the facial expressions that
> are possible to write in SignWriting. Most people can’t either ;-)
>
> We have had some photos taken in the past to use in this way, but not
> everyone could recognize in the photos exactly what the position was. This
> proves one fact that I had to learn over time…we are not just writing what
> we see…we are also writing what we feel, and what is considered necessary
> facial expressions for the understanding of the signs…not all facial
> expressions are needed. When you transcribe videos you need to know the
> difference between the facial expressions that are needed and the facial
> expressions that are not needed…that is not easy, with or without
> automation.
>
> SignWriting has many aspects. It is a handwriting. It is expressing
> signer’s expression from their own perspective. That is the Expressive
> Viewpoint which is our standard.
>
> But as you know for a decade we wrote Receptively. I was given an
> assignment, back in 1974, at the University of Copenhagen, to write what I
> saw people sign (and gesture) on a videotape of both hearing and Deaf
> people. The project was “writing the difference between hearing person’s
> gestures, and the movements of Deaf people signing Danish Sign Language”…so
> you can imagine the Deaf people were expressing their language and the
> hearing people were supporting their spoken language with minimal
> gestures…that was my beginning with writing signs and developing the
> beginning of SignWriting and it was through that project that I realized
> that the sign languages of the Deaf are real languages and it was clear
> that the hearing people did not have a sign language, but were using
> gestures to support their spoken language.
>
> But being the hearing person I am (smile), I made the mistaken assumption
> that when we started to write real sign languages, that we should write
> from videotape and write exactly what we see…not feel. That came from the
> fact that I did not know the languages I was writing from videotape. I felt
> I could not impose my lack of knowledge on the writing, so I made a strong
> rule that we just write what we see, and it was Receptive, from 1974 to
> 1984.
>
> But Receptive writing is not the same as Expressive writing - there are
> larger differences than most people realize…Later it was our Deaf staff,
> Lucinda O’Grady and Meriam Ina Schroeder, who requested switching from
> Receptive to Expressive in 1984, after three years writing articles in ASL
> in the SignWriter Newspaper in the Receptive, they became fed up with
> Receptive and demanded Expressive and I am glad they did!
>
> They explained in a meeting in 1984 that they were really writing from
> their own experience, and were not writing from videotape…and when we write
> Expressively, which is the standard now, we write what we FEEL too… My
> immediate response was “what about Facial Expressions? how can we “see
> Facial Expressions” if we are writing ourselves?” and Lucinda explained it
> is what we “feel” while we are signing and writing that matters to a person
> fluent in writing ASL...
>
> Which brings us back to video transcription - Video transcription fits
> well with Receptive, but nonetheless, most people I know use the Expressive
> to transcribe video because sooner or later the transcript must be in the
> Expressive for others to read... so most people write the videos in the
> Expressive from the beginning…
>
> And Facial Expressions truly are based on a combination of what we see and
> what we feel…so the photos you are needing are harder because not all
> facial expression photos really portray what a person feels while they are
> signing…
>
> So here are my questions to you Maria and Fabrizio…
>
> 1. Are you creating software that does an automatic transcription of sign
> language videos into SignWriting?
>
> 2. What is the name of this new software so I can refer to it easier? ;-)
>
> 3. Are you developing the software to transcribe videos in the Recpetive
> or the Expressive or both?
>
> Val ;-)
>
>
> -----------
>
>
>
>
> On Aug 27, 2014, at 2:54 AM, Maria De Marsico <demarsico at DI.UNIROMA1.IT>
> wrote:
>
> Hi Stefan,
> thanks for the quick feedback! I suppose we need the latter, since we
> would like to create an application able to process any pre-recorded viedo
> ... or almost any ;-)
>
> Maria
>
>
> 2014-08-27 11:30 GMT+02:00 Stefan Wöhrmann <stefanwoehrmann at googlemail.com
> >:
>
>>  Hi Maria,
>>
>>
>>
>> Are you looking for a match of symbols for “Woehrmann´s SpeechWriting”
>> and a picture how the mouth / face looks like or are you looking for all
>> kinds of facial expressions – tongue, eyebrows, eyes, cheecks, air in or
>> air out, teeth…
>>
>>
>>
>> All best
>>
>>
>>
>> Stefan
>>
>>
>>
>>
>>
>> Animation of performing a sign name  - the mouth moves as if saying: T I
>> M O    (German way to pronounce the word ;-)) )
>>  ------------------------------
>>
>> *Von:* SignWriting List: Read and Write Sign Languages [mailto:
>> SW-L at LISTSERV.VALENCIACOLLEGE.EDU] *Im Auftrag von *Maria De Marsico
>> *Gesendet:* Mittwoch, 27. August 2014 11:22
>> *An:* SW-L at LISTSERV.VALENCIACOLLEGE.EDU
>> *Betreff:* Face glyphs
>>
>>
>>
>> Dear all,
>> we are addressing the problem of transcribing signed videos in
>> SignWriting. Unfortunately, when automatically processing single frames, we
>> cannot use any previous knowledge of the content. For this reason, in
>> oprder to reliably reproduce face expresions through the corresponding
>> glyphs, we would need a kind og "glossary". i.e., a table of correspondence
>> between real face images and glyphs.
>>
>> Do you know if something of this kind is already available?
>>
>> Thanks!
>>
>> Maria
>>
>>
>> --
>>
>> ---------------------------------------------------------------------------
>> Maria De Marsico
>> Assistant professor
>> Sapienza University of Rome
>> Department of Computer Science
>> Via Salaria 113 - 00198 Rome - Italy
>> email: demarsico at di.uniroma1.it
>> tel: +39 06 49918312
>>
>> ________________________________________________
>>
>> SIGNWRITING LIST INFORMATION
>>
>> Valerie Sutton SignWriting List moderator sutton at signwriting.org
>>
>> Post Messages to the SignWriting List: sw-l at listserv.valenciacollege.edu
>>
>> SignWriting List Archives & Home Page
>> http://www.signwriting.org/forums/swlist
>>
>> Join, Leave or Change How You Receive SW List Messages
>> http://listserv.valenciacollege.edu/cgi-bin/wa?SUBED1=SW-L&A=1
>>  ________________________________________________
>>
>> SIGNWRITING LIST INFORMATION
>>
>> Valerie Sutton SignWriting List moderator sutton at signwriting.org
>>
>> Post Messages to the SignWriting List: sw-l at listserv.valenciacollege.edu
>>
>> SignWriting List Archives & Home Page
>> http://www.signwriting.org/forums/swlist
>>
>> Join, Leave or Change How You Receive SW List Messages
>> http://listserv.valenciacollege.edu/cgi-bin/wa?SUBED1=SW-L&A=1
>>
>>
>>
>>
>>
>>
>
>
> --
> ---------------------------------------------------------------------------
> Maria De Marsico
> Assistant professor
> Sapienza University of Rome
> Department of Computer Science
> Via Salaria 113 - 00198 Rome - Italy
> email: demarsico at di.uniroma1.it
> tel: +39 06 49918312
>  ________________________________________________
>
> SIGNWRITING LIST INFORMATION
>
> Valerie Sutton SignWriting List moderator sutton at signwriting.org
>
> Post Messages to the SignWriting List: sw-l at listserv.valenciacollege.edu
>
> SignWriting List Archives & Home Page
> http://www.signwriting.org/forums/swlist
>
> Join, Leave or Change How You Receive SW List Messages
> http://listserv.valenciacollege.edu/cgi-bin/wa?SUBED1=SW-L&A=1
>
>
> ________________________________________________
>
>  SIGNWRITING LIST INFORMATION
>
> Valerie Sutton SignWriting List moderator sutton at signwriting.org
>
> Post Messages to the SignWriting List: sw-l at listserv.valenciacollege.edu
>
> SignWriting List Archives & Home Page
> http://www.signwriting.org/forums/swlist
>
> Join, Leave or Change How You Receive SW List Messages
> http://listserv.valenciacollege.edu/cgi-bin/wa?SUBED1=SW-L&A=1
>
>

-- 
---------------------------------------------------------------------------
Maria De Marsico
Assistant professor
Sapienza University of Rome
Department of Computer Science
Via Salaria 113 - 00198 Rome - Italy
email: demarsico at di.uniroma1.it
tel: +39 06 49918312

________________________________________________

SIGNWRITING LIST INFORMATION

Valerie Sutton
SignWriting List moderator
sutton at signwriting.org

Post Messages to the SignWriting List:
sw-l at listserv.valenciacollege.edu

SignWriting List Archives & Home Page
http://www.signwriting.org/forums/swlist

Join, Leave or Change How You Receive SW List Messages
http://listserv.valenciacollege.edu/cgi-bin/wa?SUBED1=SW-L&A=1
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/sw-l/attachments/20140901/69a8c197/attachment.htm>