Robots to replace Linguistic Anthropologists

[Steve Bialostok]

But if anyone knows of software that DOES transcribe (multiple voices), I'd love to know about it. I have a lot of classroom talk that is, as everyone knows, enormously time consuming to transcribe. (And no, we don't have any money or graduate student slave labor.) Steve



________________________________
From: Scott F. Kiesling <kiesling+ at PITT.EDU>
To: LINGANTH at LISTSERV.LINGUISTLIST.ORG
Sent: Tuesday, October 28, 2008 11:51:52 AM
Subject: Re: Robots to replace Linguistic Anthropologists

I think I saw a paper based on this technology presented at an NSF
conference I was at. It was actually pretty cool when I saw it,
although it sounds like they are now stretching a bit with all the
prescriptive applications. 

The verson I saw was trying to track network patterns through the
length and intensity of interaction, and I saw another paper about
rhythmic coordination. It was all very descriptive and interesting,
especially since there was no way of getting around the IRB problem of
recording people who hadn't had a half-hour briefing on the project if
you actually recorded content. Apparently, recording prosody does not
invade your privacy...

SFK

On Tue, Oct 28, 2008 at 01:36:57PM +0800, Kerim Friedman wrote:
> From: Kerim Friedman <oxusnet at gmail.com>
> Date: Tue, 28 Oct 2008 13:36:57 +0800
> To: LINGANTH at LISTSERV.LINGUISTLIST.ORG
> Subject: [LINGANTH] Robots to replace Linguistic Anthropologists

> I, for one, welcome our new robotic converstational analysis overlords.

> - Kerim

> http://www.nytimes.com/2008/10/26/business/26novelties.html

> October 26, 2008
> Novelties
> You May Soon Know if You're Hogging the Discussion
> By ANNE EISENBERG

> PEOPLE who want to improve their communication skills may one day have
> an unusual helper: software programs that analyze the tone,
> turn-taking behavior and other qualities of a conversation. The
> programs would then tell the speakers whether they tend to interrupt
> others, for example, or whether they dominate meetings with
> monologues, or appear inattentive when others are talking.

> The inventor of this technology is Alex Pentland of the Massachusetts
> Institute of Technology, who has developed cellphone-like gadgets to
> listen to people as they chat, and computer programs that sift through
> these conversational cadences, studying communication signals that lie
> beneath the words.

> If commercialized, such tools could help users better handle many
> subtleties of face-to-face and group interactions ??? or at least stop
> hogging the show at committee meetings.

> With the help of his students, Dr. Pentland, a professor of media arts
> and sciences at M.I.T., has been equipping people in banks,
> universities and other places with customized smartphones or thin
> badges packed with sensors that they wear for days or even months. As
> these people talk with one another, the sensors collect data on the
> timing, energy and variability of their speech.

> Dr. Pentland, known as Sandy, calls his gleaning and processing of
> conversational and other data "reality mining ??? using data mining
> algorithms to parse the real life, analog world of social
> interactions."

> The tools he has developed might help people change their
> communication tactics, including those that lead to unproductive
> workplace dynamics, said David Lazer, an associate professor of public
> policy at the Kennedy School of Government at Harvard.

> Mr. Lazer praised "the richness of the data" captured by the process ???
> the "minute-by-minute, fine-grained data on whether you are talking,
> whom you prefer to talk with, what your tone is, and if you interrupt,
> for instance."

> That kind of tool is rare, Mr. Lazer said. "Our existing research
> tools for gathering this kind of data aren't very good," he said ??? for
> example, questionnaires in which people self-report on conversations.
> Reality mining may be more accurate, and has the potential to show
> "all sorts of interactive patterns that may not be obvious to
> individuals in an organization," he said.

> Many of Dr. Pentland's research studies with smartphones and badges
> with embedded sensors are discussed in his new book, "Honest Signals,"
> recently published by MIT Press. The badges use tools including
> infrared sensors to tell when people are facing one another,
> accelerometers to record gestures, and microphones and audio
> signal-processing to capture the tone of voice.

> With the array of sensors, the badges can detect what Dr. Pentland
> calls "honest signals, unconscious face-to-face signaling behavior"
> that suggest, for example, when people are active, energetic followers
> of what other people are saying, and when they are not. He argues that
> these underlying signals are often as important in communication as
> words and logic.

> For example, the badges register when listeners respond with regular
> nods or short acknowledgments like, "Right." Such responses, he
> argues, are a kind of mirroring behavior that may help build empathy
> between speaker and listener. He also examines patterns of turn-taking
> in conversations, as well as gestures and other, often unconscious
> signals.

> Future smartphones that take advantage of his technology may act as
> friendly personal assistants, automatically putting through calls from
> friends and family, but sending all others straight through to voice
> mail.

> "The phone can be like a butler who really gets to know you," he said,
> by deciding to ring brightly for an urgent call when its owner has
> forgotten to turn on the ringer.

> In the research, many steps are taken to make sure the identities of
> participants remain anonymous, said Anmol Madan, a graduate student of
> Dr. Pentland. For instance, when microphone audio data is collected,
> the microphone picks up tone and the length of speaking time but does
> not record any of the actual words spoken.

> So far, Mr. Madan has found that the data gathered by mobile phones is
> far more accurate than accounts of the same information reported by
> participants.

> "Humans have a lot of bias when they recall their behavior," he said.

> Tanzeem Choudhury, a former student and collaborator of Dr. Pentland
> and now an assistant professor of computer science at Dartmouth,
> continues to do reality mining with smartphones.

> "We spend a lot of time talking about how to improve communication
> skills," she said. "This work lets us pin down what makes
> conversations effective by analyzing people's actual conversation in
> their social networks."

-- 
Scott F. Kiesling, PhD

Associate Professor 
Department Chair

Department of Linguistics
University of Pittsburgh, 2816 CL
Pittsburgh, PA 15260
http://www.linguistics.pitt.edu
Office: +1 412-624-5916