Speaking in tongues, Part 2: Foreign language Knowledge Management technologies

[Harold Schiffman]

Speaking in tongues, Part 2: Foreign language KM technologies
By Greg Pepus - Posted Jan 2, 2009

I am a news junkie, and I don't limit myself to just U.S. news. I like
finding out what is going on across the world. One thing I find very
interesting is looking at video information about foreign countries
presented on sites like CNN, BBC, Al-Jazeera and other foreign news
sources. What also interests me is the question of whether or not
different things are being reported in the native tongue of a country
vs. on the internationalized English versions of those broadcasts.
However if you are, like me, just an English speaker, then reviewing
information provided in foreign languages is a serious challenge.
Enter, automated foreign language video exploitation tools and
services.

This article is the second of a two-part series on foreign language
knowledge management technologies that can help you meet those foreign
language challenges. In the first article (in the July/August 2007
issue of KMWorld), I focused on foreign language tools to support
unstructured text mining software, which provides users with natural
language processing, language identification, transliteration and name
normalization capabilities. In this article, I will focus on BBN
Technologies'  Broadcast Monitoring System (BMS) and some of the
underlying components that make it work.

The basics

BMS is a suite of technologies that have been integrated to provide a
comprehensive capability to monitor, search and supply alerts based on
the specific content in streaming audio and video. BBN, a 60-year-old,
DARPA-funded spinoff from Verizon, was originally formed in 1948 and
is well known for its speech-to-text conversion technologies. [DARPA
is the Defense Advanced Research Projects Agency.] BBN's technologies
are very mature and widely used in everything from telephone-based
call management systems to automated audio and video speech-to-text
transcription capabilities.

In a nutshell, BMS works by integrating various Internet video
channels into an Internet Explorer 6.0 (or later) and Windows media
player-based streaming video exploitation portal (see Figure 1, Page
8, KMWorld, Vol 18. Issue 1). Back-office servers provide media
extraction services that convert speech to text, align the text with a
given video frame, and then allow the foreign language text to be
converted to English or another supported language, using automated
machine translation. BMS' foreign language support includes Modern
Standard Arabic, Spanish, Mandarin Chinese, Persian/Farsi and English.

Core features

Some really nice features of the system include continuous monitoring
of incoming multimedia feeds and the ability to search for any text
string in either English or the target language, such as Arabic or
Chinese. The system can actually extract segments of video into either
MPEG format or as stills in JPEG format. Furthermore, users are able
to set system alerts based on text strings and keywords, which are
activated if the right trigger conditions arise as the system monitors
incoming broadcast information (see Figure 2, Page 8, KMWorld, Vol 18.
Issue 1).

A really useful design feature of BMS is that as it is playing a
particular video segment, it automatically aligns and highlights the
speech it has converted to text in the targeted foreign language, the
translation and the actual playing video. As it displays the text, it
identifies the voice that is speaking (in the text) as either male or
female and numbers each separate voice it identifies (see Figure 3 on
page 9, KMWorld, Vol 18. Issue 1).

Machine translation

One of the impressive features of BMS is its machine translation (MT)
capability. Machine translation is a difficult technical challenge,
and over the past 15 years, the focus has been on using rule-based
systems. However, the precision or accuracy of rule-based MT systems
has been extremely limited and generally not very useful. More
recently, MT technology companies have focused on statistical machine
translation approaches. In the case of BMS, BBN has partnered with
Language Weaver, which provides its machine translation capability for
the product.
Rather than use the rules of language (e.g. what's a noun, verb,
adverb, etc.) to provide the basis of converting from one language to
another, Language Weaver uses statistical measures that analyze the
frequency of phrases, sentences and relationships within the text, and
contextually convert them to the targeted foreign language.

This method, more formally attributed to Warren Weaver (Weaver Method)
and initially pioneered by IBM in the 1970s and 1980s, uses
statistical methods and large samples of documents that have already
been translated manually to build the automated underlying translation
system. It normally takes about 2 million sample documents and their
associated translations to the target language to build up the
capability to translate information from one language to another with
any reliability.

Unlike rule-based foreign language translation, which is about 35
percent accurate, statistical methods for translation can bring the
precision up to 90 percent or even higher. In the case of a system
like BMS, this means that humans can definitely look at the
information being presented in the automated translation and get the
"gist" of what is being said. If an analyst or other BMS user wants a
more perfect translation, he or she can copy the relevant section of
the video/audio/text and send it to a human translator using BMS'
built-in capabilities.

Exploiting multimedia

BBN's BMS is a leap ahead in terms of technologies that help with the
automated exploitation of Internet foreign language multimedia. It
provides a turnkey and automated (hands-off) means of monitoring
broadcast audio and/or video information and allows exploitation of
that information in real time.

Once you use BMS to extract text information from video or audio
feeds, you have a body of text that can be further exploited in an
automated fashion. In the first article in this series, I introduced
you to a range of data mining and text exploitation tools that can
automatically extract structure from unstructured information. Since
the transcription capabilities of BMS provide both the native language
text and the translated version as well, a range of text mining
activities are now possible. Those activities include extraction of
nouns, verbs, context, language identification, geospatial
information, concepts and other metadata, all of which can be stored
in a database for long-term historical access.

Though I am normally not a big fan of watching television, just for
laughs, I think I will go and watch some of the old re-runs of
Saturday Night Live that have been translated into Mandarin. I somehow
think it's going to be even funnier the second time around.

http://www.kmworld.com/Articles/News/News-Analysis/Speaking-in-tongues,-Part-2-Foreign-language-KM-technologies--52012.aspx
-- 
**************************************
N.b.: Listing on the lgpolicy-list is merely intended as a service to
its members
and implies neither approval, confirmation nor agreement by the owner
or sponsor of
the list as to the veracity of a message's contents. Members who
disagree with a
message are encouraged to post a rebuttal. (H. Schiffman, Moderator)
*******************************************