The Giant Who Walks Amongst Us (fwd)

[phil cash cash]

The Giant Who Walks Amongst Us

Jenn Director Knudsen   March 17, 2005
http://www.technologyreview.com/articles/05/03/wo/wo_knudsen021605.asp?
p=1

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: text/enriched
Size: 205 bytes
Desc: not available
URL: <http://listserv.linguistlist.org/pipermail/ilat/attachments/20050319/8e5d1225/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: knudsen021605.gif
Type: image/gif
Size: 27453 bytes
Desc: not available
URL: <http://listserv.linguistlist.org/pipermail/ilat/attachments/20050319/8e5d1225/attachment.gif>
-------------- next part --------------


Giant Jimmy Jones is a friendly, helpful giant. In fact, this book 
character is so helpful, he can make the sun shine on an otherwise gray 
village. The giant simply walks across the page, reaches up to the 
cloud cover and pushes it out of the sun's way so the villagers can 
catch some rays.

Those light rays may be virtual, but the book this scene pops out of is 
not.

Using augmented reality (AR), the technology behind the interactive 
version of Giant Jimmy Jones, New Zealand author Gavin Bishop recently 
collaborated with Mark Billinghurst and his colleagues at the Human 
Interface Technology Laboratory New Zealand (HIT Lab NZ) to turn the 
book into not only a storytelling device, but also a storytelling 
experience.

A child can flip through its pages and read it like a conventional 
book. But with a handheld display and computer vision tracking 
technology, the child can watch the story literally come to life.

"You can see animated virtual characters overlaid on the real book 
pages and hear the voice of Gavin Bishop reading the story," says 
Billinghurst, director of the HIT Lab NZ..

While Giant Jimmy Jones currently only exists in a lab setting, there 
are scores of others being developed at places such as Georgia Tech 
University?s Augmented Environments Laboratory.

Technology is not the hindrance to turning books into interactive 
devices whose readers can exist within them and manipulate their 
stories. The most difficult roadblock stems from the limitations of 
physical books, most notably the reality that embedding markers that 
can interact with VR-headgear is expensive and produces ugly visual 
results on the page.?

  Virtual Stories

Unlike virtual reality that exists only within the confines of a 
computer-generated world, augmented reality includes virtual space 
digitally, seamlessly overlaid onto a real environment, explains Kelly 
L. Dempski, a senior researcher at Chicago-based Accenture Technology 
Labs.

For example, AR can be used in tandem with a child reading from a book 
and a computer loaded with AR software. The child, then, could 
virtually place characters from his story anywhere in the room.

The challenge, of course, is to get the technology to work with the 
user.

A wearable computer in the form of a head-mounted display (HMD) is worn 
like a fighter-pilot helmet, fitting over the head and eyes, and 
projecting images within the user's visual field.

Improvements have led to devices that resemble glasses, but even these 
are unwieldy, unsanitary and limited to one user at a time. The HMDs 
also suffer tracking problems, or perfect registration, which means 
that the virtual overlay doesn't quite match up with the physical space 
upon which it is projected.

  That visual slip confuses the brain, making a child feel off balance, 
mirroring the effects of alcohol consumption, Dempski claims. The user 
would lose the ability to differentiate between what she's really 
seeing and what's being injected into a scene via the lens.

However, experiments with these head-mounted displays has paved the way 
for more practical, more commercially viable AR devices such as 
Personal Digital Assistants (PDA) and projectors.

A PDA was used to view Magic Book, demonstrated at SIGGRAPH 2000 by a 
team of researchers, including Billinghurst, then with the University 
of Washington's Human Interface Technology Laboratory.

But this book's images -- of boxy buildings and a stiff, humanesque 
figure -- weren't very sophisticated or capable of much interaction. 
Instead, the 3-D visuals were like "pop-up drawings on paper," says 
Ulrich Neumann, professor of computer science at the University of 
Southern California .

The real breakthrough for the mobile AR will come when books are loaded 
with "trigger symbols," or markers, that a PDA could track and then 
turn into organic images leaping off the page. Doing so, though, isn't 
practical and is a visual turn off to readers; the markers look like 
barcodes, says Ramesh Raskar, a senior research scientist with 
Mitsubishi's Electric Research Labs (MERL). (Ulrich, Billinghurst and 
others are working on algorithms for tracking to reduce the barcode 
look.)

That has led researchers look at alternatives such as radio frequency 
identification tags (RFIDs) and touch sensors in lieu of visible 
computers and their components.

"The point is to hide the computer," Raskar says. "The technology 
should be entirely transparent."

Building a Better Virtual Story

Spatial augmented reality makes this possible. Instead of merely 
overlaying images on top of objects within a reader's view, a projector 
makes objects and images appear to blend into your very world -- in 
front, to the side and behind you.

The advantage of this technology's use in interactive storybooks is the 
reader can create non-linear and event-driven stories.

Raskar says a child no longer would have to read his book page by page, 
and any physical action of his could change the action in and plot of 
the story.

  "To open a book and see this animation happen is counter to anyone's 
experience," Neumann says. "This is not to replace the imagination, but 
to help it along a bit" in the same way film adds dimensions to 
stories.

But Raskar says the future of spatial AR technology in books is 
limited. So AR for storytelling may leap completely off the page.

Steven Feiner, computer science professor, and his colleagues with 
Columbia University's Computer Graphics and User Interface Lab and in 
collaboration with its Graduate School of Journalism, have turned 
documentary films into more interactive and educational tools.

In one of the lab's "situated documentaries" about Columbia 
University's system of underground tunnels, a viewer hears narration 
and sees both the campus and flag-like markers that indicate where 
portions of the story are located, Feiner explains.

By selecting a flag, the viewer immerses herself in a narrated, 3-D 
surround view of one of the university's thousands of such tunnels.

Feiner's documentaries, though, are hypermedia stories embedded in the 
real world and presented using a mobile augmented reality system.

For hands- (and head-) free presentations, projectors and touch sensors 
are being used by Raskar and his MERL colleagues to graphically animate 
physical objects.

In their lab, at the University of North Carolina at Chapel Hill, 
they've created a Taj Mahal that reflects a 24-hour light cycle; in 
only a few moments' time, you can see the effect light would have on 
the building's onion domes and flanking towers as a simulated sun rises 
and then falls.

Using the same technology, a nine-year-old girl, using a white paint 
brush, white piece of paper and white bird house, demonstrates how she 
can choose a new color from a digital palette and "paint" her paper and 
bird house.

"Children are extremely attracted to these displays," Raskar says in a 
phone interview.

And adults -- the ones with money in their pockets -- are extremely 
attracted to demonstrations of spatial AR in advertising.

Paul Dietz, senior research scientist at the Mitsubishi Electric 
Research Laboratories and a colleague of Raskar's, says its MERL lab in 
Cambridge is host to a music store display.

At the store's entrance is a fancy set of speakers, playing catchy 
hip-hop music. Interest piqued, the shopper walks closer to the setup, 
and the music changes genres and the volume picks up.

Now standing next to the display and looking straight at a speaker, the 
shopper sees the inner workings of the hi-fi equipment itself. And if 
he handles a component, the display changes yet again.

Raskar explains the technology behind this exhibit, demonstrated at 
SIGGRAPH 2004 is similar to that in a photocopier; when an office 
worker walks by the copy machine, it detects her presence and clicks 
on.

These types of technologies, though, aren't cheap -- and currently make 
it difficult to move these devices into widespread public use, 
particularly in the book publishing world.

But physical books may not be its future form factor.

It may instead be a room -- or "interactive narrative playspace" -- in 
which a child creates his own adventure, such as KidsRoom, begun years 
ago at the MIT Media Laboratory.

"Our reliance on a physical book provides some limitations on the type 
of stories that can be told," says Billinghurst of HIT Lab NZ in an 
email. "Although of course it still provides traditional writers an 
exciting new medium to work in."
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: text/enriched
Size: 8540 bytes
Desc: not available
URL: <http://listserv.linguistlist.org/pipermail/ilat/attachments/20050319/8e5d1225/attachment-0001.bin>