A Future of Virtual Reality.
The Mind-Boggling Future of Virtual Reality
By Sarah Scott Saturday National Post - Canada
3-17-2
"In the next decades, we're going to have worlds and renditions of reality that
will make our modern way of thinking of the world -- perspectival and cognitive
and mathematical and certain -- seem almost medieval."
There is a place in southern Ontario, a small windowless room called the Cave,
where National Research Council scientists are trying to bring the Holodeck to
life. If you're a fan of Star Trek, you know what the Holodeck is -- a
computer-created virtual reality, so real you'd think you were there. On Star
Trek: The Next Generation, the Holodeck is a virtual playground for Captain
Picard and his staff. The Cave is a real-life approximation, a
multi-million-dollar illusion machine run by powerful computers.
Step into it, scientists say, and you will step inside a three-dimensional
image -- a city street, perhaps, or a space station flying over Earth, or even
a throbbing human heart. Other machines have promised similar experiences. What
makes this one so convincing is that you always see the image from your
perspective. Turn your head to the right, and the image is instantly redrawn --
just as in real life. You can move through the virtual world, walking or
flying, if you wish. You might even meet someone there.
So, this is virtual reality, circa 2002. Manufacturing reality has been the
dream of computer science since 1965, when Ivan Sutherland wrote a paper called
The Ultimate Display, in which he predicted that computers would one day
digitize sight, sound and the other senses so effectively that they could
control "the existence of matter." This was five years before IBM invented its
first personal computer. "With appropriate programming," Sutherland wrote,
"such a display could literally be the Wonderland into which Alice walked."
Sutherland helped launch a branch of computer science dedicated to making
virtual reality a real thing -- with the assistance of helmets, wands, gloves
and plenty of refrigerator-sized computers. But the gadgets have always seemed
too intrusive to suspend disbelief and make you feel you were there.
Then, a decade ago, a trio of scientists from the University of Illinois at
Chicago invented the Cave, named after Plato's cave. The Cave's official
manufacturer is a company based in Kitchener, Ont., called Fakespace Systems
Inc., and it has made over 30 Caves so far for more than $1-million a piece,
selling them to companies and research institutes around the world. (Another 70
copies, made by other manufacturers, are scattered around the world.) The Cave
is not the only way to immerse yourself in a manufactured reality, of course.
Virtual reality is an ever expanding world that includes everything from simple
desktop screens to three-dimensional social clubs on the Web that you (or
rather, a cartoon character acting as your proxy) can visit. But the Cave is
probably the most convincing illusion machine, the closest thing so far to the
Holodeck.
I had to see it, so I drove to the nondescript NRC building, on the wooded edge
of London, Ont., to experience the ultimate in virtual reality. The Cave,
located deep inside the building, is a 10x10x10 room with screens for walls. I
step inside and put on special glasses to enhance the 3D image. A student
working at the NRC wears the headgear; it links him to the giant computer and
will track the movements of his eyes so that the 3D image that will surround us
is always projected from his perspective. I move in behind him, and the lights
go off. Suddenly, we're floating up an art gallery lined with paintings. I can
almost forget the images are actually generated by computers and projected on
to the screens and floor of the Cave. Then we're in the front seat of a car,
and we're slowly crashing. I'm feeling a little weird at this point. After a
side trip inside the brain of the victim of the car crash, we see what the
victim sees: a blotted, obscured field of vision, the result of the brain
injury. I have that odd feeling of being disjointed from reality, as if I'm
inside a David Cronenberg movie.
There's nothing fantastical about the research director overseeing this
project. A stocky, greying 57-year-old scientist born in Trieste, Italy, Dr.
Gian Vascotto has a doctorate in biology; he has studied shellfish and shrimp,
industrial nuclear waste and acid rain. He didn't get into the
computer-visualization business until four years ago, when he came to the NRC,
where he's now director of system simulation and control research. He then
began thinking about how computers and simulation software could make
manufacturing more competitive. He decided to find out whether Canadian
manufacturers were interested in using these new technologies. "We thought it
would be a really hard sell," he says. To his surprise, companies jumped at the
idea. So the NRC created the 4,500-square-foot centre, which includes the Cave
and other visualization equipment. It opened for research, testing, and
demonstrations in September, 1999.
The NRC tests different uses of the Cave with several "collaborators" who help
pay for the research. One of them is General Motors, which makes light armoured
vehicles for the U.S. army at a nearby plant. Until recently, designers would
draw a detailed picture, which would then be turned into a physical prototype
-- or several, if the first one didn't work -- for $2-million or more apiece.
Now they create a 3D computer image of the vehicle. The team visits the NRC's
giant screening room, where they inspect the 3D full-scale image. They can walk
around the virtual vehicle, inspect it from every angle, even look at the
interior. According to Joe Attard, a manager in GM's defence division, this
reduces by about a third the time it usually takes to design these vehicles. "I
see this as a major change in the way we develop design," Attard says. GM also
uses the Cave to design cars at its vast facility outside Detroit. Legend has
it that one car executive put his coffee cup on the virtual coffee holder in
the Cave. It looked that real.
Now, inside the NRC'S Cave, the computer has churned out a new 3D image. This
time, we're in downtown London, Ont. Vascotto explains that recently, city
politicians wanted to see whether a proposed arena would overwhelm the city's
historic market. As a research project, the NRC decided to show them. The city
officials met at the Cave. The young university student at the controls charges
up the program to show me what they saw: "There's the arena on your left," she
says. She's waving her hands to move us through the virtual downtown. We swoop
down to ground level. "We can go over here and stand on the street corner," she
says, "and see, we have no problem seeing the market building." It's like a
three-dimensional IMAX universe, except I feel like a floating giant. The
buildings look like very good architectural models, built to scale, but not
quite real. Maybe it's because we're missing the other part of the picture --
the hot wind on my face, the dull sound of a plane overhead. (The student has
been trying to fix that, by giving the computer instructions to duplicate the
subtle sound of jets that you hear if you're inside a building.) Later, when I
drive back downtown to see what the scene really looks like, I look up at the
roof of the building against the blue sky and I'm surprised by what I see.
Reality is not how I had pictured it. It actually looks more like a computer
graphic than I remembered.
Not every aspect of reality has been mimicked as successfully as sight and
sound. Take touch, or haptic feedback, as the computer nerds call it.
Researchers are working on how to duplicate that sense in a virtual
environment. Surgeons at some hospitals in North America can now practise on
the image of a patient in a virtual environment, with an actual scalpel. The
instrument is attached to a machine that feeds back resistance or vibrations to
simulate cutting through bone or puncturing skin. But it's still a pretty
primitive touch machine; other sensations are harder to duplicate.
Virtual-reality scientists are still working on it. Still, they're getting
close enough to manufacturing reality to ask a crucial question: Just how real
do you want to be?
"In some sense, virtual reality is a wish to recreate an experience close to
our experience of the natural world," says Daniel Sandin, one of the Cave's
creators and the director of the Electronic Visualization Laboratory at the
University of Illinois. The "Killer App" is obviously design, he says, but the
Cave is useful for lots of other things -- the study of the Big Bang, or the
flow of water at the bottom of the ocean, the analysis of complicated things in
constant flux, or the reconstruction of ancient sites.
A group of historians, scientists and artists, for instance, is working on the
world's first virtual exhibit of a historic site -- the ancient Mogao Grottoes
in the Gobi Desert, 492 caves with murals, wall fresco paintings and more than
3,000 painted sculptures. Imagine: Instead of going all the way there, you
could see virtually the same thing in a North American museum.
But doesn't it matter that the Grottoes are virtual, and not real? I put that
question to Canadian architects Hani Rashid and Lise Anne Couture, who are
designing the Virtual Guggenheim Museum, what they call "the first important
virtual building in the 21st century." Their answer: Maybe not. "They removed
the David, the real David, from the Palazzo Vecchio, in Florence," says
Couture. "People know, but they go, 'I don't really need to see the real one
because this one is good enough.' It's in the setting, whereas the real one is
in a museum. Most people would rather see the fake one in the real space than
see the real one in the wrong space."
"Looking at the real is a nostalgic experience," notes Rashid. He spent a year
in grad school looking at pictures of Bosch's famous painting The Garden of
Earthly Delights. When he finally saw the real thing, he recalls, he wasn't
bowled over: "I thought it was going to be bigger." In any case, the line
between real and virtual is blurred these days, he says. "Marcel Duchamp told
us a long time ago that anything can be authentic. You see a real urinal or a
bottle rack in a museum. You see that stuff coveted as real, and then you walk
by the store and there's exactly the same bottle rack in the window. I went,
'Wow, let's buy it. Now do I have a real Duchamp?' "
As partners in the ultra-cool New York architectural firm Asymptote, Rashid and
Couture have built an internationally renowned practice that includes spatial
experimentation and installation, buildings and urban design, and
computer-generated environments and architecture. They work out of a
fifth-floor Soho office, three blocks from their home, that's lined with books
-- Plato, Rousseau, the Marquis de Sade, The Decline and Fall of the Roman
Empire, Gertrude Stein. They speak at cutting-edge international conferences,
such as the Doors of Perception.
To Rashid and Couture, virtual reality offers incredible possibilities for
altering our perception of space. "We are right now in a state -- with
mathematics, understanding of biological systems, and virtual-reality
technologies --that is probably unparalleled since the Renaissance," says
Rashid. Consider how people saw things before the invention of perspective in
the mid-15th century. Looking at a drawing by Giotto in the early 14th century,
"you would get this strange, flat world, and that was real to the medieval
mind." A century later, with the discovery of linear perspective, "you get an
entirely different view of what is considered to be a rendition of the real,"
he says.
"In the Renaissance, there was a new mathematical understanding of the cosmos.
We understood science; we had the printing press at our disposal." Suddenly,
the medieval world looked remarkably primitive. With the invention of virtual
reality, it's as if we're in the mid-15th century, when we suddenly start
perceiving reality in a new way, Rashid continues.
"In the next decades, we're going to have worlds and renditions of reality that
will make our modern way of thinking of the world -- perspectival and cognitive
and mathematical and certain -- seem almost medieval."
But Rashid and Couture are not interested in duplicating reality. They see in
virtual reality the chance to create entirely new possibilities.
Take the New York Stock Exchange, one of the last exchanges on the planet to
use real people to trade stock, instead of doing it by computer. The NYSE
recently hired Rashid and Couture to design a virtual stock exchange to enable
NYSE officials to track the estimated US$20-billion of trades per day. The
challenge was to track the movement of vast sums of money at incredible speeds.
"We took it on as a spatial, architectural problem," Rashid says.
They constructed a computer world that makes you feel you are going into a
three-dimensional space to retrieve information -- with the advantage of being
able to zoom from an overview to a close-up in an instant, and to retrieve
information and manipulate it in novel ways. The NYSE is a "machine for
information exchange," said Rashid. "We had to mimic that machine in virtual
space."
Down at the New York Stock Exchange, the virtual stock exchange is just off the
Main Room, on the Ramp. Anne Allen, senior vice-president of the NYSE's floor
operations, explains how it works. Her job is to make sure things run smoothly
at the stock exchange, that there are no backups or glitches in the trading.
Allen used to try to track the action on 21 computer screens filled with names
and numbers. But it was hard to detect a problem quickly; she usually heard it
first from the guys on the floor.
Now, she can watch the trading on nine screens. Although they are flat, the
screens depict the trading floors as futuristic 3D images in blue, purple and
yellow. The images match the geography of the real floor. Allen can see the
floor from any vantage point, zoom down from a bird's-eye view to ground level,
fly past the trading posts, and stop at a trading post to inspect the trades in
detail. Whenever anything unusual happens -- hot news or a stop in trading, for
instance -- flags pop up beside the virtual trading posts. Allen can fly in to
inspect, and then pull away to slice and dice the information as it's flooding
in; the charts are constantly bobbing as the numbers change.
Although the virtual NYSE includes live footage from the floor, its main
architecture doesn't look like the real thing. Rashid and Couture rebuffed
suggestions to make the walls look like marble. "We looked at them and said,
what does that have to do with information? Forget that," says Rashid. "Why not
put data in the walls. That's why the walls are liquid, living walls of
information." In other words, click on the wall and you will find numbers,
information. "The only thing they haven't been able to put in this is the
energy," says Allen. The real Main Room, a financial bazaar dating back to 1791
that buzzes under a web of high-tech equipment, is a rush, she says: "I don't
know how you can manufacture that synthetically."
I ask Rashid and Couture about the Cave, the attempt to create an artificial
reality that mimics the natural world. It's a "myopic, closed view," says
Rashid. "We think reality is interesting enough." Last year, Rashid saw a
virtual Toronto display in downtown Toronto, where he grew up. "I was trying to
figure out what was missing. It wasn't just the wind and the sun. How did it
turn into a hollow, emptied-out model? What's missing is memory," he says. So,
the way you see something is influenced by your memory of it. "The majority of
the stuff [out there now] is kind of illustrative. It's the Norman Rockwell
syndrome."
Back at the Cave in London, Vascotto is pressing ahead with an ambitious goal:
"To put people into virtual reality directly so they can interact with virtual
people." Vascotto wants to create a convincing image of a person who walks and
talks inside the Cave. Then he wants to send that 3D computer image via the
Internet to another Cave thousands of miles away, where the "virtual person"
could talk with a real person. In other words, you could be in two places at
one time, although one of you would be virtual.
Vascotto is not joking. He hopes to demonstrate that this can work sometime
next year. It would be like a video conference, only more realistic because,
instead of seeing your colleague on a flat screen -- which some companies have
tried and then abandoned -- he or she would be a moving 3D image. "Basically
five people across Canada could send their virtual selves to a meeting place,"
he says. Now, this might not cut it for a client paying the big bucks to see a
management consultant or an investment banker in person, but it might replace
other types of business travel. "If I go to Ottawa for a meeting, it's 10 hours
of my time for a two-hour meeting," says Vascotto. "An awful lot of travel
takes place today because of the value of face-to-face communication."
No one has yet managed to create a virtual person and zap him somewhere else.
That's largely because it takes a phenomenal amount of computer power to
capture the movements and expressions of humans. To make movement look fluid,
you need to create 30 frames a second, and the exercise is complicated by the
fact that every time the viewer shifts her eyes a little, the computer has to
instantly redraw the scene to accommodate her new perspective. "The facial
features are very difficult to track because of the very subtle, very small
motions," said Vascotto. At the NRC, scientists are borrowing from the world of
animation to create a virtual person.
Say that a London golf pro wants to demonstrate his swing to someone in
Calgary. The people at the NRC start by creating a 3D representation of the
golfer's body. They can use a standard skeletal form, or the body shape of the
actual golf pro. Then they photograph the golfer's swing. To do this, they
stick little quarter-inch reflective balls all over the real golfer's body and
then use special cameras to track each little ball as he swings. When it's time
to show the swing to the person in Calgary, the real golfer demonstrates the
swing in a Cave in London, and his virtual image in the Calgary Cave would
duplicate the motion, like a puppet.
Vascotto has some serious competition in the United States from Jaron Lanier,
the computer genius in dreadlocks who invented the term "virtual reality." In a
2001 article in Scientific American, Lanier described how he brought together
three people in different locations for a virtual conference. Rather than using
an animation technique, Lanier and his team use a "sea of cameras," between
seven and 60, to capture the participants, or the "moving sculptures," as he
calls them. There are problems: pixelated images, a time delay that can be
disorienting, even nauseating, the vast expense. Nevertheless, Lanier predicts
that tele-immersion will be commonplace in 10 years. The main obstacle, Lanier
says, is not cost or computer power: "It's just that the whole thing is so damn
exotic." (Others say the roadblock is human nature. Some people don't want
their real selves to be projected into cyberspace. They might just want to
amend the image a little, or put on a mask.)
It is perhaps inevitable that, at the end of this story, science reunites with
fiction. The U.S. military has used simulators for a long time to teach
trainees how to fly a plane or use other types of machinery. Now, the military
is taking simulation to a whole new level. It's set up the Institute for
Creative Technologies at the University of Southern California -- an assembly
of people from film, games, computer science and the army -- to create a
virtual training ground filled with virtual people. "The goal is to create a
prototype of something like the Holodeck," says Dr. Bill Swartout, the
institute's technical director. Some of the people at ICT even worked on Star
Trek, he says. "The Holodeck is a source of inspiration. It suggests research
paths we're going down."
ICT's aim is to teach soldiers headed for peacekeeping missions to deal with
people in crisis. For these kinds of simulations, the army used to hire actors,
but they're not always available when you need them, says Swartout. So the ICT
is developing virtual characters -- like the distraught mother with an injured
child in a Balkan village who's crying out for help. She's projected on to a
giant curved screen that gives the illusion of immersing you in another
reality. The human trainee, cast in the role of army lieutenant, has to figure
out what to do. His responses are fed back into the computer. If he turns away,
the mother gets really upset. If he tries to help, she calms down. The image is
still a little cartoonish, but that's the price you have to pay if you want the
virtual character to be able to respond to the recruit's actions, says Dr.
Norman Badler, director of the Center for Human Modeling and Simulation at the
University of Pennsylvania, who's working on the virtual characters. "The level
of realism is satisfactory enough for people to overlook the slightly synthetic
look and get engaged."
Badler wants to turn the avatars, as these virtual representations of humans
are called, into something more than mere puppets. He wants the computer
characters to start thinking for themselves. To start with, they need to learn
natural language, says Badler, co-author of a paper on the subject, Dynamically
Altering Agent Behaviors Using Natural Language Instructions. "The long-term
goal is to educate our avatars to make decisions, respond, and act in future
situations on their own," Badler says. Getting the computer characters to
express emotion is not easy, though. "We don't have an effective computational
model for what goes on inside our head," he said. "But we're now pushing our
characters to think about what they see, and react." It's digitizing emotion,
essentially, and that could be the final frontier.
http://www.nationalpost.com/artslife/story.html?f=/stories/20020316/352666.html
|