• When confronted with the task of 3D sound, how is hard is it to do with only two headphone speakers? Very hard. How the brain processes sound is not well understood.

• one can only "tune in" to a conversation if there is stereo

• A fixed white noise source is only locatable if the listener is able to move their head.

• The literal interpretation is "stereo" but in the context it means that the audio is spatially pinpointable in 3 dimensions

Systems

• Convolvotron
  • This is a desktop sized unit which costs around $25,000. The state of the art for some years.
• DSP Based systems
  • Recently the development of DSP technology has reduced the cost of such systems to about $10 and has reduced the size enough to make them embeddable.

Special Effects

• Reverberation, Echo and Realism
  • Special effects must be computed relative to the VR user's apparent position. This is very difficult, comparable to doing radiosity in the visual world.

• Feedback to user
• Also known as "Haptics"

Vibrating Joystick

• Simulates "Crash and burn"

Wand solenoids and Motors

• Provides "recoil"

Glove Bladders

• Provides feedback with pneumatic bladders that inflate inside the glove when simulated contact is made in the virtual world. Generally too slow to be effective.

Exoskeletons

• Force feedback
  • When modeling molecules in VR, an exoskeleton can simulate repulsion and attraction.
• Prevents movement.
  • Normally in a virtual world, there is nothing physically to stop you from moving through walls--unless there is an exoskeleton with sufficiently powerful motors to control movement.

Largely an unsolved problem

• Provides feedback to the machine

Walking-Like

• Movement´VR-Response
  • Provides a one-to-one "real-world" movement and the corresponding virtual world movement.
  • In some circumstances, this can be rather tiring and/or slow. It can also severely limit the travel distance without more hardware such as treadmills used in some architectural walk-throughs.

Vehicle-Like

• Constant motion
  • The user only indicates motion and speed changes. Usually found in aircraft or land vehicle simulators, or games where "flying" is required.

Wands, HMDs

Technologies

• Magnetic
  • Three magnetic field generators are placed in proximity to a sensor located on the wand. The generators fire sequentially and a computer triangulates, based on the magnetic strength, where the sensor is in 3-space.
• Ultrasonic
  • This works much like the magnetic sensors, only using sound. Instead of "strength" these sensors measure the time it takes for the sound to get from the generator to the sensor.
• Exoskeleton
  • Sensors at each joint or rotation axis are able to keep track of user movement very accurately. The trade-off is that this technology is quite cumbersome.
• Intertial
  • On HMDs, intertial sensors are oft-used for keeping track of head rotation. Unfortunately, these types of sensors are particularly unreliable for precise measurement such as that required for VR work. They are more frequently used to keep track of how far a missile has traveled.
• Optical
  • Sensors in the Wand or HMD "count" light emmissions from the environment to determine position.
• Robot Vision
  • Sufficiently sophisticated HMD-mounted pattern matching that is able to determine position and orientation by taking visual cues from the outside scenery.
  • This is not currently feasible (although highly desirable).

Gloves

• Detect finger flexion
• Fiber optics
  • Glass filaments located in the fingers of the glove that have been appropriately "slashed" leak light in proportion to the amount that the fiber has been flexed. This is detected by optoelectronics in the glove and digitized.
• Exoskeleton
• Chordgloves--Pinch Sensors
  • Determines how much pressure one is using to grab a virtual object. It imparts the potential to "crush" or deform object's shape.

Simulation

Rendering

• Raytrace
  • Models objects
  • Point light sources
  • View dependent
  • Rays of light from the eye propagate through objects.
• Radiosity
  • Models light
  • Diffuse, area lighting; realistic shadows.
  • View independent (conditionally)
  • Surface patches ("emitters") bounce light from one to the other.

SGI

If attempting to do any kind of heavy-duty VR, life begins and ends at

• Onyx RealityEngine/2
  • This is used in most of the major research involving VR such as the CAVE.
  • 

Goals

• Platform Independent
• Consistency
• Integrate with the WWW
• Large shared space
• Interactive

Defines

• Objects
  • These are the actual "things" available in the VRML space.
• Scenes
  • Where the objects are, relative to one another.
• Binding mechanism
  • Matches the 3D space to VRML servers on the network.

No pictures yet! It is largely theoretical, however the claim is that a browser is currently being developed: Labyrinth

Code Snippet:

        Cube {
             width   2     # SFFloat
             height  2     # SFFloat
             depth   2     # SFFloat
             }

Wired's VRML page

• Some of the things that VR promises are practically impossible without surgery.

• A system such as n-Vision cost about $65,000. Some U.S. Military simulation system cost $50 Million. Amortized over 10 years, this works out to be about $3000/hour. In comparison, instructor assisted flight (actual reality) at the Calgary airport is $100/hour.

• There are currently no hard standards for interfacing much of the VR peripheral equipment. For example, some of the CAVE transducer output needed to be "massaged" in a i486 machine before it was sent to their RealityEngine/2.

• In VR with an HMD display, one tends to adapt to rotating one's head rather than simply moving your eyes.
• Eye strain: your eye finds the focus of the screen nearer (usually a constant of about 3m) than the apparent focus of the object.
• Visual/Audio mismatches can cause headaches.

• Much of the sensor technology today is not particularly accurate. Even when the user is completely still there is often some "jitter" associated with many types of sensors (e.g. magnetic and ultrasonic)

• Processing the transducer input can take quite a bit of time, generating some lag time between movement and virtual response.
• The rendering process consumes a huge piece of the processing power pie. Some compromise is usually reached, such as reducing the LOD (level of detail) or partial scene rendering.

• VR users tend to be encumbered with a veritable plethora of heavy and inconvenient appendages such as suits, gloves and goggles, many attached with wires and/or arms affixed to the ceiling.