3
Why is it important? n Interaction is basic to VEs –We defined them as ‘interactive in real-time’ n No interaction => NOT a VE n Ideal interaction: –Very low latency - i.e fast –Multi-modal –Unencumbered –Intuitive n Technology falls well short of this of course

4
Tracking the human body n Large displays require position and orientation of viewer’s body to be tracked –tracking information fed to runtime system as input signal. n Most commonly tracked is head but sometimes also hands, arms, legs, eyes etc. –Head tracking used to update virtual viewpoint orientations. n Body tracking needed for lifelike interaction with objects and creatures. –say user wishes to wave at another person in the VE: their real-world motions can be tracked and replicated in the VE.

7
Tracking the human head n An essential basic requirement in immersive VR systems. n Imagine axes mounted on top of your head –pans, tilts and yaws of head measured around those axes. n HMDs often have rotation sensors to measure these three angles. n Angles passed to run-time VR software which updates viewing angles. HMD

8
Tracking devices n Many tracking devices and systems developed over the years –some aimed specifically at VR systems – others borrowed from other areas. n Some systems are portable and cheap - some require permanent installations in large rooms and are very expensive indeed. –Trackers can be magnetic, electro-magnetic, acoustic, inertial, optical, or mechanical. n Electro-magnetic trackers –transmitter generates electromagnetic signals –received by a receiver (or sensor). –Signal strength used to determine absolute position and orientation of receiver relative to transmitter.

9
Example: Polhemus FASTRAK n FASTRAK electro-magnetic sensor from Polhemus –accurately computes the position and orientation of tiny receiver as it moves through space. n Dynamic, real time six degree-of-freedom measurement of position (X, Y, and Z) and orientation (yaw, pitch, and roll) –RS-232 signal updated at 120 records/sec. n Transmitter constantly puts out a weak magnetic field. –passive receiver generates an electric signal as it is moved through the field. –Polhemus' processing electronics then amplify and analyse this signal to determine the real-world position and orientation of the receiver relative to the transmitter.

10
Polhemus FASTRAK system n Polhemous trackers well proven and widely used since the very early 1990’s. n The FASTRAK system shown here has one receiver and one transmitter. n System expanded by adding up to three more receivers –can attach receivers to different parts of body –log data for gait and limb analysis or computer animation.

11
Electromagnetic Tracking Polhemus

12
Electromagnetic Tracking Ascension Ascension market a number of systems based on DC rather than AC fields including Flock of Birds and a full gait analysis system called MotionStar.

15
Ultrasonic trackers n Two main components –transmitter generating an ultrasound signal – receiver detecting the signal. n Distance is calculated by measuring time-of-flight of ultrasonic pulse. –Three transmitters and receivers needed to calculate full 3D position and orientation. n Ultrasonic tracking used by Logitech Head Tracker (shown) and 3D mouse.

16
Ultrasonic trackers n The Power Glove made by toy company Mattel (who make Barbie) –introduced in 1989 for use with the Nintendo Entertainment System (NES). n Ultrasonic device for use in place of standard Nintendo controllers n Detected finger motion –Plus full set of buttons on the wrist. n In fact not much use for Nintendo gamers –But amazingly advanced piece of VR kit for its time.

20
Optical tracking methods n Many different forms –Often use image processing and pattern recognition and matching –Much work outside of VR: numerous ideas suitable for tracking object position and pose n For example fiducial mark detection –light sources or reflective colour markers attached to object at important locations such as joints or extremities. n Easier for image processing algorithm to track in cluttered conditions.

21
How it is done

22
Optical tracking methods n Outside-in tracker –tracking apparatus is fixed –object to be tracked (e.g. the user) is viewed from the "outside". n Inside-out systems –take tracking measurements from the object to be tracked –for instance a camera can be mounted on the HMD –images analysed to produce pose and distance estimations based on the position of fixed patterns within the environment. n Visible images or infra-red used. n Many optical systems (but not all!) are one-offs, expensive and require careful calibration procedures.

25
Eye trackers n Eye tracking systems are examples of optical tracking devices. –viewpoint in the virtual world follows the gaze of user’s eye. n Originally developed as a mouse replacement – simply look at object –interact through eye movement (such as a slow blink). n Support physically impaired users. n Combined eye and head tracking systems also exist - use in practice is complicated.

26
Mechanical trackers n Mechanical linkage system –arm-like structure of several joint, one end fixed, the other free to move with the user. n Measure position and angular orientation of free end –by measuring angles at each joint and factoring in length of each segment. n Fake Space BOOM (right)

29
Cybergloves and similar n Inherent in the folklore and hype of VR is the cyberglove - a wearable device that monitors the the position and orientation of hand and fingers. The name CYBERGLOVE ® is registered by Virtual Technologies Inc (VTi). –uses 18 or 22 patented angular sensors for tracking the position of fingers and hand.

33
Mouse as input device in VR n Normal 2D mouse can be used (as in Cortona for example). –Need user selectable modes to switch between DoF’s. n More sophisticated mice provide 3 or more DoF: these include the Spaceball (shown here) and Spacemouse. n Standard games joysticks or gamepads also used to give 2 or more DoF’s.