Communicating with trackers

This section is optional, many desktop stereo 3D systems don't use head tracking.

Communicating with head trackers is conceptually the easiest part of building a VR program. The details can be messy, and careful tuning may be needed to extract the best performance, but it's really very simple underneath.

Head tracking isn't essential to VR. If only one person is using your VR system, then head tracking will increase the sense of immersion by having the display respond to head movement. But many VR systems are for public display, with more than one person viewing at once, which defeats the whole purpose of having a head tracker. So get the other VR bits right first before worrying about tracking.

All 3D trackers (the difference between a head tracker and a 3D mouse or wand is whether it is attached to your head or not) generate a simple data stream: 3 numbers for the position in 3D space, 3 more for the orientation (angles) in 3D space, and a 7th value indicating whether any buttons have been pressed. The calculations for resolving information from acoustic or magnetic sensors into these values may be horrendous, but it's all handled by the hardware and invisible to you the programmer.

The messy part is learning how to extract these numbers from a given tracker. Typically you have to open a serial port, which means setting the communication speed, parity, and other parameters. You have to send a short initialisation string through the port. After that the tracker sends regular lines of data back, which your program must read and decode.

Because serial port IO is far slower than anything else in a VR system, you also have to work out how to handle this incoming data stream without blocking the rest of the program. My preference is to have a separate thread for the tracker data, but you might decide to use polled IO.

A 3D tracker also introduces Yet Another Coordinate Space. I wrote above about the physical positioning and placement of the actual screens relative to the viewer. The tracker will generate coordinates relative to a different origin point, and possibly use different units (eg cm instead of metres). Converting the tracker data into screen-relative data is another simple but messy task.

As with the VR display itself, try to use as much runtime configuration as possible. The translation from the tracker origin to the screen origin should certainly be a runtime value, otherwise you'll have to recompile your program if the 3D transceiver box is moved. Try to specify which serial port at runtime, so your program will still work when moved to another PC with different ports.

The ideal is to write a generic 3D tracker API for your main code and dynamically load a shared library that implements your API for a given piece of hardware. This will allow you to switch between say a Logitech or a Polhemus tracker simply by loading a different library.


Back to 3D Graphics