At this stage, virtual reality (VR) headsets and handheld controllers can only track head and hand movements. Even with so-called inverse kinematics (IK) algorithms, they can only predict elbow and torso movements relatively well, with little correction for legs. Because of this, many VR solutions simply display only the upper half of the avatar.
Video screenshot (via VR Trailers & Clips)
Even with the additional tracking solutions offered by SteamVR and HTC Vive, the cost of the three units needed for body tracking would cost more than $350.
The good news is that in a new paper (PDF) discussing QuestSim, Meta researchers demonstrate a system powered by a neural network.
It features the ability to use tracking data from the Quest 2 headset + controller to more rationally predict the wearer’s full-body posture without the need for additional trackers/external sensors.
Meta researchers even claim that the resulting accuracy and jitter is better than that of wearable IMU trackers.
Pico 4’s announced Pico Fitness Band, for example, is a device equipped only with an accelerometer and gyroscope — although the company also claims to be working on its own machine learning approach.
But here’s the catch — as the video shows, the system renders a reasonable set of full-body poses, but it doesn’t necessarily accurately reproduce the user’s actual posture.
Because of this, QuestSim is only suitable for referencing other people’s VR avatars, not for looking down on yourself at all times, and the system has a latency of 160 ms (over 11 fps at 72 Hz).
Even so, being able to see other people’s full bodies in VR interactions is much better than a half-portrait at this stage. The question that remains is — will this (or a similar) system, and when, make it to Quest 2?
Meta CTO Andrew Bosworth seemed to hint at this during last week’s Ask Me Anything interactive Q&A event, and when asked about leg tracking in Instagram, he replied.
"It's true that we get a lot of flak for legless avatars, but I think the trolling is pretty fair and funny. After all, giving an individual avatar legs that don't match reality can be very disturbing to users. But if the legs are placed on other people's avatars, there is no such disturbance."
For those interested, watch for a detailed announcement during Meta Connect’s annual AR / VR event in two weeks.
