While the researchers are able to gather behavioral data such as facial expressions and hand gestures , new technology like the HP Reverb G2 Omnicept Edition headset and SDK ( software development kit ), which Bailenson helped design tests for alongside HP researchers , will elevate his studies by providing a wealth of physiological data .

The Omnicept measures a user ’ s biometric data , including eye tracking , heart and respiratory rates , and head movement , information that can ’ t be gleaned from a typical VR setup . The researchers developed an AI that takes high-level inferences such as users ’ cognitive load — a measure of how hard the brain is working — to gain insight into the person ’ s mental state , stress levels , and attention , including inflection points at which the brain becomes overloaded .

“ Any sort of physiological information could be super valuable ,” says Dr . Erika Siegel , a psychophysiologist at HP Labs who worked on developing testing with Bailenson for Omnicept and studies how physical markers reflect our internal state . Those measurements could also provide helpful data to support educators and training across industries of all kinds , from surgical training to public speaking .

Bailenson plans to integrate the HP Omnicept into

— WALTER GREENLEAF , NEUROSCIENTIST AND VISITING SCHOLAR

largest ever taught in the medium . Researchers set up a study of the class to give a deeper understanding of how humans engage virtually in social settings — a largely unexplored domain .

In this kind of networked virtual environment , people can meet in real time and interact with one another . There are plenty of questions still to be explored , says Eugy Han , a PhD student who leads the academic study about the VR class , from how responsible people-as-avatars are for their actions to how many people is too many for one location . “ It ’ s a really hot topic ,” she says .

Bailenson envisions a day when he can gauge how well HP Omnicept headset-clad students are comprehending his lectures so he can tailor his instruction in real time .

As the lab continues to break new ground , Bailenson and the researchers are thinking about their next projects , such as telepresence systems that use computer vision to beam in a volumetric capture of another human ; studying how matching race and gender between co-learners and teachers in VR impacts social and cognitive learning ; and integrating their ocean acidification content into high school environmental education curricula . But even as they expand the field of VR studies , Bailenson and the lab have prioritized making content widely available so that others can use it .

It is the emphasis on the realworld impact of VR that sets the lab apart , says Stanford ’ s Greenleaf .

“ They ’ re looking at some very difficult issues , but important issues , in terms of both the evolution of VR technology and its impact on society .” the lab ’ s ongoing studies , revealing new types of data .

“ It ’ s going to be a game changer in the lab , where we ’ ve never been able to collect good physio data ,” says Bailenson . Typically the best devices for gathering that data require staying still . “ By definition , good VR involves movement . And the Omnicept bridges that .”

Teaching in VR This past summer , the pandemic ushered in an unexpected window of opportunity when Bailenson taught an entire curriculum to roughly 100 students in VR for the first time . This quarter , one of three courses Bailenson is teaching will also be in VR — to nearly 170 undergraduate and master ’ s degree students , the

HP / INNOVATION / FALL 2021 21