Peering at his laptop screen during a Skype video call, Professor Mark Mon-Williams declares “It’s going to change the way we live, work, and play. And a lot of that is for the better.” Mon-Williams, a cognitive psychologist at Leeds, is talking about virtual reality − the next ‘disruptive’ technology that many tech industry experts say is poised to shake the foundations of how we do things, from communicating to learning to healthcare.
“Twenty-five years ago, the idea that I could sit here now with a wireless device to talk to somebody and see their face sounded fantastical,” says Mark. But now that idea is a reality, he thinks that a more natural experience could be delivered by virtual reality. “Every day the vast majority of us are interacting with computers but in a suboptimal manner − already I’m getting sore shoulders sitting here looking at my laptop – it’s a really unnatural way of interacting.” Virtual reality, or VR, is not new of course. In the early 1990s it was much hyped, and it was around this time that Mark started to use it as a human psychology research tool. However, VR never became mainstream − it was prohibitively expensive and simply lacked the computing power. And then there were the consequences of being immersed in VR. People reported experiencing headaches, sore eyes and motion sickness when using VR systems for even just short periods of time.
Some experts believed that more powerful computing in the future could iron out the potential health problems, but Mark made a crucial discovery. His experiments revealed that the way VR presents information is at odds with how humans process visual information in the real world. The focusing and alignment mechanisms of our eyes are hardwired to work together to help us interact with what’s around us. But in VR, our eyes must focus on a 2D screen while they align towards 3D objects out in the presented virtual world. This separates, or decouples, the two mechanisms, placing stress on the brain. This is what Mark found caused adverse effects among users − something no amount of processing power would fix.
Today, the computing and technology landscape is transformed. Smartphones and the internet are ubiquitous, technology is more affordable and, with backing from the likes of Facebook, Apple and Google, VR is making a big comeback. “Now it has the foundations, it has the tech support,” says Mark. “But some of those issues we raised 25 years ago still haven’t been resolved. A big part of the work we are doing at Leeds is to engineer those problems out.”
Mark is helping to do that by taking advantage of VR itself to better understand how humans use sensory information to control their actions. “The great thing about VR is we can have complete control, so we know exactly what is happening at a millisecond basis within that virtual world,” he says. Simultaneously, the individual who is interacting with that virtual environment can be precisely tracked to obtain useful data on their behaviour. By knowing what sensory information we need, VR systems can be designed to allow our brains to enjoy more natural interactions with computers. Mark’s research has already led to software development as well as potential hardware solutions, engineering suitable optics in head-mounted displays that don’t place unnatural demands on our visual system. “The theoretical foundations are strong,” he says. “We just need to do the science to check that our ideas do remove the difficulties, in order that VR can reach its full potential.”
Brain training Some VR applications, however, don’t suffer from the visual mismatch posed by large immersive environments. Virtual worlds that are confined to really small 3D spaces can avoid those problems, and Leeds is exploring their potential. “VR is a complete game- changer from a learning perspective,” says Dr Faisal Mushtaq, a psychologist and colleague of Mon-Williams. Faisal investigates how humans learn skills and how such knowledge can aid VR systems to accelerate that learning.
He recently conducted experiments on how VR could help surgeons prepare for a procedure. Before undertaking an operation in a surgical simulator, one group prepared by interacting with a 3D VR model of a tumour. Meanwhile a different group had a traditional 2D scan of a tumour to help visualise the surgical steps required. The group using VR to prepare for the task was found to complete it more quickly and safely. With promising results, Faisal thinks that VR could provide a cost-effective solution for training surgeons, especially since the European Working Time Directive limits how much real-world practice surgeons can now get.
VR training is already benefiting Leeds dental students. A £1.2 million state-of-the-art suite of 30 VR dental simulators – the largest of its type in the UK and the second largest in the world – lets students practise on virtual cases with realistic haptic feedback provided through a hand piece.
What’s more, a pilot project, dubbed ‘patient- specific surgical rehearsal’, allows fourth and fifth year dental students to upload 3D scans of their patients into the VR system and practice complex techniques prior to performing the actual procedure. “When trainees first start interacting with patients there’s a high risk of error given they don’t yet have the experience,” says Faisal. “Using VR accelerates the training process and provides a safe environment for trainees to practise.”
Medicine and Dentistry are not the only areas benefiting. Leeds is embracing VR. Data scientists are visualising datasets that could improve hospital efficiencies, biologists are understanding protein interactions, and transport researchers are developing systems to increase road safety.
Into the unknown Meanwhile, Mark and Faisal are demonstrating VR in schools by enabling children to explore virtual environments, including the solar system, the inner workings of the human body and dinosaurs roaming the Earth. “It was just wonderful seeing children getting so excited. You just think that in a decade there is no way that these systems aren’t going to be fully exploited to better educate schoolchildren,” says Mark. The researchers also allow children to practise writing in VR using a robotic device, which could assist those who struggle with writing due to motor difficulties. “This could be a powerful tool to remove some of the inequalities that exist within our society.”
alumni.leeds.ac.uk
To avoid the risks associated with larger VR environments, the team restricts how long children can use them. The visual mismatching issue becomes a problem when VR might be used for longer periods, which could be highly disruptive to our normal physiological and psychological workings, and more so for children. “We don’t yet know what the implications are for prolonged use of VR over time for children where the visual system is still developing,” says Faisal.
The problems are especially important to rectify now because while Mark is eager to tap into VR’s educational potential for children, technology companies want to exploit VR in the gaming and movie markets. “We don’t want headsets going out into households where a child could use it for 12 hours,” Mark says. “It seems a sensible way forward to engineer the problems out of the systems from the outset.”
Computers have already changed the way we live, work and play, but VR promises to take it to another level. “Skype already allows us to see each other and talk but through VR we’ll be able to share a common space and interact in naturalistic ways with computer-generated information,” says Mark. “It’s almost impossible to believe it is not going to transform our lives for the better, and Leeds wants to be at the vanguard.”
The Institute for Transport Studies (ITS) is developing warning systems to help truck drivers be more aware of cyclists. ITS has also created a VR model of Leeds city centre as part of an initiative called VirtuoCITY, helping to design cities in a future where humans will interact with automated vehicles and robots.
Earth and Environment students can explore the topography of virtual landscapes, going on field trips from their desk.
Born in Bradford is a major project researching the reasons for poor health outcomes in the city. As part of the project, Mark Mon-Williams uses elements of virtual reality as a tool to help understand children’s development.
Engineers, scientists and clinicians have designed and built robots for uses from helping patients to recover after a stroke, to exploring chambers in Egypt’s Great Pyramid.
The University’s Cultural Institute is facilitating collaborations with the creative industry including research into virtual and augmented reality, working with bodies such as Creative England and UKIE.