'It is just stupid': Tech experts voice concern over drivers using mixed reality headsets

This still from a video shared to social media site X shows an unidentified driver behind the wheel of what appears to be a Tesla Cybertruck while wearing what looks like an augmented reality headset. (Twitter/@SecretaryPete)

Viral videos of people who appear to be driving Tesla vehicles while wearing mixed reality headsets have sparked safety concerns within the U.S. government and among Canadian tech experts.

"Reminder — All advanced driver assistance systems available today require the human driver to be in control and fully engaged in the driving task at all times," U.S. Transportation Secretary Pete Buttigieg wrote Monday in a post on X with a video that attracted more than 25 million views.

• Top headlines on electric vehicles, and all the latest automotive news

The video shows an unidentified man who appears to be wearing the new US$3,500 Vision Pro -- Apple's first mixed reality headset, which blends virtual reality and augmented reality -- while driving a Tesla electric pickup Cybertruck. He is shown gesturing as if pressing buttons in the air, with both hands off the steering wheel.

The Apple Vision Pro headset was released in the U.S. last week. An Apple employee told CTVNews.ca it was unclear when it would be released in Canada.

Dante Lentini, a computer science student at New York University, also posted a viral video on X that had over 26 million views as of Friday. He later told the tech and science website Gizmodo that the stunt was for a "skit" he was making with friends, adding that he only drove wearing the headset while the car was on autopilot for 30 to 40 seconds.

Apple did not comment directly on the videos when asked about them by CTVNews.ca, but said people should “never use the device while operating a moving vehicle, bicycle, heavy machinery, or in any other situations requiring attention to safety.”

Tesla did not respond to CTVNews.ca's request for a comment.

While Canada has not issued any recent warning about using mixed reality headsets on the road, Canadian tech experts weighed in on the hype and the prospect of drivers using these devices behind the wheel.

Safety concerns

"I have severe concerns about road safety," said Douglas Soltys, editor-in-chief of BetaKit, a Canadian startup and tech innovation online publication, in an email to CTVNews.ca. "This is the same distracted driving concern of smartphone use cranked to 11. No one should have these devices strapped to their heads while behind the wheel — including (in) vehicles with automated driving systems (like Tesla)."

Soltys said the devices are not yet popular in Canada, but that he believes they could eventually become more popular if they become more affordable.

"I don't expect the devices to become more widely adopted in the short term, particularly due to price, but Apple has a tendency to popularize pre-existing technology for the masses," he said.

Shi Cao, associate professor in the Department of Systems Design Engineering at the University of Waterloo in Waterloo, Ont., doesn't expect to see headsets such as the new Apple device commonly used on roads for practical and safety reasons.

"It is just stupid and I think that they're just buying clicks," Cao said about the viral videos in a phone interview with CTVNews.ca. "I would not worry too much. For any people who have common sense, they should avoid doing that."

But he points out that the level of distraction the headsets create depends on how much information the user adds to them.

• Top science and technology headlines, all in one place

Cao's lab owns both older and newer models of virtual reality and augmented reality headsets such as Microsoft HoloLens, Oculus Quest and Google Glass, which all have different designs. While some devices use transparent glasses to allow people to see through them, the Apple headsets do not.

Instead, they use cameras to record the view in front of the user, add virtual content into the video stream, and then play the combined result on a pair of internal displays so that the user will perceive a 3D world composed of real-world scenes and virtual content.

Just like smartphones, the virtual content could include movies, digital notes, games, and more, he added.

"The level of distraction depends on the contents being viewed inside those devices," said Cao, who studies distracted driving as well as augmented reality and virtual reality applications. "If you only wear those devices (and) maybe only a clock is shown in those devices, then the distraction will be minimum. … But if you start to do texting, or go to video, or play a game, that definitely will be a big problem."

For now, Cao sees these headsets as more useful for developing virtual training programs for new drivers or pilots. He doesn't see mixed reality headsets like Vision Pro as useful on the road, partly because they aren't transparent. He said they can be useful if a transparent glasses-like design is used, allowing drivers to see more useful information near their view of the road, such as speed, navigation and important messages.

"Technically, those devices like (Vision Pro), you could wear it and see outside the world, but only when it has power," Cao explained. "If the power runs out, of course you cannot see anything because it's not transparent."

Another concern with using the newer headsets is the smaller field of view, he added. "Those VR goggles with the camera, because they're overlaying information using video, the field of view angle as well as image resolution is limited," Cao explained in an email.

Martin Lavallière, a kinesiology professor at the University of Quebec in Chicoutimi, Que., and chair of the Canadian Association of Road Safety Professionals, said he is concerned the technology could be used inappropriately by some people who may overestimate or misunderstand its capabilities and limitations.

He said the challenge lies with augmented-reality or mixed-reality headsets, which allow users to see the real world with additional information or content added, rather than with virtual-reality headsets. Virtual reality headsets would be completely forbidden on the roads since people would not be able to see through the glasses, he explained.

"That visual information comes at a cost of … possible cognitive saturation," Lavallière said in a phone interview with CTVNews.ca. "If I keep bombarding someone with visual information while they're driving … I'm distracting the person depending on which types of information I'm giving them."

Instead of using information on traffic jams, road conditions and the state of the vehicle, Lavallière thinks people will probably use augmented- or mixed-reality headsets for purposes not related to driving, thus increasing the risk of collisions on the road.

Automation and driver distraction

Fully automated driving systems aren't commercially available in Canada, though they are being tested at the University of Waterloo, Cao said. Currently, he doesn't believe they are available anywhere in the world.

However, partially automated driving systems like those found in Tesla vehicles are available for consumers in Canada, putting the responsibility of safety on the driver, Cao added. For instance, if automation fails, it will tell the human driver to take control of the vehicle.

"The driver must be in charge of everything and monitoring the vehicle all the time," Cao said of the partially automated driving systems. "The driver cannot sleep, or watch a video or play VR. They should be monitoring the car (because) the system is not perfect."

If there are any crashes, Cao said the driver will be primarily responsible, not the car manufacturer, under the current design of partially automated driving systems.

• Watch live events and breaking news video on demand

Bruce Hellinga, professor at the Department of Civil and Environmental Engineering at the University of Waterloo, said the law in Canada commonly requires that the driver is responsible for the control of the vehicle at all times and bans distracted driving. Some Canadian jurisdictions ban the use of smartphones and cell phones unless they allow hands-free operation, he added.

"The greater the automation in the vehicle, the easier it is for a driver to engage in distracting activities, possibly because they believe that the risk is low because the vehicle is doing the 'driving,'" he said in email to CTVNews.ca. "Nevertheless, no vehicle manufacturer in the world (to my knowledge) trusts their own self-driving systems enough to sell them to the public and then assume financial/legal responsibility for the performance of the vehicle while operating with those systems."

Future of headsets and driving

Cao said right now, he only foresees a future in which people can drive safely using mixed reality headsets, under "limited" conditions, if fully autonomous cars become commercially available, which he thinks may be possible in 20 years. In those cases, people can sleep, watch videos or not pay attention to driving. He said headsets would be safer to use in specially designed driving scenarios, such as if there was a highway with a lane exclusively for fully automated vehicles that has a specific design to ensure "it's perfectly safe."

"Like if you mix an autonomous car with human drivers, pedestrians, cyclists, and other types of human road users, I don't think that's a good idea and I don't foresee that in the future," Cao explained.

With files from Reuters