Stanford works on mind reading video game controller


"You don't need to do anything extra, you can just hold the controller as you normally would," explained Stanford doctoral candidate Corey McCall.

But that controller is anything but normal.

"We're working on a video game controller that is able to sense how the player's feeling," he said.

McCall ripped the bottom off a normal Xbox controller and 3D printed a new one packed full of high-tech sensors.

"These metal plates that can sense various things, as well as the pulse oximitry sensor," he said.

It knows how fast your heart's beating, how deeply you're breathing, and how much you're moving your hands.

"So instead of actually reading the player's brain, we just read what is affected by the brain," said McCall.

Now, they're about to start clinical trials.

They'll use a device to track how much movement's on the screen. In other words, how exciting the game is. They want to know what your heart does when a surprising or scary moment happens in the game.

Eventually they want to feed all this data right back into the game.

"And we hope that the game will be able to adapt individually to each player, to make it fun for that specific person," said McCall.

So if the game senses you're bored, maybe they'll turn it up a notch to recapture your attention.

Professor of Electrical Engineering Greg Kovacs says it's a new frontier.

"That has not yet happened very much, where the game's picking up something form the user and adjusting itself, I think that's really cool," he said.

It turns out the research could also have broader implications. The ability to tell how you're feeling based on something you already have in your hands could be useful far beyond the gaming world.

"If you have the same sensors basically built into a steering wheel for example, of a vehicle, and someone's heart rate is declining, their respiration is declining, you might take a guess they're falling asleep," Kovacs said.

On public transit, he says that could protect hundreds of lives. And even on video games, it could protect peace of mind for parents.

"Parents are concerned that their kids are getting too involved in the game, maybe they're getting too hyped up about it," McCall said. "We would also be able to sense that and sort of tell the player you need to take a break."

Copyright © 2023 KGO-TV. All Rights Reserved.