New device lets players control video games by using facial expressions and head motions


Over decades, input devices in the video game industry have evolved from simple joysticks to sophisticated controllers that emit haptic feedback. But with Enabled Play, a new piece of assistive tech created by self-taught developer Alex Dunn, users embrace a different kind of input: facial expressions.

While companies such as Microsoft have sought to expand accessibility through adaptive controllers and accessories, Dunn's new device translates users' head movements, facial expressions, real-time speech and other nontraditional input methods into mouse clicks, keystrokes and thumbstick movements. The device has users raising eyebrows — quite literally.

"Enabled Play is a device that learns to work with you — not a device you have to learn to work with," Dunn, who lives in Boston, said via Zoom.

Dunn, 26, created Enabled Play so that everyone — including his younger brother with a disability — can use technology in a natural and intuitive way. At the beginning of the pandemic, the only thing he and his New Hampshire-based brother could do together, while about 70 miles apart, was play video games.

"And that's when I started to see firsthand some of the challenges that he had and the limitations that games had for people with really any type of disability," he added.

At 17, Dunn dropped out of Worcester Polytechnic Institute to become a full-time software engineer. He began researching and developing Enabled Play 2½ years ago, which initially proved challenging, as most speech recognition programs lagged in response time.


[Video not showing up above? Click here to watch » arkansasonline.com/88play/]


"I built some prototypes with voice commands, and then I started talking to people who were deaf and had a range of disabilities, and I found that voice commands didn't cut it," Dunn said.

Having already built Suave Keys, a voice-powered program for gamers with disabilities, Dunn created Snap Keys — an extension that turns a user's Snapchat lens into a controller when playing games such as "Call of Duty," "Fall Guys," and "Dark Souls." In 2020, he won two awards for his work at Snap Inc.'s Snap Kit Developer Challenge, a competition among third-party app creators to innovate Snapchat's developer tool kit.

With a wider variety of inputs, users can connect the Enabled Play device -- equipped with a robust CPU and 8 gigabytes of RAM -- to a computer, game console or other device to play games in whatever way works best for them.

Dunn also spent time making sure Enabled Play was accessible to people who are deaf, as well as people who want to use audio input, such as "ooh" or "aah," to perform an action. Enabled Play's vowel-sound detection model is based on "The Vocal Joystick," which engineers and linguistics experts at the University of Washington developed in 2006.

Dunn's artificial-intelligence-enabled controller takes into account a person's natural tendencies. If players want to set up a jump command every time they open their mouth, Enabled Play would identify the individual's resting mouth position and set that as the baseline.

In January, Enabled Play launched in six countries, its user base extending from the United States to Great Britain, Ghana and Austria. Dunn sees Enabled Play filling a gap in accessibility and pricing compared with other assistive gaming devices.

"There are things like the Xbox Adaptive Controller," he said. "There are things like the Hori Flex [for Nintendo Switch]. There are things like Tobii, which does eye-tracking and stuff like that. But it still seemed like it wasn't enough."

Dunn's AI-enabled controller — priced at $249.99 — supports a combination of game systems or computers. Speech therapists say that compared with augmentative and alternative communication (AAC) devices, which are medically essential for some with disabilities, Dunn's device offers simplicity.

Julia Franklin, a speech language pathologist at the Community School of Davidson in Davidson, N.C., introduced students to Enabled Play this summer. She said it's a better alternative to other devices on the market that are "expensive, bulky and limited" in usability. Many sophisticated systems can range from $6,000 to $11,500 for high-tech devices, with low-end eye-trackers running in the thousands. A person can also download apps on their mobile devices, which range from $49.99 to $299.99.

"For many people who have physical and cognitive differences, they often exhaust themselves to learn a complex AAC system that has limits," she said. "The Enabled Play device allows individuals to leverage their strengths and movements that are already present."

The device is also being used in schools to make computer labs more accessible to disabled students.


Upcoming Events