In Search of Communication: Robots and Autism

One in 150 children in the United States will be diagnosed with an autism spectrum disorder (ASD). ASD is a complex neuro-developmental condition whose cause isn’t completely understood. The degree to which a child with ASD is affected varies widely. What is clear, says Nilanjan Sarkar, associate professor of mechanical engineering and computing engineering, is that those with autism have difficulty communicating. 

 

“More than 70 percent of human communication is nonverbal, and 55 percent of emotional meaning is communicated nonverbally,” says Sarkar. These nonverbal signals can be explicit, such as a yawn or a grimace, or implicit, such as muscle tension or perspiration. These signals could provide important information about the mental state of a person with autism.  

 

“People with ASD don’t express themselves clearly. If we can understand their nonverbal cues, we can develop better interventional strategies,” says Sarkar, who became interested in autism after a cousin’s son was diagnosed with it. “The overarching goal of my work is to develop a robotic system that understands you and can respond to your needs.”  

 

Emotion-Sensing Basketball

 

Sarkar’s earlier work with a robotic system that senses and responds to physiological cues that are related to emotion, such as heart rate, grimacing and perspiration, is at the heart of his new work. Sarkar collaborated with Wendy Stone, an investigator in the Vanderbilt Kennedy Center for Research on Human Development and director of the center’s Treatment and Research Institute for Autism Spectrum Disorders, to explore the project’s potential and application. Through a grant from the Marino Autism Research Institute, Sarkar and his team developed a robot-based basketball game. The game includes a robotic arm that adjusts the difficulty of the game by moving the hoop based on the mental state of children with ASD through assessment of nonverbal signals obtained from wearable physiological sensors. 

 

“In this research, the robot is endowed with intelligence to decipher nonverbal communication and the ability to progressively engage and challenge a child,” says Sarkar, who presented his work in August at the IEEE International Symposium on Robot and Human Interactive Communication in Korea. 

 

A robot can present a task in an engaging way that kids like. Using a child’s physiological responses to gauge how well he likes the task, the robot can subtly moderate the task to a level the child progressively enjoys, he explains.  

 

The next step, says Sarkar, is to develop an intelligent virtual reality system. The virtual reality characters will engage the child in social interactions and use the child’s physiological feedback to build communication skills incrementally. Sarkar has been awarded a $120,000 biomedical research grant from Autism Speaks for this phase.  

 

The goal, he says, is to create an interactive technology that challenges children with autism to communicate better in a measurable way. “The technology allows us to change the interaction incrementally, to start the kids in a position of comfort and slowly scaffold their skills so they improve.”