Part of a study released this week spoke about use of animations that are optimized for synchronized audio and video. It was a follow-up experiment to one that might help to explain why individuals with autism spectrum disorders have a tendency to stare at people’s mouths.
The study found that it was a fascination with the exact match of lip movement to sound produced, according to a study in 2-year-olds with the social deficit disorder.
Individuals with autism spectrum disorders (ASD) (http://www.nimh.nih.gov/health/topics/autism-spectrum-disorders-pervasive-developmental-disorders/index.shtml) tend to stare at people’s mouths rather than their eyes. Now, an NIH-funded study in 2-year-olds with the social deficit disorder suggests why they might find mouths so attractive: lip-sync — the exact match of lip motion and speech sound. Such audiovisual synchrony preoccupied toddlers who have autism, while their unaffected peers focused on socially meaningful movements of the human body, such as gestures and facial expressions.
“Typically developing children pay special attention to human movement from very early in life, within days of being born. But in children with autism, even as old as two years, we saw no evidence of this,” explained Ami Klin, Ph.D., of the Yale Child Study Center, who led the research. “Toddlers with autism are missing rich social information imparted by these cues, and this is likely to adversely affect the course of their development.”
Klin, Warren Jones, and colleagues at Yale, report the findings of their study, funded in part by the National Institute of Health’s National Institute of Mental Health, online March 29, 2009 in the journal Nature.
For the first time, this study has pinpointed what grabs the attention of toddlers with ASDs,” said NIMH Director Thomas R. Insel, M.D. “In addition to potential uses in screening for early diagnosis, this line of research holds promise for development of new therapies based on redirecting visual attention in children with these disorders.”
A eureka moment in the research came when researchers followed up on a clue from children’s responses to audiovisual synchrony embedded in a nursery rhyme cartoon.
While it was known that people with autism do not spontaneously orient to social signals, it was unclear what early-emerging mechanism may contribute to that. Nor was it clear exactly what they were attending to instead. To find out, Klin, Jones and colleagues tracked the eye movements of two-year-olds with and without the disorder while they looked at cartoon animations on split-screen displays.
The researchers borrowed a technique from the video game industry, called motion capture. They then reduced the movements to only points of light at each joint in the body, like animated constellations. These cartoons played normally — upright and forward — on one half of the screen, but upside-down and in reverse on the other half. The inverted presentation engages different brain circuits and is known to disrupt perception of biological motion in young children. The normal soundtrack of the actor’s voice, recorded when the animations were made, accompanied the presentations.
Eye-tracking data initially showed that 21 toddlers with ASD had no preference for the upright animations, looking back and forth between the two. By contrast, 39 typically-developing toddlers and 16 developmentally delayed but non-autistic toddlers clearly preferred the upright animations.
However, responses to one animation didn’t fit the pattern. The toddlers with ASD changed their behavior and shifted their attention to the upright figure as it played a game of pat-a-cake, where the figure claps his hands repeatedly. In this animation (see movie below), unlike the others, the movements of the points of light actually cause the clapping sound. This physical synchrony — dots colliding to produce a clapping sound — only existed on the upright side of the screen, because the inverted figure played in reverse and its motions weren’t in sync with the soundtrack. The children with ASD chose the upright figure 66 percent of the time, a strong preference.
This clue led the researchers to suspect that what initially appeared to be random viewing by the ASD toddlers might actually reflect preference for audiovisual synchronies that were less obvious than the clapping. So they re-analyzed the data, factoring in more subtle synchronous changes in motion and sound.
“Audio-visual synchronies accounted for about 90 percent of the preferred viewing patterns of toddlers with ASD and none of unaffected toddlers,” said Jones. “Typically-developing children focused instead on the most socially relevant information.”
A follow-up experiment using new animations optimized for audiovisual synchrony confirmed these results.
Klin, Jones, and colleagues also recently reported (http://www.nimh.nih.gov/science-news/2008/lack-of-eye-contact-may-predict-level-of-social-disability-in-two-year-olds-with-autism.shtml) that children with autism look more at peoples’ mouths than eyes as early as age 2. Since the mouth is the facial feature with most audiovisual synchrony — lip motion with speech sound — the researchers propose that their new findings offer a likely explanation for this phenomenon.
“Our results suggest that, in autism, genetic predispositions are exacerbated by atypical experience from a very early age, altering brain development,” said Klin. “Attention to biological motion is a fundamental mechanism of social engagement, and in the future, we need to understand how this process is derailed in autism, starting still earlier, in the first weeks and months of life.”
NIMH is funding a related research project of Klin and Jones’ that explores related behaviors in infants who have older siblings already diagnosed with ASD and who, because of the genetic heritability risk in autism, have greater risk of also developing the condition.
Also participating in the research were: David Lin, now at Harvard Medical School; Phillip Gorrindo, now at Vanderbilt University; Gordon Ramsay, Ph.D., Haskins Laboratories. The study was funded through the NIH’s STAART Program (Studies To Advance Autism Research & Treatment).
Eye-tracking data shows where toddlers in each of three groups were looking during the Pat-a-Cake animation. It plays upright and forward on the left side of screens, upside down and in reverse on the right side. Red cross indicates where the child was looking. Toddler with autism is focused on audiovisual synchrony of hands clapping, while typically developing and developmentally delayed toddlers focus on face.
Pat-a-cake animation, with audio playing at half-speed and color scale indicating levels of audiovisual synchrony in child with autism. The hands clapping show the highest levels of audiovisual synchrony (red) in the figure on the right, which played upright and forward. Motion was out of sync with the sound track in the figure on the left, because it played upside down and in reverse.
The National Institute of Mental Health (NIMH) mission is to reduce the burden of mental and behavioral disorders through research on mind, brain, and behavior. More information is available at the NIMH website, http://www.nimh.nih.gov.
The National Institutes of Health (NIH) — The Nation’s Medical Research Agency — includes 27 Institutes and Centers and is a component of the U.S. Department of Health and Human Services. It is the primary federal agency for conducting and supporting basic, clinical and translational medical research, and it investigates the causes, treatments, and cures for both common and rare diseases. For more information about NIH and its programs, visit www.nih.gov.
Reference: Klin A, Lin DJ, Gorrindo P, Ramsay G, Jones W. Two-year-olds with autism fail to orient toward human biological motion but attend instead to non-social physical contingencies. 2009 Nature.