Overview – Baron-Cohen et al (1997)

Baron-Cohen, S., Jolliffe, T., Mortimore, C. and Robertson, M. (1997) Another advanced test of theory of mind: evidence from very high functioning adults with autism or Asperger Syndrome. Journal of Child Psychology and Psychiatry, 38, 813-822.

Baron-Cohen et al (1997) is a classic study in the psychology of individual differences that investigated differences in theory of mind in adults with autism or Asperger syndrome. Baron-Cohen et al tested autistic and neurotypical adults using an advanced theory of mind task – the Eyes Test – to examine whether they could infer other people’s thoughts, beliefs, and intentions from pictures of people’s eyes. The study found that adults with autism or Asperger syndrome performed significantly worse than neurotypical adults, providing evidence that difficulties in understanding other people’s mental states may be a characteristic of autism. The study supported the idea that autism involves differences in social cognition.

You can jump to different sections of this article via the links below:

• Aim
• Method (sampling and procedure)
• Results
• Conclusions
• Evaluation and methodological issues
• Relation to other psychological studies of individual differences

Baron-Cohen et al (1997)

Aim

The aim of Baron-Cohen et al (1997) was to investigate whether very high-functioning adults with autism or Asperger syndrome have difficulties with theory of mind (ToM).

Theory of mind is the ability to understand that other people have their own mental states – such as thoughts, beliefs, feelings, intentions, and desires – which may differ from our own and influence their behaviour. For example, understanding that someone can hold a false belief requires recognising that their behaviour may be based on what they think is true rather than what is actually true.

Baron-Cohen argued that a difficulty in developing theory of mind may be a core cognitive feature of autism – that people with autism may have ‘mind-blindness’ in that they have difficulties attributing mental states to other people and understanding how these mental states influence their behaviour. This could help explain some of the social and communication difficulties for autistic people because such communication often requires us to infer what another person means or intends rather than simply interpreting their words literally.

Existing theory of mind tests were designed for children and were relatively easy such that even an adult with an incomplete ToM could achieve a normal score. This created a ‘ceiling effect’: the tests were not difficult enough to reveal subtle differences in ToM ability among high-functioning adults. In other words, it was possible that autistic people continued to have deficiencies in theory of mind into adulthood but the existing tests were unable to pick this up. Happé (1994) addressed this problem by testing adults with autism or Asperger syndrome using more difficult ‘Strange Stories’ – finding that they had greater difficulty interpreting mental-state stories than matched controls. Baron-Cohen et al built on this research by developing an advanced test of ToM – the ‘Eyes Test’ – which required participants to infer complex mental states from photographs showing only the eye region of people’s faces.

So, in short, the aim of Baron-Cohen et al (1997) was to investigate whether very high-functioning adults with autism or Asperger syndrome showed impairments in theory of mind.

Method

Baron-Cohen et al (1997) used a quasi-experiment (an experiment where the IV is a naturally occurring characteristic that cannot be manipulated) with an independent measures design.

The independent variable (IV) was participant group, with participants belonging to one of 3 naturally occurring groups:

• Adults high-functioning autism or Asperger syndrome
• Adults with Tourette syndrome
• Neurotypical adults.

The main dependent variable (DV) was participants’ score out of 25 on the Eyes Test, which measured their ability to infer complex mental states from photographs of people’s eyes. Participants also completed the Strange Stories test and two control tasks to provide additional measures of theory of mind and check whether performance on the Eyes Task could be explained by more basic perceptual or emotional-recognition difficulties.

Participants and sampling

The sample consisted of 76 adults in total, divided into 3 groups:

• High-functioning autism/Asperger syndrome group:
  • 16 participants (4 with high-functioning autism and 12 with Asperger syndrome)
  • 13 males and 3 females
  • All were of normal intelligence. Participants were recruited through an advert in the National Autistic Society magazine and from clinical sources.
• Control group:
  • 50 neurotypical adults from the general population of Cambridge
  • 25 males and 25 females
  • They were age-matched to the participants in the autism/Asperger syndrome group.
• Tourette syndrome group:
  • 10 adults with Tourette syndrome
  • 8 males and 2 females
  • They were also age-matched with the other groups, had normal intelligence, and were recruited from a specialist referral centre in London.

The inclusion of the Tourette syndrome group provided an additional comparison group. This helped researchers investigate whether any difficulties found in the autism/Asperger syndrome group were specifically associated with autism-related theory of mind difficulties rather than simply being a consequence of having a clinical condition.

Procedure

Participants were tested individually in a quiet room, either in their own home, at the researchers’ clinic, or in the researchers’ laboratory at Cambridge University. They completed four tasks, which were presented in a random order.

The main measure of theory of mind was the Eyes Test:

• Participants were shown 25 standardised black-and-white photographs of the eye region of faces, with both male and female eyes represented.
• For each photograph, they were given two mental-state words and had to choose the word that best described what the person in the photograph was feeling or thinking. One word was the correct target and the other was a foil. For example, the participant might have to decide whether the eyes represented a particular emotional or mental state from the two alternatives provided.
• Participants received a score out of 25, based on the number of mental states they correctly identified.

The Eyes Test was designed to assess the ability to infer complex mental states from limited social information. Because only the eyes were shown, participants could not rely on information from the person’s whole face, body language, or the situation. This made the task a more demanding test of theory of mind than simpler tests designed for young children.

The other 3 tasks participants completed were:

• Gender Recognition Task: Participants identified the gender of the person shown in the eye photographs. This was used as a control task for basic face perception and perceptual discrimination. It helped establish whether any difficulties on the Eyes Task were specifically related to interpreting mental states rather than simply recognising information from photographs of eyes.
• Basic Emotion Recognition Task: Participants were shown photographs of whole faces displaying basic emotions and had to identify the emotion being expressed. This controlled for basic emotion-recognition ability. If participants had difficulty with the Eyes Task, this task helped determine whether the difficulty was specifically with complex mental-state inference rather than with recognising emotions generally.
• Strange Stories Task: Participants completed Happé’s Strange Stories, an advanced theory of mind task involving stories requiring participants to interpret people’s mental states. This provided a second measure of theory of mind and allowed the researchers to compare performance on the Eyes Task with an established advanced theory of mind measure.

The four tasks were presented in a random order to participants in order to reduce the possibility that order effects – i.e. the order in which the tasks were completed – would affect performance and bias the results.

Results

The autism/Asperger syndrome (AS) group had the lowest mean score on the Eyes Test, which was scored out of 25. The neurotypical and Tourette syndrome groups performed at very similar levels:

The mean score for the autism/Asperger group (16.3) was significantly lower than both the neurotypical group (20.3) and the Tourette syndrome group (20.4). However, there was no significant difference between the neurotypical and Tourette syndrome groups.

There were also differences within the neurotypical group. Neurotypical females scored significantly higher than neurotypical males, with mean scores of 21.8 and 18.8 respectively. However, neurotypical males still scored significantly higher than the autism/Asperger’s group (18.8 compared with 16.3).

The results from the other tasks provided further information:

• Strange Stories: The autism/Asperger group made significantly more errors than both the neurotypical and Tourette syndrome groups. No participants in the Tourette syndrome group made any errors on this task.
• Gender Recognition Task: There were no significant differences between the groups, suggesting that the autism/Asperger’s group’s lower Eyes Test scores could not simply be explained by difficulties with basic perception or recognising information from the eye region.
• Basic Emotion Recognition Task: There were also no significant differences between the groups, suggesting that the autism/Asperger group’s difficulties were not simply due to an inability to recognise basic emotions.
• IQ: Within the autism/Asperger group, there was no significant correlation between IQ and performance on the Eyes Task. This suggests that differences in general intelligence did not explain the group’s lower scores.

Overall, the autism/Asperger group performed significantly worse on the advanced theory of mind measures than the other groups, while performing similarly to them on the control tasks.

Conclusions

The findings led Baron-Cohen et al to two main conclusions:

Adults with autism/Asperger syndrome have an impaired theory of mind

The autism/Asperger group performed significantly worse than both the neurotypical and Tourette syndrome groups on the Eyes Task and Strange Stories task. Because the groups did not differ on the gender and basic emotion control tasks, the researchers concluded that these difficulties were specifically related to theory of mind, rather than to basic perception or emotion recognition. This provided evidence that, contrary to some previous research suggesting that adults with autism may have an intact theory of mind, theory of mind difficulties can persist into adulthood.

Theory of mind difficulties are independent of general intelligence

All participants in the autism/Asperger group had normal intelligence, and some had university degrees. Furthermore, there was no significant correlation between IQ and performance on the Eyes Task within the autism/Asperger group. This suggests that the theory of mind difficulties identified in the study cannot simply be explained by lower general intelligence. Instead, they may represent a specific cognitive difficulty associated with autism/Asperger syndrome.

Evaluation

Research methods and techniques

Baron-Cohen et al (1997) was a quasi-experiment with independent measures design. The researchers compared 3 naturally occurring groups of adults: high-functioning autism/Asperger syndrome, neurotypical, and adults with Tourette syndrome. Participants completed the Eyes Test and several control tasks designed to investigate whether any difficulties were specifically related to theory of mind rather than more general problems with face perception or emotion recognition.

• More sensitive theory of mind task: Traditional Theory of Mind tests were too easy for adults, making it hard to catch subtle difficulties (ceiling effect). The Eyes Test was designed to be more challenging, which allowed researchers to spot small differences in how autistic adults read social and emotional cues compared to adult controls.
• Control for confounding variables: The researchers included the gender recognition and basic emotion recognition tasks. These allowed them to investigate whether performance on the Eyes Test could be explained by more general difficulties with recognising faces, eyes, or basic emotions.
• Comparison group: The inclusion of adults with Tourette syndrome provided a useful clinical comparison group because it helped the researchers investigate whether any difficulties were associated with autism/Asperger syndrome specifically rather than simply being a consequence of having a psychological or neurological condition.
• Standardisation: All participants completed the same set of tasks, including the same 25 standardised black-and-white photographs in the Eyes Test. This helped ensure that differences in performance were more likely to reflect differences between the participant groups rather than differences in the testing procedure.
• Lack of random assignment (Quasi-Experiment): Because diagnostic status was a naturally occurring variable, participants could not be randomly allocated to the three groups. As a result, differences between the groups may have been caused by other factors associated with the groups rather than autism/Asperger syndrome itself.
• Artificial task: Inferring someone’s mental state from a photograph of their eyes is a highly controlled and unusual situation. This may not accurately represent how theory of mind is normally used during real social interactions.

Population and sampling

The study involved 76 adults divided into 3 groups: 16 adults with high-functioning autism or Asperger syndrome, 50 neurotypical adults, and 10 adults with Tourette syndrome. All participants were of normal intelligence and the groups were age-matched.

• Relevant participants: The autism/Asperger syndrome group consisted of people with the characteristics that Baron-Cohen et al wanted to investigate. The participants were also of normal intelligence, allowing the researchers to investigate theory of mind without focusing on general intellectual disability.
• Clinical comparison group: The Tourette syndrome group provided a useful comparison with another clinical population. This helped the researchers distinguish difficulties specifically associated with autism/Asperger syndrome from difficulties that might be associated with having a diagnosed condition more generally.
• Small autism/Asperger syndrome sample: Only 16 adults with autism or Asperger syndrome took part. This is a relatively small sample, making it difficult to generalise the findings to all autistic adults.
• Unequal group sizes: The three groups contained 16, 50, and 10 participants respectively. The much larger neurotypical group means that the groups were not evenly balanced, particularly compared with the small Tourette syndrome group.
• Unequal sex distribution: The autism/Asperger group contained 13 males and only 3 females whereas the Tourette syndrome group contained 8 males and 2 females. This means the study provides limited evidence about whether the findings apply equally to autistic men and women.
• Volunteer bias: The autism/Asperger participants were recruited through an advert in the National Autistic magazine and clinical sources. People who volunteered may have differed from autistic adults who did not volunteer. For example, they may have possessed higher cognitive functioning, stronger reading skills, or a greater awareness of their own social difficulties. As such, the sample was biased toward highly motivated individuals, reducing its representativeness.

Types of data

The study produced mainly quantitative data – particularly participants’ scores on the Eyes Test and their performance on the other tasks.

• Quantitative data: Participants were scored out of 25 on the Eyes Test and their performance on the other tasks could also be compared between groups.
  • Easy to compare: Numerical scores allowed the researchers to directly compare theory of mind performance between the autism/Asperger, neurotypical and Tourette syndrome groups.
  • Statistical analysis: The researchers were able to determine whether differences between groups were statistically significant. This provides a more objective basis for judging whether observed differences were likely to reflect genuine group differences.
  • Useful for identifying individual differences: The range of scores within each group showed that participants did not all perform identically. This provides some evidence that theory of mind ability varies between individuals.
  • Limited detail: Numerical scores show how well participants performed but do not provide much information about why an individual found a particular mental state difficult to identify.

Representativeness and generalisability

The study provides useful evidence about theory of mind in high-functioning adults with autism or Asperger syndrome, but there are limitations to how widely the findings can be generalised.

• Relevant sample: The autism/Asperger participants were directly relevant to the research question because they were adults with high-functioning autism or Asperger syndrome. Their normal intelligence also allowed the researchers to investigate whether theory of mind difficulties could occur independently of general intelligence.
• Age-matched groups: The groups were matched for age. This reduced the possibility that differences in theory of mind performance were simply caused by differences in age.
• Small sample: Only 16 adults with autism/Asperger were studied. This makes it difficult to know whether the same findings would be obtained from a much larger and more diverse sample of autistic adults.
• High-functioning participants: The participants were specifically selected because they had high-functioning autism or Asperger syndrome and normal intelligence. The findings may therefore not generalise to autistic people with intellectual disabilities or greater levels of support needs.
• Sex imbalance: The autism/Asperger group consisted mainly of males. This limits the extent to which the findings can be generalised to autistic females.
• Limited sample characteristics: The participants were recruited from specific sources, including an autism magazine and clinical services. They may therefore not be representative of all adults with autism or Asperger syndrome.

Ethical issues

• Informed consent: The participants were adults and were therefore able to provide their own informed consent to take part in the research.
• Protection from harm: The tasks were relatively low risk and involved looking at photographs and answering questions. There was no invasive procedure or obvious risk of physical harm.
• Psychological harm: Potential distress: Participants with autism or Asperger syndrome were being tested on an ability that the researchers expected might be impaired. Performing less well than other participants could potentially cause embarrassment or frustration, particularly if participants were aware of the purpose of the study.
• Privacy: The study involved participants’ diagnostic status and psychological abilities, which are potentially sensitive forms of personal information. Researchers therefore had a responsibility to keep participants’ data confidential and protect their identities when reporting the findings.
• Right to withdraw: As adults participating voluntarily in research, participants should have been able to withdraw from the study without penalty.

Overall, the study appears to have involved relatively few serious ethical concerns because the procedures were non-invasive and participants were adults. However, researchers needed to take particular care when studying autistic participants because the tasks involved assessing a socially important psychological ability.

Validity

• Internal Validity: Do the findings accurately show differences in theory of mind between the participant groups?
  • Controlled measures: The Eyes Test used 25 standardised photographs and required participants to choose between two mental-state words. This provided a consistent way of measuring performance across participants.
  • Control tasks: The Gender Recognition and Basic Emotion Recognition tasks helped rule out alternative explanations for poor performance. If the autism/Asperger group had simply struggled to recognise eyes, faces or basic emotions, this should have been reflected in their performance on the control tasks.
  • Multiple measures of theory of mind: The researchers also used Happé’s Strange Stories. Using another theory of mind measure strengthened the study because the findings did not depend entirely on performance on one task.
  • IQ controlled: All participants were of normal intelligence and there was no significant correlation between IQ and Eyes Test performance within the autism/Asperger group. This provides evidence that differences in Eyes Test performance were not simply due to general intellectual ability.
  • Quasi-experiment: The groups were naturally occurring rather than randomly assigned. Therefore, factors other than autism/Asperger, such as differences in education, personality or social experience, could potentially have contributed to differences in performance.
  • Validity of the Eyes Test: The Eyes Test requires participants to infer mental states from photographs of people’s eyes. It is possible that performance depends partly on other abilities, such as vocabulary, emotion recognition or familiarity with particular facial expressions, rather than theory of mind alone.
  • Leading interpretation: Participants were given a target mental-state word and a foil and had to choose between them. Their answer therefore depended partly on how well they understood the words used, which could introduce factors unrelated to their ability to understand another person’s mental state.
• Ecological Validity: Do the findings translate to real-world social behaviour?
  • Artificial task: Participants had to infer what someone was thinking or feeling from static photographs showing only the eye region. This is very different from normal social interaction, where people can use facial expressions, tone of voice, body language, context and previous knowledge.
  • Limited social context: The photographs provided very little information about what was happening in the situation. In everyday life, people usually infer mental states using a much wider range of contextual information.
  • Real social ability: However, the Eyes Test was specifically designed to assess the ability to infer complex mental states from subtle social information. This is relevant to an important aspect of everyday social interaction, even though the task itself was artificial.
  • Complementary measure: The use of Strange Stories provided an additional measure of theory of mind involving social situations and mental-state reasoning, making the overall investigation more relevant to real-world social understanding than the Eyes Test alone.

Reliability

• Internal Reliability: Is the procedure and measurement consistent across participants in this particular study?
  • Standardisation: All participants completed the same tasks and the Eyes Test used the same 25 standardised photographs. This increased consistency in how theory of mind was measured.
  • Forced-choice responses: Participants selected one of two mental-state words for each photograph. This provided a clear and objective way of recording their answers and reduced the subjectivity involved in scoring.
  • Randomised order: The Eyes Test, Strange Stories and the two control tasks were presented in a random order. This reduced the possibility that fatigue, boredom or practice effects would systematically affect one particular task.
  • Different testing locations: Participants were tested in their homes, the researchers’ clinic or the Cambridge University laboratory. Differences between these environments could potentially affect participants’ concentration or performance.
• External Reliability: Can the study be replicated at a different time and with different groups to produce consistent results?
  • Highly standardised Eyes Test: The use of 25 standardised photographs and fixed response options makes the main measure relatively easy for other researchers to repeat.
  • Clear procedure: The researchers clearly described the tasks used and the way in which participants’ performance was measured. This increases the replicability of the study.
  • Small clinical samples: The autism/Asperger and Tourette syndrome groups were relatively small. A replication using different participants could therefore produce somewhat different results because of individual differences within these groups.
  • Changing diagnostic categories: The study used the diagnostic categories of autism and Asperger syndrome as they were understood in the 1990s. Diagnostic criteria and terminology have changed since then, which could make exact replication and comparison with modern samples more difficult.

Ethnocentrism

The study was conducted in the UK and participants were recruited from the general population of Cambridge, an autism magazine, and clinical sources. This means that the sample was drawn from a particular Western cultural context.

• Limited cultural representation: The study did not directly compare participants from different cultures. It therefore provides limited evidence about whether the same pattern of theory of mind performance would be found in non-Western populations.
• Cultural differences in social cues: The interpretation of facial expressions and mental states may be influenced by cultural norms. People from different cultures may therefore have different expectations about how emotions and intentions are communicated through the eyes.
• Underlying cognitive ability: The ability to understand that other people have mental states is likely to be an important part of social interaction across cultures. This means that the basic concept of theory of mind is not necessarily culture-specific, even though the particular social cues used to infer mental states may vary between cultures.
• Use of facial information: The Eyes Test examines a form of social perception that is relevant to human interaction generally. Therefore, although the sample was culturally limited, it is not necessarily the case that the underlying ability being investigated is specific to Western culture.

Evaluation summary table

Relation to the psychology of individual differences more broadly

Baron-Cohen et al (1997) contributes to the psychology of individual differences by demonstrating that people can differ in their ability to understand other people’s mental states. The study provides evidence that adults with autism or Asperger syndrome may have particular difficulties with theory of mind even when they have normal intelligence and high levels of functioning in other areas. This suggests that individual differences in psychological functioning cannot necessarily be explained by general intelligence alone.

The table below summarises how this study compares with other core studies in this area and evaluates its contribution and current relevance: