Persian Version of Eyes Test: Psychometric Properties & Theory of Mind Measure, Lecture notes of Sculpture

Download Persian Version of Eyes Test: Psychometric Properties & Theory of Mind Measure and more Lecture notes Sculpture in PDF only on Docsity! ORIGINAL PAPER The ‘‘Reading the Mind in the Eyes’’ Test: Investigation of Psychometric Properties and Test–Retest Reliability of the Persian Version Behzad S. Khorashad1 • Simon Baron-Cohen2 • Ghasem M. Roshan3 • Mojtaba Kazemian4 • Ladan Khazai5 • Zahra Aghili6 • Ali Talaei7 • Mozhgan Afkhamizadeh8 Published online: 2 April 2015  Springer Science+Business Media New York 2015 Abstract The psychometric properties of the Persian ‘‘Reading the Mind in the Eyes’’ test were investigated, so were the predictions from the Empathizing–Systemizing theory of psychological sex differences. Adults aged 16–69 years old (N = 545, female = 51.7 %) completed the test online. The analysis of items showed them to be generally acceptable. Test–retest reliability, as measured by Intra-class correlation coefficient, was 0.735 with a 95 % CI of (0.514, 0.855). The percentage of agreement for each item in the test–retest was satisfactory and the mean difference between test–retest scores was -0.159 (SD = 3.42). However, the internal consistency of Persian version, calculated by Cronbach’s alpha (0.371), was poor. Females scored significantly higher than males but aca- demic degree and field of study had no significant effect. Keywords Theory of mind  Reading the Mind in the Eyes test  Reliability  Persian  Empathy  Sex differences Introduction Emotion recognition has been defined as ‘‘the ability to read subtle cues indicating the emotional state of another person’’ (Baron-Cohen and Wheelwright 2004). These cues can be visual and verbal, both revealing an internal emo- tional state. Emotion recognition is part of a broader set of cognitive capabilities for analyzing the clues on beliefs and desires of conspecifics which is called social cognition. The ability to use such social cues is sometimes referred to as employing a ‘‘theory of mind’’ (henceforth ToM); the & Simon Baron-Cohen sb205@cam.ac.uk Behzad S. Khorashad b_sorouri_k@yahoo.com Ghasem M. Roshan roshan.g2006@yahoo.com Mojtaba Kazemian kazemian2m@yahoo.com Ladan Khazai ladan_khazaee@yahoo.com Zahra Aghili zahra.aghili@gmail.com 1 Evolution and Human Behaviour Group, Psychiatry and Behavioral Sciences Research Centre, Mashhad University of Medical Sciences, No. 17, Toufigh 9 Lane, Shahid Sadeghi Blv., 91858-84714 Mashhad, Iran 2 Autism Research Centre, Department of Psychiatry, Cambridge University, Douglas House, 18B Trumpington Rd., Cambridge CB2 8AH, UK 3 Evolution and Human Behaviour Group, Psychiatry and Behavioral Sciences Research Centre, Mashhad University of Medical Sciences, No. 101, 4th Fl., Daneshju 19st, 9188977361 Mashhad, Iran 4 Evolution and Human Behaviour Group, Psychiatry and Behavioral Sciences Research Centre, Mashhad University of Medical Sciences, No. 9, Motahari Shomali 41st, 9193974567 Mashhad, Iran 5 Psychiatry and Behavioral Sciences, University of Miami, Unit 1221, 100 Lincoln Rd., Miamibeach, FL 33139, USA 6 Psychiatry and Behavioral Sciences Research Centre, Mashhad University of Medical Sciences, Unit 4, No. 134, Asrar 1st, Daneshgah Av., Mashhad, Iran 7 Psychiatry and Behavioral Sciences Research Centre, Mashhad University of Medical Sciences, Bu-Ali Sq., Amel Blvd., Mashhad, Iran 8 Department of Endocrinology, Imam Reza Hospital, Mashhad University of Medical Sciences, Imam Reza Sq., Mashhad, Iran 123 J Autism Dev Disord (2015) 45:2651–2666 DOI 10.1007/s10803-015-2427-4 capability to attribute mental states to the self and to others in order to predict behaviour (Premack and Woodruff 1978; Apperly et al. 2006; Samson 2009). It is a core aspect of cognition in the social evolution of human and non-human primates and is key for success in social interaction. Fluency in this ability confers social and vocational advantages (Begeer et al. 2010; Bender et al. 2012; Peterson et al. 2007; Woolley et al. 2010). On the other side, deficits in emotion recognition may lead to serious interpersonal and social difficulties. Several studies have found that conditions such as autism, schizophrenia and anorexia nervosa, involve a difficulty in recognizing another’s state of mind (Baron- Cohen 1995; Lind et al. 2014). Therefore, there have been numerous studies in recent years attempting to design an instrument for measuring atypical, as well as typical, var- iations in emotion recognition and social cognition. The ‘‘Reading the Mind in the Eyes test’’ (hereafter the Eyes test) is one such measure, which due to its simplicity, has been widely used. The first version of Eyes test was an effort towards developing an adult test for detecting subtle individual differences in the ability of ‘mind reading’ (Baron-Cohen et al. 1997). The task involved looking at the pictures of strangers’ faces and choosing which of two words that best describes what the person in the picture is feeling or thinking. In order to improve the test’s psycho- metric properties, it was revised by the same team (Baron- Cohen et al. 2001a). The revised version has 36 pictures of the eye region of males and females, and the participants have to choose one of the four words that best describes the mental state of the person in picture. Each correctly an- swered item is awarded one point and each incorrectly answered or unanswered item is scored as zero. The final score is sum of all acquired points. The English version of revised Eyes test has been translated into many languages, including Turkish (Yil- dirim et al. 2011), Spanish (Fernandez-Abascal et al. 2013), Japanese (Kunihira et al. 2006), German (Pfaltz et al. 2013), Swedish (Hallerback et al. 2009), French (Prevost et al. 2014) and Italian (Vellante et al. 2013); all available for free at www.autismresearchcentre.com/arc_ tests. It has also been translated into Persian and used in some studies (Nejati et al. 2012). The psychometric aspects of the Persian version, however, have never been tested in an independent systematic study. Therefore, one aim of the present study is to evaluate the psychometric properties of the Persian Eyes test. Eyes Test and Empathizing–Systemizing Theory The Eyes test, originally developed as a sensitive measure of subtle cognitive deficits in individuals with autism spectrum conditions (ASC), correlates with scores on the self-report Empathy Quotient (EQ) in the general population (Baron-Cohen 2010). Empathizing is defined as the drive to identify another’s mental states and to respond to these with an appropriate emotion (Baron-Cohen et al. 2003). Empathy encompasses two components: cognitive empathy, which is the capacity to recognize what someone else believes or feels (the same ToM); and affective em- pathy, which is the capacity to experience an appropriate emotion in response to someone else’s thoughts and feel- ings. EQ is negatively correlated to the Systemizing Quo- tient (SQ). Systemizing is defined as the drive to analyze and construct rule-based systems (Baron-Cohen et al. 2003; Wheelwright et al. 2006; Wheelwright et al. 2006). It in- volves identifying the ‘‘input-operation-output’’ rules that govern and predict how a system behaves. Systemizing is an algorithmic process: understanding systems in a relatively finite and closed fashion (Lai et al. 2012). It is, therefore, reasonable to expect Eyes test scores to be high whenever the SQ is low and EQ is high and vice versa. There are some studies supporting this notion. In a study of the psychometrics of the Italian version of the Eyes test, Vellante et al. (2013) found that performance of typical participants on the EQ is correlated with their score on the Eyes test. Baron-Cohen et al. (2003) reported that people with high-functioning autism or Asperger Syndrome, whose ability in theory of mind is impaired, score higher on the SQ compared to general population. Auyeung et al. (2009) reported that children with ASC scored significantly lower on the EQ, and significantly higher on the SQ, compared to typical samples. Grove et al. (2014) found that weak performance on the Eyes test is related to EQ in children with ASC and their relatives in comparison to typical individuals. Chapman et al. (2006), in an attempt to reveal the origins of differences in empathy, studied pre- natal testosterone and EQ and Eyes test scores and found a negative correlation between fetal androgens and both measures. These studies, in harmony with the Empathiz- ing-Systemizing Theory (E–S theory) (Baron-Cohen 2010), suggest that people can be classified based on their EQ and SQ along two dimensions of Empathizing and Systemizing. According to E–S theory, five types of cognitive style are defined: Type E (EQ[ SQ), Type S (SQ[EQ), Type B (SQ = EQ), Extreme Type S (SQ  EQ) and Extreme Type E (EQ  SQ). This E–S discrepancy is reflected in the academic interests of students. Several studies have found that students in different academic fields have dif- ferent EQ and SQ profiles (Billington et al. 2007). Indi- viduals interested in fields of sciences that are about lawful systems such as mathematics, engineering, computer sci- ences and the natural sciences are more likely to have a profile of Type S or extreme Type S, whereas individuals in the humanities and social sciences show the opposite pat- tern (Billington et al. 2007; Focquaert et al. 2007; Manson and Winterbottom 2011). Considering the correlation 2652 J Autism Dev Disord (2015) 45:2651–2666 123 in some questions may not be due to the unreliability of item, but, as noted (Vellante et al. 2013), it may be due to its difficulty and its ‘‘lower margin for differentiation’’. Similarly, items that are chosen by the majority of par- ticipants are not necessarily valid either: they may be easy enough to be answered correctly by most participants. To address this problem, we computed the difficulty and dis- crimination coefficient of each item in order to test if it yields the necessary degree of reliability and validity (Linda and Algina 2006). We also assumed that a reliable and valid item would be able to appropriately distinct between those with a higher score and those with a lower score on Eyes test. Based on Kelley (Kelley 1939), we defined the higher score and lower score groups as the upper 27 % and the lower 27 % of participants, ordered according to their final score on Eyes test. These two groups were compared to each other based on how they answered each item. Difficulty Coefficient The difficulty of an item is measured by the proportion of the persons who answer a test item correctly. The higher this proportion is the lower would be the item’s difficulty. The very easy and the very difficult items lead to whether all examinees selecting the correct answer or selecting the answer by absolute chance. In neither case, the test can discriminate various abilities of facial affect recognition among participants. To calculate the difficulty of an item the number of participants who answered it correctly is divided by the total number of participants who answered it. This proportion is indicated by the letter p, which indicates the difficulty of the item (Linda and Algina 2006). It is calculated by the following formula: Pi ¼ Ai=Ni  100 ð1Þ where Pi = difficulty index of item i; Ai = number of correct answers in upper 27 % and lower 27 % groups of participants; Ni = The sum of number of those in upper 27 % and lower 27 % groups. Difficulty levels are classified in the following way: very difficult (p\30 %); moderately difficult (31 %\p\ 50 %); medium difficulty (51 %\p\70 %); moderately easy (71 %\p\90 %); and very easy (p[90 %). The ideal distribution of items, based on their difficulty, would be as follows: 5 % very difficult, 20 % moderately difficult, 50 % medium difficult, 20 % moderately easy and 5 % very easy. The mean Di of all items should be around 50–60 % (Zainudin et al. 2012). Discrimination Coefficient The purpose of many tests is to provide information about individual differences. Therefore, the items of a valid and reliable test must be able to appropriately differentiate between participants who are relatively strong from those who are relatively weak. In many cases, such as Eyes test, there is no gold standard to compare the test with, and there is only the total score of the test itself. The goal, in such situations, is to discover items for which high-scoring participants have a high probability of answering them correctly and low-grading participants have a low prob- ability of answering them correctly. These items would be able to discriminate those participants who know the ma- terial from those who do not (Linda and Algina 2006). The Index of Discrimination is used here as a strategy to in- vestigate the validity of items in measuring various abilities in emotion recognition. It is calculated through the following formula: Di ¼ ðPu  PlÞ  100 ð2Þ where Di: index of discrimination of item i; Pu: the pro- portion of those in upper 27 % group who correctly scored item i (chose the target word); Pl: the proportion of those in lower 27 % group who correctly scored item i. Based on a simulation (Linda and Algina 2006), Ebel and Frisbie offered the following guidelines to interpret the D values: D[ 39 = Excellent; 39[D[30 = Good; 29[D[20 = Mediocre; 19[D[0 = Poor; -1[D = Worst. We used the same guideline in this study. Results Socio-Demographic Data The final sample included N = 545 participants (fe- males = 51.7 %). The mean age of the sample was 25.8 years (SD = 6.17, Min = 16, Max = 69), with no significant sex differences in age (males: 25.3 ± 6.2; fe- males: 25.2 ± 6.05; U = 33,545, p = 0.425). Of all par- ticipants, N = 503 answered the question that asked for their field of study, among whom 29.4 % (N = 160) had studied engineering, 19.4 % (N = 106) sciences, 26.8 % (N = 146) humanities and 16.7 % (N = 91) of them had studied med- icine. The distribution of academic degree among our par- ticipants was as follows: Diploma 7.3 % (N = 40), Bachelor 47.9 (N = 261), Master 23.9 (N = 130), Doctor of Medi- cine 16.7 % (N = 91) and Ph.D 4.2 % (N = 23). There was no gender differences in having an academic degree (Di- ploma: male = 8.3 %, female = 6.0 %, p = 0.30; Bache- lor: male = 48 %, female = 47 %, p = 0.725; Masters: male = 24.4 %, female = 23.0 %, p = 0.650; Ph.D: male = 4.5 %, females = 4 %, p = 0.701; Doctor of Medicine: males = 14.4 %, females = 18 %, p = 0.251). However, significant sex differences were detected in field of study. A significantly higher proportion of females had J Autism Dev Disord (2015) 45:2651–2666 2655 123 studied humanities (33.3 % in comparison to males = 19.3 %, p\ 0.0001), and a significantly lower percentage of them had studied engineering (20 % in comparison to males = 39.4 %, p\ 0.0001). There was no significant sex differences in those studied medicine (males = 14.8 %, fe- males = 18.4, p = 0.254) or sciences (males = 17.4 %, females = 21.2 %, p = 0.264). Distribution of Responses The mean score of all participants was 22.76 (SD = 3.41, min = 9, max = 31). Table 1 indicates the responses of all items of the Eyes test and the per- centage of participants who chose them. This had been used as an index of item difficulty by most studies (Baron-Cohen et al. 2001a, b) in which items were considered to be of satisfactory difficulty if at least 50 % of participants selected the target word and of no more than 25 % of participants selected one of the foils. In 6 items, the first condition was not met (7, 10, 25, 27, 28, 29) and in 15 items the second condition was not met (3, 5, 7, 10, 13, 17, 23, 24, 25, 27, 28, 29, 33, 35, 36). This was similar to other studies of the Eyes test in other countries. Table 1 Distribution of responses to Eyes test in percentages Item Test Item Test 1 2 3 4 1 2 3 4 Q1 53.8 23.1 20.4 2.8 Q19 4.2 16.1 9.9 69.7 Q2 19.6 72.1 5.3 2.9 Q20 3.7 91.4 4.4 0.6 Q3 2.2 3.3 53.4 41.1 Q21 21.3 56.1 21.8 0.8 Q4 9.4 63.7 17.4 9.5 Q22 80.7 1.7 7 10.6 Q5 32.5 8.8 58.2 0.6 Q23 2.9 2.2 51.9 42.9 Q6 10.8 78.3 8.4 2.4 Q24 64.6 29.4 1.7 4.4 Q7 3.1 38.7 18.7 39.4 Q25 7.3 32.5 20.9 39.3 Q8 72.7 20.2 1.7 5.5 Q26 4.4 2.8 78 14.9 Q9 20.4 6.8 11.7 61.1 Q27 0.7 47.5 30.1 21.7 Q10 43.9 26.8 13.8 15.6 Q28 47 7 11.4 34.7 Q11 9.9 22 52.1 16 Q29 12.5 30.6 18.7 38.2 Q12 16.1 0.9 74.3 8.6 Q30 6.2 86.8 5.7 1.3 Q13 64 4.2 2.9 28.8 Q31 4.4 58.3 10.6 26.6 Q14 3.3 1.8 1.3 93.6 Q32 79.1 5.1 9.7 6.1 Q15 80.7 2.8 10.5 6.1 Q33 2.6 26.1 4.8 66.6 Q16 13.4 59.4 3.9 23.3 Q34 5 17.1 65.1 12.8 Q17 57.8 27.9 1.7 12.7 Q35 28.8 53 5 13.2 Q18 82.4 6.2 5.1 6.2 Q36 1.5 4 68.6 25.9 Correctly identified descriptors are marked in bold The highlighted items have failed to fulfill either first condition or second condition 2656 J Autism Dev Disord (2015) 45:2651–2666 123 Validity Comparison of Items Among Different Studies To assess the validity of the new Persian version of Eyes test, we compared it with other non-English versions of Eyes test. Table 2 shows a comparison of the percentage of participants choosing the target word in different studies. The italicized items failed to fulfill the first condition of item difficulty (Baron-Cohen et al. 2001a, b). Since we lacked the detailed data of other studies, we could not in- vestigate whether the differences among various studies are statistically significant or not. However, the mere obser- vation of percentage of target-choosing participants may be Table 2 A comparison of the percentage of participants who chose the target in different version of the eyes test Item Persian Italian (Vellante et al. 2013) German (Pfaltz et al. 2013) French (Prevost et al. 2014) Spanish (Fernandez- Abascal et al. 2013) English (Baron- Cohen et al. 2001b) Q1 53.8 69.5 65.8 84 66.9 85.2 Q2 72.1 56 49.4 70 63.8 78.7 Q3 53.4 65 85.1 93 75.2 86.1 Q4 63.7 65.5 74.2 54 81.1 73 Q5 58.2 84 64.5 71 92.5 77 Q6 78.3 69 72.9 80 75.2 80.3 Q7 18.7 42.5 49 33 64.6 68 Q8 72.7 67 77.4 68 88 67.2 Q9 61.1 90.5 78.6 81 81.9 77 Q10 43.9 63.5 76 60 71 73 Q11 52.1 71 74.3 57 74.1 68 Q12 74.3 71.5 87.7 75 80.8 87.7 Q13 64 63.5 55.8 34 80.8 69.7 Q14 93.6 80 73.4 85 88.9 80.3 Q15 80.7 83 84.5 84 86.9 69.7 Q16 59.4 76 76 79 85.8 77 Q17 57.8 54 50.3 48 54.3 65.6 Q18 82.4 92 81.9 86 96.4 58.2 Q19 69.7 52.5 57.4 43 39 69.7 Q20 91.4 73.5 81.3 92 89.4 88.5 Q21 56.1 73 39.4 86 75.2 73.8 Q22 80.7 90.5 72.9 87 70.8 79.5 Q23 51.9 62.5 61.7 37 65.5 77.9 Q24 64.6 58.5 57.4 84 73.5 73.8 Q25 39.3 67 42.6 76 70.5 71.3 Q26 78 76.5 78.1 68 75.2 65.6 Q27 47.5 63 67.1 49 64.1 65.6 Q28 47 70 63.9 73 83.6 66.4 Q29 38.2 66.5 69 66 81.1 77.9 Q30 86.8 91 86.5 80 88.6 91 Q31 58.3 66.5 32.3 69 57.1 51.6 Q32 79.1 73 66.5 80 78 50 Q33 66.6 54 77.4 60 61.3 58.2 Q34 65.1 71 71 63 72.1 77 Q35 53 36.5 60.6 47 77.7 65.6 Q36 68.6 76.5 85.8 71 87.5 76.2 Mean 22.76 24.85 24.5 24.8 27.18 26.2 The italicized items have failed to fulfil the first condition in Persian Eyes test J Autism Dev Disord (2015) 45:2651–2666 2657 123 the Pearson correlation between the scores of Eyes test the first and second time was r = 0.60, p\ 0.01) and Spanish studies (in which the Spearman Rho correlation between test and retest for each was positive and significant except item 18), the test and retest total scores of Persian Eyes test, as well as each item, were positively and sig- nificantly correlated (Vellante et al. 2013; Fernandez- Abascal et al. 2013; Hallerback et al. 2009). Taking the Bland–Altman approach, we could demonstrate that most responses on all items were consistent from test to retest, mean differences were 0 and most differences fell with 95 % CI. According to the plot, any individual result can vary in the range of ±6 out of 36 when the test is re- peated. As Hallerback et al. (2009) emphasized, this means that any obtained test score must be regarded as an approximation. Percentage of agreement between the participant’s per- formance in test and retest was also measured for each item (Table 5). Following the procedure used by others (Hutchins et al. 2008), the minimum accepted value of agreement was considered to be 70 %. Based on this cri- terion, 11 items (items 9, 10, 11, 16, 21, 24, 25, 27, 28, 29, 31 and 35) failed to achieve the acceptable reliability, among these item 29 was the only item in which less than 50 % of participants selected the same answer twice; item 24 was answered in the same way by 54 % and all the other ten items had been answered by more than 60 % and less than 70 % of participants. This is in concordance with results of German (10 items), Turkish (19 items) and Swedish (8 items out of 28 items) studies. Items 16, 21, 29 and 31 failed to fulfill the 70 % criterion in both the Persian and German studies, as well as items 9, 10, 21, 24, 27, 28 and 31 in both Persian and Turkish version. Since the Swedish study had used the child version of Eyes test, it was not possible to identify the shared failing items. Considering all of these together it seems that the re- liability of Persian Eyes test is as acceptable as other studies. In addition, as Table 8 shows, the exclusion of items that failed to fulfill the 70 % criterion (Hutchins et al. 2008), did not change either the internal consistency of the Eyes test or any of the results. The validity of Eyes test, as different authors have previously mentioned, is difficult to investigate since there is no gold standard to be used (Hallerback et al. 2009; Prevost et al. 2014). In order to solve this problem, we developed two strategies to validate the Persian version: First, to compare the results of Persian Eyes test as a whole, as well as its each item, to similar studies investigating the psychometrics of Eyes test. Second, to analyze each item of Persian Eyes test according to discrimination and difficulty indices. Comparing the Persian Eyes test to other versions showed that our results were weaker compared to other studies. As previously mentioned, in our sample, the target word in 6 items (Items 7, 10, 25, 27, 28 and 29) was selected by less than 50 % (the first condition). In the French version (Prevost et al. 2014) seven items (Items 7, 13, 17, 19, 23, 27 and 35), in the German version (Pfaltz et al. 2013) five items (items 2, 7, 21, 25 and 31), in the Turkish version (Yildirim et al. 2011) four items (Items 19 and 21, plus items 25 and 35 that had been excluded during the pilot study), in the Italian version (Vellante et al. 2013) two items (items 7 and 35) and in the Spanish version (Fernandez-Abascal et al. 2013) just one item (item 19) had the same condition. Consistently, the number of items in other versions of Eyes test in which some foils were selected by more than 25 % of participants (the second condition), were similar to ours. In the French version 10 items (items 4, 7, 11, 13, 17, 19, 23, 27, 34 and 35), in the German version 6 items (items 2, 7, 13, 17, 19, 24 and 35), in the Turkish version 4 items (items 17, 19, 21 and 23), in the Italian version 7 items (items 3, 4, 7, 17, 19, 24 and 35) and in the Spanish version 4 items (items 17, 19, 31 and 33) failed to meet that criteria, as did 13 items of the Persian Eyes test (items 3, 5, 7, 10, 13, 17, 23, 24, 25, 27, 28, 29 and 33). It should be noted, however, that in com- parison to other studies our sample size was clearly larger and, also, demographically more varied (see above). Thus the wider range of scores and the lower mean score, alongside the less acceptable items (in terms of fulfilling the first and second conditions) in our results may reflect the more diverse strata of general population participated in our study. In addition, there are similarities in the list of prob- lematic items among different studies. For example item 7 failed to meet the first condition in the French (33 %), German (49 %), Italian (42.5 %) and Persian (18.7 %)1 studies, and item 17, failed to meet the second condition in the Persian (27.9 % of participants chose the foil ‘‘affec- tionate’’), German (40 % chose the foil ‘‘affectionate’’), French (39 % chose the foil ‘‘affectionate’’), Italian (32 % chose the foil ‘‘affectionate’’) and Spanish (27.3 % chose the foil ‘‘affectionate’’) studies. Seemingly, participants tend to mistake ‘‘affectionate’’ to ‘‘doubtful’’ regardless of their language or nationality. More interestingly, Persian participants had chosen the foils 2 (friendly) and 4 (dispirited) in the item 7 considerably more frequent than foil 1 (apologetic); this is similar to how Italians (25.2 % for foil 2 and 10 % for foil 4), French (39 % for foil 2 and 16 % for foil 4), Germans (22.2 % for foil 2 and 15 % for foil 4) and Turkish (12 % for foil 2 and 23.9 % for foil 4) had answered item 7. Examples such as item 7 show that though there are differences in percentage of participants 1 The number in the parenthesis is percentage of participants choosing the target in item 7 in each study. 2660 J Autism Dev Disord (2015) 45:2651–2666 123 answering to Eyes test, the overall pattern of answering is highly similar between various studies. Other items hap- pened to be problematic in some and not others; such as item 19 which is problematic in Italian, French and Spanish studies but not in Persian and German studies. Instead of translational errors or cultural differences, such similarities point to inherent features of items which make them dif- ficult for various participants. An analytic comparison of each item between different versions of Eyes test in future studies seems to be very crucial to illuminate the nature of these similarities and differences. Another aspect of comparing Persian Eyes test to other versions is the possible differences among subgroups of participants. First, men scored significantly worse than females, echoing the frequently replicated advantage of females over males (Hallerback et al. 2009; Baron-Cohen et al. 1997; Vellante et al. 2013; Yildirim et al. 2011). This can be taken as supporting the validity of the Persian Eyes test. This is specifically notable given that male and female participants of our study had no significant difference in age and academic degree, suggesting that sex-dependent performance in Eyes test is not influenced by age and educational level—which may be cautiously interpreted as overall intelligence. In addition, there are items on which males and females performed significantly different (Table 6). Females per- formed better on items anticipating, decisive, friendly and defiant (items 13, 18, 20 and 23) whereas males outper- formed females in recognizing playful, insisting and in- terested faces (items 1, 4 and 25). Previously there have been some reports on sex differences in detecting various states of mind. In one study females were faster in detecting happy facial expressions while men spent a considerably longer time viewing the nose and mouth (Vassallo et al. 2009). Other studies propose that these sex differences are based on the type of emotion. It is suggested that females are better at recognizing facial expressions of fear and sadness (Mandal and Palchoudhury 1985; Nowicki and Hartigan 1988), and males are superior at detecting anger (Mandal and Palchoudhury 1985; Rotter and Rotter 1988; Wagner 1986; Sawada et al. 2014). Evolutionary hy- potheses have also been provided to explain the origins of such differences (Hampson et al. 2006). Although it seems that females do score ser than men when it comes to emotion recognition, whether these differences are seen for all emotions remains to be answered (Kret and De 2012). This is crucially important to determine that the widely replicated female superiority in Eyes test is an indication of an overall female psychological advantage over males or just a reflection of capability of detecting some emotions. If the latter is the case, the advantage of females in the Eyes test would be due to the higher frequency of items in which females perform better, not a general ability in reading minds. Since prenatal testosterone and sex chromosomes are physiological differences between men and women, an interesting idea for future research is to investigate the Eyes test results among patients with disorders of sex de- velopment (DSD), conditions where androgen or chromo- somal profiles are atypical. This may be helpful for revealing the possible origin of sex differences in perfor- mance in Eyes test. Further support for the validity of the Persian Eyes test comes from the comparison of subgroups with various educational levels. As noted earlier there was no significant difference between those with diploma, bachelor, Master, Ph.D and M.D degrees. This shows that higher education does not affect ToM, as measured by the Persian Eyes test. This is in opposition to Yildirim’s study that showed those with university education score significantly higher than those with primary and high school education. However, it should be considered that in our study the lowest educa- tional degree was Diploma. One can speculate that although different levels of higher education do not affect the performance in Eyes test, but there is a minimum level below which the performance of participants deteriorates significantly. Considering the E–S theory, our study has led to some notable findings. First of all, among those studied Hu- manities and engineering there were significant sex differ- ences, with females scoring higher in the first and males in the second. This is consistent with E-S theory that predicts those with the S type brain, which is more common among men, tend to ‘analyze and construct rule-based systems’ (such as is involved in engineering) rather than to identify another’s mental states and to respond to these with an ap- propriate emotion (which is involved in the humanities) which is an aspect of those with E type brain, and more common among women. However, opposite to what theory predicts, we found no significant difference among various fields of study in performance on the Eyes test. Although the medicine subgroup had scored significantly better than other groups, post hoc comparison between groups showed that there was no overall main effect and the significant advan- tage of those studied medicine is nominal. The mean score among other three subgroups (humanities, engineering and sciences) were very close. There are several explanations for this inconsistency. First, one might assume that there is no correlation between performance in Eyes test and EQ. Re- viewing the relevant literature shows that this is unlikely (Manson and Winterbottom 2011; Focquaert et al. 2007; Billington et al. 2007). Secondly, it might be due to an un- controlled third factor varying between the subgroups, such as overall and verbal intelligence. This is important consid- ering the studies investigating the correlation between verbal J Autism Dev Disord (2015) 45:2651–2666 2661 123 intelligence and performance in Eyes test (Peterson and Miller 2012). When comparing sex differences we could cautiously infer that there is no significant difference be- tween our male and female participants regarding their educational level. But it is not the same considering the subgroups in various academic fields. This is because most Iranian students have to choose their academic field of study according to their rank in the National University Ex- amination which is taken at the end of the last year of high school and is very competitive. This causes most of the students to choose their academic field of study based on their available options, and not their preferences and interests. And those with highest grades mostly, if not ex- clusively, regard ‘medicine’ as their first option. Thus, one might claim that various academic subgroups in Iran may not be similar based on their general intelligence. In addition, it has been noted that performance on the Eyes test is correlated with social skills and affective as well as cognitive empathy (Grove et al. 2014). Thirdly, it has been suggested that medical students have higher abilities on the Eyes test. However, this is not consistent in the literature either (for a review see (Pedersen 2010)). Although most of these studies used measures other than the Eyes test to evaluate the ToM but some used the Eyes test (Dehning et al. 2013). Table 3 Percentage of participants in the upper and lower 27th percentile choosing the target in each item item Lower 27 % Upper 27 % Item Lower 27 % Upper 27 % Q1** 41.5 66.1 Q19** 55.9 78.8 Q2* 60.2 87.3 Q20** 81.4 97.5 Q3** 30.5 67.8 Q21** 35.6 72 Q4** 48.3 77.1 Q22** 61.9 92.4 Q5** 44.9 71.2 Q23* 45.8 63.6 Q6** 67.8 89.9 Q24* 52.5 69.5 Q7** 9.3 30.5 Q25** 27.1 49.2 Q8** 50.8 86.4 Q26** 50 91.5 Q9 55.9 62.7 Q27** 41.5 56.8 Q10** 37.3 58.5 Q28** 31.4 66.9 Q11** 39.8 62.7 Q29* 28.8 44.9 Q12** 62.7 86.4 Q30** 73.3 94.9 Q13* 55.1 74.6 Q31** 46.6 72 Q14** 81.4 99.2 Q32** 68.6 89 Q15** 59.3 89.8 Q33** 54.2 85.6 Q16** 49.2 75.4 Q34** 47.5 76.3 Q17** 41.5 72 Q35** 38.1 72 Q18** 62.7 94.9 Q36** 49.2 86.4 * p\ 0.01; ** p\ 0.0001 Table 4 Item analysis of Eyes Test: Difficulty index (Pi)a and index of Discrimination (Di)b for each item Item Di Pi Item Di Pi Item Di Pi Item Di Pi Q1 24.58 53.81 Q12 23.73 74.58 Q23 17.80 54.66 Q34 28.81 61.86 Q2 27.12 73.73 Q13 19.49 64.83 Q24 16.95 61.02 Q35 33.90 55.08 Q3 37.29 49.15 Q14 17.80 90.25 Q25 22.03 38.14 Q36 37.29 67.80 Q4 28.81 62.71 Q15 30.51 74.58 Q26 41.53 70.76 Q5 26.27 58.05 Q16 26.27 62.29 Q27 15.25 49.15 Q6 22.03 78.81 Q17 30.51 56.78 Q28 35.59 49.15 Q7 21.19 19.92 Q18 32.20 78.81 Q29 16.10 36.86 Q8 35.59 68.64 Q19 22.88 67.37 Q30 21.19 84.32 Q9 6.78 59.32 Q20 16.10 89.41 Q31 25.42 59.32 Q10 21.19 47.88 Q21 36.44 53.81 Q32 20.34 78.81 Q11 22.88 51.27 Q22 30.51 77.12 Q33 31.36 69.92 aVery difficult (p\ 30 %); Moderately difficult (31 %\ p\ 50 %); Medium difficulty (51 %\ p\ 70 %); Moderately easy (71 %\ p\ 90 %); Very easy (p[ 90 %) b D[ 39: Excellent; 39[D[ 30: Good; 29[D[ 20: Mediocre; 19[D[ 0: Poor; -1[D: Worst 2662 J Autism Dev Disord (2015) 45:2651–2666 123 Acknowledgments This study was part of a medical doctorate dissertation which was fully supported by Mashhad University of Medical Sciences. SBC was supported by the Autism Research Trust during the period of this work. This article is thoroughly the original work of authors without any financial interest or benefit. Conflict of interest It is strictly ascertained that there would be no matter of concern as a subject for conflict of interests. References Adams, R. B., Jr., Rule, N. O., Franklin, R. G., Jr., Wang, E., Stevenson, M. T., Yoshikawa, S., et al. (2010). Cross-cultural reading the mind in the eyes: An fMRI investigation. Journal of Cognitive Neuroscience, 22(1), 97–108. doi:10.1162/jocn.2009. 21187. Ahmed, F. S., & Stephen, M. L. (2011). Executive function mechanisms of theory of mind. Journal of Autism and Devel- opmental Disorders, 41(5), 667–678. doi:10.1007/s10803-010- 1087-7. Apperly, I. A., Riggs, K. J., Simpson, A., Chiavarino, C., & Samson, D. (2006). Psychological Science, 17, 841. Auyeung, B., Baron-Cohen, S., Ashwin, E., Knickmeyer, R., Taylor, K., Hackett, G., et al. (2009). Fetal testosterone predicts sexually differentiated childhood behavior in girls and in boys. Psycho- logical Science, 20(2), 144–148. doi:10.1111/j.1467-9280.2009. 02279.x. Baron-Cohen, S. (1995). Mindblindness: An essay on autism and theory of mind. Cambridge, MA: MIT Press. Baron-Cohen, S. (2010). Empathizing, systemizing, and the extreme male brain theory of autism. Progress in Brain Research, 186, 167–175. Baron-Cohen, S., Jolliffe, T., Mortimore, C., & Robertson, M. (1997). Another advanced test of theory of mind: Evidence from very high functioning adults with autism or asperger syndrome. Journal of Child Psychology and Psychiatry, 38, 813–822. Baron-Cohen, S., Richler, J., Bisarya, D., Gurunathan, N., & Wheelwright, S. (2003). The systemizing quotient: An investi- gation of adults with Asperger syndrome or high-functioning autism, and normal sex differences. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 358, 361–374. Baron-Cohen, S., & Wheelwright, S. (2004). The empathy quotient: An investigation of adults with Asperger syndrome or high functioning autism, and normal sex differences. Journal of Autism and Developmental Disorders, 34, 163–175. Baron-Cohen, S., Wheelwright, S., Hill, J., Raste, Y., & Plumb, I. (2001a). The ‘‘Reading the Mind in the Eyes’’ Test revised version: A study with normal adults, and adults with Asperger syndrome or high-functioning autism. Journal of Child Psy- chology and Psychiatry, 42, 241–251. Baron-Cohen, S., Wheelwright, S., Skinner, R., Martin, J., & Clubley, E. (2001b). Journal of Autism and Developmental Disorders, 31, 5. Begeer, S., Malle, B. F., Nieuwland, M. S., & Keysar, B. (2010). Using Theory of Mind to represent and take part in social interactions: Comparing individuals with high-functioning aut- ism and typically developing controls. European Journal of Developmental Psychology, 7, 104–122. Bender, L. C., Linnau, K. F., Meier, E. N., Anzai, Y., & Gunn, M. L. (2012). Interrater agreement in the evaluation of discrepant imaging findings with the Radpeer system. AJR: American Journal of Roentgenology, 199, 1320–1327. Billington, J., Baron-Cohen, S., & Wheelwright, S. (2007). Cognitive style predicts entry into physical sciences and humanities: Questionnaire and performance tests of empathy and systemiz- ing. Learning and Individual Differences, 17, 260–268. Bland, J. M., & Altman, D. G. (1986). Statistical methods for assessing agreement between two methods of clinical measure- ment. Lancet, 1, 307–310. Chapman, E., Baron-Cohen, S., Auyeung, B., Knickmeyer, R. F., Taylor, K. F., & Hackett, G. (2006). Fetal testosterone and empathy: Evidence from the empathy quotient (EQ) and the ‘‘reading the mind in the eyes’’ test. Social Neuroscience, 1(2), 135–148. Dehning, S., Gasperi, S., Tesfaye, M., Girma, E., Meyer, S., Krahl, W., et al. (2013). Empathy without borders? Cross-cultural heart and mind-reading in first-year medical students. Ethiopian Journal of Health Sciences, 23, 113–122. Domes, G., Heinrichs, M., Michel, A., Berger, C., & Herpertz, S. C. (2007). Oxytocin improves ‘‘mind-reading’’ in humans. Biolo- gical Psychiatry, 61(6), 731–733. Fernandez-Abascal, E. G., Cabello, R., Fernandez-Berrocal, P., & Baron-Cohen, S. (2013). Test-retest reliability of the ‘Reading the Mind in the Eyes’ test: A one-year follow-up study. Molecular Autism, 4, 33. Focquaert, F., Steven, Megan S., Wolford, George L., Colden, Albina, Miller, L. S., & Gazzaniga, Michael S. (2007). Emphatizing and systemizing cognitive traits in the sciences and humanities. Personality and Individual Differences, 43, 619–625. Grove, R., Baillie, A., Allison, C., Baron-Cohen, S., & Hoekstra, R. A. (2014). The latent structure of cognitive and emotional empathy in individuals with autism, first-degree relatives and typical individuals. Molecular Autism, 5(1), 42. doi:10.1186/ 2040-2392-5-42. Guillemin, F., Bombardier, C., & Beaton, D. (1993). Cross-cultural adaptation of health-related quality of life measures: Literature review and proposed guidelines. Journal of Clinical Epidemi- ology, 46(12), 1417–1432. Hallerback, M. U., Lugnegard, T., Hjarthag, F., & Gillberg, C. (2009). The Reading the Mind in the Eyes test: Test–retest reliability of a Swedish version. Cognitive Neuropsychiatry, 14, 127–143. Table 8 The results after excluding problematic items according to each criterion Criterion Descriptive statistics Cronbach’s alpha Sex Degree Field of study Excluded items Mean Range U p value p value p value Eyes test 22.76 9–31 0.371 33,583 \0.05 0.076 0.235 – Agreement 16.6 4–18 0.437 31,946 \0.05, 0.01 0.375 0.458 9, 10, 11, 16, 21, 24, 25, 27, 28, 29, 31, 35 [25 % 17.422 8–25 0.303 36,527 0.761 0.295 0.76 3, 5, 13, 17, 23, 24, 33, 35, 36 \50 % 20.46 8–28 0.416 32,025 \0.05, 0.01 0.164 0.408 7, 10, 25, 27, 28, 29 27th percentile 22.18 9–30 0.385 33,591 \0.05 0.113 0.241 9 J Autism Dev Disord (2015) 45:2651–2666 2665 123