Effectiveness of Open-Book, Closed-Book, and Cheat Sheet Exams: A Comparative Study, Exams of Psychology

Download Effectiveness of Open-Book, Closed-Book, and Cheat Sheet Exams: A Comparative Study and more Exams Psychology in PDF only on Docsity! Psychology Research, ISSN 2159-5542 August 2012, Vol. 2, No. 8, 469-478 Cheat Sheet or Open-Book? A Comparison of the Effects of Exam Types on Performance, Retention, and Anxiety∗ Afshin Gharib, William Phillips, Noelle Mathew Dominican University of California, California, USA The differences between open-book, cheat sheet, and closed-book exams were examined in two different types of psychology courses. A total of 297 students enrolled in eight sections of Introductory Psychology and 99 students enrolled in four sections of Statistics participated in this study. Exam types were counterbalanced across sections of the same course. Students were given either open-book, cheat sheet, or closed-book exams, took a surprise quiz two weeks after the exams to measure retention of course material, completed a preference questionnaire, and took a pre-test measure of test anxiety on open-book and cheat sheet tests. While students did slightly better on open-book exam than on closed-book exams, they also much preferred open-book and cheat sheet exams over closed-book exams, and had lower levels of anxiety when taking open-book exams compared to cheat sheet exams. Based on these results, open-book exams may be a superior style of examination for a variety of psychology courses. Keywords: test anxiety, test style, student assessment, academic performance, open-book tests, closed-book tests This study developed from an ongoing debate in our department, one which is probably happening in many academic departments. The question is which type of exam is best. All teachers want their students to learn and retain the material they cover and at the same time enjoy the process of learning. One way to check how well students have learned is by giving exams. There are many teachers who consider the traditional closed-book and notes time-limited exam the best measure of student learning, while others argue for alternatives, such as cheat sheet or open-book tests. While a closed-book exam is probably the most common type of exam in psychology courses, alternative exam types, such as cheat sheets (or crib notes) where the students can prepare notes to use on the exam ahead of time, and open-book exams where the student can use all their material during the test, are increasingly accepted. Students themselves prefer alternative forms of testing, for example, Williams and Wong (2009) found that students prefer open-book exams compared to closed-book tests. Students believed that being able to use notes, cheat sheets and texts during an exam would improve grades and decrease their anxiety. Are these expectations justified? The purpose of this project was to examine the effectiveness of various exam types: open-book, closed-book, and cheat sheet. There is a relatively long history of trial-and-error efforts to improve testing procedures (Kalish, 1958; J. M. Stalnaker & R. C. Stalnaker, 1934). Surprisingly, there is little consensus among investigators as to which testing style may be the ∗Acknowledgements: The authors wish to thank Lara Corkerey, Stephanie Lemp, and Lindsay Sander for their assistance. This research was supported in part by an Academic Excellence Grant from Dominican University of California. Afshin Gharib, Ph.D., Department of Psychology, Dominican University of California. William Phillips, Ph.D., Department of Psychology, Dominican University of California. Noelle Mathew, Department of Psychology, Dominican University of California. DAVID PUBLISHING D EFFECTS OF EXAM TYPES ON PERFORMANCE, RETENTION, AND ANXIETY 470 most effective. J. M. Stalnaker and R. C. Stalnaker (1934) suggested that while the closed-book exam is the norm in higher education, in fact, an open-book test may be preferable since it puts less emphasis on memorization of facts and encourages a deeper engagement with the course material on the part of the students. Similarly, Eilertsen and Valdermo (2000) argued that an open-book exam encourages greater engagement and improves understanding of course material. Feller (1994) further suggested that the open-book exam is superior to closed-book exams as it is more realistic—similar to problem-solving situations students are likely to face outside of academia. And of course, students prefer open-book to closed-book exams and find them less stressful (Philips, 2006). As Kalish (1958) pointed out, however, open-book test may encourage less studying. The evidence for the benefits of open-book test is mixed. Kalish (1958) found that grades do not improve on open-book tests compared to closed-book exams, but other researchers have found higher grades on open-book tests, although the improvement is more modest than what might be expected (Krarup, Naeraa, & Olsen, 1974). Students who are struggling with the course material are particularly likely to see an improvement in their grades on an open-book exam (Philips, 2006). Those higher grades do not necessarily mean better learning of the material. For example, Agarwal, Karpicke, Kang, Roediger, and McDermott (2008) found that while students performed better on an open-book test, there was no difference on a later retention test between open- and closed- book tests. In observing students taking open-book examinations, Boniface (1985) found that weaker students relied more on their notes and texts than academically stronger students, and students that used their notes most ended up doing more poorly on the exam, suggesting that open-book exams may actually lead to lower scores. Moore and Jensen (2007) found that while exam scores in an Introductory Biology class were higher on open-book tests, the open-book format also encouraged lower attendance and less effort (fewer extra credit assignments and lower attendance at review sessions), and lower retention of information over the long term. Heijne-Penninga, Kuks, Hofman, and Cohen-Schotanus (2008) found that in medical students, a closed-book exam actually encourages more in-depth engagement with the course material. An alternative to either closed-book or open-book exams is a cheat sheet exam, where the student is given the opportunity to prepare a sheet of notes ahead of time to use on the exam. Erbe (2007) had argued that the preparation of cheat sheets improve and deepen learning by helping students organize their study time. Some authors found an improvement in performance when students use cheat sheets (Skidmore & Aagaard, 2004), while others found no effect (Dickson & Miller, 2005; Hindman, 1980; Whitley, 1996). Funk and Dickson (2011), and Dickson and Bauer (2008) argued that cheat sheets are more of a crutch than an aid in preparing for an exam and found that students who prepared cheat sheets for an exam showed no improvement in learning on a pre-test without their cheat sheets. One important benefit of alternatives to closed-book exams is that they may lower test anxiety. Test anxiety—worrying about the outcome of the test and experiencing negative emotions during the test—is associated with poor performance on exams (Morris, Davis, & Hutchings, 1981; Tryon, 1980). Students prefer open-book and cheat sheet exams because they feel they would be less anxious during those types of exams and therefore perform better (Zoller & Ben Chaim, 1988). Theophilides and his colleagues (Theophilides & Koutselini, 2000; Theophilides & Dionysiou, 1996) found that students reported that they were more optimistic and less anxious when taking open-book exams. On the other hand, Dickson and Miller (2005) found that students did not feel that being able to use cheat sheets decreased their test anxiety, although in their study, students reported their impressions at the end of the term, not during the exam itself. EFFECTS OF EXAM TYPES ON PERFORMANCE, RETENTION, AND ANXIETY 473 = 7.74, p < 0.05). Paired-samples t-tests revealed that closed-book exams resulted in lower scores than either open-book exams (t(68) = 4.00, p < 0.05) or cheat sheet exams (t(67) = 2.43, p < 0.05), and cheat sheet exams resulted in lower scores than open-book exams (t(286) = 3.45, p < 0.05). In Statistics, the difference in exam scores between open-book and cheat sheet exams was not significant (t(98) = 1.98, p > 0.05) . Table 1 Exam and Quiz Scores Class Open-book Cheat sheet Closed-book Introductory Psychology Exam scores 80.11 (10.92) N = 287 77.85 (12.14) N = 287 72.52 (11.81) N = 69 Quiz scores 6.41 (1.94) N = 78 6.44 (1.88) N = 79 6.38 (1.94) N = 34 Statistics Exam scores 80.73 (13.05) N = 99 77.48 (14.89) N = 99 Quiz scores 6.18 (1.75) N = 169 6.31 (1.83) N = 174 Notes. Values are means (and standard deviations in parentheses) for the different types of exams calculated across all sections. For the exam scores, N reflects the number of students in all the sections of the two classes that took the exams; For the quiz scores, N reflects the total number of quizzes returned. 5 0 7 5 1 0 0 5 0 7 5 1 0 0 O p e n - B o o k C he at S he et 5 0 7 5 1 0 0 5 0 7 5 1 0 0 C h e a t S h e e t C lo se d- B oo k 5 0 7 5 1 0 0 5 0 7 5 1 0 0 O p e n - B o o k C lo se d- B oo k ( A ) ( B ) ( C ) Figure 1. Correlations between different exam types across all sections and both courses. (A) Scores on open-book and cheat sheet exams (N = 396); (B) Scores on closed-book and cheat sheet exams (N = 67); (C) Scores on closed-book and open-book exams (N = 69). EFFECTS OF EXAM TYPES ON PERFORMANCE, RETENTION, AND ANXIETY 474 Figure 1 shows the overall correlations between the three exam types. Pearson correlations reveal that scores on different types of exams are positively correlated in both classes. In Introductory Psychology, there is a positive correlation between scores on open-book and cheat sheet exams (r(285) = 0.54, p < 0 .05), open-book and closed-book (r(67) = 0.66, p < 0.05), and cheat sheet and closed-book scores (r(66) = 0.67, p < 0.05). In Statistics, there is a positive correlation scores on open-book and cheat sheet exams (r(97) = 0.32, p < 0.05). Students who do well on one type of exam do well on the other two types. Cheat Sheets The quality of the cheat sheets prepared by students was rated on a 10-point scale on the organization of the cheat sheet (whether there are headings and subheadings, highlights, etc.) and amount of detail (number of words) by a rater blind to the students’ exam scores. A Pearson correlation found that the quality of the cheat sheet was not related to scores on cheat sheet exams in Introductory Psychology, but there was a positive correlation between cheat sheet quality and cheat sheet exam score in Statistics (r(43) = 0.34, p < 0.05). Retention Quizzes As it can be seen in Table 1, there were no differences in retention quiz performance across exam types in either course: In Introductory Psychology, an ANOVA comparing quizzes after open-book, cheat sheet, and closed-book exams finds F(2, 105) = 0.03, p > 0.05; In Statistics, an independent samples t-test comparing quizzes given after open-book and cheat sheet exams finds t(184) = 0.86, p > 0.05. Test Anxiety A Pearson correlation shows that the test anxiety measured right before the exam was negatively correlated with scores on the exam (r(528) = -0.15, p < 0.05). Figure 2 shows the differences in anxiety level between open-book and cheat sheet for the two classes. A paired-samples t-test finds higher anxiety scores during cheat sheet exam compared to open-book exam in both Introductory Psychology (t(220) = 1.98, p < 0.05) and Statistics (t(46) = 4.87, p < 0.05). Introductory Statistics 15 20 25 30 Open-Book Cheat Sheet 22.92 23.98 20.62 26.86 t(220)=1.98 t(46)=4.87 Class A nx ie ty S co re Figure 2. Mean and standard error of anxiety scores in Introductory and Statistics courses. Values are the total scores for the Worry-Emotionality scale. Paired-samples t-tests compared the two sets of scores. Differences are significant at p < 0.05. Student Preferences Table 2 summarizes the results of the student preferences questionnaire. As it can be seen, the students in EFFECTS OF EXAM TYPES ON PERFORMANCE, RETENTION, AND ANXIETY 475 both classes predicted that they would do better on open-book or cheat sheet exams compared to closed-book exams (χ2 (3) = 238.67, p < 0.05). Comparisons of the actual exam scores of those who predicted they would do best on open-book exams (N = 192) and those who predicted they would perform best on cheat sheet exam (N = 136) were made. Those who predicted that they would do best on open-book exam scored an average of 79.3% (SD = 15.0) on open-book exam and 77.0% on cheat sheet exam (SD = 16.3), those who predicted that they would do best on cheat sheet exam scored an average of 80.2% (SD = 12.0) on open-book exam and 76.6% (SD = 13.3) on cheat sheet exam. An independent samples t-test finds no difference between the two groups of students with scores of the two types of exams. Table 2 Student Preference Questionnaire Introductory Psychology (%) Statistics (%) Overall (%) Predicted best Closed-book 6.3 2.4 5.4 Open-book 53.0 47.1 51.6 Cheat sheet 33.8 48.2 37.1 No preference 7.0 2.4 5.9 Study time Closed-book 76.0 65.9 73.7 Open-book 5.6 4.7 5.4 Cheat sheet 6.6 9.4 7.3 No preference 11.8 20.0 13.7 Preferred exam Closed-book 7.3 2.4 6.2 Open-book 51.7 41.7 49.5 Cheat sheet 35.0 54.8 39.5 No preference 5.9 1.2 4.9 Notes. Values are percent of participants reporting a preference for that exam type. Students were asked before the first exam to: (1) predict which exam type they would do best on; (2) which exam type they think they would study most for; and (3) which exam type they preferred. N = 372 total (287 participants in Introductory Psychology and 85 in Statistics). Students believed that they would study most for closed-book exams (χ2 (3) = 475.38, p < 0.05). Students were asked on the first exam to report how much time they had actually spent on studying for that test. For Introductory Psychology, the average study time reported for open-book exams is 3.97 hours (SD = 4.42, N = 160), for cheat sheet exams study time is 4.04 hours (SD = 3.65, N = 103), and for closed-book exams reported study time is 3.32 hours (SD = 3.32, N = 23). Independent-samples t-tests find no differences between exam types in reported study time. In Statistics, students reported spending an average of 2.60 hours (SD = 1.66, N = 36) on the open-book exam and 3.77 hours (SD = 6.86, N = 49) on the cheat sheet test. This difference is also not significant. Finally, students were asked which exam of the three exam types they would prefer to take. As it can be seen in Table 2, the students in both classes preferred open-book and cheat sheet exams over closed-book test (χ2 (3) = 231.71, p < 0.05). To see if preferences are related to exam performance, the exam scores of those who preferred open-book exam (N = 183) were compared to those who preferred cheat sheets (N = 144). Those who stated a preference for open-book exam scored an average of 78.6% (SD = 14.4) on open-book test, and 76.0% (SD = 16.3) on cheat sheet tests. Those who preferred cheat sheet exams scored an average of 80.7% (SD = 13.4) on open-book tests and 77.5% (SD = 13.2) on cheat sheet exams. An independent samples t-test reveals