Peer Feedback and Descriptive Writing: A Study on Expository Literacy Skills, Study notes of History of Science and Technology

Download Peer Feedback and Descriptive Writing: A Study on Expository Literacy Skills and more Study notes History of Science and Technology in PDF only on Docsity! Geraldine Mongillo, Hilary Wilder An Examination of At-Risk College Freshmen’s Expository Literacy Skills Using Interactive Online Writing Activities This qualitative study focused on at-risk college freshmen’s ability to read and write expository text using game-like, online expository writing activities. These activities required participants to write descriptions of a target object so that peers could guess what the object was, after which they were given the results of those guesses as feedback on their writing. Findings suggested that these online writing activities can improve at-risk students’ expository literacy skills. Specifically, findings emphasized the importance of the writers’ description of salient features and word choice when writing for an online (distant) reader, and the importance of knowing the audience they are addressing. The participants’ feedback concerning how and why they made a particular choice may provide a lens to view how at-risk readers utilize and apply reading strategies. Further research is recommended to determine if the reading behaviors of at-risk students can be better understood by examining the rationale described in the feedback. There is an ongoing need to create educational settings that address the cognitive, social, and emotional needs of at-risk college freshmen who have limited literacy skills. The need for developmental reading instruction is widespread and affects most higher education institutions, disproportionately affecting Expository Literacy Skills 27 28 Journal of College Reading and Learning, 42(2), Spring 2012 historically underserved populations including low-income, first-gener- ation, and minority students (Green, 2006; McDonough, 1997). Further, there is a need for systematic research that provides reliable results about interventions in order to inform and guide educators’ practice. According to the National Center for Education Statistics (2011), 36.2% of students entering American colleges and universities require at least one developmental course and 48% did not meet the reading benchmark for college readiness (ACT, 2010). The cost of remediation at the col- lege level is estimated at a staggering $3.7 billion a year (Wise, 2009). As the research suggests, there is an urgent need to address this issue where a significant number of high school students graduate without the necessary skills to succeed at college-level work. Specifically, attention needs to be paid to the development of higher order reading, writing, and critical thinking skills required to tackle today’s ever increasing literacy demands. This study focused on the ability to read and write expository text, the genre that is generally understood to constitute the majority of college- based reading. Specific skills and strategies are required in order to be an effective and proficient reader and writer within this genre, includ- ing knowledge of text structure (Flood, Lapp, & Farnan, 1986; Gunning, 2010). There are several types of expository text structures that serve to organize the material, and the most common are often identified as time sequence, description, explanation/process, comparison-contrast, problem-solution, and cause and effect (Gunning, 2010). This study focused on the ability to comprehend and compose descriptive writ- ing. Descriptive writing is defined as the author’s ability to list charac- teristics, features, and examples to describe the salient features of the selected topic (Blasingame & Bushman, 2005; Tompkins, 2005). This skill is particularly important in today’s society where the increased use of online technology heightens the need to understand expository writing because in an online environment the selection of words to cre- ate a visual representation is essential to the reader’s understanding. Therefore, writers must use appropriate descriptive language to get their message across, using words that would allow the reader to interpret the author’s message. This study builds on previous work done by the authors in which a game-like, online expository writing activity was used to help preservice teachers develop descriptive writing skills (Wilder & Mongillo, 2007). Specifically, the activity required participants to write descriptions of a target object in such a way that peers could guess what it is, and then to use the results of those guesses as feedback on their writing. This repeated feedback was shown to help participants hone their descriptive Expository Literacy Skills 31 ing and writing interact, or function reciprocally” (Leki, 2001, p. 184). Spivey (1990) discussed the reciprocal nature of reading and writing stating that “readers and writers…construct meaning from texts through reading and for text through writing” (p. 1). In addition, the construc- tion of meaning is influenced by one’s sociocultural background and individual understandings. When students have an opportunity, such as provided in this study, they may collaborate and co-construct mean- ing (Spivey, 1997) by reading the descriptions of others and attempting to make sense of these descriptions as seen and described through another’s perspective. The following questions guided our investigation: a) how do repeated expository literacy tasks influence participants’ ability to compose and comprehend descriptive text?; b) what impact does peer feedback have on the quality of descriptive writing and the participants’ ability to make accurate guesses?; and c) how effective is online discussion board tech- nology in facilitating the writing and reading activities? Method Participants Participants in this study were freshmen at a mid-sized state university who were enrolled in a developmental reading course. The majority (69%) of the participants were from ethnic and/or racial minority and/ or non-English-speaking families, from nearby disadvantaged urban school districts. All were in their late teens to early twenties. Of the 28 students enrolled in two sections of the course during the spring 2009 semester, four students were selected as a purposive sample (Plano Clark & Creswell, 2008) based on the following criteria: • participated in all of the assigned activities that included writing descriptions, reading peer descriptions and making guesses based on those descriptions, completing a modified cloze activity, and writing a descriptive paragraph on a selected topic • increased their ACCUPLACER® reading comprehension test score by at least 4 points (assessed just prior to and at the end of the semester), as compared to the class average gain of 3.9 points (on a 35-point score), allowing us to look at correlations between the study activities and above-average reading comprehension growth • earned a passing grade in the course • demonstrated (reported) that they read peer feedback Data were collected from a total of 28 students over five weeks from both sections of Basic Reading Instruction (BRI) (The university has since changed the name of this course to College Reading). 32 Journal of College Reading and Learning, 42(2), Spring 2012 The instructors had taught the course for at least two semesters and reported that one major reason they agreed to participate in this study was that they felt the inclusion of these writing activities would supple- ment an area of need because their primary focus was teaching reading strategies. Procedures The writing and reading activities were presented as an online com- ponent that was integrated into the class curriculum and completed by the students as part of their homework assignments, which they were able to submit any time over a two-day period, from any internet-con- nected computer. In weeks 1 and 2, participants wrote descriptions of a target picture for their peers to identify from a picture set of six similar objects. Separate online discussion forums were set up each week in the university’s Blackboard® learning management system. After the descriptions were submitted, they were then posted anonymously for peers to read. Each participant read peer descriptions of three different target objects and posted their guesses for the “right one” based on these descriptions. In addition, participants were instructed by their instruc- tor, as well as by email from the researchers, and in announcements posted on Blackboard®, to provide feedback explaining why they made their choice. All guesses were then reposted (again anonymously) in a response thread to each writer’s original description posting. In weeks 3 and 4, participants completed an expository cloze activity where they provided two descriptive words to complete the paragraph. Again, this was done using weekly Blackboard® discussion forums. In this case, peers were given three similar paragraphs to choose from and asked to select the paragraph into which the two words best fit. Again, they were instructed to provide feedback. In the fifth and final week, participants were instructed to write a paragraph about a contemporary news topic (e.g., steroid abuse; hybrid cars) where they were instructed to describe the characteristics without actually mentioning or naming the topic. Peers then guessed the topic based on these descriptions and explained how they arrived at that guess. Data Sources and Analysis The participants’ online descriptions and responses were printed and used as a primary data source. Additional data were collected and compared to assure reliability of the emerging patterns through triangulation (Leedy & Ormrod, 2010). Data sources included the transcripts (via Blackboard®) of participant descriptions and responses Expository Literacy Skills 33 (guesses and feedback). The ACCUPLACER® entrance and exit reading comprehension examination scores were used to select participants for a purposive sample as described above. This test consisted of 35 multiple- choice questions focused on reading comprehension and sentence structure and is used by the university as a precourse placement test and as a postcourse measure for passing the course. The participants also completed an attitudinal survey (see Appen- dix A), which was distributed in class at the end of the data collection period. The survey consisted of 12 questions related to the participants’ perspectives of the efficacy of the interactive activity. A writing rubric (see Appendix B) that assessed descriptive word choice, feature set, conciseness, and text structure was used to assess the participants’ writing samples submitted as part of the activities in weeks 1, 2, and 5. Descriptive word choice was defined as “only uses audience appro- priate vocabulary.” Feature set was defined as the ability to “describe all salient features based on prototypical feature set for item with no comparisons to other examples (i.e., could not mention relative posi- tion of the target item as a clue).” Conciseness was considered the ability to write without using extraneous details; and text structure was defined as “coherently structured showing ability to accurately list characteristics, features, and details about objects through appropriate use of semantics and syntax.” Another data source was the research- ers’ reflective field notes (Creswell, 2008) that recorded insights and emerging themes. Qualitative procedures were used in this inquiry, employing content analysis to systematically identify patterns and themes. Data analysis was ongoing and inductive, employing the constant-comparative method (Corbin & Strauss, 2008) that was used to recursively review the data and identify patterns between and among the data. Participants’ written descriptions and responses were coded and recursively compared to the survey and field notes to identify emerging patterns across sources. Continuous comparisons helped revise codes into discrete patterns that determined themes. The participants’ writing samples were evaluated by three raters and the scores were averaged. The mean for each category was determined based on the assessment of the participants’ responses where 0 to 3 points were assigned, 3 being the highest (exceeds expectation), 2 (meets expectation), 1 (below expectation), and 0 the lowest (does not meet expectation). Three raters independently evaluated the data, and in cases of disagreement consensus was reached through discussion. 36 Journal of College Reading and Learning, 42(2), Spring 2012 Analyses of the written descriptions as well as the feedback to these descriptions revealed insights to the participants’ selection process of target items. Serena was the most successful: her description prompted four of four correct guesses. Based on her description of a Greek urn in a set of similar items, it appears that she captured the salient features that helped peers guess correctly. She wrote, “it has three people on it one looks like they have wings.” Two of the six urns had three figures, but Serena distinguished the one she was describing from the others by indicating it had wings. Three of the four respondents mentioned the clue “have wings” in their feedback, a feature that only the target urn had. Jed was the least (50%) successful in writing the descriptions for the week 2 trial. Jed’s description of the targeted African mask (see Figure 1) did not make mention of the salient features that would have made his mask easier to identify. He wrote, “This mask simply has little squigley [sic] facial hair and has a cut-off mouth feature.” The use of the word “squigley” was descriptive but not particularly helpful or accurate and the same could be said for “cut-off mouth feature.” His inappropriate word choice also caused ambiguity; for example, one respondent said, “I am guessing #4 that is the only one with facial hair…I think not very sure.” Fifty-seven percent of Andy’s peers guessed the correct target item during week 2. The picture he described was similar to other pictures of moths presented. Andy wrote this description: “The base color of the wings are light beige. The corners of the wings are brown but with spots. The wings look like they have light grey veins.” What Andy called “spots” could have been more accurately described as lacework or webbing, which would have distinguished it from the one that actu- ally had brown spots. Further, all of the pictured moths had light gray veins; however, Andy did not select the salient features that would help his peers distinguish the target item from other moths pictured in the group. Although four of seven did guess correctly, the other three guess- ers expressed confusion. A peer responded, “I think it’s butterfly 2 or 5 because they both have brown spots and grey veins.” At least four of the pictured moths had wings that were light beige and five had brown corners; therefore, his description did not help peers identify the target item. Another peer replied, “the butterfly that I chose would be butterfly #6 because the description was very brief.” Their feedback demonstrated that the participants were using the descriptive clue words to identify the target item, but they also told the author why his description was not helpful by pointing out that more than one item had both brown spots and gray veins and that the description did not provide enough detail. The following example also demonstrates how the feedback directly commented on how the writer’s word choice helped peers make a more Expository Literacy Skills 37 informed guess. Jessica’s description of an African mask yielded two of three correct guesses. She wrote, “the mask is really big, with big eyes, small mouth, and has a little triangle shape on the top of the head.” Jes- sica used adjectives (big, small, little, top) to describe each feature of the mask, which may have helped peers make the distinction between masks. Andy’s feedback confirms this: “It’s mask one. I say this because its [sic] big compared to the others and it has a triangle thing on tope [sic] of its head.” Analysis of the participants’ writing samples by category. Analy- sis showed that each participant demonstrated improvement in each of the four categories. The aggregated scores for the four participants’ written descriptions of the target picture items (Tasks 1 and 2) and the final concept paragraph (Task 5) are provided in Table 2. Table 2 Participants’ Mean Scores on Writing Tasks over Time Serena Jed Jessica Andy Task1 Task2 Task5 Task1 Task2 Task5 Task1 Task2 Task5 Task1 Task2 Task5 Word Choice 1 2 2.3 2 1 2.6 1.5 2 2.6 2 2.3 2.3 Feature Set 0 0 1.6 2 1 2 1 2 2 2 1.3 2 Con ciseness 2 2 2.6 2 2 2.6 2.3 2.6 2.3 2.3 2 2.3 Text Structure 1 1.6 2 2 2 2.3 2 2.3 2 2 2 2.3 Overall, the four participants improved in each category over time and met expectations as measured by the rubric in all but the feature set for Task 2. The participants struggled with describing salient fea- tures (feature set) in some of the more complex objects; however, they demonstrated improvement by the last task. The greatest improvement in scores was in the word choice category, which was defined on the rater’s rubric as “uses audience-appropriate vocabulary.” Specifically, it is the ability to use understandable and meaningful designations for features and aspects when specific expert terminology is not known or vocabulary is not universal or generally shared (Wilder & Mongillo, 2007). This suggested that the participants 38 Journal of College Reading and Learning, 42(2), Spring 2012 knew their audience and used descriptive words that were meaningful and accessible to their audience. This result also suggests that partici- pants became more adept at selecting descriptive words over time. For example, Jessica scored a 1.5 (below expectation) on word choice for Task 1, for which she was asked to describe the target flower in a way that would allow her peers to choose the correct one from a group of six. Jessica wrote, “the flower has three fully bloomed flowers that are at the top with many pedals [sic] below.” Although she used accessible and appropriate descriptive language to describe the target flower (bloomed) and provided visual cues (top, below), she mistakenly used the word “pedals” for “petals” in describing the flower. In Task 2 Jessica was asked to write a description for Mask 1. She included words that described size (really big, small mouth, little) shape (triangle), and position (top). These word choices were appropri- ate and accessible, and she was graded a 2.0. Although this was better than her description for Task 1, it still lacked specificity and defining characteristics. Only two of the three peers guessed correctly based on her description. For Task 5, Jessica was instructed to write a descriptive paragraph (without actually mentioning or naming the concept) about steroid usage. Jessica wrote the following: The use of this drug has become very popular with many people playing sports. Many famous baseball players have been caught using this illegal drug to enhance their body. Many boys think it will make them stronger and better than all other players. One baseball player that was recently caught using it was Alex Rodriguez. The combined score of the three evaluators for this paragraph was 2.2, and Jessica received a 2.6 for word choice. Illegal steroid use was a cur- rent topic in the news during this study, and Jessica used her background knowledge to make connections for her readers. She used meaningful designations to describe steroid use such as “popular” and “illegal” to refer to the drug in question. She also used the word “enhance,” which is associated with steroid use and distinguishes steroids from other popu- lar drugs. She also associated steroid use with athletic ability by using words such as “baseball players,” “stronger,” and “better” and capitalized on one well-publicized case by referring to the baseball player involved by name. In doing this, Jessica used descriptive language that her audi- ence would understand as defined in the rubric as “only uses audience appropriate vocabulary.” In another example, Serena’s overall score for Task 5 was 2.1 and her score on word choice was 2.3, an increase in score from those she received for Task 1, where she scored 1.0. In Task 1, she described one Expository Literacy Skills 41 Jessica’s responses for the cloze tasks (3 and 4) demonstrated that she employed metacognitive strategies to monitor reading comprehension by frequently using the phrases “make sense” and “sound right.” The basic principle of reading (Snow, Burns, & Griffin, 1998) is to construct meaning from the text and Jessica did this by asking herself, “Does this make sense and does this sound right?” Effect of Peer Feedback on Literacy Skills Although the participants’ feedback was often limited, it was sufficient for the writer to ascertain if the description helped the participants cor- rectly identify the target item. Their remarks provided some insight into how the readers made decisions. For example, Andy’s response to Serena’s description of the target apple stated, “I think its [sic] apple 5 because it seems smaller than the other ones and it doesn’t seem to be leaning to any side.” In his remarks, it becomes clear that he read the descriptive clues (smaller, leaning), but he did not find the information sufficient to make a distinction (he guessed the wrong item). The attitudinal survey contained questions directly related to reading and writing feedback. Participants (27 of 28) reported that they read their peer feedback and found it useful (26% agree; 26% strongly agree; 33% neutral). When asked if making guesses and giving feedback was hard to do, most of the participants disagreed (52% disagree; 7% strongly disagree; 30% neutral), and most agreed (37% agree; 18% strongly agree; 41% neutral) that getting feedback was an important motivation to com- pose better descriptions that would enable their peers to make correct guesses. Although more than half of the participants found feedback to be an integral part of these activities, it was also clear that a large por- tion of the survey respondents were neutral or not in agreement. For example, Jessica reported, “It [feedback] helped me know if I needed to put more information or not,” and another student said, “I learned that I needed to have more detail in descriptions.” However, another participant stated, “I read some of them but they didn’t really effect [sic] my writing.” One of the reasons for negative or neutral responses may be that the participants did not understand the potential of using peer feedback as a method to improve their writing or to help writers to connect to their intended audience (Blair, 2003). The majority of the participants agreed (44%) or strongly agreed (22%) that guessing the target item/concept was fun. It was obvious to the classroom instructors and the researchers that most of the participants enjoyed the activities, and because it was presented as a game with prizes (McDonald’s gift certificates earned for participation), their overall attitude was cooperative. Findings show that reading 42 Journal of College Reading and Learning, 42(2), Spring 2012 peer feedback may not have played a significant role in improving participants’ writing; however (and somewhat surprisingly), reading the descriptions written by peers and having to make guesses based on them may have helped participants become more cognizant (as readers) of the importance of appropriate word choice and description of salient features in facilitating correct interpretations of the descriptions. Signs of this latter effect are seen in both the transcript content analysis and the attitudinal survey. At the same time, responses on the attitudinal survey as well as researcher field notes indicated that participants did find the feedback beneficial and they enjoyed the activity and saw it as a fun way to learn. Effect of the Online Discussion Technology in Facilitating the Writing and Reading Activities Based on the responses on the survey, 85% of the respondents reported that the technology was not difficult to use. Using threaded discussion made it easy to identify peers’ feedback (posted as a response, indented below participants’ initial description). Comments such as “easy,” “very user friendly,” and “it was very simple to use” suggest that participants were comfortable using the technology; however, one participant reported that “Blackboard does not let me log in all the time” and another wrote, “It wasn’t hard, but it was hard for me to get on a computer sometimes.” Although it is easy to overgeneralize that this generation of “digital natives” (Prensky, 2001) has anywhere/anytime access to the technology, this may not always be the case for all students. Conclusion Biancarosa and Snow (2006) reported that nearly six million secondary students read below grade level. The majority of these students graduate high school, and those who enter college are unprepared to succeed, hence the extraordinary need for college developmental reading courses. This research explored online expository literacy tasks that required students to read and write descriptive text for a specific purpose. Par- ticipants were asked to compose descriptive text for the purpose of having peers guess an object or subject. Findings suggested that these online activities improved at-risk students’ expository literacy skills in the categories of description of salient features and word choice. When writing in an online environment, it is important that writers not only select appropriate words but also know their audience (Rijlaarsdam et al., 2009). Participants had the opportunity to experience “reader-text- writer transaction” (Kucer, 2009; Rosenblatt, 1938/1995) through their participation in the online activities. By reading peers’ responses, the Expository Literacy Skills 43 participants were exposed to diverse and varying viewpoints, which may have helped them to better understand their audience. Although most participants indicated that peer feedback was important, we found feedback was often limited to responses such as “I think it is number 5.” However, reading the descriptions of others appeared to influence some participants as evidenced in their subsequent writing. This finding is similar to Larson’s (2009) in that the online design allowed time for participants to read through peers’ responses and consider alternate viewpoints. Further, the time afforded by asynchronously reading online may have fostered the reading-writing connection (Spivey, 1990) where participants constructed meaning both from reading others’ descriptions and writing their own. This study also indicates that there is a need for teachers to explain the use and purpose of peer feedback if it is to be an effective learning strategy for students to gain a more accurate understanding of their intended audience (Blair, 2003). Clearly, a method that attracts this population of struggling read- ers and writers through an engaging activity is a valuable and much needed learning tool. At-risk readers often disengage when presented with expository text, yet we know that many of them are proficient users of technology, surfing the Internet for information when moti- vated. Using Blackboard® facilitated this game-like activity, and the participants reported it was easy to use. However, not all students have access to computers and Blackboard®, and as some participants reported, it is not always reliable. Further research that includes using a smartphone application may be more successful in ensuring greater participation, as this is a technology that is being adopted by more and more students. The online activities outlined in this study can be easily replicated by practitioners to improve expository literacy skills of their students. The activity that requires them to describe objects can be used for any subject or topic. It can be introduced as a game-like activity for extra credit to be completed as homework or as an additional assignment. Content related to the curriculum may be used in these activities to review subject area knowledge, such as those used in the cloze tasks in this study, that include social studies, health, and science. In addition, the cloze activity assesses reading comprehension. The use of technology provides an alternate method for struggling and disinterested students to practice reading and writing tasks. The game-like activities may engage students who often find reading and writing expository text both boring and difficult. The instructors reported that this was a useful intervention to target strategies that utilize the 46 Journal of College Reading and Learning, 42(2), Spring 2012 Vygotsky, L. S. (1978). Mind in society: The Development of higher psychological processes. Cambridge, MA: Harvard University Press. Wilder, H., & Mongillo, G. (2007). Improving expository writing skills of preservice teachers in an online environment. Contemporary Issues in Technology and Teacher Education, 7(1). Wise, B. (2009). Adolescent literacy: The cornerstone of student success. Journal of Adolescent & Adult Literacy, 52(5), 369-375. Yule, G. (1997). Referential communication tasks. Mahwah, NJ: Lawrence Erlbaum Associates. Geraldine Mongillo is an associate professor of literacy in the Department of Educational Leadership and Professional Studies at William Paterson University of New Jersey. She also co-coordinates the First Year Foundations reading program. Her research interests include developmental reading processes and adolescent literacy. Hilary Wilder is an associate professor of learning technologies at William Paterson University of New Jersey. Her research interests include the use of social media and new technologies to promote writing literacies and afford effective written communication skills. Expository Literacy Skills 47 Appendix A Student Attitudinal Survey Please answer the following 12 questions (front and back) indicating whether you agree or disagree with the statement by marking an “X” in the appropriate box. Feel free to add comments as well. 1. Guessing the target item/concept was fun. Strongly Agree Neutral Disagree Strongly Agree Disagree Comments: 2. I always read my peers’ guesses and feedback. If so, did the feedback guesses from others based on your descriptions help you write better. If not please explain why you didn’t read them. Strongly Agree Neutral Disagree Strongly Agree Disagree Comments: 3. It was hard to make guesses and give feedback on others’ descriptions. Strongly Agree Neutral Disagree Strongly Agree Disagree Comments: 48 Journal of College Reading and Learning, 42(2), Spring 2012 4. Reading others’ descriptions helped me write better. Strongly Agree Neutral Disagree Strongly Agree Disagree Comments: 5. The technology (Blackboard discussion board) was hard to use. Strongly Agree Neutral Disagree Strongly Agree Disagree Comments: 6. This would be a good way to teach students how to write descriptions. Strongly Agree Neutral Disagree Strongly Agree Disagree Comments: 7. Writing online descriptions was easy. Strongly Agree Neutral Disagree Strongly Agree Disagree Comments: