Assessment of Student Learning in IN250 U.S. Studies at Millikin University (2007-08) - Pr, Exams of Sociology of Religion

Download Assessment of Student Learning in IN250 U.S. Studies at Millikin University (2007-08) - Pr and more Exams Sociology of Religion in PDF only on Docsity! Student Learning in IN250 U.S. Studies Academic Year 2007-08 Assessment Millikin University IN250 Coordinator Larry Troy May 20, 2008 Executive Summary The 2007-08 academic year was again a time of transition. With a new Coordinator, the methodology used in assessing student learning was refined, and an interim report based on just the fall semester was developed to smooth the transition between the two plans. A copy of that report is appended to this document (Appendix C). US Studies faculty were also conscious of the changes to the MPSL that will begin in the 2008-2009 academic year. However, these changes will not have meaningful consequences for US Studies until the 2009-2010 academic year, because incoming first-year students will take US Studies classes in their sophomore year. Nevertheless, pilot IN251 classes will be offered during the 2008-2009 academic year to ensure that transfer students will be able to satisfy their MPSL requirements. Assessment of student learning in IN250 for the 2007-08 academic year demonstrates that learning goals are being satisfied and that IN250 is a strong component of the sequential program of the MPSL. Goals The IN250 learning goals are: By the end of IN250 U.S. Studies, students will be able to 1. Critically respond to texts by or about some of the diverse cultures and people living in the United States; 2. Use discipline-appropriate sources to inform their critical analyses of diverse individuals’ or cultures’ experiences and perspectives; and 3. Write at least one well-developed critical analysis about cultural, structural, or institutional issues specific to the discipline of the course. Snapshot During the 2007-08 academic year, 25 sections of IN 250 were offered by 17 faculty members (see Appendix A):  12 sections by eight History faculty—three full-time tenured or tenure-track faculty; five adjunct faculty without terminal degree  5 sections by three Political Science faculty—two full-time tenure-track; one adjunct faculty  2 sections by one Sociology faculty—full-time, tenured  1 section by one Communication faculty—full-time, tenured  2 by one Theatre faculty—full-time, tenure track  1 by one Exercise Science faculty—full-time, tenured  2 by two English faculty—one full-time tenure-track, one full-time without terminal degree The Learning Story In all sections of IN250, students learn about the experiences and perspectives of some of the diverse cultures and people living in the United States. Courses may focus on particular groups (African Americans, women) or particular issues (poverty and welfare, Vietnam, Civil Rights, Watergate), but all faculty are committed to inculcating in their students sensitivity and empathy, as well as sharper critical thinking and writing skills. In all sections, students use discipline-specific sources in their writing, which ranges from the formal essay to informal discussion threads in Blackboard. Competencies that promote professional success and understanding of others are emphasized (as per MPSL prepares 1 and 2). Assessment Methods and Data The IN250 Coordinator collected artifacts at the end of the Fall 2006 and Spring 2007 semesters. The IN250 self-study committees (Monroe and Troy in the Fall, and Monroe, McKenna, and Troy in the Spring) randomly selected 5 artifacts each from 6 (out of 8 possible) of the Fall IN250 classes and 3 artifacts each from 11 (out of 13 possible) of the Spring IN250 classes. The artifacts included term papers, essay exams, and assignments. The committees used a single rubric (see Appendix B) to assess all artifacts on a 1-5 scale (1 indicating Nominal, 5 indicating Excellent) on each of the three goals. They then calculated average scores for each learning goal, for all artifacts in total and for each specific class. The following table shows the range of possible scores and interpretation. Green Excellent 3.67-5.00 Yellow Adequate 2.34-3.66 Red Nominal 1.00-2.33 One difference between the Fall and Spring semesters concerns how the committees interpreted the learning goals. The Fall committee noted uncertainty about how the three learning goals should be interpreted. On the first learning goal, for example, the members of the committee were uncertain whether they were to assess how well the student critically responded to texts or how well the student showed an understanding of diverse cultures. Similarly, on the third goal, they were unclear about whether they were to assess a “well-developed critical analysis” or the coverage of “cultural, structural, or institutional issues.” Based on this confusion, the members of the Spring committee decided:  On the first learning goal, they would emphasize how well students showed understandings of “diverse cultures and people living in the United States.”  On the second learning goal, they would emphasize how well students used “discipline-appropriate sources.”  On the third learning goal, they would emphasize how well-written the students’ critical analyses were. Appendix A IN250 Courses, AY 2007-08 Date Course Title Cross-listed Department & Course number FT Tenured Or t-track, W/terminal Degree FT Contract Faculty w/ Terminal Degree FT Contract Faculty w/o Terminal Degree Adjunct Faculty Fall 2007 Criminal Law: 4,5,6th Amendments PO260 ● Civil Rights, Vietnam, & Watergate PO260 ● World Religions in America HI210 ● History of American Workers HI210 ● U.S. History To 1865 HI203 ● U.S. History To 1865 HI203 ● Drama & Representation of US TH331 ● Sport in America ● Spring 2008 Rhetoric-Violence in America CO260 ● Political Participation and Democratic Citizenship PO260 ● Group Influence in America (2 sections) PO260 ● Drama & Representation of US TH331 ● US History Since 1865 (2 sections) HI204 ● US History Since 1865 (2 sections) HI204 ● The Outsider in Film & Fiction EN220 ● Poverty and Welfare (2 sections) SO224 ● Women in America HI210 ● State and Local History HI210 ● Native American Myths and Folklore ● Christianity in the US HI210 ● World Religion in US HI210 ● Appendix B IN250 Assessment Rubric Millikin University IN250 U.S. Studies Student Learning Evaluation—REVISED June 26, 2007 By the end of IN250/U.S. Studies, students will be able to 1. Critically respond to texts by or about some of the diverse cultures and people living in the United States; 2. Use discipline-appropriate sources to inform their critical analyses of diverse individuals’ or cultures’ experiences and perspectives; and 3. Write at least one well-developed critical analysis about cultural, structural, or institutional issues specific to the discipline of the course. Item Evaluated: Variable—may be an essay, exam, or response paper Evaluation by: Self-Study Assessment Team Member Excellent (5 points) Adequate (3 points) Nominal (1 point) Critical Response An excellent response demonstrates consistent reflective thinking in using previous reading, research, and/or experience as a prompt not only to describe and explain, but also to develop awareness of the self in relation to other cultures. An adequate response demonstrates inconsistent reflective thinking in using previous reading, research, and/or experience as a prompt not only to describe and explain, but also to develop awareness of the self in relation to other cultures. A nominal response lacks a connection between uses of reading and writing and the development of self-awareness in relation to others. Use of Sources to Inform Critical Analysis An excellent analysis demonstrates the student’s consistent ability to use discipline- appropriate sources. The student interprets, rather than summarizes, relevant evidence. Student can formulate a well-positioned and well-supported argument or opinion by critically synthesizing multiple perspectives. An adequate analysis demonstrates the student’s inconsistent attempt to use discipline- appropriate sources. The student inconsistently attempts to interpret, rather than summarize, relevant evidence. Student attempts to formulate a well-positioned and well- supported argument or opinion by critically synthesizing multiple perspectives. A nominal analysis demonstrates no attempt to interpret evidence. Well-developed Critical Analysis An excellent analysis demonstrates the student’s consistent ability to compose a well- organized, properly- documented, and carefully-edited piece in a confident voice. An adequate analysis demonstrates the student’s inconsistent attempt to compose a well- organized, properly- documented, and carefully-edited piece in a confident voice. The student may make errors in documentation and/or grammar and mechanics. A nominal analysis demonstrates no attempt to edit or document. There are many errors in documentation and/or grammar and mechanics. Appendix C IN250 Interim Assessment Results Fall 2007 March 17, 2008 This report outlines the result of an assessment made for IN250 classes taught in Fall 2007. This interim report is important because it allows us to track changes more closely than simply on a yearly basis, and because the method of analysis has changed from previous years to more accurately reflect a holistic conception of assessment. The Learning Goals The learning goals for IN250, as most recently revised in 2007 are: By the end of IN250 U.S. Studies, students will be able to: 1. Critically respond to texts by or about some of the diverse cultures and people living in the United States; 2. Use discipline-appropriate sources to inform their critical analyses of diverse individuals’ or cultures’ experiences and perspectives; and 3. Write at least one well-developed critical analysis about cultural, structural, or institutional issues specific to the discipline of the course. Snapshot During the Fall 2007 semester, 8 sections of IN 250 were offered by 7 faculty members: 1 section by one Religion faculty--adjunct 3 sections by three History faculty—two full-time tenured or tenure-track; one adjunct 2 sections by one Political Science faculty—adjunct faculty 1 by one Theatre faculty—full-time tenure-track 1 by one Exercise Science faculty member—full-time tenured The Learning Story In all sections of IN250, students learn about the experiences and perspectives of some of the diverse cultures and people living in the United States. Courses may focus on particular groups (African Americans, women) or particular issues (poverty and welfare, Vietnam, Civil Rights, Watergate), but all faculty are committed to inculcating in their students sensitivity and empathy, as well as sharper critical thinking and writing skills. In all sections, students use discipline-specific sources in their writing, which ranges from the formal essay to informal discussion threads in Blackboard. Competencies that promote professional success and understanding of others are emphasized (as per MPSL prepares 1 and 2). Assessment Methods and Data All faculty teaching IN250 classes in Fall 2007, with the exception of those taught by the adjunct faculty members in Religion and History, submitted artifacts, including final exams, term papers, or written assignments, at the end of the Fall 2007 semester. The IN250 self-study committee randomly selected five artifacts from each of the six classes. The committee used a single rubric (see Appendix A) to assess all the artifacts, scoring each artifact on a 1-5 scale (1 indicating Nominal, 5 indicating Excellent) on each of the three goals. We then calculated average scores for each learning goal, for all artifacts in total and for each specific class. The following table shows the range of possible scores and interpretation.