Investigation Report: NRHS's Utilization of Apex Learning Program for Credit Recovery, Lecture notes of Physical education

Download Investigation Report: NRHS's Utilization of Apex Learning Program for Credit Recovery and more Lecture notes Physical education in PDF only on Docsity! .fiu INTRODUCTION APEX Leorning is a privately held online leaming platform that was utilized by New Rochelle High School ('NRHS'), during the period 2014 through the summer of20l8, for the purpose of affording NRHS seniors the opportunity to make up failed or incomplete coursework and earn credits required for graduation ("credit recovery"). The undersigned was appointed by the Board of Education of the New Rochelle City School District as special counsel for the purpose of conducting an investigation regarding NRHS's utilization of such program for the purpose of credit recovery, I The undersigned was tasked with, (i) reviewing whether the Apex Leorning credit recovery program (the "Apex Program" or the "Program") was administered in compliance with the Regulations of the Commissioner of Education pertaining to credit recovery, and (ii) determining whether any School District employees who performed administrative responsibilities for such Program engaged in misconduct in connection with grading and/or awarding credit to studenls. The Board of Educati and The undersigned was charged with conducting a separate investigation into the matter and reviewing the overall manner in which the Apex Program was administered during the four school years the Program was offered at NRHS (academic years, 2014-2015,2015-2016,2016-2017, and 2017-2018). The undersigned was also asked to review NRHS's implementation plan for Edgenuity, the new online platform through which credit recovery is presently offered at NRHS. The undersigned's investigation consisted ofi review of relevant laws and regulations, interviews ofSchool District personnel, including retired and current employees who were involved with the Program, review of various records fumished by the witnesses who were interviewed (and/or located with their assistance), verbal and written inquiries to Apex Learning representatives, review ofextensive data furnished by Apex Learning,and review ofthe newly adopted guidelines pertaining to credit recovery at NRHS. l. Relevant Laws & Requlations Dioloma Reouirements: Pursuant to Section 100.5 of the Regulations of the Commissioner of Education (the "Commissioner's Regulations"), students in the general education program are required to eam at least22 units ofcredit to receive a high school diploma, inclusive of:4 units of credit in English; 4 units ofcredit in social studies (inclusive of 7z unit ofcredit in U.S. Govemment and % unit ofcredit in Economics); 3 units ofcredit in science; 3 units ofcredit in mathematics; I unit ofcredit in languages other lhan English ("LOTE"); I unit of credit in visual art, music, dance 2 for an I Apex karning was not utilized by the School District exclusively for credit recovcry. It was also utilized as I means for providing AIS, unit recovery, instructional support to students, as well as online coursc offerings to "homebound" and College-level students during the period 2014 through 2018 and prior. and/or theatre; 2 units of credit in physical education ("PE"); % unit of credit in health; and 3 % units of credit in electives. See 8 N.Y.C.R.R. $100.5(a). As per Section 100.1(a) and (b) ofthe Commissioner's Regulations, aunit of credil is eamed by "the mastery of the learning outcomes set forth in a New York State-developed or locally developed syllabus for a given high school subject, after a student has had the opportunity to complete a unit o/ sludy in the given subject matter area", Unit of study is defined as, "at least 180 minutes of instruction per week throughout the school year, or the equivalent." See 8 N.Y.C.R.R. $100.1(a) and (b). Credit Recovery: Section 100.5(d)(8) of the Commissioner's Regulations authorizes school districts to offer students who previously failed a course required for graduation, the opportunity to make up incomplete or failed courses and obtain credit. 8 N.Y.C.R.R. $100.5(dX8). In regard to make up credit programs, also known as credit recovery programs, the Commissioner's Regulations authorize school districts to select the specific formal ofthe program(s) through which credit-recovery is offered, and the following differing approaches are listed as acceptable options: . repeating an entire course; . taking the course again as part ofa summer school program; o receiving intensive instruction in the deficiency areas ofthe coursel or . *digital leaming (online study) that: o is comparable in scope and quality to regular classroom instruction; o provides for documentation of satisfactory student achievement; and o includes regular and substantive interaction between the student and the teacher who is certified in the subject matter area and providing direction and/or supervision. The above list is not exhaustive, meaning, it's within the discretion of local school districts to off€r a credit recovery program that differs from the approaches listed above, However, the regulations specifically state that any credit recovery program offered to students must: o be aligned with the applicable New York State leaming standards; o satisfactorily address the student's course completion deficiencies and individual needs; and o ensure that the student receives equivalent, intensive instruction in the subject matter area provided, under the direction and/or supervision of a teacher who is certified in the subject matter area. The Commissioner's Regulations further provide that a student's participation in a credit recovery program must be approved by a school-based panel consisting oi at a minimum, the principal, a teacher in the subject area for which the student must make up credit, and a guidance director or other administrator. 8 N.Y.C.R.R. $100.5(dX8). Online Courses Generall],: ln regard to online courses taken for credit, Section I 00.5(d)( l0) of the Commissioner's Regulations similarly states, in pertinent part, that school districts must ensure that: o such courses are aligned with the applicable New York State leaming standards; r such courses provide for documentation ofstudent mastery ofthe learning outcomes; 3 For the 2017-2018 school year, there were a total of 200 course completions. for 198 courses. assigned the final grade for 2 courses. The Course Completions Report indicates a total of 54 course completions (in various different subject areas) over the four-year period for which no Time Spent (00:00) was recorded by the Apex system. (For 15 of such 54 courses, the student Enrollment Dale and Completion Dale are lhe same.) For the remaining 435 courses completed, lhe Time Spenl (pu completed course) ranges from 2 minutes, to 8l hours and l5 seconds. From the Course Completions Report,l also noted, (i) a number of instances of students being enrolled in a course in one school year, and completing such course during the following school year, and (ii) instances ofstudents being enrolled in the same classes as students from prior school years (i.e.,2017-2018 Apex students enrolled in the 2016-2017 Physical Education course), The undersigned asked Mrs. Kamana'o if this apparent practice ofenrolling students in the same courses, year after year, without opening new sections could impact th€ reliability ofthe Apex data provided, Specifically, I asked, if modifications were made to a course's requirements over time (i.e., certain units, lessons or activities, which werc once requirements, were later eliminated), would the modifications implemented take retroactive effect and impact any data that was previously recorded for earlier enrolled studentsf Mrs. Kamana'o replied that modifications to courses are not retroactive - when a course is modified or updated, a new version ofthe course is released. Mrs, Kamana'o stated that new classrooms must be created, and students must be enrolled in order for the new version ofthe course to be used. I asked Mrs, Kamana'o ifthere are any actions that could be laken by a school districl administrator that would impact the reliability of Apex's data. Mrs. Kamana'o replied, ?fo I have not encountered that belore.l also asked Mrs. Kamana'o how a scenario of same enrollment and completion date, with no Time Spent, but a passing final grade is possible, Mrs. Kamana'o replied that she has seen this scenario when, (i) students complete all course work offline, and (ii) a student completed an Apex course at one school and then transferred to a new school - the new teacher chose to add scores for the student so it would be recorded in the system of the new school. I asked if there is any other possible explanation - ifa technological glitch (on Apex's end or on a school district's end) could result in the elimination of time that a student spent working on the Apex system? Mrs. Kamana'o did not acknowledge this as a possibility - she stated she could not provide any further explanation, other than the two possibilities noted above. I then asked Ms. Saunders Ortiz ifthere was anyone at the company, perhaps from a different department or function, who might have more information regarding my areas of inquiry. She responded by stating "they" believe the data to be "factual and accurate," and noting, "interpretation ofthe data is really dependent on the implementation so \ve would refer you to the school on that." However, she noted that, in a course like Physical Educalion, a good percentage ofthe overall score comes from offline activities - a teacher could o o 6 print Activity Logs and hand those to the students, and the students could be "working" in the course without ever logging in to the system. Aoex C onso I idote d Act ivi I ie s Re oorl ln addition to the Course Completions Report, the undersigned was given access to the ?{e}, Rochelle Consolidoted Aclivilies report (hereinafter the "Consolidated Acliviti$ Reporl"), which provides comprehensive information regarding the individual activities completed by students (organized by course, unit, lesson, and activity number) including, for every activity assigned: points eamed, points possible, the number of times a student attempted the activity (Activity Attempts), the time the student spent on the activity displayed in seconds (Time Spent Seconds), a description of whether a student completed the activity and if it was graded by a leachet (Score Detalls), and the name ofthe teacher who entered the score for the activity in Grade Boot if such activity was "teacher entered" and not scored by lhe computer (Score Awarded By). This report was primarily used by the undersigned to cross-check and better understand information reported in the Course Completions Report andlhe Grade Book Audit Report. (Copies of the Consolidoted Aclivities Reporl pertaining to the 2016-2017 and2017-2018 school years are provided as Exhibits 3c and 3d.) Anex Grade Book Audit Reoort In response to my request for data that would indicate and isolate score changes and score entries by School District personnel for activities that should have been scored by the Apex system ("computer-scored"), Apex granted me access to the Grade Book Audit Report for each of the school years under review. Grade Book is the record within which all scores eamed by students are recorded (by course), inclusive of the scores earn€d for diagnostics (DU), quines (QUQ, computer scored tests (CSI), and final and mid-term examinations (EXM). Mrs. Kamana'o explained thatthe Grade Book Audit Report displays, (i) all computer-scored activities completed by a student for which the score eamed by the student was changed by a teacher, and (ii) all compuler-scored activities not completed by the student, but which had a score assigned by a teacher. Mrs. Kamana'o explained that the term "override" is utilized to refer to either ofthe above two actions. (Copies of the undersigned's working copies of the Grade Book Audit Report pertaining to each of the school years under review (sorred by Override Performed By (column U)) are provided as Exhibits 4a, 4b, 4c, and 4d.)7 For each ovenide documented in the Grade Book Audit Report, the following pertinent information is reported: the name ofthe studenl, the course title, the activity ty pe (i.e., QUZ, CST or EXM), the points possible for such activity, the score earned by the Student (Student Earned Score),lhe date such score was earned (Student Score Earned Or), the student's score after the override (Grade Book Score After Override), the name of the individual who performed the ovenide (Override Performed By), and the date and time such override was performed (Overuide 7 The undersigned's working copics of lhe Grade Book A dit Repo show overrides performed in the context of compleled courses only. The undersigned did not examine ovenides performed in the context ofcourses for which the Enrollment Staus is indicated as llilhdrawn or Active. 7 Dote).Mrs. Kamana'o explained that ifno score is listed in column S ofthe report (Student Earned Sdore), then the student did not complete the activity to eam a computer-generated score. Mrs, Kamana'o explained that while the override function could be used for improper purposes (such as grade inflation or to award credit when no credit was eamed), there are valid justifications for the utilization of such featurc. She is aware of overrides being utilized, (i) for special needs students who are taking course content offline - in such cases, teachers may print course work, and students may write their answers on paper; (ii) for students who are taking course content ollline for other reasons (i.e,, students who are having technical issues on the computer they are utilizing); and (iii) in connection with mastery-based leaming, as a means to permit a student to progress in a course after multiple failed atlempts at a particular activity - rather than have the student make additional attempts to pass a quiz or test, teachers will sometimes assign additional work o{Iline and award the credit needed for progression to the next unit utilizing the ovenide function. Mrs. Kamana'o explained that, to perform ovenides, users must engage in a multi-step process. Users must enter the Grade Book, select the unit they wish to review, review the scores eamed for the activities within such unit, double-click on the score he or she wishes to change (or enter, if no score is recorded), enter the new score and press, "save". o Grade Book Audit Renort. 2014-2015 School Year The Grade Book Audit Report for the 2014-2015 school year (Exhibit 4a) shows 5 "ovenides" (score enlries forE computer-scored - 3 ovenides by and 2 overrides by student, Accordin to information ided the School District (Exhibit 'lhe Grade Book Audit Report for lhe 2014-2015 school year demonstrates that forty percent (40%) ofthe overrides performed were performed on or after June lsrofthe school year. o Grade Book Audit Report. 2015-2016 School Year The Grade BookAudit Report forlhe 2015-2016 school Exhibit 4b similar or one student - I resulted in no change to the student's eamed score,e I decreased the student's eamed score from 2ll30 to utilization of the override function l5130, and I was a score entry for a C.lI; plus 2 overrides in no change to the students' eamed scores,) shows minimal (3 ovenides by - both resulted This report also shows 23 overrides performed by during the months of May and June 2016, rtaln ln to 3 different students. I ofthe 23 ovenides ed was for a student (See Exhibit 5.) The remaining 22 overrides byErr,ere for consisted of- 3 overrides increasing the students' earned scores for DU and CS? activities and l9 score enties for DIA and QUZ activities. 8 The student's score was exact same rt override was performed. "r nt II zt TI 9t a a a a LI 8l IZ zz o tZ 9Z LZ 8Z Report for details regarding this student's activities on Apex for the course in queslion, Biologt Literacy Advanlage, The undersigned noted that all ofthe scores awarded to such student werc Teacher Dnlered on 6121117, as per rows 18749 - I 8856 of such report, with no Time Spent or Activity Attempts by the student. The undersigned further noted from the Grade Book Audit Reporl, the student's course enrollment date, course completion date, and the dates ofthe overrides performed arc the same (6/2U17).) In fu(her response to the 7 overrides the 77 ovenides performed unde During this investigation, the undersigned spent a significant amount of time studying Apex data, and revisiting such data as witnesses were interviewed, in an effort to reconcile it with the statements and evidence provided. The undersigned also conesponded and spoke with Apex Learning representatives to request clarification as I uncovered inconsistencies and information that could not be reconciled. Ofparticular concem was the data inthe Course Completions Report indicating a total of 54 course completions (in various different subject areas) over the four-year period in question for which no Time Spent (00:00) was recorded by the Apex system. I found this data particularly conceming given that Apex is an online credit recovery platform. As noted in Section Il of this report, the undersigned inquired about various possibilities that could impact the 3l hour of 9:00 AM on June 3:00 PM and 4:45 IV, Accuracv and Reliabilitv ofAnex Data Provided In fight of the foregoing, the undersigned requested a copy of the Consolidated Activities Report for the 2014-2015 school year (Exhibit 3a), and cross-checked the information reported therein for I*ittr the information report;d in the Course Completions Report (Exhibit 2). The undersigned observed that the Corsolidaled Activities Repoil, (i) showed values for "Date Grade Entered' (Column S) and"Points Earned' (Column T) for substantially all ofthe course activities assigned to this student, (ii) there were no instances of"Teacher Enlered'scores, and (iii) the were a substantial number ofactivities indicated as, "tested out." (See Exhibit 3a, rows I l8l7 - I 1985). (The undersigned further observed that the above scenario repeated itself for other Apex students,) On Ocrober i0,20i9, rhe un<iersrgne<i conta ueo Apex Leariingand specrricaliy ari.iO, iif lid not compl€te or attempt any course activities online (as the reports indicate), and if a teacher did not enter the scores (as there are no overrides indicated for this studentinlhe Grade Book Audit Reporr),how did the student obtain lhe points that were awarded? Also, ifE.rt.d-ouf' of various units (as the Consolidated Activities Report indicates), why wasn't he credited "Time Spenf' for the amount of time he spent completing such pre-tests? (The undersigned's email correspondence with Apex is provided with this report as Exhibit 16.) reliability of Apex's data, and Apex Learning initially maintained the data was "factual and accurate." Notwithstanding lpex Learning's insistence, the undersigned continued to doubt the accuracy and reliability of the data provided, particularly in light of, (i) the dau in the Course Completions Reporl perlaining to the 2014-2015 school year showing that, for l8 ofthe 20 courses completed, no Time Spenl was recorded by the Apex system (see Exhibit 2), in contrast with (ii) data in the Grade Book Audit Reporl showing only 5 overrides for such school year (all pertaining to the same student) (see Exhibit 4a). The above in On November l, 2019, in response to the above query, Ms. Saunders Ortiz of Apex Leorning notified the undersigned that the Course Completions Reporl for the period 2014 through 2018 (Exhibit 2) and the Consolidared Activities Reports for the 2014-2015 and 2015-2016 school years (Exhibits 3a and 3b) "were inconect," explaining: We discovered that Time spent, Aclivity Atlempl, and score details were slorecl differently prior to 8/ l/2015 ond we didn'l accounl for lhal in the previous versions of the reporls. lle spenl yesterday updating the reporls and Bernee will be sending you lhe links lo these soon. On the afternoon of November 1,2019, Apex Learning gave me access to purportedly comected versions of the Course Completions Report and Consolidaled Aclivilies Reports; however, my inspection of such data revealed that it was still flawed. Specifically, information reported in the "corrected" version of the Course Completions Reporl was inconsistent rvith information reported 32 in the "conected" version of the Consolidoted Activities Report for the 2014-2015 school year. (i.e., Row 410 of the "corrected" Course Completions Report showed the completion of Health Opportunities through Physical Education (HOPE) (201r,-@ 00:00 Time Spenr, 78o/o Aclivity Complete studen0, but the updated Corso liclated Activities Report for the20l4-2015 school year contained no information at all regarding this student.) On November l, 2019, I advised Apex Learning that the "corrected" reports still seemed to be incorrect. (See Exhibit 16.) ln response, on November 6,2019, the company notified me it was conducting its own investigation and explained: It appears that certain oldet activity detail data (including, unfurtunately, Time Spent ond Activity Attempls) has expired. So a "0" for Tine Spent may mean "dala unavailable" as opposed to zero seconds spenl llre ore determining now if il can be retrieved. I should hove an updote for you by the end of the day today or lomorrow- In response, I notified Apex Learning that it was imperative that I know the scope ofthe issue(s) impacting the data provided, as the 00:00 Time Spent course completion occunences appear in the Apex reports pertaining to all four school years. (Seg Exhibit 17.) Apex agreed to pass my request on to the person(s) investigating the issue. (See Exhibit 18.) As ofthe date ofthe undersigned's issuance of this report, no further information or updates have been received from Apex. Thus, the nature of the enor and the scope of the error impacting the Apex data which was examined during the course ofthis investigation is unknown. Issue #l: lTas the Apex Learnlng Onllne Credlt Recovery Prugram adminlstered ln compllonce wllh lhe Regulallons of lhe Commlssloner of Educallon governlng credil recovery? Based on the information learned from the available witnesses who were willing to participate in this investigation and the documents reviewed, the undersigned finds that the Apex Credit Recovery Program at NRHS was not administered in compliance with the Regulations of the Commissioner of Education, at any point in time, during the years the Program was in operation. The foregoing finding is largely driven by the lack ofcertified teacher involvement with students enrolled in Apex credit recovery course offerings during the period 2014 through 2018. As explained in Section I of this report, Section 100,5(d)(8) of the Commissioner's Regulations expressly requires that online credit recovery programs, (i) ensure that the student receives equivalent, intensive instruction in the subject matter area provided under the direction and/or supervision of a teacher who is certified in the subject matter area," and (ii) include regular and substantive interaction between the student and a teacher who is certified in the subject matter area and providing direction and/or supervision. See 8 N.Y.C.R.R. $ I 00.5(dX8); see also, 8 N.Y.C.R.R. $ 100.5(dX I 0), applicable to online course offerings. V. Findines. Ooinions & Recommendations 33 9t c Issue #3: Newly Adopled Guldellnes Appllcable lo Online Crcdll Recovery 'dgenuity was launched during the spring of20l9 and utilized by a cohort of approximately eight (8) graduating seniors 37 The foregoing investigative findings and opinions are referred to Ceneral Counsel for discussion with the Board of Education. Dated: December 3,2019 Miller Place, New York Yours, etc., By: INA M. CAFA A, Esq. Law O(fices ofDouglas A. Spencer, PLLC 85 Echo Avenue, Suite 6 Miller Place, New York I 1764 (63 1) s09 - il 20 38