Study on Online End-of-Course Testing for Computer-Based Driver Education in Virginia, Exercises of Technology

Download Study on Online End-of-Course Testing for Computer-Based Driver Education in Virginia and more Exercises Technology in PDF only on Docsity! Online End-of-Course Testing Study 2018 Report Virginia Department of Motor Vehicles December 2018 Online End-of-Course Testing Study 2 EXECUTIVE SUMMARY During the 2018 General Assembly Session, two bills were introduced to permit DMV licensed driver training schools to conduct remote online end-of-course testing for computer-based driver education courses. DMV supported studying this issue because the agency has minimal experience in administering online testing in remote environments, and needed to better understand the technology necessary to ensure the security and integrity of the test, if remote online testing were permitted. Subsequently, Chairmen of the House and Senate Committees on Transportation charged DMV with studying the feasibility of administering remote online end-of-course testing for computer-based driver education courses, what standards would be needed for such testing, and how to implement standards that would maintain test integrity and security.1 As a result of the need to study the issue, House Bill 1077 (HB 1077) was left in the House Committee on Transportation and Senate Bill 889 (SB 889) was stricken at the request of the patron. To fulfill the General Assembly’s request, DMV formed an internal team to gather and analyze relevant research and information. DMV convened a study group consisting of the DMV internal team and relevant stakeholders from the Virginia Department of Education (VDOE), the Virginia State Police, the Virginia Sheriffs’ Association, representatives from the insurance industry, highway safety advocates, owners and operators of driver training schools licensed to offer driver education courses in Virginia, and representatives from the Virginia Driver Education and Traffic Safety Schools (VADETS). The stakeholders met twice to discuss the General Assembly’s directive and whether or not to recommend changes to the current end-of-course testing standards for computer-based driver education courses. The stakeholder group was provided with information related to the driver education program administered by DMV licensed driver training schools, secure testing and the Standards of Learning (SOL) Assessment Program administered by VDOE, and Commonwealth of Virginia Technology Risk Management Standards. DMV and the stakeholder group also considered information related to how and if remote online testing has been administered in other jurisdictions, a measure of the programs’ success, issues that arose when implementing such programs, auditing and monitoring methods used to ensure the integrity of the tests, and the overall cost to the licensing agency. Additionally, DMV and stakeholder participants discussed various information technology (IT) security and monitoring methods that would help maintain the security and integrity of end-of-course tests for computer-based driver education courses, should driver training schools be allowed to conduct remote online testing. After analysis of the relevant research and stakeholder discussion, the study team could not reach a consensus to allow remote online end-of-course testing for computer-based driver education courses. DMV and a majority of the members of the stakeholder group believe the current standards and Code of Virginia provisions governing computer-based driver education provide a sufficient level of oversight, protection, and control over the end-of-course test. Though numerous IT security methods and controls were considered by the stakeholder group that would potentially enhance remote online test integrity, DMV and a majority of stakeholders did not feel those controls or methods would provide an equal level of assurance that is demonstrated by the current standards requiring in-person, proctored end-of-course tests. Thus, DMV and the study group recommend no changes be made to the current standards related to end-of-course testing for driver education, believing the current standards are needed to maintain the program security and integrity. 1 See Appendix A for charge letters from Chairmen of the House and Senate Committees on Transportation. Online End-of-Course Testing Study 3 If the General Assembly moves forward with legislation allowing for remote online end-of-course testing for computer-based driver education courses, DMV, in accordance with stakeholder feedback and input, has identified recommended IT security compliance and risk standards. These recommended controls, standards, and methods are included to assist in providing some degree of test integrity, security, and control should remote online testing be authorized. However, DMV and the majority of the stakeholders believe these methods would not eliminate the potential for students to receive or obtain assistance on the final exam, thus, undermining the integrity of the test. Online End-of-Course Testing Study 4 1. INTRODUCTION 1.1 Background During the 2018 General Assembly Session, two bills were introduced by Delegate Filler-Corn (HB 1077) and Senator Marsden (SB 889) that proposed allowing computer-based driver education providers to conduct remote online end-of-course testing. Current Code of Virginia provisions provide the DMV Commissioner with authority to develop testing standards. DMV end-of-course testing standards for computer-based driver education courses require all tests to be administered in a DMV approved driver training school facility, while being proctored by a DMV licensed instructor. HB 1077 (Filler- Corn) and SB 889 (Marsden) proposed amendments to the Code that would no longer require all end-of- course tests for computer-based courses, conducted by DMV approved driver training schools, to be completed in-person, proctored by a DMV licensed instructor, at DMV licensed facilities. DMV supported studying this issue because the agency has minimal experience in administering online testing in remote environments, and needed to better understand the technology necessary to ensure the security and integrity of the test, if remote online testing were permitted. As a result of the need to study the issue, HB 1077 was left in the House Committee on Transportation and SB 889 was stricken at the request of the patron. Chairmen Yancey and Carrico, of the House and Senate Committees on Transportation, requested that DMV study what standards are necessary and how to implement such standards to maintain test integrity and security, should the stakeholder group desire to allow remote online end-of-course testing for computer-based driver education courses. DMV conducted internal team meetings to gather and analyze relevant information to aid stakeholders in making a recommendation regarding computer-based driver education testing standards. In accordance with the study charge, DMV convened an internal team along with a group of key stakeholders including the Virginia Department of Education (VDOE), the Virginia State Police, the Virginia Sheriffs’ Association, representatives from the insurance industry, highway safety advocates, owners and operators of driver training schools licensed to offer courses in Virginia, and representatives from the Virginia Driver Education and Traffic Safety Schools (VADETS). DMV’s internal group included subject matter experts in driver training school administration, IT security and VITA standards, highway safety, law enforcement, driver’s licenses and other DMV issued credentials, and Virginia legislative processes.2 The stakeholders examined: • Current Virginia laws, regulations, and standards governing end-of-course testing for computer-based driver education courses administered by DMV licensed driver training schools; • Information on the Standards of Learning (SOL) Secure Assessment Program administered by VDOE and the Virginia Board of Education; • Commonwealth of Virginia Technology Risk Management Standards and IT security compliance requirements; • Proposed secure testing, proctoring, and monitoring methods to ensure test taker identity, and concerns with proposed methods; • Research of other jurisdictions including: o How and if online testing has been administered in other jurisdictions; 2 See Appendix B for full list of study participants. Online End-of-Course Testing Study 7 ensure that the test administration is successful, appropriate, and secure. There are a number of logistics that need to occur before, during and after a test including (i) test security agreements, (ii) transmittals for secure test materials, and (iii) affidavits after each test session. These procedures help ensure every student in Virginia taking SOL tests has a similar experience so they are not, in any way, advantaged or disadvantaged by the testing environment. Though VDOE does not administer an end-of-course SOL assessment for driver education courses, formative and summative assessments are conducted in driver education courses administered in the Virginia public schools. DMV looked to VDOE testing requirements when establishing the in-person end-of-course testing standards for computer-based driver education. 3. DRIVER TRAINING SCHOOLS & END-OF-COURSE TESTING To provide stakeholders a better understanding of where the end-of-course test for driver education is in the standard driver’s licensing process, the following steps were presented during stakeholder meetings to demonstrate what requirements a person under 18 years of age must satisfy to be eligible for a standard driver’s license. Persons under the age of 18 must: • Take a knowledge exam and vision test with DMV to receive a learner’s permit; • Complete the driver education classroom course; • Complete the behind-the-wheel driver education course; • Complete the 90 minute parent/student course (if required); and • Attend a licensing ceremony conducted by the courts to receive the license. 3.1 Driver Training School Program Overview DMV licensed driver training schools are authorized to conduct driver education, which includes both the behind-the-wheel component and the driver education classroom course component. The end-of-course test being discussed throughout this report is the summative assessment of the classroom course component of driver education. The classroom course component of driver education and behind-the-wheel may be taken simultaneously if they are both administered by the same driver training school. However, if the student elects to complete the classroom course component and behind-the-wheel component of driver education at different driver training schools, he or she must complete the driver education classroom course component as a prerequisite to participating in behind-the-wheel. Although driver’s license applicants 18 years and older are not required to complete driver education, they may opt to complete the classroom course and behind-the-wheel in order to waive the 60-day learner’s permit waiting period, normally required for driver’s license applicants over the age of 18, and have the road skills test administered by a DMV licensed driver training school. In 2016, the General Assembly passed legislation (HB 748) to allow DMV licensed driver training schools the option to conduct computer-based driver education courses.3 This bill amended § 46.2-1701 of the Code of Virginia to provide that upon application of a driver training school licensed in accordance with Chapter 17 of Title 46.2, the Commissioner may license such driver training school 3 HB 748, patroned by former Delegate Greason, ultimately passed both the full House and Senate uncontested. Online End-of-Course Testing Study 8 “using criteria established by the Commissioner pursuant to § 46.2-1702 to provide computer-based driver education courses using curricula approved by the Commissioner.” Driver training schools participating in this stakeholder study noted the convenience computer-based courses provide to students in regards to completing the course requirements. Specifically, computer-based driver education provides students increased flexibility in satisfying the course requirements due to the remote online nature of the course. Currently, 295 driver training schools are authorized to provide in-person driver education in Virginia, 27 of which are also authorized to offer computer-based driver education courses. For a map displaying the geographical locations of DMV licensed driver training schools authorized to provide both in-person and computer-based driver education courses, see Appendix D. 3.2 Approving Driver Training Schools to Conduct Computer-Based Driver Education Once a driver training school has been licensed by DMV to conduct driver education in the Commonwealth, and once the school is providing in-person classroom instruction, the school’s owner may apply to be authorized to conduct computer-based driver education courses. To apply for approval, the school’s owner must provide proof that the testing location it uses has adequate infrastructure to conduct end-of-course testing meeting DMV’s current standards and proof of an agreement from the computer-based course vendor indicating a VDOE approved curriculum is being provided. Driver training schools seeking to provide computed-based driver education instruction are subject to the same licensing requirements as driver training schools providing in-person driver education only, aside from the additional licensure fee of $100 according to §46.2-1701 of the Code. As part of the process of approving driver training schools to conduct in-person and computer- based driver education courses, DMV requires the owners to submit and agree to DMV forms that address driver training school standards and requirements. Driver training schools seeking authorization to conduct computer-based driver education must submit an Online Driver Education License Application and Agreement form, otherwise known as a DTS60, to DMV prior to its approval. The DTS60 form ensures a driver training school has an adequate testing location, with sufficient classroom material and internet services, to successfully administer the end-of-course assessment according to DMV standards. Additionally, this form sets out standards related to computer-based course requirements, driver training school business practices, web site services, test site requirements, monitoring instructions, and audit requirements. DMV directs driver training schools on administering and monitoring end-of-course tests for computer-based driver education courses. Driver training schools must comply with test administration directions set out in DMV forms DTS65A and DTS65B.4 In addition, an IT security and overall risk assessment of the school is conducted by DMV. DMV considers the Commonwealth of Virginia Technology Risk Management Standards as technological compliance requirements for this assessment. The purpose of these standards is to assist Virginia State Agencies in assessing the risks to its sensitive IT systems and data, and protect the resources that support the Commonwealth’s mission. The Commonwealth of Virginia Technology Risk Management Standards mainly relate to data hosted and maintained by the Commonwealth for employees and contractors, and maintaining data of current Commonwealth of Virginia contracted vendors. By performing this assessment, DMV will determine if a driver training school has sufficient confidentially controls, the appropriate technology to provide the course, and controls to maintain the student’s privacy. From this 4 See Appendix C for forms related to end-of-course testing standards for computer-based driver education courses. Online End-of-Course Testing Study 9 assessment, DMV will produce an analysis of the driver training school’s overall level of IT security and associated risks. Driver training schools are given the option to provide computer-based driver education using a VDOE approved curriculum they developed themselves or a curriculum provided through a third party vendor. As discussed prior, all driver training schools are required to use VDOE authorized curricula developed in accordance with the Curriculum and Administrative Guide for Driver Education to administer in-person driver education courses. However, driver training schools seeking authorization to conduct computer-based courses must obtain separate approval from VDOE, whether the curriculum is developed by a third party vendor or the driver training school itself. To date, driver training schools approved to conduct driver education courses in Virginia have only opted to use course curricula produced by third party vendors. 3.3 Record Retention and Audit Upon licensing, a driver training school is permitted to provide training to students. Under current Virginia law, during the tenure of the school’s licensure, the school is required to maintain business records pertaining to student training and assessments. To ensure that schools are maintaining the appropriate business and testing practices, driver training schools are required to submit to regular auditing by DMV. DMV conducts three types of auditing to maintain oversight of the driver training school program: (1) initial licensure audits, (2) annual audits, and (3) random audits. These audits are conducted by driver license quality assurance (DLQA) personnel. If, as a result of any one of these audits, DLQA personnel find a deficiency in a school’s business practices, DMV may take action to suspend, revoke, cancel, or refuse to renew a school’s license. It should be noted, record retention and audit requirements are the same for schools authorized to provide both in-person or computer-based courses. As it relates to this study, DMV and stakeholders discussed how record retention and audit requirements may be impacted should remote online testing be allowed for computer-based driver education providers. DMV and stakeholders agreed that driver training schools should remain subject to at least the same record retention and audit requirements in place today, should changes be made to the end-of-course testing standards. DMV added that computer-based driver education course providers would become subject to more stringent record retention and audit requirements should remote online end-of-course testing be allowed, to ensure overall integrity and security of the test is maintained. It was noted that increased requirements may cause increases in technological expenditures for driver training schools. DMV’s internal cost to audit driver training schools may also be impacted should end-of-course testing be permitted for computer-based driver education courses. 3.4 End-of-Course Test Driver training schools that provide in-person and computer-based driver education courses must administer the end-of-course test at a DMV licensed location, proctored by a DMV approved instructor. The process for completing end-of-course tests for driver education is outlined in the following section. On the day of the test for computer-based and in-person driver education courses, the applicant must present valid photo identification to prove identity. If the student is under 18 years old and the course was completed online, he or she must also bring a signed Parental Consent for Online Driver Education Examination form; this is not required for in-person driver education courses.5 After the 5 See Appendix C for example of Parental Consent for Online Driver Education Examination form. Online End-of-Course Testing Study 12 while being proctored. At this level, before students enter the room, they are required to empty their pockets and remove all technology, books, and notes; additionally at this level, a proctor is required to be physically present to monitor the test. Based on the security levels presented, implementation of the proposed 2018 legislation would adjust the level of security from level four to either level two or three. 5. RESEARCH 5.1 Jurisdiction Survey and Research As part of the study, DMV was charged with researching how and if remote online end-of-course testing for driver education courses has been implemented in other jurisdictions. In doing so, DMV sought information related to the success of these programs, issues that arose when implementing the programs, auditing and monitoring methods used to ensure the test integrity, and the overall costs to implement and maintain the online testing programs. DMV surveyed other North American driver licensing or motor vehicle agencies by using the survey tool provided by the American Association of Motor Vehicle Administrators (AAMVA).7 The survey consisted of ten questions to determine if other jurisdictions that require driver education allow classroom instruction through computer-based driver training schools and whether those schools conduct remote online end-of-course testing to evaluate students. Thirty-two U.S. States and one Canadian province provided a survey response. According to the survey responses, nine jurisdictions allow remote online end-of-course testing for courses equivalent to the classroom component of Virginia’s driver education program. For a complete list of jurisdictions that reported allowing remote online end-of-course testing for driver education courses, please see Appendix E. Among the jurisdictions that provided survey responses, the total number of computer-based driver training schools ranges from one to 78, with an average of 29 schools. It should be noted that some jurisdictions do not track or monitor the current number of driver training schools in their jurisdictions. The jurisdictions that reported conducting remote online end-of-course testing for driver education courses have been doing so anywhere from two years to 12 years. Based on the information provided by survey respondents, remote online end-of-course testing in other jurisdictions is administered in a variety of ways, using several security and monitoring controls. As discussed previously, research on other jurisdictions revealed several methods used to ensure test taker identity. For example keystroke memory technology, pre-established security questions, parent proctoring, “secret shopper” audits, and enhanced record keeping systems were discussed in survey responses to help maintain program security and integrity. Both positive and negative comments were submitted about the reliability of these methods. The survey also requested jurisdictions provide information on how the success and security of the remote online testing program is measured. According to survey respondents, student pass/fail rates, the total number of students completing the 7 AAMVA is an organization that represents driver, vehicle, and law enforcement administrators in North America. Its work includes the creation of model policies and best practices on topics of interest to the motor vehicle administrator community. Online End-of-Course Testing Study 13 course, and a student’s ability to ultimately pass the DMV administered road test are considered to measure the success, integrity, and security of remote online testing. 5.2 Non-Commercial Driver Training School Survey As a result of stakeholder group consensus that additional information would be beneficial to making a recommendation, DMV surveyed the non-commercial driver training schools in Virginia. Of the 295 driver training schools, 65 schools provided a survey response. The survey consisted of ten questions to gain input from driver training schools in regards to the overall study topic, appropriate monitoring and auditing methods, and end-of-course test pass/fail rates for both in-person and computer-based courses. The goal of the survey was to gauge driver training schools’ opinions on allowing remote online end-of- course testing for computer-based driver education courses and the proposed security methods to ensure test taker identity, if it were allowed. The driver training school survey results did not reveal a definitive opinion of the overall driver training school market in regards to the study topic, based on the survey questions. While the survey responses did not indicate strong support for changes to the current end-of-course testing standards, responses partially demonstrated support for the proposed proctoring and monitoring methods, if remote online end-of-course testing were allowed. However, comments included in the survey responses noted both positions as it relates to the feasibility of the proposed security methods for ensuring test taker identity. The survey questions requested driver training schools to provide pass/fail rates for computer- based and in-person driver education courses, but the data provided did not reveal conclusive evidence supporting changes to the current computer-based driver education end-of-course test standards. In general, the results of this survey were inconclusive. For the complete survey results, see Appendix E. 6. STUDY RECOMMENDATION 6.1 Portion of Market Desires Online End-of-Course Testing for Driver Education Though DMV and a majority of the stakeholder group felt the current standards for driver education end-of-course testing provide the Commonwealth a sufficient level of oversight, protection, and control over the test security and integrity, a portion of the group felt allowing remote online testing would be beneficial to the Commonwealth. The proponents of this study argue that current driver education testing standards create an unnecessary burden for students enrolled in computer-based driver education courses by requiring the student to complete the end-of-course test at a DMV licensed driver training school. As noted by multiple driver training schools that are both supportive and unsupportive of allowing remote online end-of-course testing, computer-based driver education allows a higher level of convenience to students enrolled in the course. This is because computer-based driver education courses allow students to complete the course during any timeframe, from a remote location. According to proponents of allowing remote online testing, by not requiring the end-of-course test to be administered at a DMV licensed driver training school location, DMV standards would be allowing a greater level of convenience to the student enrolled in the course because he or she would be able to complete the final assessment in any location, at any time. DMV and the stakeholder group felt this argument revolved primarily around a measure of convenience Online End-of-Course Testing Study 14 to driver education students versus the overall safety of users on Commonwealth roadways. Stakeholders agreed that safety outweighs the need for convenience. The advocate for the legislation prompting this study, an owner of a driver training school in Virginia, indicated that significant populations of the driver training school market desire the availability to complete end-of-course driver education tests online, in a remote environment. Additionally, the proponent of the changes to the current laws and standards asserted that sufficient security controls are available to maintain complete integrity and security of the end-of-course test. Though this information was discussed throughout the study, the research and the majority of the stakeholder group did not indicate demand for online end-of-course testing nor indicated strong support for the proposed security methods. Stakeholders believe no changes to the current standards would be in the best interest to the Commonwealth, at this time. Though the study group recommends maintaining the current end-of-course testing standards, the following items address recommended end-of-course IT security controls, methods, and practices should the General Assembly desire to move forward in allowing remote online testing: 1. A DMV licensed instructor should be required to proctor all end-of-course tests, whether in- person or remote online testing. 2. The required end-of-course test should be administered by DMV licensed driver training school or authorized vendor. 3. Legislation authorizing online testing should prescribe minimum security standards in accordance with the Commonwealth of Virginia Technology Risk Management Standards. 6.2 No Changes to Program Because of current VDOE practices regarding secure assessments and the process of approving driver education curriculum, DMV will not authorize driver training schools to conduct remote online end-of-course testing for computer-based driver education courses. This is because VDOE’s current practices do not permit any assessments to be administered in environments where the proctors are not monitoring the testing taker in person, nor do they allow approval of driver education curricula that permits the end-of-course test to be administered in a remote location. To date, DMV has not received a request similar to what was proposed by the legislation prompting the study charge, nor has DMV observed significant public outcry or complaints requesting changes to the driver education testing standards and requirements. From DMV’s perspective, the testing standards for the computer-based driver education courses in Virginia do not require further changes for the test to be administered successfully and securely. DMV and a majority of the members of the stakeholder group believe the current standards, Code of Virginia provisions, and regulations governing driver training schools provide a sufficient level of oversight, protection, and control over the driver education end-of-course tests. By requiring all end-of- course tests for driver education to be completed at a DMV approved driver training school facility, while being proctored by a DMV licensed instructor, the current driver training school standards best ensure that the person enrolled in the course is the person taking the test and that no outside aid is being provided to the test taker. Though numerous IT security methods and controls were considered by the stakeholder group that would help enhance online test integrity, DMV and a majority of stakeholders did not feel those controls or methods would provide an equal level of assurance that is demonstrated by the current standards. Online End-of-Course Testing Study 17 Appendix A: Charge Letters April 13, 2018 Mr. Richard D. Holcomb Commissioner Virginia Department of Motor Vehicles P.O. Box 27412 2300 West Broad Street Richmond, Virginia 23269 Dear Commissioner Holcomb: During the 2018 General Assembly session two bills were proposed to amend the Code of Virginia to authorize the Commissioner of the Department of Motor Vehicles (DMV) to allow computer-based driver education courses to offer virtual testing for the end-of- course tests, House Bill 1077 patroned by Delegate Filler-Corn and Senate Bill 889 patroned by Senator Marsden. Current law provides in VA. Code §46.2-1702 that the Commissioner may approve driver education courses offered by driver training schools if he finds that the courses are of comparable content and quality to that offered by the Commonwealth’s public schools under §46.2-334 of the Code. The Commissioner may authorize driver training schools to be computer-based driver education providers, and may establish minimum standards for testing students who have enrolled in computer- based driver education courses. In conversations with both patrons and those advocating for the bills, DMV indicated that it does not currently allow driver training schools to test students in any capacity other than in person testing monitored by a licensed instructor. DMV suggested that since it has no experience with the type of testing proposed by the bills that the agency be allowed to study the issues regarding virtual testing and the standard by which the Commissioner should approve such testing in the Commonwealth. DMV requested to have the opportunity to gather additional data, obtain the input of multiple stakeholders, and have time to investigate any fiscal impact allowing virtual testing may have on the Commonwealth. Therefore, I respectfully request that the Department of Motor Vehicles study what standards are necessary and how to implement such standards for approving computer- [Type text] based virtual testing for the end-of-course assessment for driver education courses. The Department should determine what changes are necessary to maintain the safety of all drivers on Virginia highways. I request that DMV convene a working group of interested parties to conduct such a study. I ask that the group of stakeholders include the Virginia Department of Education, the Virginia Information Technologies Agency, the Virginia State Police, representatives of the Virginia Sheriff’s Association, representatives from the insurance industry, highway safety advocates, owners and operators of driver training schools licensed to offer courses in Virginia, representatives from the Virginia Driver Education and Traffic Safety Schools (VADETS) and other stakeholders identified by the Department. I ask that the working group research how and if such testing has been implemented in other states, the success of those programs, any issues that have arisen with implementing such programs, auditing methods to ensure the integrity of the program, and the cost to the licensing agencies, the insurance industry, and other similar stakeholders in considering standards for approving such testing. I request that you report back to the Senate Transportation committee in December of 2018 with the results of the study and the working group’s recommendations. As part of the report, the working group should provide for each item it proposes an analysis of the feasibility, the cost to the Commonwealth, and its cost-effectiveness. Also include any proposed legislation that would be necessary in order to pursue the recommendations. Sincerely, Charles W. Carrico, Sr. Chairman, Senate Transportation cc: The Honorable David W. Marsden, Senator The Honorable Shannon R. Valentine, Secretary of Transportation Online End-of-Course Testing Study 18 Appendix B: Stakeholders and Study Participants Online End-of-Course Testing Study 19 Appendix C: Forms and Reports OTS 60 (01/10/2017) Page 3 of 4 VIRGINIA DEPARTMENT OF MOTOR VEHICLES ONLINE DRIVER EDUCATION SCHOOL AGREEMENT Upon submission of this signed application, the owner or authorized representative of the Online Driver Training School agre ‘es to meet the following requirements. Online Requirements 1. 2 3. 4 5. 6. Busi aARwhy > 6. 7. 8. Web NOORONS fo} g. Test 4. iB Submit annual application. Pay annual license application fee. Provide DMV with a copy of contract or agreement (showing expiration date) authorizing use of approved online driver education curriculum from vendor that has been approved by the Department of Education (DOE) Provide approved classroom lacatian far the administration of final examination Provide computer for each student, where applicable. Comply with driver training school licensing requirements pursuant to Virginia Code Chapter 17 Driver Training Schools and Virginia Administrative Code (Chapter 121 Virginia Driver Training Schools Regulations). iness Practices Only use online curriculums approved by the Department of Education Maintain license as a Driver Training School throughout the licensure period. Use name as shown on the license issued by the Department. Test and monitor students at test sites approved by DMV. The Department must be notified in writing, 30 days prior ta a change of address for the licensed location of the business office or test site. The Department will issue a revised license reflecting such change. The school must return the current license to the Department when yau receive the revised school license. All business office and test sites must be approved by the Department prior to use. Provide DMV with written verification authorizing use of a facility to conduct testing and monitoring throughout license tenure and maintain a copy of verification. Maintain a record of business and student test records for three years. No public WiFi; must be a private Internet connection. Site: Display name of online driver training school. Display working telephone number. Display working email address. Post DMV's toll-free number. Display logo for approved Online Curriculum Vendors. List addresses for all DMV-approved classroom location test sites. Post the schoal's hours for testing in accordance with “Norrnal Business Hours* (9:00 a.m. - 5:00 p.m.) of the Department. Post testing procedures. Post valid DMV-issued school license. Site Have DMV-approved test site in the Commonwealth of Virginia. Have at least one DMY licensed Instructor to monitor testing. Testing must be conducted at the scheduled test site. Any changes in testing site must be mutually agreed upan by student. parent (if student is under age 18) and instructor. Comply with all local business license and zoning regulations and with federal. state, and local health, fire, and building code requirements. Seating arrangements and writing surfaces for each student attending at any given time Have working camputer(s) with Internet connection. No student owned camputers. Have working printer at each test site. Have working electrical outlets for computer support Past the school's hours for testing in accordance with “Nerrnal Business Hours" (9:00 a.m. - 5:00 p.m.) of the Department in a conspicuous location . Post testing procedures in a conspicuous location inside the test site Post DMV-issued schol license Past sign displaying DMV's toll-free number in a conspicuous location inside the test site. . Have clean and accessible restroam facilities. Have adequate interior and exterior lighting and adequate parking spaces. DTS 80 (01/10/2017) Page 4 of 4 Monitoring Instructor: 1. Maintain DMV-issued license with driver training schaal thraughaut the school's licensure period. 2. Ensure that the test is only administered to students that signed up through the school that offered the online course. 3. Prior to testing, Instructor must verify the following a. Parental Consent for Online Driver Education Examinatian (DTS 62) granting permission for student ta take the final examination online with appropriate signatures Proof that the student successfully completed the online driver education course. Proof that student completed the 90-minute Parent/Teen course (Students under age 18 in Planning District 8 (counties of Arlington, Fairfax, Laudaun and Prince William and the cities of Alexandria, Fairfax, Falls Church, Manassas and Manassas Park) along with a parent or guardian, must participate in an additional 90-minute driver education component as a part of the in-classroom portion of the driver education curriculum, d. Verify student's identity through DMV-issued learner's permit, DMV-issued identification card, school issued identification card, valid passport or military identification. 4. Read DMV-issued testing instructions to students before beginning test. Advertising/Notice to DMV Requirements 1. Use the words “Licensed by DMV" as the sole reference ta the relationship between the online driver training school and DMV. Do not use the DMY logo on any form of advertising. 2. Online school advertisernents cannot be placed in the DMV customer service centers. 3. School licenses are not transferable. The license cannot be sold, loaned, bartered or given by a licensee or an agent of a licensee ta another school, individual, association, partnership or corporation. 4. Achange in ovener will require an application for an original license along with the documents and fees required under the Virginia Cade, which must be submitted ta the department at least 30 days in advance of the effective date of the change. The online schoal will not operate under the change in ownership until an original license has been issued by the department reflecting the new ownership. 5. Inthe event of cessation of business, the online school must submit a written statement indicating the business is closing, and forward to the department within 30 calendar days after cessation of business, the online license, all instructars' licenses, student records and any materials furnished to the anline school by the department Conduct of Instructors The online school owner shall be responsible for the acts of any instructors while performing within the scope of his/her duties. They shall not conduct themselves in a manner that is not suitable or campatible with school-related activities pursuant to 24VAC20-121-30 (12) of the Virginia Administrative Code (Regulations) Scheduled and Unannounced Audits 1. Allow DMV ta conduct audits with ar without prior notice. At least one audit will be canducted per year. 2. Online driver training records must be open and available for inspection by any officer or employee of DMV or any law enforcement officer during normal business hours. DMV may secure and remove these records for the purpose of conducting audits or investigations. The department will return those records after review or at the conclusion of any DMV or related court action, when used for that purpose. 3. DMV will prepare a written repart on the results of each inspection and audit, and provide a copy of the report to and review it with the owner or authorized representative of the online driver training school. At the conclusion of the review of the repart, the owner or business must sign the documentation that indicates the school has received and reviewed the report. 4. Respond to and/or correct deficiencies/violations noted on audit report within 30 working days as directed. CERTIFICATION Ive hereby make application for an online driver training school and certify that all information contained on this application is true. IAwe understand that if licensed I/we are subject to the current statutes pertaining to operation of the online driver training school. By signing this document, | certify that | am an owner or designated representative of the school and that | am authorized to enter into binding agreements on behalf of the online driver training school. | agree to abide by the terms and conditions specified above and the current statutes. This agreement shall become effective upon signing and shall expire an the date indicated below. Either party may terminate this agreement by giving written notice within 30 working days. | understand that failure to comply with any of the terms of this agreement or the submission of false or inaccurate information pursuant to this agreement or application may result in suspension, cancellation or revocation of the online schoal license. APPLICANT NAME (print) APPLICANT SIGNATURE TITLE DATE (mmiddiyyyy> REPRESENTATIVE NAME (print) REPRESENTATIVE SIGNATURE TITLE DATE (mraiddl'yyyy) Virginia Driver Training DTS 65 (10/14/2016) ONLINE DRIVER EDUCATION MONITORING INSTRUCTOR EXAMINATION SCRIPT Purpose: To provide the driver training school Instructors with instructions for monitoring the final examination for the online driver education course. Instructions: The monitoring Instructor must read the instructions to the students before starting the examination. EXAMINATION INSTRUCTIONS. General Information: 1. You are here to take the final examination of the online driver education course. 2. No paper testing is allowed. 3. Be sure that you have signed the log sheet prior to taking this examination. Failure to sign the log sheet may result in your not receiving credit for the course and final examination. 4. This is not an opened book examination. 5. Turn off and put away all other personal electronic devices. 6. Do not start the examination until you have been instructed to do so. To Begin the Examination: 1. Login to the driver training school website where you registered for the online course. 2. Click on the logo of the online course you completed. 3. Locate the instructions for taking the final examination. 4. Atthe Instructor's prompt, you may begin the examination. During the Examination: 1. There is to be no talking or disruption during the examination. 2. Once the examination has started, an Instructor will be moving around the classroom to monitor testing environment. If you have any questions during the examination, you must raise your hand. Assistance will be given for technical problems only. 3. You may use headphones to listen to the audio of examination questions and answers. Use the audio player to pause/ play, advance/replay, and control the volume of the audio. 4. Do not toggle away from the examination material. 5. If itis discovered that you toggled away from the examination material, your examination will be immediately cancelled and you will have to schedule another examination session. 6. Once the examination begins, there will be no breaks or recess time. After the Examination: 1. When you finish the examination, exit the examination screen and return to the desktop so that you can log off. 2. If you pass, the online vendor will mail you the certificate of completion as proof of your completion of the online driver education curriculum. . If you fail, you will not receive a certificate of completion. You must schedule another examination session. . After the examination, you must sign the Online Driver Education Examination Monitoring Record (DTS 63). . Are there any questions? Aa nw . You may begin. Virginia Driver Training DTS 63 (10/14/2016) Gamy,,, .. ONLINE DRIVER EDUCATION oe EXAMINATION MONITORING RECORD Purpose: Use this form to document student attendance. Instructions: Type cr print in ink. Keep this report with the student's records. Important Information: Online driver education schools must not issue a certificate of completion to a student in Planning District 8 prior to receiving proof of completion of the additional minimum 90-minute parent/teen driver education component pursuant to § 22. 1-205 of the Va. Code. SCHOOL INFORMATION ‘SCHOOL NAME SCHOOL CODE TEST DATE INSTRUCTOR NAME (last) frst) (mi) INSTRUCTOR LICENSE NUMBER STUDENT INFORMATION Form of Photo ID: [_ DMV-tssued Learner's/identification Card [7] Passport —] SchoolID — Government-Issued ID (describe) ‘STUDENT NAME (last) rs im TIME IN TIME OUT | _| aM _] PM _| PM Verified completion of 80-minute parent/student driver if'ye cchecicanplicable education component (Planning District 8) Clyes [J]no CJ wa aS PP (J otseo [F] bec-1 : : document (check either YES, NO or N/A}: TEST RESULTS {| Student Passed {[] Student Failed CERTIFICATION \ certify and affirm that all information presented in this form is true and correct. | make this certification and affirmation under penalty of perjury and | understand that knowingly making a false statement or representation on this form is a criminal violation ‘STUDENT NAME (print) STUDENT SIGNATURE DATE (mrnvddiyyyy) INSTRUCTOR NAME (print) INSTRUCTOR SIGNATURE DATE (mmiddiyyyy)  Online End-of-Course Testing Study 20 Appendix D: Driver Training School Locations in Virginia Online End-of-Course Testing Study 21 Appendix E: Survey Results Online End-of-Course Testing Study 22 AAMVA Survey Jurisdictions that Administer Computer-Based DE # of Computer- Based Schools in Jurisdiction Jurisdictions that administer End-of-Course Test Online Online End-of-Course Test Start Year Alberta, Canada 2 Yes 2014 California Unknown* No N/A Colorado 10 No N/A Florida 70 Yes 2013 Georgia 65 Yes 2006 Kentucky 1 Yes 2008 Nebraska Unknown* No N/A North Dakota 4 Yes Unknown* Ohio 9 Yes 2015 Pennsylvania 64 Yes 2015 Texas 16 Yes 2009 Utah 2 No N/A Wisconsin 78 Yes 2016 *Survey response did not specify. Driver Training School Survey 1. Do you have any concerns with the current standards and requirements for Driver Training School's end-of- course- testing? Yes 10.94% 7 No 89.06% 57 Comments: 1. Without a proctored setting there is no way to truly test an individuals knowledge without relying on the honor code. The concern would be someone else would take the test for the student and the student would be able to move on without sufficient knowledge to be safe behind the wheel. 2. Test should be done on laptop or anywhere the student and instructor wants, physical classroom should not be needed. 3. Existing requirement to have students report to a center bottlenecks the process and capacity constrains driver training schools. 4. With technology and security mechanism available in the market, remote testing just like they do it at colleges and universities should be permitted instead of asking driving schools to proctor in person and in a facility. Testing of this type can be done remotely and online highly secured and verifiable. 5. I believe it's more cost effective to allow the schools to test the students after they finish the course. 6. The BRC Knowledge Test is too easy and does not test the student's knowledge. 2. Should remote online end-of-course testing be allowed for computer-based driver education courses? Online End-of-Course Testing Study 25 3. WE strongly feel that all driving education should be done in a classroom or in-the-car and on-the-road environment. 4. If required. 5. If it doesn't require an extra expense. 6. Many students would not be able to work with this process. Too complicated. 7. Yes, as part of an entire menu of security applications. 8. The Russians would get hold to the individual's keystrokes. 9. Don't know enough about keystroke to make a positive decision on it. 10. Although one can be taking the test with another proving the answers. It would still be the person...something to consider. 11. At no cost to me. 12. I have no idea how to do that. 6. Requiring pre-established security questions that would be randomly prompted throughout the end-of-course test is a security method to ensure the person taking the test is the person enrolled in the course and there is no cheating on the test. Would your organization be supportive of this requirement if online testing is allowed? Yes 69.84% 44 No 30.16% 19 Comments: 1. How do you ensure the questions aren’t shared to allow another person to cheat on the test but the person. 1. Randomly prompted questions would create a distraction; the security question should be asked prior to beginning the test. 2. WE do not support any online education for new drivers. 3. If online testing is approved, I would support all security measures. But I do not support online testing at home. 4. If required. 5. Only if other security measure aren't being used. 6. The driver could be there to give the security answers but someone else may be taking the test. 7. Yes. As part of a menu of security options. 8. I need more information. 9. I still think that there would be space for cheating as the person can receive assistance and be the real- actual person taking the test. 10. That information is easily shared. 11. Again, this would be tricky to implement technically. 7. Requiring parent proctoring is a security method to ensure the person taking the test is the person enrolled in the course and there is no cheating on the test. Would your organization be supportive of this requirement if online testing is allowed? Yes 51.56% 33 No 48.44% 31 Comments: 1. Some parents will help their child answered questions, at least they would see firsthand what their child’s weaknesses are. 1. Requiring parent proctoring will again create a distraction. 2. We strongly feel that all driving education should be monitored by a DMV licensed instructor in a classroom environment. 3. Parents would not insure fairness. 4. Same reasons as above. Online End-of-Course Testing Study 26 5. Not all parents will be trustworthy. 6. However, this method of security should not be absence from other reinforcing methods already suggested in this survey to be utilized as security method to verify the authenticity of the person taking tests remotely. 7. Let the DMV approved schools handle all testing, including the road skills testing. 8. Relationship is to close. 9. Parents take the test for them, no way. 10. Parents do not like to disappoint their children. Not sure that’s as effective. 11. I do think that private schools could proctor themselves, and home school associations might be able to proctor themselves, but not the random parent who is more likely to "help the student". 12. Some parents are cheaters. 13. Establish a tight structure now, then review in a year. 8. If your Driver Training School provides computer-based driver education courses, please enter the number of customers who passed and the number of customers who failed the end-of-course test for "computer-based" driver education courses for each of the following months listed below. (Provide answers in the following format: 150/10) July 2017 - June 2018 Answered: 11 Computer Based – 186 passes; 45 fails. 9. Please enter the number of customers who passed and the number of customers who failed the end-of-course test for the classroom component for the “in-person" driver education courses for each of the following months listed below. (Provide answers in the following format: 150/10) July 2017 - June 2018 Answered: 11 In Person – 104 passes; 2 fails Comments: 1. No records for final test failure. We correct all chapter quiz mistakes so they are prepared for final exam. If they do fail they will then make corrections and retake exam. I will document in the future. 2. We did not keep records for failures. 10. Please use this space for providing any additional comments you may have about online end-of-course testing for computer-based driver education. Answered: 23 1. We feel that online driving education of any kind is not necessary. Commercial driving schools and DMV licensed driving instructors are the best way for a new driver to get the best and most through driving education possible. Computers don't drive...people do. Therefore, people should administer all testing and provide all driving education for any new driver, no matter the new driver's age. 2. How can I get established to host online classes and end-of-course testing for computer-based education? 3. This is not supported by our school. 4. Test security should be a priority. Also person to person contact and communication is important. Also who will be monitoring the video and what happens when there are issues with the program or computer as happens now. 5. If the student can do 36 periods online, he/she should be able to do the test online as well. He/she should not need to be at a classroom Driver training school just for the test. Test can be done on a laptop anywhere, just like the 36 periods. 6. I think any of those measures would be a good thing. I also believe the 90 minute teen parent class should be allowed online as well. 7. There are many options available in the market that can achieve this goal. This option should be made available since it makes it much easier for students to complete the training on their schedule and at their pace. Parents and guardians have routinely requested that this training be available from end to end online. The prevalence of online training is only going to grow in the coming years and the driver education industry in Virginia should be ahead of the curve. 8. I presently do not offer the online course, but am considering it for future use. Online End-of-Course Testing Study 27 9. I have not provided the online course and testing to date. This is the reason I was not able to report on the previous two questions. However, should the opportunity and facility become supportive to have end testing using some of the methods envisioned I will like to consider starting the online and testing services as well. 10. I don’t have that capacity at this time. 11. Administration of the online course should be done by the schools to include end of course testing. Otherwise, get rid of the program completely. 12. We currently do not offer computer based programs 13. My answer for the majority of the questions was no. My reasoning for answering no is because I do not offer any online courses. My instructors and myself believe that Driver Education is a course that is most valuable to a teenager’s driving development when they get to work with an instructor in a classroom environment. Thank you for the survey. 14. Online Driver Education is pointless. Students are not learning anything, just logging in to meet the time requirement. Parents, siblings and friends are signing in to help complete activities and quizzes. By the time they get to Behind the Wheel, they have learned absolutely nothing. Accepting "at home" tests is just confirmation that you really don't have to take this class seriously 15. It needs to remain the way it is, we to ensure the individual does not receive any assistance during the testing. We need to ensure the individual knows and understands the Drivers Manual of Virginia. 16. Support your local driver clinics and pass on these services to us. We have been serving the community for years and this would keep the financial growth in the local areas and promote jobs! 17. I believe there could be some changes to make it easier for the customer, but the test needs to be proctored. 18. Thanks for asking for input regarding this matter. I welcome your questions and hope to offer more valuable input. 19. The available curricula aren't very good and we have a wonderful product that we can't get approved because the procedure is almost impossible. Make it easier for us to get our program approved.