comparative validity of three english oral language proficiency ..., Exams of English

Download comparative validity of three english oral language proficiency ... and more Exams English in PDF only on Docsity! THE BILINGUAL RESEARCH JOURNAL Winter 1996, Vol. 20, No. 1, pp. 55-68 COMPARATIVE VALIDITY OF THREE ENGLISH ORAL LANGUAGE PROFICIENCY TESTS Fredrick A. Schrank Riverside Publishing Company Todd V. Fletcher University of Arizona Criselda Guajardo Alvarado University of Houston, Clear Lake Abstract The validity of three English oral language proficiency tests was examined in terms of Cummin's BICS/CALP distinction. The tests studied included the Idea Oral Language Proficiency Test (IPT-I; Ballard. Tighe, & Dalton, 1989), the Language Assessment Scales (LAS; DeAvila & Duncan, 1991), and the Woodcock Language Proficiency Battery--Revised (WLPB- R:Woodcock, 1991). An examination of test content and pattern of correlations between subscales is used to describe the way oral language proficiency has been conceptualized and operationalized for each of the three oral language proficiency tests. These three tests measure similar and dissimilar aspects of oral language proficiency. Some of the similar and dissimilar aspects support the BICS/CALP distinction. Implications for evaluating oral language proficiency tests are discussed. Oral language proficiency tests are frequently used to determine if language minority school-aged children are able to meet the academic task demands of monolingual instruction in English. As a consequence, the tests provide results that are used to establish or deny eligibility for 56 BILINGUAL RESEARCH JOURNAL/Winter 1996 instruction in a language other than English. In practice, language- minority children scoring above a defined criterion on English oral language tests are presumed to be sufficiently proficient to participate, and function successfully, in a program of English-only instruction. Those falling below a defined criterion are eligible for a program of bilingual or English-as-a-second-language (ESL) instruction. Because these tests are used to make high stakes, irreversible decisions about the program of instruction in which a child is best suited to learn, it is important to critically examine their validity. One way to address the issue of validity is to compare the tests to one another in terms of what each measures. It has been suggested that language proficiency may have two aspects that can be differentiated by context. For example, language proficiency has been defined as "the student's ability to communicate in an informal social setting as well as the ability to function in a more formal, cognitively demanding academic setting" (Hamayan, Kwiat, & Perlman, 1985, p. 21, italics ours). Cummins (1984) is often credited with formalizing a distinction between these two aspects of language proficiency. He described one aspect of language proficiency as basic interpersonal communication skills (BICS), or a repertoire of communication skills developed and used in everyday social situations. He described the other aspect of language proficiency as cognitive- academic language proficiency (CALP), or a type of language facility developed through, and needed for success in, formal classroom settings. His distinction has been posited to explain why teachers report that some language-minority students appear to be able to communicate in English but lack proficiency in language-related academic learning. This article contains a description of the content for each of the subtests that constitute the English forms of the Idea Oral Language Proficiency Test (IPT-I; Ballard, Tighe, & Dalton, 1980), the Language Assessment Scales (LAS; DeAvila & Duncan, 1991), and the Woodcock Language Proficiency Battery-Revised (WLPB-R; Woodcock, 1991). Each test's reported concurrent validity data are summarized. It presents the results of an oral language concurrent validation study among these tests. Similarities and differences in test content are shown to exist. Cummins' distinction between BICS and CALP is used to explain some of the differences. Schrank, Fletcher, & Alvarado/ORAL LANGUAGE PROFICIENCY TESTS 59 instructional goals for developing language competencies. The instrument is appropriate for individuals aged 2 to over 90 years of age. For interpretive purposes, each WLPB-R provides cluster scores for Broad Ability, Oral Language Ability, Reading Ability, and Written Language Ability. (The broad ability, reading, and writing measures were not included as part of this study.) The WLPB-R Oral Language Cluster contains five subtests: • Memory for Sentences measures the ability to remember and repeat phrases and sentences presented auditorily. • Picture Vocabulary measures the ability to name familiar and unfamiliar pictured objects. • Oral Vocabulary measures knowledge of synonyms and antonyms. • Listening Comprehension measures the ability to comprehend a passage and supply the single word missing at the end in an oral cloze procedure. The test begins with simple verbal analogies and associations and progresses to a higher level of comprehension involving the ability to discern implications. • Verbal Analogies measures the ability to comprehend and verbally complete a logical word relationship. Although the vocabulary remains relatively simple, the relationships among the words become increasingly complex. The WLPB-R manual presents evidence of concurrent validity as established through moderate to strong correlations between the WLPB- R and measures of cognitive abilities, achievement, and language proficiency, including the Wechsler Intelligence Scale for Children- Revised (Wechsler, 1974), the Stanford-Binet Intelligence Scale-Fourth Edition (Thorndike, Hagen, & Sattler, 1986), the Kaufman Assessment Battery for Children (Kaufman & Kaufman, 1983), the McCarthy Scales of Children's Abilities (McCarthy, 1972), the Kaufman Tests of Educational Achievement (Kaufman & Kaufman, 1985), the Peabody Individual Achievement Test (Dunn & Markwardt, 1970), the Test of English as a Foreign Language (Educational Testing Service, 1987), the Clinical Evaluation of Language Fundamentals-Revised (Semel, Wug, & Secord, 1987), the IDEA Oral Language Proficiency Test I (Ballard, Tighe, & Dalton, 1980) and the Test of Written Language (Hammill & 60 BILINGUAL RESEARCH JOURNAL/Winter 1996 Larsen, 1983).The WLPB-R Oral Language yields a cluster score for determination of oral language proficiency. The Language Rating Scale. The LRS (see Figure 1) is a teacher rating of language proficiency. The LRS was obtained for use from the Houston Independent School District. It consists of a Likert-type rating for six areas of language ability, including sentence structure, vocabulary ability, word recall, telling stories, idea formulation, and speech production. For the purposes of this study, it is included as evidence of cognitive-academic language proficiency as measured by teacher judgment. (Item F, Speech, was not included in the data set, as this was presumed to be a measure of speech fluency). Figure 1 Houston Independent School District Language Rating Scale English Spanish NAME________DOB________CAMPUS________IDF#_________ DATE________ 1 2 3 4 5 A. SENTENCE STRUCTURE: Always uses incomplete sentences with grammatical errors Frequently uses incomplete sentences and/or numerous grammatical errors Uses correct grammar; few errors of omission or incorrect use of prepositions, verb tenses or pronouns Above average oral language; rarely makes grammatical errors Always speaks in grammatically correct sentences B.VOCABULARY ABILITY: Always uses immature or improper vocabulary Limited vocabulary including primarily simple nouns; few precise, descriptive words Adequate vocabulary for age and grade Above average vocabulary; uses numerous precise descriptive words High level vocabulary; always uses precise words to convey message; uses abstractions Schrank, Fletcher, & Alvarado/ORAL LANGUAGE PROFICIENCY TESTS 61 C.RECALLING WORDS: Unable to call forth the exact words Often gropes for words to express himself Occasionally searches for correct word but adequate for age and grade Above average ability; rarely hesitates on the word Always speaks well; never hesitates or substitutes words D. TELLING STORIES: Unable to relate isolated facts Has difficulty relating ideas in logical sequence Average ability to tell stories Above average; uses logical sequence Exceptional ability to relate ideas in a logical and meaningful manner E. IDEA FORMATION: Unable to relate isolated facts Has difficulty relating ideas in logical sequence Usually relates facts into meaningful ideas; adequate for age and grade Relates facts and idea well Outstanding ability in relating facts appropriately F. SPEECH: Speech is always unintelligible and/or dysfluent Speech is difficult to understand; sometimes dysfluent; draws attention to itself Speech is intelligible; fluent; adequate for age Speech is usually clear and easy to understand Speech is always distinct and fluent Procedure Kindergarten sample . Data were collected during the Fall of 1991 (September, October, and November) on 77 kindergarten students in five different classrooms of one elementary school in a large urban district. The subjects selected were identified as having a primary home language other than English on school enrollment forms and home language surveys. They were assessed for English and Spanish language proficiency using the PRE-LAS and the WLPB-R (the Spanish- language correlations are not included in this analysis). Three trained personnel administered the oral language proficiency tests in a counterbalanced order as a control for a possible ordering and learning effect. After the testing was completed, the Language Rating Scale was explained to the kindergarten teachers and they were asked to complete a form for each student in the study. 64 BILINGUAL RESEARCH JOURNAL/Winter 1996 10 intercorrelations). Correlations among LA S-0 subtests range from .39 to .95 with a median correlation of .81 (based on 6 intercorrelations). This result is due to the rather low correlations of the LAS-O Pronunciation subtest with the other LAS-O subtests. Among the correlations between the WLPB-R Oral Language and LAS-O subtests, the highest correlations include .88 between the WLPB-R Listening Comprehension and the LAS-O Vocabulary and .86 between the WLPB-R Picture Vocabulary and LAS-O Vocabulary scores. The lowest correlation is .44, between the WLPB-R Verbal Analogies and the LAS-O Pronunciation. The test correlations obtained from this study provide overall evidence of concurrent validity among the three tests. The IPT-I, LAS- 0, and WLPB-R Oral Language total scores correlated highly at the Kindergarten level and well at the Grade 2 levels. Differences among correlational patterns were found, however, when the subtests from one test were compared to those of another. For example, at the Kindergarten level, these correlations provide evidence to suggest that the WLPB-R Oral Vocabulary and the Pre-LAS Simon Says subtests measure different aspects of oral language proficiency (correlation .36). The LA 5-0 Simon Says is a receptive test of ability to follow instructions; this may be explained as a component of BICS. The WLPB-R Oral Vocabulary subtest measures knowledge of synonyms and antonym 5; this may be explained as a component of CALP. At the Grade 2 level, differences were also found. The WLPB-R Verbal Analogies and LAS-O Pronunciation appear to be measuring different aspects of language proficiency, as suggested by a low correlation (.44). The LAS-O Pronunciation test measures auditory discrimination of minimal pair items and ability to listen to and repeat specific phonemes embedded in words, phrases, and short sentences. This is a component of communicative language, and can be interpreted as a component of BICS. The WLPB-R Verbal Analogies measures the ability to comprehend and verbally complete a logical word relationship. This is an ability necessary for success in school, and can be described as a component of CALP. The Language Rating Scale (LRS) was used as a teacher rating of each student's language proficiency. At both the Kindergarten and Grade 2 levels, the WLPB-R Oral Language correlated most strongly Schrank, Fletcher, & Alvarado/ORAL LANGUAGE PROFICIENCY TESTS 65 with the teacher's ratings of language proficiency. This may also provide evidence that the WLPB-R Oral Language tests measure CALP better because these tests showed the strongest relationship with the teacher's ratings of English language proficiency. Implications In the final analysis, test validity can only be established within the context of the purpose of assessing oral language proficiency - to determine if children are able to meet the academic task demands of a mainstream English instruction classroom. Individuals responsible for selecting oral language proficiency tests for use in making high stakes, irreversible decisions regarding service eligibility either for instruction in a language other than English or a transition to English-only instruction should carefully examine test content to ascertain if the test is measuring cognitively-demanding academic content, or CALP. Measures of BICS may lead educators to assume that a student possesses CALP. This is not an innocuous assumption, as it may lead to incorrect placement decisions, denial of services, or a failure to succeed with language-related academic learning tasks. If oral language proficiency tests are to be used to determine if language-minority school-aged children are able to meet the academic task demands of monolingual instruction in English, then the test's task demands should also be academic in nature. Test users should examine each test manual for evidence of concurrent validity to measures of cognitive ability and academic achievement, as well as to other measures of language proficiency. If such evidence is not presented, users have no way of knowing if the test correlates well with established measures of cognitive-academic language proficiency. References Ballard, W. S., Tighe, P. L., & Dalton, E. F. (1980). IDEA Oral Language Proficiency Test I. Brea, CA: Ballard & Tighe. Burt, M. K. Dulay, H. C., & Hernandez Chavez, E. (1978). Bilingual Syntax Measure. San Antonio, TX: The Psychological Corp. Cummins, J. (1984). Bilingualism and special education: Issues in assessment and pedagogy. Austin, TX: Pro-Ed. 66 BILINGUAL RESEARCH JOURNAL/Winter 1996 De Avila, E. A., & Duncan, S. E. (1991). Language assessment scales. San Rafael, CA: Linguametrics. Dunn, L. IA., & Markwardt, F. C. (1970). Peabody Individual Achievement Test. Circle Pines, MN: American Guidance Service. Educational Testing Service. (1987). Test of English as a foreign language. Princeton, NJ: Author. Hamayan, B., Kwiat, J., & Perlman, R. (1985). The identification and assessment of language-minority students: A handbook for educators. Illinois Resource Center, Arlington Heights, Illinois. Hammill, D. D., & Larsen, S. C. (1983). Test of Written Language. Austin, TX: Pro-Ed. Herbert, C. H. (1979). Basic Inventory of Natural Language. San Bernadino, CA.: CHECpoint Systems. Houston Independent School District. Language Rating Scale. Houston, TX Kaufman, A. S., & Kaufman, N. L. (1983). Kaufman Assessment Battery for Children. Circle Pines, MN: American Guidance Service. Kaufman, A. S., & Kaufman, N. L. (1985). Kaufman Tests of Educational Achievement. Circle Pines, MN: American Guidance Service. McCarthy, D. A. (1972). McCarthy Scales of Children's Abilities. San Antonio, TX: Psychological Corporation. Semel, E., Wug, E., & Secord, W. (1987). Clinical Evaluation of Language Fundamentals-Revised. San Antonio, TX: Psychological Corporation Thorndike, R. L., Hagen, E. P., & Sattler, J. M. (1986). Stanford-Binet Intelligence Scale-Fourth Edition. Chicago: Riverside. Wechsler, D. (1974). Wechsler Intelligence Scale for Children-Revised. New York: Psychological Corporation. Woodcock, R. W. (1991). Woodcock Language Proficiency Battery Revised. Chicago: Riverside.