Measuring Family Background: History, Indicators, and Importance in Education Research, Lecture notes of Religion

Download Measuring Family Background: History, Indicators, and Importance in Education Research and more Lecture notes Religion in PDF only on Docsity! 21 3. Measuring Family Background When measuring students’ family backgrounds, researchers and analysts need to take several aspects into account. Examples are the inclusion of those backgrounds within the broader theoretical framework, as well as diverse operational issues such as question formats and who to interview (i. e., students themselves, their parents, or other people connected to the students). Conducting international large-scale assessments in education brings to the fore other specific issues. The international scope of these assessments necessitates checking for cross-country validity—that is, determining if the measures really are measuring something that is comparable across the countries participating in the assessments. Assessing students in schools leads one to query if these students are the most knowledgeable persons to provide information about their families or parents, a concern which then poses questions about the validity of the students’ answers. In this chapter, we provide an overview of these aspects of measuring students’ family background, keeping our focus throughout on international large-scale assessments in education. It needs to be noted here that most of the research done on the measurement of family background is based on the work of researchers from the United States and Europe. It is therefore possible that indicators fit better in these parts of the world than in others. This consideration is especially pertinent for international large-scale assessments because the participation of countries from diverse regions of the world is likely to raise issues of validity. Any international assessment needs to be aware of this matter and should carefully validate the indicators that are used. 3.1 the Development of Measuring Family Background The following section presents a short review of the history of conceptualizing and measuring the characteristics of family background. Included are the occurrence of different approaches and aspects, their theoretical underpinnings, and approaches to measurement that have been evolving over the decades. Some of these characteristics are interrelated because ambiguous theoretical borders sometimes cause researchers and authors to use certain terms interchangeably. Theorists can also differ in how they define different constructs, as can researchers as they seek to operationalize and define the variables used to measure the different constructs constituting the broad concept of “family background.” 22 IERI MONOGRAPH SERIES: ISSUES AND METHODOLOGIES IN LARGE-SCALE ASSESSMENTS SPECIAL ISSUE 3.1.1 Social class and Social Stratification Social stratification is one of sociology’s central terms. It is used to describe the society (or community) in which particular individuals, families, or groups are ranked in hierarchical structures that help determine who controls access to socially valued attributes such as power, wealth, and status. The position of the individual in the hierarchy relates to his or her SES (Mueller & Parcel, 1981). Modern societies can be described as internally divided (or stratified) into groups based on certain characteristics of their members. One of the principal bases and, at the same time, outcomes of such stratification is class. Hobsbawm (1972) realized that classes establish themselves when groups of people acquire the consciousness of themselves as classes. The separation into classes comes into play when the classes relate to one another through interaction in society (Bond, 1981). Max Weber saw social class as a quasigroup or aggregation of individuals governed by its own principles and in close relation to the market. This association of class with the market shifts the meaning of class to the economic position of its members. The individuals share common traits and occupy certain positions in society. According to Warner (cited in Bond, 1981), social class comprises people ordered in socially superior and inferior positions and who are thereby ranked and perceived as different by the members of the community as a whole. It is a widespread opinion that members of a given social class and their children will reproduce the class itself, with that process relevant not only for individuals but sometimes for whole institutions. Thus, within the sphere of education, schools located in working-class areas tend to prepare their students for working-class jobs (McGregor, 1997). Only a few such students are likely to have the ambition and resources to continue their education in universities (Pearce, Down, & Moore, 2008). The relationship between social-class position and academic achievement in schools was particularly observable until at least the middle of the 20th century in England and in some other European countries. Class origin had considerable influence on children’s success in applying for “grammar schools,” despite protestations that any child who passed the selection examination could gain entry (Bond, 1981). Regardless of whether social class has a direct effect on student achievement, differentiation based on social classes may influence, among other things, the individual’s aspirations for higher levels of education. As Hatcher (1998) pointed out, even when working-class students have the same level of abilities and achievement as children from higher social classes, they tend to be less motivated to continue their education. Furthermore, parents of children from higher social classes generally have higher expectations than parents from lower classes in regard to their children’s achievement (Coleman, 1985), a characteristic that influences school choice and often leads to social and ethnic segregation across schools. Various studies in education have included social stratification measures, most of which use a single variable, typically occupation, income, or education. Others use scales that combine the information associated with different variables such as education, source of income, type of housing, and type of neighborhood. Striker 25 MEASURING FAMILy BACKGROUND The fourth argument used to explain variation in student achievement focuses on unequal treatment at school. According to this approach, formal schooling will maintain class inequalities as long as class differences are present in society. Thus, a child who comes from a poor family will demonstrate lower performance in school. According to Bowles (1997), society seeks to maintain its social separation of labor and control over production. Formal education, as embodied in schools, is a means of maintaining this division and thereby ensures political and social stability. This mechanism not only gives more distinct shapes to the classes in a society but also solidifies the values and traits of each, such that they become subcultures (Bond, 1981, p. 245). 3.1.3 cultural and Social capital Beyond economic indicators such as family income, family background is also thought to encompass more ambiguous but important “capital,” such as cultural and social capital. Bourdieu (1986) is considered to be the father and the main developer of the theory of forms of capital, although the term “social capital” appeared as early as 1920. In Bourdieu’s understanding, capital is accumulated labor that can occur in three forms: economic (can be converted into money and institutionalized into property), cultural (can be institutionalized in educational qualifications and converted into economic capital), and social (is made up of social obligations or connections and can be converted into economic capital and institutionalized). According to Bourdieu (1986), cultural capital exists in three interrelated states. The embodied state describes the long-lasting dispositions of the mind and the body. Its acquisition depends on class, period, and society and is unconscious. In parent–child relationships, children’s unconscious adoption of their parents’ dispositions is part of the reproduction effect that many critical theorists stress. Furthermore, because cultural circumstances shape the acquisition of cultural capital in its embodied state, individuals conducting crossnational research need to take differences in cultures and societies into account. In the objectified state, cultural capital objectifies itself in material objects that are seen as the media of the culture, such as paintings, writings, and monuments. But, unlike the embodied form, they can be transferred, just like economic capital. These cultural products can be acquired both materially (as economic capital) and symbolically. In the third state, the institutionalized state, cultural capital is objectified in the form of academic qualifications. Academic qualifications are certificates of cultural competence that confer guaranteed cultural value on their owners. Academic qualifications provide their holders with a means of comparing themselves with and against others (Bourdieu, 1986). Social capital is an aggregate of relations (even only potential ones) within a social network (either institutionalized or informal, or simply presumed) and group membership(s). The groups may be ones that are established institutionally and labeled with a name, for example, family name, school name, political party affiliation. These relations are based on material and symbolic exchanges (Bourdieu, 1986). 26 IERI MONOGRAPH SERIES: ISSUES AND METHODOLOGIES IN LARGE-SCALE ASSESSMENTS SPECIAL ISSUE The central part of Bourdieu’s theory focuses on social networks. Network connections are not ones given by default, naturally, or by social means, but are the product of permanent effort (either conscious or unconscious). The reproduction of social capital presupposes expenditure of time, energy, and economic capital, but cannot be effective without investment of specific competence. For Bourdieu, it was clear that economic capital is the root of all other types of capital. The transformation of economic capital, however, does not happen automatically or spontaneously without labor. Instead, it is a time-consuming pursuit that needs attention and care (Bourdieu, 1986). Another author who is considered a founder of the theory of social capital is Coleman (1988). He argued that social capital is intangible and exists in three forms: the level of trust evidenced by obligations and expectations, information channels, and norms and sanctions aimed at promoting good for all rather than just for the individual. Like Bourdieu, Coleman stressed the importance of networks, especially the connections across generations. He gave as an example parents who know the parents of their children’s friends. In this kind of setting, he saw a social structure that facilitates the emergence of norms. Coleman (1988) challenged Bourdieu for not distinguishing social capital and the resources or means by which it is obtained. Coleman argued that having greater access to social resources because of their availability within one’s own network does not necessarily mean that someone without those advantages in his or her social network is less eligible to procure them. Coleman also disagreed with the notion that the dominant class is able to reproduce itself because of its members’ ability to secure social capital. According to Coleman (cited in Dika & Singh, 2002), social capital is social control, but trust, norms, and information channels are also characteristics of a society. As Lareau (2001) explains, Coleman stressed the family’s responsibility to adopt certain norms so that their children benefit by having better chances in life, whereas Bourdieu maintained that structural constraints and factors such as class, gender, and race are what influence access to institutional resources. Compared to the other theories/constructs described so far (SES, for example), the concepts of social and cultural capital are relatively new. Their influence on the field of education, as well as the onset of their measurement, came later, at the end of the 1980s. Nevertheless, as Dika and Singh (2002) point out, the measurement of social capital that took place in educational research was comparatively easy and straightforward. In 1988, Coleman used data from the High School and Beyond (HSB) study to explore the relationship between social capital and educational outcomes. He used different measures (both parents at home, number of siblings, parental educational expectations, closeness between the generations) to argue that higher accrual of social capital leads to lower rates of school dropout. Many subsequent educational studies adopted Bourdieu’s approach for measuring social capital. The researchers involved in these studies included, as measures of social capital, language used in classroom instruction, career decisionmaking, academic 27 MEASURING FAMILy BACKGROUND discourse, and relations between schools and families. Educational sociologists have also applied Bourdieu’s approach to social and cultural capital in their research directed toward explaining differences in schools, and using class, gender, race, and ethnicity as the characteristics of interest (Dika & Singh, 2002). De Graaf, De Graaf, and Kraaykamp (2000) investigated the effects of parental beaux arts participation (i.e., attending art galleries, museums, opera, ballets, plays, cabarets, and classical music concerts) and parental reading behavior (as aspects of parental cultural resources) on educational attainment. Lee and Bowen (2006) used different aspects of parental involvement at school to explore the influence of these aspects on students’ achievement. They distinguished levels of parental involvement in terms of differences in parents’ habitus and considered the extent to which the effects of parental involvement reflected differences in cultural capital. Although our paper concentrates solely on quantitative approaches, it is pertinent to note that Bourdieu’s and Coleman’s concepts of economic, social, and cultural capital have also been used in qualitative and ethnographic approaches (see, amongst others, Lane & Taber, 2012; Lareau, 1987; Lareau & Weininger, 2003). As Bourdieu (1986) himself stated, the best measure of cultural capital is the time invested in obtaining it. The reason why it is the best relates to the fact that time needs to be invested to transform economic capital into cultural capital, which, in turn, again needs economic capital. The cultural capital of a family is thus obtained by investing time, and time can also be spent spreading it among family members— on transferring it from one family member to another (Bourdieu, 1986). However, for Vryonides (2007), measuring both social and cultural capital raises the issue of operationalization of the constructs to be measured and the selection of variables. Vryonides delineates the different understandings of social capital between the two major theorists (Bourdieu and Coleman) in the following way: Bourdieu saw social capital as an aggregate of resources (actual or potential), linked to possessions steadily accumulated within a network of institutionalized relationships, whereas Coleman located social capital within the family (as a structure and as relationships between generations) and outside of it (the social ties outside the family that create trust and obligations and impose norms). Sullivan (cited in Vryonides, 2007) contradicts Bourdieu’s assumption regarding cultural capital that higher parental education indicates higher levels of cultural capital. She states that such an assumption is inadequate and misses the broadness of the concept. Cultural capital, she continues, has been operationalized in many different ways by many different researchers. As Vryonides (2007) points out, the most likely reason for this diversity is that the concept of cultural capital is very broad and not easy to quantify. Some researchers at the end of the 20th century and the beginning of the 21st century used measures such as frequency of visiting “high” cultural events (e.g., classical concerts, opera, museums, and art galleries) as well as reading habits in the family. For Vryonides (2007), this kind of approach is too narrow a way of measuring cultural capital at home, especially with respect to Bourdieu’s definition of the concept. 30 IERI MONOGRAPH SERIES: ISSUES AND METHODOLOGIES IN LARGE-SCALE ASSESSMENTS SPECIAL ISSUE Another problem that arises in regard to measuring household income lies with missing data due to nonresponding participants, an absence which also affects the reliability of the measure. Participants in surveys can be very sensitive about questions regarding their economic situation and often refuse to answer when they are asked to provide more precise information. For example, in the National Survey of Family and Households reported by Demo and Acock (cited in Ensminger et al., 2000), 17.5% of the mothers did not report their family income. Another challenge with income as an indicator lies with comparability across countries participating in international comparative research. The income of families can easily be compared on a national level. But at the international level, absolute income is difficult to compare because of different currencies, let alone fluctuations in currency exchange rates. In educational research, an additional issue regarding the administration of questions about family income emerges. Children might simply not know about their parents’ income. Thus, the reliability of the information provided becomes questionable. This issue could be addressed by surveying the parents directly. However, along with potential problems arising out of respondents’ unwillingness to provide information about such sensitive data, there is the additional effort needed to develop and administer a parental questionnaire, not to mention the likely significant rise in administration costs. Overall, there seems to be a good many issues associated with using income as a variable of family background. Researchers have attempted to address these various problems by using certain indicators as a proxy for income, such as poverty status, housing tenure, or participation in school-based free lunch programs. However, as Hauser (1994) discussed, these proxy measures introduce additional problems that make their utility less than ideal, with crosscountry validity being the most critical one. 3.2.2 occupation Another often-used indicator of family background is occupation. The traditional approach when measuring occupation (along with other characteristics) is to use only the occupation of the head of the household. Usually, this person is assumed to be the father or the “father figure”—the main person in the household responsible for the socioeconomic wellbeing of the family. When the father’s occupation is not available, the mother’s occupation is often taken as a substitute. But the relevance of this approach has become increasingly questionable. The employment situation in households has changed significantly over the last decades, especially with mothers increasingly contributing to family income. Nevertheless, as Entwisle and Astone (1994) have pointed out, even though women in modern society have jobs, and often the job is one with relatively high occupational prestige, they still tend to be paid less than men. Today, the theory and practice of measuring SES-related components emphasizes the considerable importance of occupational status compared to financial resources and education. The stability of this indicator, as already discussed, is the main reason 31 MEASURING FAMILy BACKGROUND for this change. A stable occupation status shows a certain position in the hierarchy of a society and also shows the economic situation. However, May (2002, p. 130) suggests that more variant indicators contain additional important information related to educational outcomes and wealth. Different family indicators may reveal important facts about the current economic situation that change over time, while occupational status scales may mask this important variation. Usually, for analysis purposes, researchers group similar occupations into larger categories that can mask differences in educational attainment or the current economic situation underlying these occupations. Scales that contain family-specific indicators normally reduce the measurement error arising from the aforementioned problem. Earlier in this paper, we stated that occupation is a preferable measure compared to income because it is more stable over time. But is occupational status really so stable? In 1995, Swinnerton and Wial published their research on job stability among low- seniority workers in the U.S. The data showed that occupational stability in general can decline for certain periods of time depending on the factors related to what the two authors called a “business cycle.” The authors suggested that there is a general tendency toward increasing job instability in the U.S. (Swinnerton & Wial, 1995, p. 303). Nevertheless, occupational status still remains one of the most stable indicators of family socioeconomic status. Occupation has huge potential as a measure of family background. It could be operationalized as an indicator of family wealth or prestige. It could also relate to social capital, given that some occupations imply a higher level of connectedness with other people or societal institutions. That said, obtaining the information about parental occupation has some challenges. While respondents usually do not see their occupation as sensitive information, unlike income, and (older) students can usually provide reliable occupation information with regard to their parents (see Chapter 3.5), occupational data always need to be transformed into a measure that can be used for comparisons. Usually, occupation data are grouped and afterwards scaled (see Chapter 3.3). Depending on the purpose of the research, this grouping can differ from study to study, reducing comparability. Furthermore, in international settings, the ascription of value, prestige, or other characteristics used for comparison purposes can vary around the world. Using occupation at the international level therefore requires knowledge about these differences. Furthermore, these ascriptions generally change over time. Lastly, the business world is constantly developing, creating new jobs and changing the world of labor, which poses yet another challenge to comparisons across time. Nevertheless, data on occupation are relatively easy to collect, researchers worldwide are addressing the challenges mentioned above (see especially the authors deriving indices and scales from the occupation data mentioned in Chapter 3.3), and information on parental occupation has shown, overall, to be a valuable indicator of family background both in the past and currently. 32 IERI MONOGRAPH SERIES: ISSUES AND METHODOLOGIES IN LARGE-SCALE ASSESSMENTS SPECIAL ISSUE Industry Several authors, among them Duncan and Magnuson (2003), Entwisle and Aston (1994), Hauser (1994), and Hauser and Warren (1997), argue that using only occupational status and job type is not sufficient, and that researchers should also collect information about the industry in which each parent works. The issue that is of interest here relates to occupational prestige. As Entwisle and Astone (1994) state, for some occupations, job prestige depends on the industry itself. Researchers must know the industry to properly assign the occupational prestige of parents. Industry, according to Hauser and Warren (1997), is an abstract category. It is used to group and classify products and services, despite there being complex interdependencies between job and industry classifications. Hauser and Warren discuss in detail the necessary coding procedures, which, as they demonstrate, can be complex and time consuming. The problems related to collecting data on industry are manifold. As Entwisle and Astone (1994) remind us, industry is rarely, if ever, used directly in analyses of data, other than in terms of defining the prestige of an occupation. Furthermore, children cannot be reliably expected to provide the name of the industry or industries their parents work in. Finally, the issue of comparability of industry classifications at the international level remains. Categories of industry are likely to have a different connotation or even a different prestige in different countries. 3.2.3 Household possessions When discussing cultural capital, we mentioned that its objectified state concerns the material objects that serve as a media for a culture (e. g., paintings, writings, monuments, etc.). According to yang (2003), the number of books at home (among other cultural possessions) is hypothesized to be a measure of the cultural and educational resources at home, although these indicators vary greatly across countries with regard to the cultural value they represent. Furthermore, the number of books at home indicates parents’ emphasis on education or intellectual activities beyond books used as educational resources. As Elley (1992, p. 14) pointed out, “The availability of books is a key factor in reading literacy. The highest scoring countries typically provide their students with greater access to books in the home, in nearby community libraries and book stores, and in the school.” In large-scale education studies, students are often administered questions concerning their home possessions, household composition, and parents. yang and Gustafsson (2004) argue that data on home possessions collected from young children are much more reliable compared to the information they provide about their parents’ education, jobs, and income. It seems that children find it much easier to grasp the notion of the material possessions they have daily contact with at home than to understand notions of parental education, job, and income. So, regardless of the criticism about the reliability of data on possessions provided by students, we can assume it is a valuable source of information in the absence of parents either not being asked or not answering questions about these matters. 35 MEASURING FAMILy BACKGROUND 3.2.6 educational attainment of parents There is a clear indication from research in education that educational attainment of parents is related to students’ academic achievement: The higher the parental attainment is, the higher their children’s achievement tends to be. Some studies from the past show that the mother’s educational attainment has a stronger relationship with student achievement than does the father’s educational attainment (Gorman & yu, 1990). Additional to the effect on student achievement are the strong correlations between parental educational attainment, parental teaching styles, home-learning environment, and student behavior (Duncan & Magnuson, 2003). Studies on educational achievement usually also seek information about the educational attainment of parents. PIRLS, TIMSS, and PISA, for example, all include questions about the highest level of education attained by each student’s parents. Parents’ educational attainment is one of the central characteristics of family background with regard to explaining variation in student achievement. It is an indicator of the institutionalized state of cultural capital, because educational achievement is usually awarded with certificates, which then become prerequisites for access to labor markets. Collecting information at the international level on parental educational attainment has become a standard practice in comparative educational research. In order to allow valid comparison of the variety of educational attainments and (school) education pathways evident worldwide, UNESCO developed a classification scheme, the International Standard Classification of Education (ISCED), which is well proven, accepted, and commonly used around the world. It was last updated in 1997 (UNESCO, 1997). 3.2.7 Neighborhood Neighborhood characteristics, such as neighborhood income and SES, as well as the level of social (dis-)organization are also important characteristics of a student’s family life. Neighborhood SES is usually measured as the proportion of people of 20 years of age who have, according to their nation’s census, not completed high school, but this measure is usually seen as a weak one. Neighborhood SES and family SES tend to be closely related. Family SES, for example, generally indicates the type of area in which the family resides as well as the school that the children will attend. Family, as previously discussed, also directly or indirectly determines the provision of home resources and social capital—the supportive relationships among structural forces and individuals. Some studies have found that the predictive power of neighborhood characteristics on achievement is stronger than the predictive power of family SES (see, for example, Sirin, 2005). Similarly, schools that have higher percentages of students from high- SES families maintain a better peer culture and have a better learning environment, advantages which often lead to higher academic performance (yang-Hansen, 2008). Additionally, relationships appear to exist between the characteristics of the 36 IERI MONOGRAPH SERIES: ISSUES AND METHODOLOGIES IN LARGE-SCALE ASSESSMENTS SPECIAL ISSUE community, the school’s SES, and the academic achievement of students, with these relationships being strongest in decentralized education systems (yang-Hansen, 2008). The theory of social disorganization states that the structural characteristics of the neighborhood define its social organization (see Kohen, Brooks-Gunn, Leventhal, & Hertzman, 2002). Such characteristics are poverty, residential instability, ethnic heterogeneity, and single parenthood. The social organization of a neighborhood defines the norms and values within it as well as the behavior of the residents and the state of public order. Most empirical studies use mainly census data to elucidate the sociodemographic characteristics of the neighborhoods in which families live. These studies adopt the variables mentioned above and include unemployment. Some researchers have criticized this approach in terms of it gathering incomplete data (see Kohen et al., 2002). The alternative approach focuses on collecting information through community surveys and systematic social observations. Kohen et al. (2002), among other researchers, suggest surveying cities in order to examine neighborhood context. They also promote conducting ethnographic studies in order to examine in depth the social structure of particular neighborhoods. Such ethnographic studies usually focus on low income and urbanization. The criterion of neighborhood in international large-scale education studies is also conceptualized as an attribute of the school rather than of the individual. In PIRLS and TIMSS, school principals are asked to provide information on the proportional distribution of some family background characteristics among the students attending the school. For example, TIMSS 2007 collected information about the percentages of students coming from economically disadvantaged homes, as perceived by the school principal. On average across the participating countries, this criterion showed a statistically significant negative relationship with mathematics achievement (Mullis et al., 2008, pp. 317–318.). However, because it is an aggregate measure at the school level, it cannot be used as an indicator of family background at the individual level. 3.2.8 Religion When measuring socioeconomic status from a crossnational perspective, Wolf (2005) included variables concerning religiosity. The relationship between individuals and religious groups can be described in two ways—first, whether the respondent is a member of a religious group and, second, whether the respondent identifies himself or herself with a religious group. The term “membership” needs to be refined, however, and not left to the respondent to interpret its meaning. Religion was included in IEA’s International Civic and Citizenship Education Study (ICCS) 2009 assessment as an indicator for students, not families (see Schulz, Ainley, Fraillon, Kerr, & Losito, 2010). But because religious affiliation is usually transferred from parents to their children and because that transference may not hold for the student’s actual religious beliefs, it might better serve as an indication of the family’s tradition regarding religious affiliation. 37 MEASURING FAMILy BACKGROUND Similar to income, information about religion is sometimes seen as sensitive. Additionally, data protection regulations in several countries do not allow researchers or other agents to collect this kind of information. In ICCS 2009, for example, questions about religion were designed as national options: 28 of the 38 participating countries chose to administer these questions (Schulz et al., 2010, p. 33). 3.3 indices, Scales, and other combinations of components In order to collect information on family background, researchers have developed various scales designed to measure various family characteristics. The broadest one in terms of concept is socioeconomic status (SES), which is a more or less overarching name for various and quite different measures, including all possible combinations of social and economic indicators. Much more clearly defined indices and scales exist that include parental occupation. Occupational positions in the stratification system of sociology are measured using one of the following three approaches: prestige ratings, sociologically derived class measures, and socioeconomic status scores. For each of these approaches, measures exist that are widely used in international comparative research. One such is the Standard International Occupational Prestige Scale (SIOPS). It measures occupational prestige according to a rating of occupations. Another measure, the EGP (named after its authors—Erikson, Goldthorpe, and Portocarero), is an occupational class scheme. The International Socio-Economic Index of Occupational Status (ISEI) follows the third approach—a socioeconomic status score (Ganzeboom, De Graaf, & Treiman, 1992). Two additional types of scales associated with PIRLS and PISA receive more consideration at the end of this section. Suffice to say here that the PIRLS Early Home Literacy Activities (EHLA) scale was developed to measure activities and resources in the child’s family relative to the child’s reading capability (Martin, Mullis, & Kennedy, 2007, p. 201), while PISA’s Index of Economic, Social, and Cultural Status (ESCS) endeavors to capture a combined measure of all three forms of capital (OECD, 2009, p. 346ff.). 3.3.1 Socioeconomic Status Socioeconomic status (SES) is by far the most prominent and widely used latent construct for measuring family background. It is also the least well-defined concept. Introduced into the U.S. in the late 1960s, SES as a construct was recognized, by the 1980s, as problematic because researchers were underpinning it with a variety of different combinations of variables, creating ambiguity in the interpretation of research results. The same conclusion holds nearly a quarter of a century later (see Sirin, 2005). As we discussed earlier, many researchers continue to use the terms SES and social class interchangeably when referring to the social and economic characteristics of the student (or rather the student’s family), but they neither clarify their understanding of these terms nor provide a rationale for using them in this way. According to Merola (2005), measures of SES are usually derived from educational attainment, occupational status, and financial resources. However, not all studies use all three components. For example, the U.S.’s National Assessment of Educational 40 IERI MONOGRAPH SERIES: ISSUES AND METHODOLOGIES IN LARGE-SCALE ASSESSMENTS SPECIAL ISSUE administrators, and higher-grade professionals to agricultural workers. As the creators of this class schema admit, problems of distinction are apparent across some of the levels (Erikson et al., 1979). In the early 1990s, Erikson and Goldthorpe (1992) reworked the scheme, using data from the Comparative Analysis of Social Mobility in Industrial Nations (CASMIN) project. They connected the EGP scheme to the ISCO, a change that requires users to first code the occupation into the ISCO and then adjust it according to information about self-employment and number of employees supervised. The revised scheme consists of 11 classes, ranging from “higher managerial and professional workers” through to “manual supervisors” and “self-employed farmers.” The use of the EGP scheme in education studies poses the challenge of obtaining the information needed. In general, students are highly unlikely to have at hand information on parental self-employment, let alone the number of employees their parents supervise. Therefore, the information needs to be provided by the parents themselves, which necessitates administering a parent questionnaire with the known implication of additional development and administration costs. 3.3.5 index of early Home literacy activities (eHla) As mentioned before, the concept of cultural capital not only concerns possession of cultural items at home, but can exist in three states. One of the three, the embodied state of cultural capital, can be described as the long-lasting dispositions of the mind and the body being inevitably related to the person (Bourdieu, 1986). Bourdieu saw the time investment as the best measure of cultural capital. Following this definition, the researchers involved in some studies attempt to attain information about this embodied state of cultural capital. For example, in the PIRLS 2006 assessment framework (Mullis, Kennedy, Martin, & Sainsbury, 2006), the background context was divided into different aspects of reading literacy. One of these—the home context— consists of a separate aspect called “social and cultural resources.” For this aspect, the authors of the framework delineated the necessity of collecting data on the literacy activities of parents with their children that would likely foster positive attitudes among the latter toward reading. These activities, identified as direct guidance, modeling, and support of children’s behavior, convey parents’ beliefs and attitudes toward literature and reading (Mullis et al., 2006). The framework authors then took the information on the frequency of occurrence of these early literacy activities with the children at home (reading books, telling stories, singing songs, playing with alphabet toys, playing word games, reading aloud signs and labels, etc.) and used it to derive a new index called the Index of Early Home Literacy Activities (EHLA) (Martin et al., 2007, p. 201). These items and the EHLA are meant to measure the building of embodied cultural capital directed toward or related to literature, reading, and language. Because the information needed relates to the time before children enter school, it is highly likely only the parents will recall if they carried out such activities with their child. Again, additional effort, through the use of a parent questionnaire, is needed 41 MEASURING FAMILy BACKGROUND to collect the data. But even then, the longer the span of time that has elapsed since early childhood, the more difficult it is likely to be for parents to recall the activities, resulting in a higher proportion of missing data. In terms of association with learning outcomes or achievement in school, the influence of early home literacy activities will likely wane as the child gets older (i.e., the longer the child attends school), with the most obvious examples of possible influence encompassing schooling. But there is a special feature of the EHLA. Unlike the other indices mentioned so far, the EHLA is directed toward a specific (and desired) outcome of education—the reading ability of the child. Although reading or reading literacy can be seen as the most basic ability shaping much of a person’s living conditions (Mullis et al., 2006), reading-literacy-related activities are clearly goal-oriented and targeted at the child. As such, these activities clearly relate to a specific learning area. Another difference in terms of the family background measures mentioned previously is that the decision of whether or not to engage in these activities is a relatively easy one. In contrast, deciding to change a job usually poses a much higher burden on a person because it can involve uncertainty about future employment and salary, and also because other people need to agree to the change (e.g., to employ the person). Even less self-determined is the increase in one’s income, assuming the change in employment eventuates. 3.3.6 the piSa index of economic, Social, and cultural Status (eScS) The research team involved in PISA derived an index from various indicators representing economic, social, and cultural status. The composition of the index, originally developed for the first cycle of PISA in 2000, was changed for the 2003 and 2006 cycles (see OECD, 2009, p. 346ff.). The index uses information on home possessions and also takes into account the highest-level occupation and the highest- level education of both parents, expressed as years of schooling. The PISA Index thus consists of the “classical” components of an SES scale (see Chapter 3.3.1). Home possessions has continued to serve in the scale as a proxy for income, because no direct data on income are available. In the PISA cycles, all the information included in the ESCS index is gathered from student responses. Although a parent questionnaire is offered as an international option, few countries administer it. Because home possessions (as the substitute for the information on income) can be collected from student responses, the index involves few, if any, major challenges for survey administration. 3.4 the Multilevel Nature of Family Background As already mentioned, family background can be seen as comprising different levels. The usual approach focuses on the individual’s family background. Studies that involve the participation of students in school add two levels: the classroom and the school. Indicators at the individual level can also be aggregated to classroom or school level— as averages, for example. This section addresses the potential of taking the different levels into account. 42 IERI MONOGRAPH SERIES: ISSUES AND METHODOLOGIES IN LARGE-SCALE ASSESSMENTS SPECIAL ISSUE 3.4.1 individual, classroom, School, and Neighborhood During the past decades, multilevel techniques for analyzing hierarchical data made great progress. Their need arose due to the fact that individuals are part of a broader system. Students are nested in classes that are nested in schools. This kind of grouping in a hierarchical structure requires analyzing the effect of the higher levels on the lower ones. For example, the “school effect” would refer to the influence of certain school characteristics on student test scores or any other educational outcome at the student level. From the 1960s until the end of the 1980s, the main methods of analyzing school effects encompassed regression methods or analysis of variance. This approach inevitably posed methodological problems such as confounding the variation of the lower and higher levels of the hierarchy. Multilevel methods that were developed later helped overcome these disadvantages. For example, they allowed the variance to be partitioned between the higher and the lower levels, yielded more precise estimates with less bias, and produced more reliable information concerning within and between school effects (Everson & Millsap, 2004). However, a multilevel or hierarchical structure can be found not only in education systems but also in the social environment of individuals outside school. Each individual is acting in and is a member of multiple environments—families, groups of peers, communities, and neighborhoods. And each one of these environments shapes the individual’s behavior, beliefs, and values. For example, the higher the SES of an individual’s neighborhood is, the more “extra value” the neighborhood adds to his or her social and material resources (yang & Gustafsson, 2004, p. 262). Even social institutions treat residents from diverse neighborhoods differently, according them various priorities in varying kinds of social situations. This occurrence also influences education. Children coming from communities with high incomes will most likely attend schools of higher quality and meet other children with similar socioeconomic backgrounds, with the interaction between them likely increasing their occupational and educational aspirations (yang & Gustafsson, 2004, p. 262). Usually, children attend schools in their neighborhood. The characteristics of the school will simply reflect the community’s characteristics (sociocultural, economic, and ethnic). As yang and Gustafsson (2004) state, many previous studies suggest the need for multilevel ecological contexts and for investigation of the effects of micro- and macro-levels on educational achievement: “A multilevel structural equation modeling approach can decompose the total variation into different sources (i. e., dimensions) of variations at different levels” (yang & Gustafsson, 2004, p. 262). yang and Gustafsson (2004) identified different dimensions of home-possession items at both the individual and the school level across the countries participating in IEA’s Reading Literacy Study (RLS). The authors identified two different dimensions— academic and cultural, and economic and material—at the individual level for the majority of countries, but used different variables for measuring these latent background aspects. Nevertheless, they could identify only a single general capital factor at the within-school level in a good many of the participating countries, probably because the representation of different indicators of cultural capital was 45 MEASURING FAMILy BACKGROUND 3.5 administration issues In addition to these content-related issues and thoughts, researchers need to consider additional aspects when trying to obtain information about students’ family background. Survey design, questionnaire development, sampling, and respondent level are examples of organizational (as opposed to content-related) issues that might affect the quality of the measurement of family background. We briefly introduce these aspects in this section. 3.5.1 Reliability of Student information about parents As already mentioned, information about characteristics of students’ parents (such as their occupation) are generally obtained from the students. The question that needs to be asked here is whether student information on parental characteristics is reliable. As research shows (Baratz-Snowden, Pollack, & Rock, 1988), reliability depends, to a large extent, on the age of students: the older they are, the more likely they are to provide the same answer as their parents. A large proportion of students in elementary school do not know or simply do not answer questions concerning parental education. Referencing data from the U.S.’s High School and Beyond (HS&B) survey, Fetters, Stowe, and Owings (1984) analyzed student against parent responses and found that students provide highly reliable and valid data about the educational attainment of their parents. According to Hauser (1994), the information provided by students at ages 14 and 15 is about as reliable as that given by their parents. Evidence in support of this conclusion is also provided by validation studies regarding the agreement between students and their parents conducted in Canada, the Czech Republic, France, and the United Kingdom during the PISA 2000 cycle. These studies found that data on parental occupation provided by 15-year-old children could be used as a valid indicator of parental occupation (Adams & Wu, 2002, p. 220). There are, however, other issues regarding the extent of agreement between parents and children on family-background characteristics (see, for example, Baratz-Snowden et al., 1988). One of them concerns the question format itself: open-ended questions about parental occupation yield more valid data than multichoice questions. Another concern is that higher-achieving students provide more valid responses than lower- achieving students. Also, children of higher-educated parents give more accurate information. Issues related to ethnicity also influence the validity of home background information, and female students’ responses are generally more valid than male students’ responses. Another gender issue is that boys provide more valid information about their father’s than their mother’s education whereas girls provide more valid information about their mother’s than their father’s education. This outcome may be because the child identifies most with the parent of the same sex. The PISA 2006 technical report states that data from the study show a relatively high consistency between student and parent data on questions about education, and a somewhat lower consistency on occupation data (OECD, 2009, p. 57). Schulz (2006) argues that the inconsistency of the responses between students and their parents 46 IERI MONOGRAPH SERIES: ISSUES AND METHODOLOGIES IN LARGE-SCALE ASSESSMENTS SPECIAL ISSUE from the 15 countries that used the parental questionnaire in the PISA 2006 field trial is due to several reasons: a lack of precision in the instructions for filling out the questionnaire, the tendency to give socially desirable responses, and problems in coding the open-ended responses to questions about education and occupation. Schulz furthermore argues that parents’ responses could also be biased, and that this possibility needs to be acknowledged: “Apart from coding problems, lack of precision in the job descriptions and also tendencies to give socially desirable responses might affect the reliability or validity of parent responses” (Schulz, 2006, p. 15). Despite the considerable variation in the reliability of some of the PISA data, Schulz also observed in an earlier work that, in general, student reports about parental occupation are more reliable than the ones about parental education (Schulz, 2005). 3.5.2 Study and Questionnaire Design International large-scale assessments in education collect huge amounts of data. Family background is only one aspect of interest related to achievement. This fact makes the task of collecting relevant data in a limited time with instruments of a limited length and a limited number of questions a challenge. The difficulty of the task becomes even more apparent when one realizes the breadth and number of the different aspects that such research projects need to cover. Collecting data therefore needs to focus on the most important characteristics, and the survey instrument needs to contain a limited number of items that are measured very precisely. As Hauser (1994, p. 1541) said, the burden of data to be collected should be kept in bounds, and at the same time the focus should be “on characteristics that will be relatively easy to measure, that can be measured for every child in the survey, and that will probably not vary greatly over the short term.” The problem associated with the variation in background variables over time and their impact on achievement is influenced by countries’ general characteristics. We need to acknowledge that a universal recommendation about which and how many variables should be included in a questionnaire is not possible. Instead, the selection of variables should be done carefully and with regard to the aim(s) of the study. As Hauser (1994, p. 1545) further noted, “a standard set of racial/ethnic and socioeconomic variables, however well measured, cannot serve as all-purpose statistical controls for family background.” Large-scale assessments in education, as well as in any other area, are very time consuming and expensive. Some of them, such as PIRLS, TIMSS, and PISA, are international comparative studies that collect data from a large number of countries all over the world. Paper and pencil questionnaires serve very well in these huge endeavors of data collection that are conducted within a limited period of time. But at the same time, they have, as is the case with any other measurement tool, some disadvantages. Paper questionnaires handed out to parents in order to obtain information about family background characteristics are self-assessment tools. Self- assessment is subjective. At times, the quality of the instruments can also contribute to the bias. As already mentioned, the PISA 2000 student questionnaires have been criticized with respect to the precision of the occupational job descriptions, which 47 MEASURING FAMILy BACKGROUND posed a problem in terms of consistency of the data (Schulz, 2006). Due to restrictions in time and budget, personal interviews with parents are rarely conducted in large- scale assessments in education. Nevertheless, personal interviews could contribute to the quality of the background information collected. But they would also pose a big challenge for survey administration procedures. Because of our focus in this paper on the TIMSS, PIRLS, and PISA assessments, we now offer a brief overview of the most recent developments in these assessments with regard to the measurement of family background. As trend studies, both TIMSS and PIRLS need to employ, to a large extent, measures and items already used in previous cycles. Not to do so limits ability to report on trends. Nevertheless, there is some room for change or development. The TIMSS 2011 countries that administered TIMSS to Grade 4 students and that administered PIRLS to the same students were able to use the data from the PIRLS home questionnaire (Mullis, Martin, Ruddock, O’Sullivan, & Preuschoff, 2009). Building on experience gained in previous PIRLS cycles, the PIRLS research team administered the home questionnaire to students’ parents. Parents were asked to provide information, such as their occupation, additional to the information on family background gathered from students through the student questionnaire. This approach has enabled TIMSS to report, to some extent, on associations between occupational data and achievement, in line with one of the areas of interest delineated in the TIMSS 2011 framework. The framework’s authors set out evidence from the research literature on the strong association between student achievement and parental occupation (Mullis, Martin, Ruddock et al., 2009, p. 114). PISA relies also on gaining information from students about family background. With the exception of the 2003 cycle, countries participating in PISA can choose to administer a parent questionnaire to the parents of the participating students. Since 2006, PISA has asked parents to provide information on household income and expenditures for educational services (OECD, 2009, p. 60). IEA’s International Civic and Citizenship Education Study (ICCS), which collected data from mainly Grade 8 students in 2009, asked these young people about their knowledge of and attitudes toward democracy, civics, and citizenship. For the first time in recent IEA studies, the research team collected information about the occupation of the students’ parents via an open-ended question format that allowed students to enter the job title and a description of that job for both mother and father (or caregivers). Coding of the open-ended answers into an occupation classification scheme was followed by the assignment of ISEI scale scores (see Chapter 3.3.3). In addition, the research team created an Index of Family Socioeconomic Background, using information relating to parental occupation and level of education as well as the number of books at home (see Schulz et al., 2010, p. 222). All scales and indices based on occupation data showed strong relations with civic knowledge, indicating that the effort required to collect this kind of information is worthwhile.