Exploring Measurement & Uncertainty in Intro Physics: Study on Students' Misconceptions, Lab Reports of Physics

Download Exploring Measurement & Uncertainty in Intro Physics: Study on Students' Misconceptions and more Lab Reports Physics in PDF only on Docsity! Teaching Measurement in the Introductory Physics Laboratory Saalih Allie and Andy Buffler, University of Cape Town, South Africa Bob Campbell and Fred Lubben, University of York, UK Dimitris Evangelinos, Dimitris Psillos, Odysseas Valassiades, Univ. of Thessaloniki, GreeceTraditionally physics laboratory courses at thefreshman level have aimed to demonstratevarious principles of physics introduced in lectures. Experiments tend to be quantitative in nature with experimental and data analysis techniques interwoven as distinct strands of the laboratory course.1 It is often assumed that, in this way, students will end up with an understanding of the na- ture of measurement and experimentation. Recent research studies have, however, questioned this as- sumption.2,3 They have pointed to the fact that fresh- men who have completed physics laboratory courses are often able to demonstrate mastery of the mecha- nistic techniques (e.g., calculating means and standard deviations, fitting straight lines, etc.) but lack an appreciation of the nature of scientific evidence, in particular the central role of uncertainty in experi- mental measurement. We believe that the probabilis- tic approach to data analysis, as advocated by the In- ternational Organization for Standardization (ISO), will result in a more coherent framework for teaching measurement and measurement uncertainty in the in- troductory physics laboratory course. Over the past few years we have researched4–7 fresh- man physics students’ understanding of the nature of measurement. The group at Thessaloniki has probed students’ views of a single measurement. They con- cluded that, after completing a traditional laboratory course, the majority have ideas about measurement394 DOI: 10.1119/1.1that are inconsistent with the generally accepted scien- tific model.4 For example, a large fraction of students view the ideal outcome of a single measurement as an “exact” or “point-like” value. A sizeable minority feel that since the ideal is not attainable, only an unquantified “approximate” value can be obtained in practice. Only if a measurement is considered really “bad” would it then be reported in terms of an interval.5 The studies carried out by the Cape Town-York group have focused on aspects of dispersion in data sets. A model of student thinking has been developed that has been termed “point” and “set” paradigms.6 In brief, in the “set” paradigm the ensemble of data is modeled by theoretical constructs from which a “best estimate” and the degree of dispersion (an interval) are reported. However, the majority of students who arrive at university operate within the “point paradigm.”6 They subscribe to the notion that a “cor- rect” measurement is one that has no uncertainty asso- ciated with it. For many students, therefore, the ideal is to perform a single perfect measurement with the utmost care. When presented with data that are dis- persed, they often attempt to choose the “correct” val- ue (for example, the recurring value) from amongst the values in the ensemble. It was found7 that even af- ter a carefully structured laboratory course,8,9 most students had not shifted completely to “set” paradigm thinking. 616479 THE PHYSICS TEACHER
Vol. 41, October 2003 What Is Wrong with Traditional Teaching About Measurement? Although one of the most important aspects of putting together a teaching sequence is bringing to- gether the philosophy, logic, and modes of thinking that underlie a particular knowledge area, introducto- ry measurement is usually taught as a combination of apparently rigorous mathematical computations and vague rules of thumb. We believe that this is a conse- quence of the logical inconsistencies in traditional data analysis, which is based on analyzing the frequen- cy distribution of the data. This approach, often called “frequentist,” is the one used or implied in most introductory laboratory courses. In the frequentist approach, “errors” are usually introduced as a product of the limited capability of measuring instruments, or in the case of repeated measurements, as a consequence of the inherent ran- domness of the measurement process and the limited predictive power of statistical methods. These two different sources of “error” cannot be easily recon- ciled, thus creating a gap between the treatments of a single reading and ensembles of dispersed data. For example, the theory applicable to calculating a mean and a standard deviation is premised on the assump- tion of large data sets (20 or 30). Yet, when students perform an experiment in the laboratory, they often take five or fewer readings. Furthermore, there is no logical way to model statistically a single measurement within this approach. Traditional instruction usually emphasizes random error for which there is a rigorous mathematical mod- el, while systematic errors are reduced to the technical level of “unknown constants” that have to be deter- mined by examining the experimental setup. The concept of a “scale reading error,” usually taught at the beginning of the course, cannot be related to either random or systematic errors that are taught during the treatment of repeated measurements. Moreover, the term “error” misleads students by suggesting the exis- tence of true and false experimental results, possibly endorsing the naive view that an experiment has one predetermined “correct” result known by the instruc- tor, while students’ measurements are often “in error.” Readers will be all too familiar with the phrase “due to human error” often used by students in order to ex- plain unexpected results! THE PHYSICS TEACHER
Vol. 41, October 2003In short, we consider that the logical inconsisten- cies in the traditional approach to data treatment, together with the form of instruction that ignores stu- dents’ prior views about measurement, further culti- vate students’ misconceptions about measurement in the scientific context. What Should be Done? The need for a consistent international language for evaluating and communicating measurement re- sults prompted (in 1993) the ISO (International Organization for Standardization) to publish recom- mendations for reporting measurements and uncer- tainties10 based on the probabilistic interpretation of measurement. All standards bodies including the U.S. National Institute of Standards and Technology (NIST) have adopted these recommendations for re- porting scientific measurements. A number of docu- ments currently serve as international standards. The most widely known are the so-called VIM (Interna- tional Vocabulary of Basic and General Terms in Metrol- ogy)10 and the GUM (Guide to the Expression of Uncertainty in Measurement),11 with a U.S. version12 distributed by NIST. A shorter version of the latter is publicly available as NIST Technical Note 1297.13 We believe that the probabilistic approach advocat- ed by the ISO will help in setting up a systematic teaching framework at the freshman level and beyond, and promote a better understanding of the nature of measurement and uncertainty. In addition, the coher- ence of the approach will foreground the central role of experiment in physics and highlight the interplay between scientific inferences based on data and theory. A Probabilistic and Metrological Approach to Measurement The recommended approach10,11 to metrology is based on probability theory for the analysis and inter- pretation of data. A key element of the ISO Guide is how it views the measurement process. In paragraph 2.1 of TN1297 it is stated that, “In general, the result of a measurement is only an approximation or esti- mate of the value of the specific quantity subject to measurement, that is, the measurand, and thus the re- sult is complete only when accompanied by a quanti- tative statement of its uncertainty.”13 Uncertainty 395 us = =  0
.0 3 0  5  = 0.0029 V. We now have to convert the rated accuracy of the voltmeter, given as
1%, to a standard uncertainty ur. This can be achieved by assuming a uniform pdf (as suggested in the GUM11,12), in which case the half-width of the distribution will be (0.01) (2.47) = 0.0247 V. The standard uncertainty ur is then given by ur =  0.
02 3 47  = 0.0143 V. The combined uncertainty uc is therefore uc =
u2s + u2r =
(0.0029)2 + (0.0143)2 = 0.0146 V. In practice this uncertainty estimate would be larger if some of the other sources of uncertainty, neglected here, are included in the uncertainty budget. We also note here the practice used in the GUM of quoting two significant figures for all final uncertain- ty estimates. Therefore, the measurement result is expressed as Vresult = 2.470
0.015 V. Should one wish to emphasize the aspect of prior information during teaching, one can proceed as follows. Students are presented with a 3-V battery and a voltmeter, and asked to describe (model) the in- formation available about the measurand (V) before measuring it. Reasoning about the measurand before obtaining data is an essential feature of the Bayesian approach. This stage aims at demonstrating that any conclusion about the measurand has the form of an interval, in this case from 0 V (depleted battery) to nominal 3 V (full battery). Students are then asked to draw a graph of the probability of the statement “the value of the voltage is x,” where x is in the interval [0, 3]. We have found that most students have little diffi- culty in drawing a rectangular probability distribution (similar to Fig. 2), an intuitive conclusion compatible with the Bayesian principle of insufficient reason. A subsequent single measurement, as illustrated above, serves to demonstrate how new information modifies half of the width of the rectangle 
3398the existing knowledge of the measurand, reduces un- certainty, and narrows the probability of the posterior distribution. Finally, successive measurements with analogue or digital meters demonstrate that despite the gradual reduction of uncertainty, absolute knowl- edge of the measurand is not possible. There are some indications that such a teaching approach may be fruitful when dealing with students’ initial tendency to view single measurement results as “exact” or “point-like.”4,7 The treatment outlined for dealing with direct sin- gle measurements is of course applicable irrespective of the type of instrument used. As an extension to the previous example, we can consider the case if the same voltage were measured by an analog voltmeter. Then the scale uncertainty would again be modeled by a pdf (e.g., uniform or triangular). In this case, the limits of the pdf depend on both the least-count division of the instrument being used and the judgment of the exper- imenter in reading the scale. As before, this scale un- certainty would be combined with the uncertainty arising from the accuracy rating of the instrument. (b) An ensemble of repeated readings that are dispersed Consider an experiment where we make 20 repeat- ed observations of a time t under the same conditions, for example, in measuring the period of a pendulum with a stopwatch having a resolution of 1 ms and rated accuracy of 0.1 ms. The 20 readings are summa- rized and represented as a histogram of relative fre- quencies [Fig. 3(a)]. According to the traditional approach, the measured values ti are modeled as values of a random variable tmeasured. The 20 values are con- sidered to be sampled from an idealized Gaussian dis- tribution, which would occur if the data were infinite and the histogram bins were reduced to zero width. From our sample we can estimate the parameters of this idealized Gaussian through the familiar quantities of the arithmetic mean t– of the N = 20 observations as t– =  N 1   N i = 1 ti , and the experimental standard deviation s(t) of the observations, THE PHYSICS TEACHER
Vol. 41, October 2003 . The calculations for the data in question yield that t– =1.015 s and s(t) = 0.146 s. Based on the result from the central limit theorem that the sample means are distributed normally, the experimental standard deviation of the mean s(t– )11,12 is given by s( t– ) = 
s(t N ) , which yields s( t– ) = 0.033 s in the present example. In the traditional approach s( t– ) is often termed the “standard error of the mean” and is denoted by m. The interpretation of this result according to math- ematical statistics is that “we are 68% confident that the mean (of any future sample taken) will lie within
0.033 s of the measured mean of 1.015 s” (Conclu- sion I). Physicists tend to interpret Conclusion I in accor- dance with their needs for making an inference about the true value as follows: “we are 68% confident that the ‘true value’ (of the measurand) lies in the interval 1.015
0.033 s” (Conclusion II). However, Conclusion II cannot easily be justified in the traditional approach since t– and s( t– ) are cal- culated from observed values, and can only summa- rize what we know about the data since there is no for- mal link between knowledge of the measurand (Con- clusion II) and knowledge of the data (Conclusion I). Thus, the measurement result cannot be represented directly on Fig. 3(a) because the relative frequency his- togram and the predicted Gaussian of infinite mea- surements [Fig. 3(a)] are plotted against tmeasured. In the probabilistic approach, however, all infer- ences about the measurand are expressed via the pdf of Fig. 3(b), which is plotted against ttrue. Using the con- cepts of prior information and data at hand, we are able to conclude in a straightforward and logically consistent way the final result as follows: “The best es- timate of the value of the time is 1.015 s with a standard uncertainty of 0.033 s, and there is a 68% probability that the best estimate of the time lies with- in the interval 1.015
0.033 s, assuming that the dis- tribution of measured times is Gaussian.” In practice, of course, the uncertainty budget for this measure- 2 1 1( ) ( ) 1 N i i s t t t N = = − − ∑THE PHYSICS TEACHER
Vol. 41, October 2003ment of t would include a number of additional sources of uncertainty, each of which would be esti- mated using a Type B evaluation, so that the com- bined uncertainty would be larger than 0.033 s. When teaching the case of repeated measurements, the most important objective is to bring students around to the notion that an ensemble of dispersed values obtained by repeated observations must be modeled by theoretical constructs that represent the ensemble as a whole.2,7 Regarding the shape of the Gaussian, this can be made plausible by constructing histograms of relative frequencies of simulated or ac- tual data and showing how the distribution tends to- ward a bell-like shape as the number of readings in- creases and the bin width decreases. Making the con- ceptually correct link between relative frequencies, based on past experience, and probabilities, for infer- ence, is an important step at this stage.15 Conclusion The ISO approach solves one of the key problems associated with the traditional frequentist approach to measurement, namely that the statistical formulae lead logically only to statements about the data them- Fig. 3. (a) Distribution of relative frequencies for the time readings tmeasured. The dotted line represents the predicted Gaussian distribution of the population from which the 20 readings were sampled. (b) A Gaussian pdf used to model the measurement result. The final result tresult indicated assumes that all other sources of uncertainty are negligible (see text).399 selves. Therefore, it is not valid to make the logical jump that is usually made in laboratory manuals to in- terpret a standard error as a standard uncertainty. Stu- dents (and others) have difficulties understanding this discontinuity in logic. The probabilistic approach, as outlined, leads directly to inferences about the mea- surand in a natural way, in both cases of single and repeated measurements. In addition, representing the states of knowledge graphically as pdfs, and not as numbers or intervals, provides a persuasive and consistent explanatory framework for all cases of mea- surement. Experimentation and measurement lie at the heart of physics, and it is important that students develop an understanding of these concepts. However, the way in which these have been dealt with does not ap- pear to have been effective. Two possible reasons are, first, that students’ prior knowledge about the nature of measurement has not been taken into account and, second, that there has been no logically consistent framework that could be used to teach the basic con- cepts. By adopting the probabilistic approach, the lat- ter can be effectively addressed (apart from the fact that this is what research scientists have to adhere to!). In addition, the guidelines suggested by the ISO, such as the concept of an uncertainty budget and the level of calculational detail to be reported, should also assist pedagogy. It should be noted that there now exist software packages16 that can be used to perform the sometimes tedious calculations required for a given uncertainty budget. We argue that by adopting the view that the intro- ductory laboratory course should be focused on exper- imentation and intelligent data analysis based on probability theory, the experimental aspects of physics can be placed at the center of the course rather than relegated to an “add on” to the theoretical aspects. The concepts of probability and uncertainty should be addressed as early as possible in the teaching as fun- damental to physics, highlighting the uncertain and tentative, yet quantifiable, nature of scientific knowl- edge. The groups involved in the present paper are developing and refining various laboratory teaching materials based on this approach. Finally, the language of probabilistic metrology of- fers access to other areas of physics such as quantum mechanics and statistical mechanics, as well as to cur-400rent technologies such as image processing. From a broader perspective, an understanding of the interpre- tation of data, and hence of evaluating “scientific evi- dence” is an essential life skill in the present informa- tion age. Acknowledgments Our work has been partially funded by the Universities of Cape Town, Thessaloniki, and York, and by the National Research Foundation (South Africa), the British Council, and the European Union (LSE project PL 95-2005). We wish to thank professors Joe Redish, Craig Comrie, Roger Fearick, and Sandy Perez for useful discussions. References 1. F. Tyler, A Laboratory Manual of Physics, 6th ed. (Ed- ward Arnold, London, 1988). 2. M-G. Séré, R. Journeaux, and C. Larcher, “Learning the statistical analysis of measurement error,” Int. J. Sci. Educ. 15 (4), 427–438 (1993). 3. J.G. Giordano, “On reporting uncertainties of the straight line regression parameters” Eur. J. Phys. 20 (5), 343–349 (1999). 4. D. Evangelinos, O. Valassiades, and D. Psillos, “Under- graduate students’ views about the approximate nature of measurement results,” in Research in Science Educa- tion: Past, Present and Future, edited by M. Komorek, H. Behrendt, H. Dahncke, R. Duit, W. Gräber, and A. Cross (IPN Press, Kiel, 1999), pp. 208–210. 5. D. Evangelinos, D. Psillos, and O. Valassiades, “An in- vestigation of teaching and learning about measure- ment data and their treatment in the introductory physics laboratory,” in Teaching and Learning in the Sci- ence Laboratory, edited by D. Psillos and H. Niederrer (Kluwer Academics, Dordrecht, 2002). 6. F. Lubben, B. Campbell, A. Buffler, and S. Allie, “Point and set reasoning in practical science measurement by entering university freshmen,” Sci. Educ. 85 (4), 311–327 (2001). 7. A. Buffler, S. Allie, F. Lubben, and B. Campbell, “The development of first year physics students’ ideas about measurement in terms of point and set paradigms,” Int. J. Sci. Educ. 23 (11), 1137–1156 (2001). 8. S. Allie and A. Buffler, “A course in tools and proce- dures for Physics I,” Am. J. Phys. 66 (7), 613–624 (1998).THE PHYSICS TEACHER
Vol. 41, October 2003