Statistical Inference: Understanding Confidence Intervals and Hypothesis Tests, Study notes of Health sciences

Download Statistical Inference: Understanding Confidence Intervals and Hypothesis Tests and more Study notes Health sciences in PDF only on Docsity! 1 1 REVIEW OF BASIC STATISTICAL CONCEPTS M 384E I am assuming that you are familiar with confidence intervals and some form of hypothesis testing. However, these topics can be taught from more than one perspective, and there are some common misconceptions regarding them, so it is worthwhile to give a review that will also lay a firm foundation for further work in statistics. I also want to introduce some notation that I will use in the course. So please read these notes carefully! They may contain details, perspectives, cautions, notation, etc. that you have not encountered before. I will, however, leave out some details that I assume you are familiar with -- such as the formulas for sample mean and sample standard deviation. In statistics, we are studying data that we obtain as a sample from some population. (For example, we might be studying the population of all UT students and take a sample of 100 of those students.) The procedures in an introductory statistics course usually assume that our sample is a simple random sample. That means that the sample is chosen by a method that gives every sample of the same size an equal chance of being chosen. (For example, we might choose our 100 UT students by assigning numbers to each UT student, and use a random number generator to pick a random sample of 100 numbers; our sample would then consist of the 100 students with those numbers.) Typically we are interested in a random variable defined on the population under consideration. (For example, we might be interested in the height of UT students.) Typically we are interested in some parameter associated with this random variable. (For example, we might be interested in the mean height of UT students.) We will illustrate with the example of mean as our parameter of interest. Notation: Let Y refer to the random variable. (e.g., height). Then • The population mean (also called the expected value or expectation) of y is denoted by either E(Y), or µ, or µY. • We use our sample to form the estimate ! y of E(Y). More generally: • We use the word parameter to refer to constants that have to do with the population. We will often refer to parameters using Greek letters (e.g., σ) • We use the word statistic (singular) to refer to something calculated from the sample. So ! y is a statistic. (However, not all statistics are estimates of parameters. In particular, we will deal with test statistics, which are not estimates of parameters.) • Consistently with the textbook, I will try to use capital letters to refer to random variables. Models: Most statistical procedures are based on model assumptions -- that is, one or more assumptions about distributions, or how data is selected, or about relationships between our variables. We will see several types of models in this course. In order to use statistical inference or form confidence intervals for means, we need to have a model for our random variable. In the present context, this means we assume that the random variable has a certain (type of) distribution. Just what model (distribution) we choose 2 2 depends in part on what we know about the random variable in question, including both theoretical considerations and available data. The choice of model is also usually influenced by information known about distributions -- we can deduce more from a distribution that has a lot known about it. In working with models (which we will do often in this course), always bear in mind the following quote from the statistician G.B.E. Box: All models are wrong - but some models are useful. For our example of height, we will use a normal model -- that is, we proceed under the assumption that the height of UT students is normally distributed, with mean µ and standard deviation σ. We will use the notation Y ~ N(µ, σ2) as shorthand for "Y is normal with mean µ and standard deviation σ." The values of µ or σ are unknown; in fact, our aim is to try to use the data to say something about µ. Note: If we are just considering students of one sex, both theory and empirical considerations indicate that a normal model should be a pretty good one; if we are considering both sexes, then data, theory, and common sense tell us that it isn't likely to be as good a choice as if we are just considering one sex. However, other theoretical considerations suggest that it probably isn't too bad. Sampling Distributions: Although we only have one sample in hand when we do statistics, our reasoning will depend on thinking about all possible simple random samples of the same size n. Each such sample has a sample mean. We thus have a new random variable ! Y . Notes to clarify distinctions: 1. We are using ! Y to refer to the random variable "sample mean" and ! y to refer to the sample mean for our particular sample. 2. The value of the random variable ! Y depends on the choice of sample, whereas in the example above, the value of the original random variable Y depends on the choice of student. 3. We call ! y an estimate and the underlying random variable ! Y an estimator of µ. Since the value of ! Y depends on the sample, the distribution of ! Y is called a sampling distribution. Mathematics (using our assumption that the distribution of Y is normal with mean µ and standard deviation σ) tells us that the distribution of ! Y is also normal with mean µ, but its standard deviation is ! " n . In shorthand notation: ! Y ~ N(µ, σ2/n). Consequently, ! Y varies less than Y. (See the demo Distribution of Mean at http://www.kuleuven.ac.be/ucs/java/index.htm under Basics for an illustration of this. ) This makes ! Y more useful than Y for estimating µ! In fact, since ! Y ~ N(µ, σ2/n), we know that ! Y " µ # n is standard normal. If we knew σ, we could get a kind of margin of 5 5 Note: 1. A hypothesis test cannot prove a hypothesis. Therefore it is wrong to say, "the null hypothesis is false," or "the alternate hypothesis is true," or "the null hypothesis is true," or "the alternate hypothesis is false" on the basis of a hypothesis test. 2. Although is it arguably reasonable to say "we reject the null hypothesis" on the basis of a small p-value, there is not as sound an argument for saying "we accept the null hypothesis" on the basis of having a p-value that is not small enough to reject the null hypothesis. To see this, imagine a situation where you are doing two hypothesis tests, with null hypotheses just a little different from each other, using the same sample. It is very plausible that you can get a large (e.g., around 0.5) p-value for both hypothesis tests, so you haven't really got evidence to favor one null hypothesis over the other. So if your p-value is not small enough for rejection, all you can legitimately say is that the data are consistent with the null hypothesis. (This discussion assumes that by "accept" you mean that the data provide adequate evidence for the truth of the null hypothesis. If by "accept" you mean accept µ0 as a good enough approximation to the true µ, then that's another matter -- but if that's what you are interested in, using a confidence interval would probably be more straightforward than a hypothesis test.) 3. The p-value is, roughly speaking, the probability of obtaining a sample at least as extreme as the sample at hand, given that the null hypothesis is true. What many people really would like (and sometimes misinterpret the p-value as saying) is the probability that the null hypothesis is true, given the data we have. Bayesian analysis aims to get at the latter conditional probability, and for that reason is more appealing than classical statistics to many people. However, Bayesian analysis doesn't quite give what we'd like either, and is also often more difficult to carry out than classical statistical tests. Increasingly, people are using both kinds of analysis. I encourage you to take advantage of any opportunity you can to study some Bayesian analysis. Framework II (In terms of rejection criteria): Many people set a criterion for determining what values of p will be small enough to reject the null hypothesis. The upper bound for p at which they will reject the null hypothesis is usually called α. Thus if you set α = 0.05 (a very common choice), then you are saying that you will reject the null hypothesis whenever p < 0.05.. This means that if you took many, many simple random samples of size n from this population, you would expect to falsely reject the null hypothesis 5% of the time -- that is, you'd be wrong about 5% of the time. For this reason, α is called the type I error rate. Note: 1. If you set a type I error rate α, then to be intellectually honest, you should do this before you calculate your p-value. Otherwise there is too much temptation to choose α based on what you would like to be true. In fact, it's a good idea to think about what p- values you are willing to accept as good evidence before the fact -- but if you are using p- values, you may think in terms of ranges of p-values that indicate "strong evidence," "moderate evidence," and "slight evidence," rather than just a reject/don't reject cut-off. 6 6 2. If you do set a type I error rate α, then you don't really need to calculate p to do your hypothesis test -- you can just reject whenever the calculated test statistic ts is more extreme ("more extreme" being determined as above by your alternate hypothesis) than tα., where tα. is the value of the t-distribution that would give p-value equal to α. 3. If you are going to publish any scientific work, the second option is not a good choice; instead, you should calculate and publish the p-value, so others can decide if it satisfies their own criteria (which might be different from yours) for weight of evidence desired to reject the null hypothesis. 4. When an α has been chosen for determining when the null hypothesis will be rejected, and when the null hypothesis has indeed been rejected, many people say that the result of the hypothesis test is "statistically significant at the α level." It is important not to confuse "statistically significant" with "practically significant." For example, the improvement on a skill after a training session may be statistically significant , but could still be so small as to be irrelevant for practical purposes. By taking a large enough sample, almost anything can be shown to be statistically significant. 5.There is another variation of hypothesis testing. In this variation, you are trying to decide between two competing hypotheses, the null hypothesis H0: µ = µ0 and an alternate hypothesis Ha: µ = µa (still assuming we are testing means). Note that in this setting the alternate hypothesis specifies one value rather than being defined in terms of an inequality. Thus the null and alternate hypotheses play symmetric roles in the initial formulation of the problem. In this setting, you will either accept H0 (and reject Ha ) or accept Ha (and reject H0). You determine a rejection region. For values of the test statistic in the rejection region, you will reject the null hypothesis and accept the alternate hypothesis; otherwise you will accept the null hypothesis and reject the alternate hypothesis. In determining the rejection region, you take into account both the type I error rate α and the type II error rate ( the probability of accepting H0 when Ha is true). Hypothesis tests of this sort are appropriate for situations such as industrial sampling. The costs of errors one way or the other as well as the costs of sampling are taken into account in determining the rejection region and the sample size. We will discuss the related concept of power from a slightly different perspective in Section 3.6.2 Important advice: Statistics gives many tools for obtaining information from data. However, it doesn't tell us "the answers." We need to combine what statistics tells us with careful thinking, caution, and common sense.