Understanding Statistical Hypothesis Testing: Significance, Power, and Errors, Assignments of Data Analysis & Statistical Methods

Download Understanding Statistical Hypothesis Testing: Significance, Power, and Errors and more Assignments Data Analysis & Statistical Methods in PDF only on Docsity! Sections 6.2 - 6.4 I. Tests of Significance (IPS section 6.2 pages 435-461) A. Test of Significance - is intended to assess the evidence provided by data against a null hypothesis Ho and in favor of an alternative hypothesis Ha. B. Null Hypothesis (Ho) – The statement being tested in a test of significance is called the null hypothesis. The test of significance is designed to access the strength of the evidence against the null hypothesis. Usually the null hypothesis is a statement of “no effect” or “no difference.” C. Alternative Hypothesis (Ha) – the statement in a test of significance that we suspect or hope is true instead of the null hypothesis. D. Test Statistic – measures compatibility between the null hypothesis and the data. We use it for the probability calculation that we need for our test of significance. It is a random variable with a distribution that we know. 1. The test is based on a statistic that estimates the parameter that appears in the hypothesis. Usually this is the same estimate we would use in a confidence interval for the parameter. When Ho is true, we expect the estimate to take a value near the parameter value specified by Ho. 2. Values of the estimate far from the parameter value specified by Ho give evidence against Ho. The alternative hypothesis determines which directions count against Ho. E. P-Values – The probability, computed assuming that Ho is true, that the test statistic would take a value as extreme or more extreme than that actually observed is called the P-value of the test. The smaller the P- value, the stronger the evidence against Ho provided by the data. F. Statistical Significance – If the P-value is as small or smaller than α, we say that the data are statistically significant at level α. G. Steps common to all tests of significance include the following: 1. State the null hypothesis Ho and the alternative hypothesis Ha. The test is designed to assess the strength of the evidence against Ho; Ha is the statement that we will accept if the evidence enables us to reject Ho. 2. Calculate the value of the test statistic on which the test will be based. This statistic usually measures how far the data are from Ho. 3. Find the P-value for the observed data. This is the probability, calculated assuming that Ho is true, that the test statistic will weigh against Ho at least as strongly as it does for these data. 4. State a conclusion. One way to do this is to choose a significance level α, how much evidence against Ho you regard as decisive. If the P-value is less than or equal to α, you conclude that the alternative hypothesis is true; if it is greater than α, you conclude that the data do not provide sufficient evidence to reject the null hypothesis. Your conclusion is a sentence that summarizes what you have found by using a test of significance. H. z Test for a Population Mean – To test the hypothesis Ho: μ = μ0 based on an SRS of size n from a population with unknown mean μ and known standard deviation σ, compute the test statistic Moore, David and McCabe, George. 2002. Introduction to the Practice of Statistics. W. H. Freeman and Company, New York. 365-413. 0 / x z n     In terms of a standard normal random variable Z, the P-value for a test of Ho against : Ha: μ > μ0 is P(Z  z) Ha: μ < μ0 is P(Z  z) Ha: μ  μ0 is 2P(Z  |z|) These P-values are exact if the population distribution is normal and are approximately correct for large n in other cases. I. Confidence Intervals and Two-sided Tests – A level α two-sided significance test rejects a hypothesis Ho: μ = μ0 exactly when the value μ0 falls outside a level 1- α confidence interval for μ. J. Critical Value – A value z* with a specified area to its right under the standard normal curve is called a critical value of the standard normal distribution. II. Use and Abuse of Tests (IPS section 6.3 pages 461-469)  P-values are more informative than the reject-or-not result of a fixed level α test. Beware of placing too much weight on traditional values of α, such as α = 0.05.  Very small effects can be highly significant (small P), especially when a test is based on a large sample. A statistically significant effect need not be practically important. Plot the data to display the effect you are seeking, and use confidence intervals to estimate the actual value of parameters.  On the other hand, lack of significance does not imply that Ho is true, especially when the test has low power.  Significance tests are not always valid. Faulty data collection, outliers in the data, and testing a hypothesis on the same data that suggested the hypothesis can invalidate a test. Many tests run at once will probably produce some significant results by chance alone, even if all the null hypotheses are true. III. Power and Inference as a Decision (IPS section 6.4 pages 469-483) A. Power – The probability that a fixed level α significance test will reject Ho when a particular alternative value of the parameter is true is called the power of the test to detect that alternative. Moore, David and McCabe, George. 2002. Introduction to the Practice of Statistics. W. H. Freeman and Company, New York. 365-413.