Statistical Hypothesis Testing: Designing a Test with Given Alpha and Power, Exams of Probability and Statistics

Download Statistical Hypothesis Testing: Designing a Test with Given Alpha and Power and more Exams Probability and Statistics in PDF only on Docsity! A statistical hypothesis test acts on data to arrive at a decision to either reject or not reject a stated null hypothesis. One important focus is to design a test achieving a given alpha. Then there is the matter of drawing a picture to describe how the test is expected to perform. Finally, one can choose the sample size to achieve needed performance goals. BE ALERT FOR CORRECTIONS TO THIS KEY Note: In b2 and b4 above, there should be a minus sign on each of zALPHA, and tALPHA. d1. Identify the null hypothesis, significance level alpha, power at mu = 18, ideal power at mu = 18 in the figures below. Label the axes mu and P(rej null | mu) respectively. c. Determine the statistical significance for this data. It is the probability that xBAR would have been more unfavorable to the null that it is for this data (calculated when mu is 100, the boundary between null and alternative). So if your test statistic is –1.87 (it is not) you would report statistical significance equal to the area under the z-curve to the left of –1.87 (more unfavorable than you observed). ANS. pSIG = area left of –2.47 = 0.0068. So the same test (4ab) rejects H0 is pSIG < alpha. Since 0.0068 < 0.05 we (as above) reject H0. d. From a lab, go to file bootstrap5.nb on the website you will find there a new function zTail which can calculate tail areas under the z-curve, e.g. zTail[1.96] will give 0.025. Use it to verify your answer (c). ANS. zTail[2.47]~0.0068. e. Use your statistical significance from (c) to conduct the test. That is, reject the null if the significance falls below alpha = 0.02. State the action taken. This should agree with (b), which conducted the test in another, but equivalent, way. ANS. See (4c). f. Suppose instead that the sample was only n = 50 (not n = 400) and the sample mean was xBAR = 98.2 with sample sd equal to s = 14.6. IF THE POPULATION IS KNOWN TO BE IN CONTROL (which is typical for IQ scores) we are entitled to conduct a t-test. Do so, stating the DF and consulting the t-table to determine a nearest entry for the rejection threshold for the test statistic. From a lab, go to the file bootstrap5.nb which has a function tTail[t, DF] for which the command tTail[1.87, 49] would give the statistical significance if the test statistic were –1.87 (it is not). ANS. Test statistic = (98.2 – 100)/(14.6/20) = -2.46575. From the computer, pSIG = tTail[2.46575, 49] = 0.00861068. Reject H0 if pSIG < alpha = 0.05. Therefore reject H0. 5. Let p denote the fraction of voters favoring the Republican candidate. For the null hypothesis that Republicans have at least 50% of the vote versus alternative p < 0.5, and significance level alpha = 0.04, freehand sketch the general shape of the curve P(rej null | p) versus p in the range 0 to 1 for a possible test of this null hypothesis. Identify alpha clearly and label axes properly. Impose on your sketch the ideal curve P(rej null | p) over the same range of p. Also impose upon your sketch another P(rej null | p) curve for a better test, based upon more data, that shares the same alpha with your first one. Indicate which one of your curves is for more data. ANS. Take d2 as your model. The better curve is lower on H0 and higher on H1 while passing through the point (0.5, 0.04). 6. Let p denote the fraction of customers buying the red label product. Historically this has been at p = 0.38. For the null hypothesis p = 0.38, versus the alternative p not equal to 0.38, and significance level alpha = 0.05, freehand sketch the general shape of the curve P(rej null | p) versus p in the range 0 to 1. Identify alpha clearly and label axes properly. Impose on your sketch the ideal curve P(rej null | p) over the same range of p. Also impose upon your sketch another P(rej null | p) curve for a better test, based upon more data, that shares the same alpha with your first one. Indicate which one of your curves is for more data. ANS. Take d5 as your model. 7. Refer to (6). Suppose a random with-replacement sample of n = 100 customers yields the sample fraction pHAT = 0.47 who favor the red label product. a. Calculate the test statistic for the z-test of the hypothesis p = 0.38. Note: when calculating the test statistic it is common practice to use root(.38 .62)/root(100), not the value root(.47 .53)/root(100) as would used for the CI. This is because doing so leads to a test having slightly more desirable performance against alternatives close to p = 0.38. Ordinarily, there will be little difference between doing it one way or the other since for fractions f between 0 and 1 the function root(f (1-f)) varies slowly as f is changed about, at least for values of f not too close to 0 or 1. ANS. (pHAT – p0)/(root(p0 q0)/root(n)) = (0.47 – 0.38)/(root(0.38 0.62)/root(100)) = +1.85419. b. Use the z-table to determine a threshold for rejection of the null. Make sure to take into account that this is a two-sided test that will reject the null if the test statistic is either too large or too small. ANS. z(ALPHA/2) = z(0.025) = +1.96. The test rejects H0 if the test statistic 1.85.. > 1.96. So we fail to reject H0. c. Conduct the test stating the action taken (reject null or fail to reject null). ANS. See (b) above. d. For the given data, calculate the statistical significance. Note: this is a two-sided test so, for a test statistic value of +1.87 (it is not) we would report significance level equal to the combined areas left of –1.87 and right of 1.87, or (simply put) twice the area right of 1.87 as “more extreme than observed.” ANS. pSIG = 2 P(Z > 1.85) = 0.0643135. e. Using your statistical significance (d) conduct the test by comparing it with alpha = 0.05. State the result of this comparison and the action taken (reject null or fail to reject null). ANS. Reject H0 if pSIG < alpha. Since 0.0643135 < 0.05 we fail to reject H0. 8. Refer to problem (7). Suppose we desire a test having alpha = 0.05 and also power P(rej null | p = 0.42) = 0.96. If it can be achieved we then have a test whose chance of falsely rejecting p = 0.38 (if it is true) is only 0.05 but also one whose chance of rejecting p = 0.38 is 0.96 if truly p = 0.42. Free hand sketch P(rej null | p) as p varies between 0 and 1 for such a test. ANS. Use p5 as your model. 9. The formula at the top of page 319 may be used to determine a sample size that will support the test described in (8). The formula tells us the required sample size n for these specifications. If we agree to use this recommended n then the test for alpha = 0.05 will automatically have the desired power also. Evaluate the formula using z0 = zalpha/2 = z0.025 = 1.96 (since the test is two-sided) z1 = zbeta = z0.04 = 1.75 p0 = 0.38 p1 = 0.47 sd known, or from a preliminary sample The third to last line is not a typo since the chance of false rejection of null, for p1 = 0.47, can safely ignore the small contribution of doing so by accidentally producing a pHAT far below 0.38. ANS. (1.96 root(0.38 0.62) + 1.75 root(0.47 0.53))^2 / (0.38 – 0.47)^2 ~ 412.