Factorial ANOVA in Psychological Statistics I - Lecture Slides | PSYC 502, Study notes of Psychology

Download Factorial ANOVA in Psychological Statistics I - Lecture Slides | PSYC 502 and more Study notes Psychology in PDF only on Docsity! 1 Factorial ANOVA Advanced Psychological Statistics I Psychology 502 November 6, 2007 2 Overview ! Finish up posthocs ! Overview of simple ANOVA procedures ! Factorial ANOVA ! Basic concepts ! Sums of squares ! Interpreting interactions " Simple main effects " Contrasts " Posthocs ! Probably won!t get through all of it, which is fine 3 Dunnett Test ! Something in between a posthoc and a planned comparison ! Compare all other groups against a control group C.V . = t d 2MSE n ! Table for td in the textbook (p. 688) " Need !, k, and dfe ! If the difference between a treatment and the control group exceeds the critical value, reject " Conclude that treatment differs from control 4 More on Dunnett ! Good power, but very specific requirements ! Will not test whether treatment groups differ from one another ! SPSS (and others) do it as a posthoc, you have to tell it which group is the control group 9 Posthocs in SPSS ! Both ONEWAY and UNIANOVA support posthocs " ONEWAY recall BY cond /POSTHOC = SNK TUKEY LSD QREGW ALPHA(.05). " UNIANOVA recall BY cond /POSTHOC=cond(SNK TUKEY LSD QREGW). # COND tells it which I.V. to do posthocs on ! The names are different posthoc tests " LSD is Fisher!s LSD " SNK is Student-Newman-Keuls " TUKEY is Tukey!s HSD " QREGW is Ryan-Einot-Gabriel-Welsch " There are bunches of others ! Usually you!ll only supply one, not all of them 10 Pairwise Output Multiple Comparisons Dependent Variable: RECALL -6.200* 1.992 .012 -11.139 -1.261 -.600 1.992 .951 -5.539 4.339 6.200* 1.992 .012 1.261 11.139 5.600* 1.992 .024 .661 10.539 .600 1.992 .951 -4.339 5.539 -5.600* 1.992 .024 -10.539 -.661 -6.200* 1.992 .004 -10.287 -2.113 -.600 1.992 .766 -4.687 3.487 6.200* 1.992 .004 2.113 10.287 5.600* 1.992 .009 1.513 9.687 .600 1.992 .766 -3.487 4.687 -5.600* 1.992 .009 -9.687 -1.513 (J) COND 1 2 0 2 0 1 1 2 0 2 0 1 (I) COND 0 1 2 0 1 2 Tukey HSD LSD Mean Difference (I-J) Std. Error Sig. Lower Bound Upper Bound 95% Confidence Interval Based on observed means. The mean difference is significant at the .050 level.*. 11 Subsets Output RECALL 10 21.200 10 21.800 10 27.400 .766 1.000 10 21.200 10 21.800 10 27.400 .951 1.000 10 21.200 10 21.800 10 27.400 .766 1.000 COND 0 2 1 Sig. 0 2 1 Sig. 0 2 1 Sig. Student-Newman-Keuls a,b Tukey HSD a,b Ryan-Einot-Gabriel-Welsch Range b N 1 2 Subset Means for groups in homogeneous subsets are displayed. Based on Type III Sum of Squares The error term is Mean Square(Error) = 19.837. Uses Harmonic Mean Sample Size = 10.000.a. Alpha = .050.b. 12 Interpreting Groupings ! Sometimes, the output of the posthoc procedures is less than totally helpful ! Example: " M1 M2 M3 M4 M5 ! What can you say from this? " Means 1 and 2 are different from Means 4 and 5 " Can!t say anything about Mean 3, though ! This can often be difficult to interpret, particularly when there are many cells and multiple overlapping groupings of means 13 Putting It Together ! We!ve seen a lot of different things to do in ANOVA. How should these all be combined? ! If you have specific hypotheses, you should use planned comparisons " Bonferroni (or Sidak) error rate adjustments are appropriate # Some people have argued that if all the contrasts are orthogonal, the Bonferroni adjustment is not necessary # Common in practice # Technically, you should do Bonferroni adjustments for orthogonal contrasts " For non-orthogonal contrasts # Technically, should do Sheffé # Must at least do Bonferroni 14 Putting It Together ! If you do not have specific hypotheses in advance, just do the omnibus ANOVA " If the overall F-test is not reliable, you!re done testing " Maybe compute power for observed effect size ! If the omnibus ANOVA is reliable " Look for pairwise differences with some kind of posthoc procedure # Which one? # Up to you, but like Howell I recommend the Ryan, et al. procedure # You will see some of the other ones " Or, do contrasts with Scheffé adjustments # Test whatever the heck you want # As many as you want 19 Example (fictional) ! Looking at performance on a game-like task involving motor skill " High scores indicate better performance ! Two populations of subjects " Professional athletes " College students ! Three levels of task difficulty " Hard, medium, easy " The harder the task, the more difficult it is to score " When scores are earned, they!re higher # High risk, high reward 20 Questions ! 60 total subjects (10 per cell) ! Does difficulty affect performance? " Perhaps performance is not affected by difficulty in a kind of speed-accuracy tradeoff ! Does the population affect performance? " Pro athletes actually better? ! Do they interact? " For example, maybe pro athletes are only better in the higher-risk, higher-reward conditions 21 Results Hard Medium Easy Athlete 46.4 30.2 17.8 31.47 College 36.8 37.8 21.4 32.00 41.60 34.00 19.60 31.73 22 Evaluating the Model ! How do we evaluate these three hypotheses? ! Conceptually: " Compute an error term from within-cells variance # Need to assume homogeneity " Compute terms for main effects # Average over other variable (that is, pretend it doesn!t exist) # Then do the same as the one-way and compute the sum of squares between " Compute terms for interaction # If no interaction, can predict cell means based on main effects # Take difference between predicted cell mean and actual cell mean # Convert to a sum of squares 23 Sums of Squares ! Sum of squares total " Sum of all squared deviations from the grand mean: SStotal = SST = (xijk ! x ) 2""" ! Sum of squares for factor A " Sum of all squared deviations of each marginal mean from the grand mean " Weighed by the number of observations contributing SSA = Kn! j 2 = Kn (x j• " x ) 2## " Why is this K and not J? 24 Getting J vs. K ! Consider this simple design " Number in each cell is n for that cell ! Note that there are 30 observations at each level of A ! 20 at each level of B B1 B2 B3 A1 10 10 10 30 A2 10 10 10 30 20 20 20 60 29 More Mean Squares ! Expected value of the mean square for B: E(MS B ) =! e 2 + Jn " k 2# K $1 ! Expected value of the mean square for interaction (or AxB): E(MSAxB ) =! e 2 + n "# jk 2$$ (J %1)(K %1) ! We have four estimates of the error variance " One is stable regardless of the status of the null " The others are not 30 F-ratios ! Form F-ratios by dividing mean square for each effect by the mean square for error ! Consider the F-ratio for A: F(J !1,N ! JK) = MS A MS e ! Now, expected values: E(FA ) ! E(MSA ) E(MSe ) = " e 2 + Kn # j 2$ J %1 " e 2 ! What happens when the null is true? When it!s false? ! Sampling distribution under the null conforms to standard F distribution 31 Results Hard Medium Easy Athlete 46.4 30.2 17.8 31.47 College 36.8 37.8 21.4 32.00 41.60 34.00 19.60 31.73 32 Sum of Squares for A “Population” ! Grand mean is 31.73 ! Marginals are 31.47 and 32.0 ! n = 10, J = 2, K = 3 SSA = Kn ! j 2 = Kn (x j• " x ) 2## ! Estimated !1 = 31.47 - 31.73 = -0.26 ! Estimated !2 = ? SSA = Kn (x j• ! x ) 2" = 3(10) !0.262 + 0.262#$ %& ! SSA = 4.056 33 Sum of Squares for B “Difficulty” ! Marginals: 41.60, 34.0, 19.60 ! Estimated #1 = 41.60 - 31.73 = 9.87 ! Estimated #2 = 34.0 - 31.73 = 2.27 ! Estimated #3 = 19.60 - 31.73 = 0 - 9.87 - 2.27 = -12.14 SS B = Jn (x•k ! x ) 2" SS B = 2(10) 9.87 2 + 2.27 2 +12.14 2[ ] = 4,998.988 34 Sum of Squares for Interaction ! Est. !#11 = 46.4 - 31.47 - 41.60 + 31.73 = 5.06 ! Est. !#12 = 30.2 - 31.47 - 34.00 + 31.73 = -3.53 ! Est. !#13 = 17.8 - 31.47 - 19.60 + 31.73 = -1.54 ! Est. !#21 = 36.8 - 32.00 - 41.60 + 31.73 = -5.07 ! Est. !#22 = 37.8 - 32.00 - 34.00 + 31.73 = 3.53 ! Est. !#23 = 21.4 - 32.00 - 19.60 + 31.73 = 1.53 SSAxB = n !" jk 2 = n (x jk # x j• # x•k + x ) 2$$$$ SS AxB = 10(5.06 2 + 3.53 2 + ... + 1.53 2 ) = 809.428 39 Output, Part 2 Estimated Marginal Means of SCORE DIFF EasyMediumHard E s ti m a te d M a r g in a l M e a n s 50 40 30 20 10 POP Athle te Coll ege 40 Output, Part 3 Estimated Marginal Means of SCORE POP CollegeAthle te E s ti m a te d M a r g in a l M e a n s 50 40 30 20 10 DIFF Hard Medium Easy 41 Interpretation ! Interaction is reliable " At least one !#jk is not zero ! Main effect of population is not reliable ! Main effect of difficulty is reliable " At least one #k is not zero ! What does all this mean? " Often a sticky problem " Many (most) stats folks would argue that you should worry about the interaction first, and only consider main effects in the context of any interactions " Going back to the example should help 42 Interpreting Interactions ! Many ways to interpret most interactions ! A viable interpretation here might be: " Effect of difficulty occurs only when the population is “athlete” 43 Interpreting Interactions ! Alternative interpretation: " “Effect of population is reversed for high difficulty” 44 General Problem ! There is no cut-and-dried formula for interpreting interactions ! There are three reasonably common approaches: " Eyeball it " Simple main effects " Interaction contrasts ! Eyeballing it (the “interocular trauma test”) " What does the interaction look like? " You must plot the means for this " Probably the most common " Probably the worst " But, even if you plan on doing something more sophisticated, you should always start with this 49 Tests of Between-Subjects Effects Depend ent Variable: SCORE 169 0.400 a 2 845.200 69.65 9 .000 307 20.000 1 30720.000 253 1.868 .000 169 0.400 2 845.200 69.65 9 .000 327.600 27 12.13 3 327 38.000 30 201 8.000 29 Source Corrected Model Intercept DIFF Error Total Corrected Total Type II I Sum of Squares df Mean Squar e F Sig. R Squar ed = .838 (Adjusted R Squared = .826)a. “Athlete” Population ! Reliable here as well ! Thus, not especially helpful 50 Contrasts ! Can still do contrasts in factorial designs ! Two kinds of contrasts: " Main effects contrasts " Interaction contrasts ! Main effects contrasts " Can do contrasts on each factor, ignoring the other factor(s) " For example, could contrast “hard” vs. “medium” and “easy” with contrast (-2 1 1) " Independent of other factors 51 Main Effects Contrasts ! Same kind of contrast statements as with one-way ANOVA " UNIANOVA score BY pop diff /CONTRAST(diff) = SPECIAL(-1 0 1) /CONTRAST(diff) = SPECIAL(-2 1 1) /CONTRAST(pop) = SPECIAL(-1 1). ! Must remember how your means are ordered in the data file 52 SPSS Contrast Output Contrast Results (K Matrix) -2 2.000 0 -2 2.000 1.091 .000 -2 4.188 -1 9.812 Contrast Estimate Hypothesized Value Dif ference (Estimate - Hypothesized) Std. Error Sig. Lower Bound Upper Bound 95% Conf idence Interval for Dif ference DIFF Special Contrast L1 SCORE Dependent Variable Test Results Depend ent Variable: SCORE 4840.000 1 4840.000 406 .343 .000 643 .200 54 11.911 Source Contrast Error Sum of Squar es df Mean Square F Sig. ! You!ll get two tables for each contrast (this is the set from the first contrast) " Just like with one-way ANOVA 53 Interaction Contrasts ! Think of design as a giant one-way ! Generate a contrast on factor A ! Generate another contrast of factor B ! Interaction contrast would be the product of those two A,+ A A,- C,+ C C,- Hard vs. others 2 -1 -1 2 -1 -1 Athlete vs. College 1 1 1 -1 -1 -1 Interaction 2 -1 -1 -2 1 1 54 Interaction Contrasts ! This tests the second interaction hypothesis that I proposed: " “Effect of population is reversed for high difficulty (hard)” ! Does everyone see why? A,+ A A,- C,+ C C,- Hard vs. others 2 -1 -1 2 -1 -1 Athlete vs. College 1 1 1 -1 -1 -1 Interaction 2 -1 -1 -2 1 1 59 Building the Contrast ! Linear contrast on display size is (-3 -1 1 3) ! Linear contrast on icon type is (1 0 -1) ! The idea: " Line best fits effect of display size " The lines for different icon types have different slopes " That!s an interaction # icons 6 12 18 24 Simple -3 -1 1 3 1 Cmplx 0 0 0 0 0 Blank 3 1 -1 -3 -1 -3 -1 1 3 60 Another Example ! What would we want to test? ! What would that contrast look like? 61 Another Example ! Tedious but doable ! Everyone clear on this idea? 210-1-2 1210-1-24 1210-1-23 -1-2-10122 -1-2-10121 54321Group 62 Interaction Contrast Pros and Cons ! Pros " Allow you to test very specific interaction effects " Good power # Overall error term is still the same MSE # No loss of degrees of freedom " Generally easier to interpret ! Cons " Can still have trouble finding one that makes sense " Can be hard to explain clearly even when you do find them " Can still find more than one is reliable " Can still find none are reliable ! Most sophisticated and sensitive method 63 Interaction Contrasts ! It turns out that SPSS will not run an interaction contrast correctly with UNIANOVA ! Best strategy is to break it into a one-way ! Need a new variable, “cell,” which represents the six cells ! BE CAREFUL: Make sure that the cell numbering and your contrast weights match up " Very common error " Useful to compute the contrast by hand to make sure output is right 64 Forming the CELL Variable ! Assume pop is coded 0, 1 and diff is coded 0, 1, 2 ! Here!s the SPSS code to make this happen (long way): " COMPUTE cell = 0. IF pop = 0 and diff = 0 THEN cell = 0. IF pop = 0 and diff = 1 THEN cell = 1. IF pop = 0 and diff = 2 THEN cell = 2. IF pop = 1 and diff = 0 THEN cell = 3. IF pop = 1 and diff = 1 THEN cell = 4. IF pop = 1 and diff = 2 THEN cell = 5. ! Here!s the short way: " COMPUTE cell = diff + 3*pop. " Where!d the “3” come from? Could we have used 10? " Would this still work if data were coded 1, 2 and 1, 2, 3?