Multiple Linear Regression: Gauss-Markov Assumptions and Hypothesis Testing, Study notes of Introduction to Econometrics

Download Multiple Linear Regression: Gauss-Markov Assumptions and Hypothesis Testing and more Study notes Introduction to Econometrics in PDF only on Docsity! Multiple linear regression Multiple linear regression ECO 5314 Dr. Peter M. Summers Texas Tech University September 9, 2008 Multiple linear regression I Verbeek, ch. 2 I Greene, chs. 2-7 Multiple linear regression The Gauss-Markov assumptions I Assumptions 3 & 4 imply Var() = σ2In, also known as spherical disturbances I Var(i ) = σ 2 ∀i ⇒ homoskedasticity I Cov(i , j) = 0 ∀i 6= j ⇒ no serial correlation I Assumption 2 means none of the X ’s has any effect on any of the ’s I ⇒ E (|X ) = 0 and Var(|X ) = σ2In I X ’s can be deterministic or stochastic Multiple linear regression The Gauss-Markov assumptions Under assumptions 1-4 (the Gauss-Markov assumptions), I The OLS estimator β̂ = (X ′X )−1X ′y is unbiased: E (β̂) = β I Var(β̂) = σ2(X ′X )−1 I Gauss-Markov theorem: β̂ has minimum variance among the class of all linear, unbiased estimators (β̂ is BLUE). If β̃ is any other linear, unbiased estimator, then Var(β̃) ≥ Var(β̂) = σ2(X ′X )−1 Multiple linear regression Small-sample properties of OLS estimators We don’t know σ2, so we need an (unbiased) estimator for it: s2 = 1 n − k n∑ i=1 e2i So V̂ar(β̂) = s2(X ′X )−1. For the kth coefficient, we have V̂ar(β̂k) = s 2ckk , where ckk is the k th diagonal element of (X ′X )−1. If we have spherical disturbances, then se(β̂k) = s √ ckk is the standard error of β̂k (needed for statistical inference). Multiple linear regression Small-sample properties of OLS estimators I Gauss-Markov assumptions and Normally-distributed errors means that, for an individual coefficient βk , zk = β̂k − βk σ √ ckk is a standard Normal variable (i.e., z ∼ N(0, 1)). I Replacing σ by s means that this isn’t true any more. However, tk = β̂k − βk s √ ckk has a student-t distribution with N − k degrees of freedom Multiple linear regression −3 −2 −1 0 1 2 3 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 N(0,1) t(3) t(3) Multiple linear regression Hypothesis testing General approach to hypothesis testing: I State null and alternative hypotheses I Compute a test statistic whose distribution is of a known form under the null hypothesis (ie, if the null is true). I Specify a significance level for the test. I Construct a confidence interval, compare the test statistic to the appropriate critical value(s) and/or compute the p-value for the test. I Draw conclusions about the likelihood of the null hypothesis being true. Multiple linear regression Hypothesis testing Example: Wages and gender wage = β1 + β2 ×male +  I Reject H0 : β2 = 0 in favor of H1 : β2 6= 0 if |t2| > 1.96 I Reject H0 : β2 = 0 in favor of H1 : β2 ≥ 0 if t2 > 1.645 Multiple linear regression Hypothesis testing Example: Wages and gender wage = β1 + β2 ×male +  I Since t2 ∼ t(N − k) under the null, then with probability 1− α, −t(N − k, α/2) ≤ t2 ≤ −t(N − k , α/2) −t(N − k, α/2) ≤ β̂2 − β2 se(β̂2) ≤ −t(N − k , α/2), or β̂2 − t(N − k, α/2)se(β2) ≤ β2 ≤ β̂2 + t(N − k, α/2)se(β2) I This is the (1− α)% confidence interval Multiple linear regression Goodness-of-fit Example: Wages and gender wage = β1 + β2 ×male +  I Given that β2 is significantly different from 0 (or just significant), how well is the variation in wages explained by gender differences? I Consider y = Xβ + . The total variation in y is∑N i=1(yi − ȳ)2 = TSS (total sum of squares) I The model predicts y using ŷ = X β̂. The variation in ŷ is∑N i=1(ŷi − ȳ)2 = ESS (explained sum of squares) I A measure of goodness-of-fit is R2 = ESS TSS Multiple linear regression Goodness-of-fit Other things equal, higher values for R2 are better. But: I Sometimes R2 = 0.2 is high, R2 = 0.9 is low I Trying to maximize R2 can be bad practice I R2 can’t fall when a new variable is added Adjusted R2: R̄2 = 1− 1/(N − k) ∑N i=1 e 2 i 1/(N − 1) ∑N i=1(yi − ȳ)2 = 1− N − 1 N − k SSR TSS Multiple linear regression Goodness-of-fit Adjusted R2: I Penalty term for additional variables: (N − 1)/(N − k) I SSR must fall enough (R2 must increase enough) to offset penalty I R̄2 will increase if and only if |tk+1| > 1 Multiple linear regression More general hypothesis testing Suppose we want to test whether some subset of the β’s (say, the last J of them) are all equal to zero: H0 : βk−J+1 = · · · = βk−1 = βk = 0 against the alternative that at least one of these is not zero. We can estimate both models and compare the change in SSR: I SSR from the larger model will be lower I if H0 is true, the change will be small Multiple linear regression More general hypothesis testing Finally, S0 − S1 is independent of s2. Therefore, if H0 is true, F = (S0 − S1)/J S1/(N − k) ∼ F (J,N − k) Large values of F mean that the unrestricted model fits the data significantly better than the restricted model (ie, S1 is substantially smaller than S0). This would be evidence against H0. We can also rewrite F in terms of the R2’s from the two models: F = (R21 − R20 )/J (1− R21 )/(N − k) Multiple linear regression More general hypothesis testing Suppose now that we have a set of J different linear restrictions on the coefficients β. We can write these restrictions as Rβ = q, where R is a J × k matrix and q is a J × 1 vector. For example, the restrictions β2 + β3 + · · ·+ βk = 0 and β2 = β3 can be put in this form with R = [ 0 1 1 · · · · · · 1 0 1 −1 0 · · · 0 ] , q = ( 1 0 ) Multiple linear regression More general hypothesis testing Testing H0 : Rβ = q vs H1 : Rβ 6= q I Could proceed as before: estimate both models and use an F -test to compare R2’s I An alternative, only requiring 1 model, is the Wald test I Another result: β̂ ∼ N(β,V (β̂)) ⇒ Rβ̂ ∼ N(Rβ,RV (β̂)R ′) where V (β̂) = σ2(X ′X )−1