Download Solving Likelihood Equations using Newton's Method in Statistical Inference and more Assignments Statistics in PDF only on Docsity! Stat 411 – Homework 03 Due: Friday 02/03 1. Problem 6.1.3 on pages 317–318. 2. Problem 6.1.6 on page 318. [Hint: Refer back to Problem 6.1.3(c).] 3. Problem 6.1.9 on page 318. [Hint: Refer back to Problem 6.1.3(a).] 4. Problem 6.1.11 on page 319. 5. Let θ be a scalar (one-dimensional) parameter and let `(θ) denote the log-likelihood function. Then the MLE θ̂ is a solution of the likelihood equation `′(θ) = 0. It may happen that there is a unique solution to the likelihood equation, but there’s no formula for it (e.g., in a gamma shape parameter problem). In such cases, the solution must be found numerically. A powerful method for such problems is Newton’s method, covered in early calculus courses. Describe how Newton’s method can be used to solve the likelihood equation. You may assume that `(θ) is at least twice differentiable. 6. (Graduate Only) Consider independent samples from two normal populations: X11, . . . , X1n1 iid∼ N(θ1, 1) and X21, . . . , X2n2 iid∼ N(θ2, 1), independent throughout. Suppose that θ1 ≤ θ2. Find the MLE of (θ1, θ2). You may find it helpful to sketch the parameter space Θ = {(θ1, θ2) : θ1 ≤ θ2}. 1 Stat 411 – Homework 03 Solutions 1. Problem 6.1.3. (a) Let X1, . . . , Xn be iid with PMF fθ(x) = e −θθx/x!, x = 0, 1, 2, . . . (a Poisson distribution). Then the likelihood function is Lx(θ) = n∏ i=1 e−θ θxi xi! = const · e−nθθx1+···+xn . Taking log and then derivative gives 0 set = ∂ ∂θ logLx(θ) = ∂ ∂θ [ const− nθ + (x1 + · · ·+ xn) log θ ] = −n+ nx θ . From here it’s clear that the MLE is θ̂ = x. (b) Let X1, . . . , Xn be iid with PDF fθ(x) = θx θ−1, x ∈ (0, 1) (a beta distribution). The likelihood function is Lx(θ) = n∏ i=1 θxθ−1i = θ n ( n∏ i=1 xi )θ−1 . Taking log and then derivative gives 0 set = ∂ ∂θ logLx(θ) = ∂ ∂θ [ n log θ + (θ − 1) n∑ i=1 log xi ] = n θ + n∑ i=1 log xi. Therefore, the MLE is θ̂ = −n( ∑n i=1 log xi) −1. (c) Let X1, . . . , Xn be iid with PDF fθ(x) = (1/θ)e −x/θ, x > 0 (an exponential distribution with mean θ). The likelihood equation is Lx(θ) = n∏ i=1 1 θ e−xi/θ = 1 θn e−(x1+···+xn)/θ. Taking log and then derivative gives 0 set = ∂ ∂θ logLx(θ) = ∂ ∂θ [ −n log θ − nx/θ ] = −n θ + nx θ2 . Then the MLE is clearly θ̂ = x. (d) Let X1, . . . , Xn be iid with PDF fθ(x) = e −(x−θ)I[θ,∞)(x) (a shifted exponential distribution). Here, like in the uniform problem from class, we use the indicator function because the support set depends on θ. The likelihood function is Lx(θ) = n∏ i=1 e−(xi−θ)I[θ,∞)(xi) ∝ enθI[θ,∞)(x(1)). For the right-hand side, the logic is as follows. First, the proportionality constant is e−nx, which can be ignored because it has nothing to do with θ. Second, the product of indicators is 1 if and only if all the indicators are 1; all the indicators are 1 if and only if all the Xi’s are ≥ θ; all the data are ≥ θ if and only if X(1), the sample minimum, is ≥ θ. But enθ is strictly increasing and, since θ ≤ x(1), it must be that θ̂ = x(1). 1
