Download Solutions to Homework 3 - Image Understanding | ENEE 731 and more Assignments Electrical and Electronics Engineering in PDF only on Docsity! 1 Solution to HW 3 Solution: 1) f(x)/g(x) = σ exp ( −x 2 2 + x2 2σ2 ) = σ exp ( −x 2 2 (1 − 1 σ2 ) = σ exp ( −γ x 2 2 ) (1) letting γ = (1 − 1/σ2). - Note that if γ < 0 (or, when σ < 1, then for x ∈ R, the ratio f(x)/g(x) is unbounded, implying the conditions of step 1 cannot be satisfied. - If γ ≥ 1(σ ≥ 1), then f(x)/g(x) does have a maxima for x ∈ R. This happens for x = 0. Hence, the smallest value of c = σ. 2) Prob. of sample x getting rejected = 1 − f(x)/(c.g(x)). Prob. that sample x is generated = g(x)dx Overall probability of sample rejection = ∫ x∈R ( 1 − f(x)cg(x) ) g(x)dx = ∫ x=R (g(x) − f(x)/c) dx = 1 − 1/c 3) Figures 1 - 3 show the plots. Code can be downloaded from http://www.umiacs.umd.edu/˜pturaga/ENEE731/papers/HWs/HW3/ar.m 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 Sample no. M ea n Mean Mean +/− Std/3 Fig. 1. Problem 3: Convergence of Mean 4) Figures 4 - 6 show the plots. Code can be downloaded from http://www.umiacs.umd.edu/˜pturaga/ENEE731/papers/HWs/HW3/ar2.m Main note for parts 3 and 4: Note the initial transient part when the statistics are far away from the nominal values of the distribution (mean 0, var 1, kurtosis 3). As with any sample based representation, an individual sample is meaningless in a statistical sense. However, aggregate statistics of samples are more informative and useful. Further, depending on the particular sampling method used (along with initializations if any), the generated samples might converge to the desired density at different rates. Controlling this rate of convergence is important, especially if you are working with finite samples. The whole notion of importance sampling is built to maximize this convergence using domain knowledge. 5) Tail decay rate of the proposal function g(x) has to be slower than the posterior p(x). Check part 1 for another example of this. It is noteworthy that this property is required of almost ALL sampling based methods. 2 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 0 0.2 0.4 0.6 0.8 1 1.2 1.4 Sample no. V ar . Mean Mean +/− Std/3 Fig. 2. Problem 3: Convergence of Variance 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 100 101 102 103 104 105 106 107 108 109 Sample no. K ut os is . Mean Mean +/− Std/3 Fig. 3. Problem 3: Convergence of Kurtosis 6) f(x)/g(x) = 1/ √ 2π 1/2 e−x 2/2 e−|x| , x ∈ R = √ 2 π e−x 2/2 e−x , x ∈ R + = √ 2 π e − 1 2 (x2−2x) = √ 2 π e − 1 2 (x2−2x+1−1) = √ 2 π e − 1 2 ((x−1)2−1) ≤ √ 2 π e − 1 2 (−1) = √ 2e π (2) Equality is obtained at x = ±1
