Markov Chains - Stochastic Hydrology - Lecture Notes, Study notes of Mathematical Statistics

Download Markov Chains - Stochastic Hydrology - Lecture Notes and more Study notes Mathematical Statistics in PDF only on Docsity! MARKOV CHAINS Docsity.com •  A Markov chain is a stochastic process with the property that value of process Xt at time t depends on its value at time t-1 and not on the sequence of other values (Xt-2 , Xt-3,……. X0) that the process passed through in arriving at Xt-1. 4   Markov Chains [ ] [ ]1 2 0 1, ,.....t t t t tP X X X X P X X− − −= Single step Markov chain   Docsity.com Transition Probability Matrix(TPM): 7   Markov Chains 1 2 3 . . m 11 12 13 1 21 22 23 2 31 1 2 . . . . . . m m m m mm P P P P P P P P P P P P P ⎡ ⎤ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ = ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎣ ⎦ 1 2 3 . m . m x m   t t-1 Docsity.com •  Elements in any row of TPM sum to unity •  TPM can be estimated from observed data by enumerating the number of times the observed data went from state ‘i’ to ‘j’ •  Pj (n) is the probability of being in state ‘j’ in time step ‘n’. 8   Markov Chains V i 1 1 m ij j P = =∑ 1 ˆ ij ij m ij j n P n = = ∑ Docsity.com •  pj(0) is the probability of being in state ‘j’ in period t = 0. •  If p(0) is given and TPM is given 9   Markov Chains ( ) ( ) ( ) ( )0 0 0 0 1 2 1 . . m mp p p p × ⎡ ⎤= ⎣ ⎦ …. Probability vector at time 0 ( ) ( ) ( ) ( ) 1 2 1 . .n n n nm mp p p p × ⎡ ⎤= ⎣ ⎦ …. Probability vector at time ‘n’ ( ) ( )1 0p p P= × Docsity.com •  As the process advances in time, pj(n) becomes less dependent on p(0) •  The probability of being in state ‘j’ after a large number of time steps becomes independent of the initial state of the process. •  The process reaches a steady state at large n •  As the process reaches steady state, the probability vector remains constant 12   Markov Chains p p P= × Docsity.com Example – 1 13   Consider the TPM for a 2-state first order homogeneous Markov chain as State 1 is a non-rainy day and state 2 is a rainy day Obtain the 1.  probability that day 1 is a non-rainy day given that day 0 is a rainy day 2.  probability that day 2 is a rainy day given that day 0 is a non-rainy day 3.  probability that day 100 is a rainy day given that day 0 is a non-rainy day 0.7 0.3 0.4 0.6 TPM ⎡ ⎤= ⎢ ⎥ ⎣ ⎦ Docsity.com Example – 1 (contd.) 14   1.  probability that day 1 is a non-rainy day given that day 0 is a rainy day The probability is 0.4 2.  probability that day 2 is a rainy day given that day 0 is a non-rainy day p(1), in this case is [0.7 0.3] because it is given that day 0 is a non-rainy day. 0.7 0.3 0.4 0.6 TPM ⎡ ⎤= ⎢ ⎥ ⎣ ⎦ No rain rain No rain rain ( ) ( )2 1p p P= × Docsity.com Example – 1 (contd.) 17   Steady state probability [ ] [ ] 0.5714 0.4286 0.5714 0.4286 0.5714 0.4286 0.5714 0.4286 np p P= × ⎡ ⎤ = ⎢ ⎥ ⎣ ⎦ = For steady state, [ ]0.5714 0.4286p = Docsity.com Difficulties in using Markov chains in hydrology •  Determining the number of states to use. •  Determining the intervals of the variable under study to associate with each state. •  Assigning a number to the magnitude of an event once the state is determined. •  Estimating the large number of parameters involved in even a moderate size Markov chain model. •  Handling situations where some transitions are dependent on several previous time periods while others are dependent on only one prior time period. 18   Markov Chains Ref: Statistical methods in Hydrology by C.T.Haan, Iowa state university press Docsity.com MULTIPLE LINEAR REGRESSION 19   Docsity.com •  ‘n’ equations are written for each observation as y1 = β1x1,1 + β2x1,2 + …….. + βpx1,p y2 = β1x2,1 + β2x2,2 + …….. + βpx2,p . . yn = β1xn,1 + β2xn,2 + …….. + βpxn,p •  Solving ‘n’ equations for obtaining the ‘p’ parameters. •  ‘n’ must ne equal to or greater than ‘p’, in practice ‘n’ must be at least 3 to 4 times large as ‘p’. Multiple Linear Regression 22   Docsity.com •  If yi is the ith observation on y and yi,j is the ith observation on the jth independent variable, the generalized form of the equations can be written as •  The equation can be written in matrix notation as Multiple Linear Regression 23   , 1 p i j i j j y xβ = =∑ ( ) ( ) ( )1 1n n p pY X× × ×= × Β Docsity.com 1,1 1,2 1,3 1, 11 2,1 2,2 2,3 2, 22 3,1 33 ,1 ,1 , . . . . . .. . .. p p n n n p pn x x x xy x x x xy xy x x xy β β β β ⎡ ⎤ ⎡ ⎤⎡ ⎤ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ = ×⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦ ⎣ ⎦ 24   Multiple Linear Regression Y is an nx1 vector of observations on the dependent variable, X is an nxp matrix with n observations on each p independent variables, Β is a px1 vector of unknown parameters. Docsity.com •  is a pxp matrix and rank is p •  is made up of sum of squares and cross products of the independent variables and •  This matrix plays an important role in estimating Multiple Linear Regression 27   ( ) ( ) ( ) 1 1' ' ' ' 1' ' ˆ ˆ X Y X X X X X X X Y X X − − − = Β = Β ( ) 1'X X − ( )'X X Β̂ ( ) 1' 'ˆ X X X Y−Β = or Docsity.com •  Suppose if no. of regression coefficients are 3, then matrix is as follows Multiple Linear Regression 28   ( ) 2 ,1 ,2 ,1 ,3 ,1 1 1 1 ' 2 ,1 ,2 ,2 ,3 ,2 1 1 1 2 ,1 ,3 ,2 ,3 ,3 1 1 1 n n n i i i i i i i i n n n i i i i i i i i n n n i i i i i i i i x x x x x X X x x x x x x x x x x = = = = = = = = = ⎡ ⎤ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ = ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ ∑ ∑ ∑ ∑ ∑ ∑ ∑ ∑ ∑ ( )'X X Docsity.com •  Assumption: An independent variable cannot be a perfect linear function of any other independent variable •  For the rank of to be p, an independent variable cannot be linearly dependent on any linear function of the remaining independent variables. •  If there is a linear dependence, the calculation of may involve round off errors or loss of significance leading to non-logical estimates for Multiple Linear Regression 29   ( ) 1'X X − ( )'X X Β̂ Docsity.com The regression model is as follows Q = β1 + β2A + β3R Where Q is the mean flood in thousand m2/sec, A is the watershed area in thousand hectares and R is the average annual daily rainfall in mm This is represented in matrix form as Example – 1 (Contd.) 32   ( ) ( ) ( )12 1 12 3 3 1Y X× × ×= × Β Docsity.com Example – 1 (Contd.) 33   1 2 3 0.44 1 324 43 0.24 1 226 53 2.41 1 1474 48 2.97 1 2142 50 0.7 1 420 43 0.11 1 45 61 0.05 1 38 81 0.51 1 363 68 0.25 1 77 74 0.23 1 84 71 0.1 1 46 71 0.054 1 38 69 β β β ⎡ ⎤ ⎡ ⎤ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎡ ⎢ ⎥ ⎢ ⎥= ×⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦ ⎤ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎣ ⎦ To obtain coefficients this equation is to be solved Docsity.com The coefficients are obtained from Example – 1 (Contd.) 34   ( ) 1' 'ˆ X X X Y−Β = ( ) 2 ,1 ,2 ,1 ,3 ,1 1 1 1 ' 2 ,1 ,2 ,2 ,3 ,2 1 1 1 2 ,1 ,3 ,2 ,3 ,3 1 1 1 n n n i i i i i i i i n n n i i i i i i i i n n n i i i i i i i i x x x x x X X x x x x x x x x x x = = = = = = = = = ⎡ ⎤ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ = ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ ∑ ∑ ∑ ∑ ∑ ∑ ∑ ∑ ∑ Docsity.com Example – 1 (Contd.) 37   ( ) 1' ' 4 4 7 6 6 4 5 ˆ 3.35 6.1 10 0.05 8.06 6.1 10 2.9 10 7.9 10 10642 0.05 7.9 10 7.5 10 417 0.0351 0.0014 5.0135 10 X X X Y − − − − − − − − Β = ⎡ ⎤− × − ⎡ ⎤ ⎢ ⎥ ⎢ ⎥= − × × × ×⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥− × × ⎣ ⎦⎣ ⎦ ⎡ ⎤ ⎢ ⎥= ⎢ ⎥ ⎢ ⎥×⎣ ⎦ Docsity.com Therefore the regression equation is as follows Q = 0.0351 + 0.0014A + 5.0135*10-5R From this equation, the estimated Q and the corresponding errors are tabulated Example – 1 (Contd.) 38   Docsity.com Example – 1 (Contd.) 39   Q A I e 0.44 324 43 0.49 -0.05 0.24 226 53 0.35 -0.11 2.41 1474 48 2.10 0.31 2.97 2142 50 3.04 -0.07 0.7 420 43 0.63 0.07 0.11 45 61 0.10 0.01 0.05 38 81 0.09 -0.04 0.51 363 68 0.55 -0.04 0.25 77 74 0.15 0.10 0.23 84 71 0.16 0.07 0.1 46 71 0.10 0.00 0.054 38 69 0.09 -0.04 Q̂ Docsity.com