Data Generation - Stochastic Hydrology - Lecture Notes, Study notes of Mathematical Statistics

Download Data Generation - Stochastic Hydrology - Lecture Notes and more Study notes Mathematical Statistics in PDF only on Docsity! DATA GENERATION 3   Docsity.com Necessity : 1.  3.  Use of historical record alone gives no idea of the risks involved. 4.  Exact pattern of flows during the historical period is extremely unlikely to recur during the economic life of the system. Length of historical record Economic life of the project 4   Data Generation Docsity.com Data Generation •  Given a distribution, to generate data belonging to that distribution 7   Randomly picked up values of F(y) follow a uniform distribution u(0, 1) u     f(u)     0     1     Choose a random F(y) from uniform distribution, get corresponding y. y   F(y)     Docsity.com Data Generation Ru: uniformly distributed random no.s in the interval (0,1) Most scientific programs have built-in functions for generating uniformly distributed random numbers. 8   ( ) ( ) ( ) ( ) y y u F y f y dy F y R f y dy −∞ −∞ = = = ∫ ∫ Docsity.com Data Generation An algorithm for random number (Ru) generation: Xi=(a+bXi-1) Modulo M {Xi/M} are the required random numbers e.g., M = 10, a = 5, b = 3 Let X0 = 2, then X1 = (3*2+5) Modulo 10 = 11 Modulo 10 = 1 X1 = (3*1+5) Modulo 10 = 8 Modulo 10 = 8 9   m Modulo n = Remainder of (m/n)   Docsity.com Generate 10 values from exponential distribution with λ = 5 Example-1 12   S.No. Ru y 1 0.026 0.729932 2 0.85 0.032504 3 0.654 0.08493 4 0.805 0.043383 5 0.205 0.316949 6 0.957 0.00879 7 0.035 0.670481 8 0.285 0.251053 9 0.996 0.000802 10 0.549 0.119931 Σ 2.258755 ln 2.26 0.226 10 1ˆ 1 0.226 4.43 uRy y y λ λ = − = = = = = … generated values   Docsity.com Data Generation •  Analytic inverse transform not possible for some distributions (eg., Normal distribution, Gamma distribution) •  Numerically generated tables of standard normal deviates (RN) available •  Given RN, data is generated by y = σRN+ µ •  Most scientific programs have built-in functions to generate standard normal deviates (RN) . 13   Docsity.com Generate 10 values from N(10, 152) Example-2 14   2 ˆ 14.869 ˆ 191.65 ˆ 13.8 yµ σ σ = = = = S.No. RN y 1 0.335 15.025 2 -0.051 9.235 3 1.226 28.39 4 -0.642 0.37 5 0.377 15.655 6 2.156 42.34 7 0.667 20.005 8 -1.171 -7.565 9 0.28 14.2 10 0.069 11.035 Σ 148.69 y = σRN+ µ y = 15 RN +10   RN obtained from: Statistical methods in Hydrology by C.T.Haan Iowa State University Press 1994 Table No.-E.11   Docsity.com TIME SERIES ANALYSIS 17   Docsity.com Time Series Analysis •  Sequence of values of a random variable collected over time is time series. •  Discrete time series: measured at discrete time intervals •  Continuous time series: recorded continuously with time •  Single time series : A realization •  Ensemble: collection of all realizations {xt}1, {xt}2……. {xt}m 18   Docsity.com Time Series Analysis 19   t     {xt}1   Realization-1 t    Realization-2 {xt}2   t     {xt}m   Realization-m Ensemble: collection of all realizations {xt}1, {xt}2……. {xt}m Docsity.com Time Series Analysis •  Deterministic component is a combination of a long term mean, trend, periodicity and jump. •  Time scale of time series – either discrete or continuous •  Discrete time scale: observations at specific times separated by Δt. (eg., average monthly stream flow, annual peak discharge, daily rainfall etc.) •  Continuous time scale: data recorded continuously with time (eg., turbulence studies, pressure measurements) 22   Docsity.com Time Series Analysis •  The pdf of a stochastic process X(t) is f(x; t) •  f(x; t) describes the probabilistic behavior of X(t) at specified time ‘t’ •  The time series is said to be stationary, if the properties do not change with time. •  f(x; t) = f(x; t+τ) v t •  for stationary series, pdf of Xt is same as Xt+ τ 23   Docsity.com Time Series Analysis Time average for a realization n is no. of observations Ensemble average at time t m is no. of realizations 24   ( ){ } 1 1 1 n j j X t X n == ∑ ( ) 1 m i i t X t X m == ∑ t     {Xt}1   Realization-1 t     Realization-2 {Xt}2   t     {Xt}m   Realization-m t1   t1   t1   Docsity.com Time Series Analysis •  Auto correlation indicates the memory of a stochastic process 27   k     ρk   Correlogram Docsity.com Time Series Analysis •  Auto covariance matrix Γn is symmetric and +ve definite matrix 28   X1 X2 X3 . . Xn 0 1 2 1 1 0 1 2 2 1 1 0 . . . . . . n n n n n γ γ γ γ γ γ γ γ γ γ γ γ − − − − ⎡ ⎤ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ Γ = ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎣ ⎦ X1 X2 X3 . Xn . Docsity.com Time Series Analysis •  Dividing the matrix by γo, Ρn is symmetric and +ve definite matrix 29   1 2 1 1 1 2 2 0 1 2 1 . . 1 . . . . . . . 1 n n n n n n ρ ρ ρ ρ ρ ρ ρ γ ρ ρ − − − − ⎡ ⎤ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥Γ Ρ = = ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥⎣ ⎦ Docsity.com Time Series Analysis •  Auto correlation function (rk) If it is purely stochastic (random) series, ρk = 0, v k = 1, 2, 3,…….. rk = may not be zero (because rk is a sample estimate) 32   10,kr Normal Distribution N ⎛ ⎞ ⎜ ⎟ ⎝ ⎠ : k         rk   Correlogram Docsity.com Time Series Analysis 33   1.96 1.96 krN N − ≤ ≤ + +z -z 95% k         rk   1.96 N 1.96 N − Statistically insignificant z = 1.96 Docsity.com Obtain Auto correlation for k=1 Example-4 34   S.No. Xt Xt+1 1 97 -10.50 110 2.5 -26.25 2 110 2.50 121 13.5 33.75 3 121 13.50 117 9.5 128.25 4 117 9.50 79 -28.5 -270.75 5 79 -28.50 140 32.5 -926.25 6 140 32.50 75 -32.5 -1056.25 7 75 -32.50 127 19.5 -633.75 8 127 19.50 90 -17.5 -341.25 9 90 -17.50 119 11.5 -201.25 10 119 11.50 Σ 1075 -3293.75 ( )tx x− ( )1tx x+ − ( )( )1 t t x x x x+ − × − Docsity.com DATA EXTENSION & FORECASTING 37   Docsity.com Data Extension & Forecasting e.g., Stream flow records for reservoir planning Data forecasting + Random component 38   1970 2010 2040 Available data Extension of data We are here Inflow, It+1 (Forecast) Time, t Time, t+1 Inflow, It (known) [ ]1 1ˆ , ,...........t t tI f I I+ −= Docsity.com Data Extension & Forecasting Use first ‘T’ values to build the model, use rest of data to validate it FT+1 FT+2 ……… FN forecasts obtained from the model (XT+1 – FT+1) (XT+2 – FT+2) . . (XN – FN) 39   X1 Calibration data Test data XT XN XT-2 XT-1 XT+1 Forecast errors   Docsity.com Smoothening technique: Moving Average (MA) • As new observation is available, new average is computed by dropping the oldest observation and including the newest one. • No. of data points in each average remains constant • Deals with the latest ‘T’ periods of known data Data Extension & Forecasting 42   T+1 T T T+2 Docsity.com Example-7 43   Data MA (3) MA (3, 3) 105 - 115 - 103 - 108 120 97 110 121 117 79 107.67 108.67 110.33 108.33 109 109.33 116 108.89 109.11 109.22 108.89 111.44 Docsity.com