Probability Plotting - Stochastic Hydrology - Lecture Notes, Study notes of Mathematical Statistics

Download Probability Plotting - Stochastic Hydrology - Lecture Notes and more Study notes Mathematical Statistics in PDF only on Docsity! Graphical construction: •  Graphical construction is done by transforming the arithmetic scale to probability scale so that a straight line is obtained when cumulative distribution function is plotted. •  The transformation technique is explained with the normal distribution. •  Consider the coordinates from the standardized normal distribution table. 3   Probability Plotting Docsity.com 4   z 0 2 4 6 8 0 0 0.008 0.016 0.0239 0.0319 0.1 0.0398 0.0478 0.0557 0.0636 0.0714 0.2 0.0793 0.0871 0.0948 0.1026 0.1103 0.3 0.1179 0.1255 0.1331 0.1406 0.148 0.4 0.1554 0.1628 0.17 0.1772 0.1844 0.5 0.1915 0.1985 0.2054 0.2123 0.219 0.6 0.2257 0.2324 0.2389 0.2454 0.2517 0.7 0.258 0.2642 0.2704 0.2764 0.2823 0.8 0.2881 0.2939 0.2995 0.3051 0.3106 0.9 0.3159 0.3212 0.3264 0.3315 0.3365 1 0.3413 0.3461 0.3508 0.3554 0.3599 Normal Distribution Tables z (Partial tables shown here) Docsity.com 0.1   0.2   0.4   0.5   0.6  0.05   0.7   0.8   0.95  0.9   0.99  0.01   Transformation plot: 7   Probability Plotting 0.3   F(Z) (probability scale) F(Z) (arithmetic scale) Docsity.com Normal probability paper (probabilities in percentage) : 8   Probability Plotting 99.9 99.0 95.0 90.0 80.0 70.0 50.0 30.0 10.0 5.0 1.0 0.1 50.0 99.0 Redrawn from source: http://www.weibull.com/GPaper/ p(X  <  xm)   Probability      p(X  >  xm)   90.0 1.0 Docsity.com •  The purpose is to check if a data set fits the particular distribution. •  The plot can be used for interpolation, extrapolation and comparison purposes. •  The plot can be used for estimating magnitudes with specified return periods. •  Extrapolation must be attempted only when a reasonable fit is assured for the distribution. 9   Probability Plotting Docsity.com •  Arrange the given series of data in descending order. •  Assign a order number to each of the data (termed as rank of the data). •  Let ‘n’ is the total no. of values to be plotted and ‘m’ is the rank of a value, the exceedence probability (p) of the mth largest value is obtained by various plotting position formulae. •  The return period (T) of the event is calculated by T = 1/p •  Plot magnitude of event verses the exceedence probability (p) or 1 – p or the return period T. 12   Plotting Position Docsity.com Formulae for exceedence probability: California Method: Limitations –  Produces a probability of 100% for m = n 13   Plotting Position ( )m mp X x n ≥ = Docsity.com Modification to California Method: Limitations –  Formula does not produce 100% probability –  If m = 1, probability is zero 14   Plotting Position ( ) 1m mp X x n − ≥ = Docsity.com Most plotting position formulae are represented by: where b is a parameter –  e.g., b = 0.5 for Hazen’s formula, b = 0.5 for Chegodayev’s formula, b = 0 for Weibull’s formula –  b = 3/8 0.5 for Blom’s formula –  b = 1/3 0.5 for Tukey’s formula –  b = 0.44 0.5 for Gringorten’s formula 17   Plotting Position ( ) 1 2m m bp X x n b − ≥ = + − Docsity.com –  Cunnane (1978) studied the various available plotting position methods based on unbiasedness and minimum variance criteria. –  If large number of equally sized samples are plotted, the average of the plotted points for each value of m lie on the theoretical distribution line. –  Minimum variance plotting minimizes the variance of the plotted points about the theoretical line. 18   Plotting Position Ref: Cunnane, C., Unbiased plotting positions – a review, J,Hydrol., Vol. 37, pp.205-222, 1978 Docsity.com –  For normally distributed data, Blom’s plotting position formula (b = 3/8) is commonly used. –  For Extreme Value Type I distribution, the Gringorten formula (b = 0.44) is used. •  All the relationships give similar values near the center of the distribution but may vary near the tails considerably. •  Predicting extreme events depend on the tails of the distribution. 19   Plotting Position Docsity.com The Weibull plotting position formula meets all the 5 of the above criteria. 1. All the observations can be plotted since the plotting positions range from 1/(n+1) (which is greater than 0) to n/(n+1) (which is less than 1). 2. The relationship lies between (m – 1)/n and m/n for all values of m and n. 22   Plotting Position ( ) 1m mp X x n ≥ = + Ref: Statistical methods in Hydrology by C.T.Haan, Iowa state university press Docsity.com 3.  The return period of the largest value is (n+1)/1, which approaches n as n tends to infinity and the return period of the smallest value is (n+1)/n, which approaches 1 as n tends to infinity. 4.  The difference between the plotting position of the (m+1)st and mth value is 1/(n+1) for all values of m and n 5.  The formula is simple and easy to use. 23   Plotting Position Ref: Statistical methods in Hydrology by C.T.Haan, Iowa state university press Docsity.com Consider the annual maximum flow of a river (in MCM) for 60years. 1. Perform the probability plotting analysis using Hazen’s formula. 2. Compare the plotted data with the normal distribution. 24   Example – 1 Docsity.com 27   Arranged data Rank (m) p(X > xm) Arranged data Rank (m) p(X > xm) Arranged data Rank (m) p(X > xm) 3256 1 0.008 2265 21 0.342 1557 41 0.675 3171 2 0.025 2243 22 0.358 1540 42 0.692 3143 3 0.042 2200 23 0.375 1518 43 0.708 2806 4 0.058 2180 24 0.392 1515 44 0.725 2721 5 0.075 2078 25 0.408 1484 45 0.742 2670 6 0.092 2064 26 0.425 1430 46 0.758 2653 7 0.108 2016 27 0.442 1339 47 0.775 2619 8 0.125 2005 28 0.458 1331 48 0.792 2591 9 0.142 1996 29 0.475 1325 49 0.808 2532 10 0.158 1982 30 0.492 1303 50 0.825 2512 11 0.175 1945 31 0.508 1274 51 0.842 2469 12 0.192 1920 32 0.525 1252 52 0.858 2466 13 0.208 1903 33 0.542 1246 53 0.875 2407 14 0.225 1894 34 0.558 1218 54 0.892 2387 15 0.242 1877 35 0.575 1141 55 0.908 2381 16 0.258 1860 36 0.592 983 56 0.925 2347 17 0.275 1773 37 0.608 966 57 0.942 2291 18 0.292 1736 38 0.625 915 58 0.958 2277 19 0.308 1705 39 0.642 804 59 0.975 2268 20 0.325 1642 40 0.658 583 60 0.992 Docsity.com 28   Example – 1 (Contd.) p( X   <   x m )   Pr ob ab ili ty       p( X   >   x m )   0.01   0.5   0.99   0.9   0.1   0.25   0.75   MATLAB Syntax: normplot(data) Docsity.com •  When probability plots of hydrologic data are made, one or more extreme events are present that appear to form a different population because they plot far off the line defined by the other points. •  The separate treatment is required for these outliers. •  Benson (1962c) has stated that these extremes can be treated if the historical information is available. 29   Plotting Position Ref: Benson, M.A., 1962c. Evolution of methods for evaluating the occurrence of floods, water supply paper 1580-A, U.S. Geological survey, Washington, D.C. Docsity.com •  Two tests are discussed Ø  Chi-Square test Ø  Kolmogorov – Smirnov test 32   Tests for Goodness of Fit Docsity.com Chi-Square Goodness of fit test: •  One of the most commonly used tests for goodness of fit of empirical data to specified theoretical frequency distributions. •  The test makes a comparison between the actual number of observations and the expected number of observations that fall in the class intervals. •  The expected numbers are calculated by multiplying the expected relative frequency by total number of observations. 33   Tests for Goodness of Fit Docsity.com Steps to be followed: •  Divide the data into k class intervals. •  Ni is the number of observations falling in each class interval. •  Select the distribution whose adequacy is to be tested. •  Determine the probability with which the RV lies in each of the class interval using the selected distribution. •  Calculate the expected number of observations in any class interval Ei, by multiplying the probability with the total length of sample. 34   Tests for Goodness of Fit Docsity.com 37   Tests for Goodness of Fit 0.995 0.95 0.9 0.1 0.05 0.025 0.01 0.005 1 3.9E-05 0.004 0.016 2.71 3.84 5.02 6.63 7.88 2 0.010 0.103 0.211 4.61 5.99 7.38 9.21 10.60 3 0.072 0.352 0.584 6.25 7.81 9.35 11.34 12.84 4 0.21 0.71 1.06 7.78 9.49 11.14 13.28 14.86 5 0.41 1.15 1.61 9.24 11.07 12.83 15.09 16.75 6 0.68 1.64 2.20 10.64 12.59 14.45 16.81 18.55 7 0.99 2.17 2.83 12.02 14.07 16.01 18.48 20.28 8 1.34 2.73 3.49 13.36 15.51 17.53 20.09 21.95 9 1.73 3.33 4.17 14.68 16.92 19.02 21.67 23.59 10 2.16 3.94 4.87 15.99 18.31 20.48 23.21 25.19 20 7.43 10.85 12.44 28.41 31.41 34.17 37.57 40.00 30 13.79 18.49 20.60 40.26 43.77 46.98 50.89 53.67 α ν Docsity.com •  The number of class intervals should be at least 5. •  If the sample size n is large, the number of class intervals may be approximately fixed by •  It is better to choose non-uniform class intervals so that at least 5 observations in each class interval. •  If more than one distribution passes the test, then the distribution which gives the least value of χ2 is considered. 38   Tests for Goodness of Fit ( )10 1.33lnk n= + Docsity.com Consider the annual maximum discharge Q (in cumec) of a river for 40years, Check whether the data follows a normal distribution using Chi-Square goodness of fit test at 10% significance level. 39   Example – 2 S.No. Q (x106) S.No. Q (x106) S.No. Q (x106) S.No. Q (x106) 1 590 11 501 21 863 31 658 2 618 12 360 22 672 32 646 3 739 13 535 23 1054 33 1000 4 763 14 644 24 858 34 653 5 733 15 700 25 285 35 626 6 318 16 607 26 643 36 543 7 791 17 686 27 479 37 650 8 582 18 411 28 613 38 900 9 529 19 556 29 584 39 765 10 895 20 512 30 900 40 831 Docsity.com From the table, χ2 = 4.448 No. of class intervals, k = 8 No. of parameters estimated, p = 2 Therefore ν = k – p – 1 = 8 – 2 – 1 = 5 Significance level α = 10% = 0.1 42   Example – 2 (Contd.) Docsity.com From the Chi-square distribution table, The hypothesis that the normal distribution fits the data can be accepted. 43   Example – 2 (Contd.) 2 0.1,5 9.24χ = 2 2 0.1,5χ χ< 0.9 0.1 0.05 3 0.584 6.25 7.81 4 1.06 7.78 9.49 5 1.61 9.24 11.07 6 2.20 10.64 12.59 α ν Docsity.com Kolmogorov – Smirnov Goodness of fit test: •  This is an alternative to the Chi-square test. •  The test is conducted as follows •  The data is arranged in descending order of magnitude. •  The cumulative probability P(xi) for each of the observations are calculated using the Weibull’s formula. •  The theoretical cumulative probability F(xi) for each of the observation is obtained using the assumed distribution. 44   Testing – Goodness of Fit of Data Docsity.com •  The advantage of Kolmogorov-Smirnov test over the Chi-square test is that it does not lump the data and compare only the discrete categories. •  It is easier to compute Δ than Χ2. •  This test is more convenient to adopt when the sample size is small. 47   Testing – Goodness of Fit of Data Docsity.com Consider the annual maximum discharge Q (in cumec) of a river for 20years, Check whether the data follows a normal distribution using Kolmogorov-Smirnov goodness of fit test at 10% significance level. 48   Example – 3 S.No. Q (x106) S.No. Q (x106) 1 590 11 501 2 618 12 360 3 739 13 535 4 763 14 644 5 733 15 700 6 318 16 607 7 791 17 686 8 582 18 411 9 529 19 556 10 895 20 831 Docsity.com Mean, = 619.62 Standard deviation, s = 153.32 Data is arranged in the descending order and a rank is assigned. The probability is obtained using Weibull’s formula 49   Example – 3 (Contd.) x ( ) 1m mP X x n ≥ = + Docsity.com General comments on Goodness of fit tests: •  Many hydrologists discourage the use of these tests when testing hydrologic frequency distributions •  The reason is the importance of the tails of hydrologic frequency distributions and insensitivity of these tests in the tails of the distributions. •  The -------- (TO BE CHECKED) 52   Testing – Goodness of Fit of Data Ref: Statistical methods in Hydrology by C.T.Haan, Iowa state university press Docsity.com INTENSITY-DURATION-FREQUENCY (IDF) CURVES Docsity.com •  In many of the hydrologic projects, the first step is the determination of rainfall event –  E.g., urban drainage system •  Most common way is using the IDF relationship •  An IDF curve gives the expected rainfall intensity of a given duration of storm having desired frequency of occurrence. •  IDF curve is a graph with duration plotted as abscissa, intensity as ordinate and a series of curves, one for each return period •  Standard IDF curves are available for a site. 54   IDF Curves Docsity.com Procedure for developing IDF curves: Step 1: Preparation of annual maximum data series •  From the available rainfall data, rainfall series for different durations (e.g.,1H, 2H, 6H, 12H and 24H) are developed. •  For each selected duration, the annual maximum rainfall depths are calculated. 57   IDF Curves Docsity.com Step 2: Fitting the probability distribution •  A suitable probability distribution is fitted to the each selected duration data series . •  Generally used probability distributions are –  Gumbel’s Extreme Value distribution –  Gamma distribution (two parameter) –  Log Pearson Type III distribution –  Normal distribution –  Log-normal distribution (two parameter) 58   IDF Curves Docsity.com Statistical distributions and their functions: 59   IDF Curves Distribution PDF Gumbel’s EVT-I Gamma Log Pearson Type-III Normal Log-Normal –∞ < x < ∞ ( ) ( ) ( ) ( ) 1 yy e f x x β λ εβλ ε β − − −− = Γ log x ε≥ ( )2 2ln 21( ) e 2 x xx x f x x µ σ π σ − −= 0 x< < ∞ 21 1( ) exp 22 xf x µ σπσ ⎧ ⎫−⎪ ⎪⎛ ⎞= −⎨ ⎬⎜ ⎟ ⎝ ⎠⎪ ⎪⎩ ⎭ x−∞ < < ∞ ( ) 1 ( ) n xx ef x η λλ η − − = Γ , , 0x λ η > ( ) ( ){ }( ) exp expf x x xβ α β α α⎡ ⎤= − − − − −⎣ ⎦ Docsity.com Using frequency factors: •  The precipitation depths is calculated for a given return period as where is mean of the data, s is the standard deviation and KT is the frequency factor 62   IDF Curves T Tx x K s= + x Docsity.com Using CDF of a distribution: •  Using the parameters of the distribution which are caluclated in the previous step, a probability model is formulated. •  The formulated probability model is inversed and xT value for a given return period is calculated by 63   IDF Curves ( ) ( ) ( ) ( ) 1 1; 1 1 1 11 ; 1 T T T T P X x P X x T T TF x F x T T T ≥ = − < = − − = = − = Docsity.com •  The precipitation depths calculated from the annual exceedence series is adjusted to match the depths derived from annual maximum series by multiplying a factor •  No adjustment of the estimates is required for longer return periods (> 10 year return period) 64   IDF Curves Return Period Factor 2 0.88 5 0.96 10 0.99 Docsity.com •  From the hourly data, the rainfall for different durations are obtained. 67   Example – 4 (Contd.) Docsity.com Rainfall in mm for different durations. 68   Example – 4 (Contd.) Year 1H 2H 6H 12H 24H 1969 44.5 61.6 104.1 112.3 115.9 1970 37 48.2 62.5 69.6 92.7 1971 41 52.9 81.4 86.9 98.7 1972 30 40 53.9 57.8 65.2 1973 40.5 53.9 55.5 72.4 89.8 1974 52.4 62.4 83.2 93.4 152.5 1975 59.6 94 95.1 95.1 95.3 1976 22.1 42.9 61.6 64.5 71.7 1977 42.2 44.5 47.5 60 61.9 1978 35.5 36.8 52.1 54.2 57.5 1979 59.5 117 132.5 135.6 135.6 1980 48.2 57 82 86.8 89.1 1981 41.7 58.6 64.5 65.1 68.5 1982 37.3 43.8 50.5 76.2 77.2 1983 37 60.4 70.5 72 75.2 1984 60.2 74.1 76.6 121.9 122.4 Year 1H 2H 6H 12H 24H 1986 65.2 73.7 97.9 103.9 104.2 1987 47 55.9 64.8 65.6 67.5 1988 148.8 210.8 377.6 432.8 448.7 1989 41.7 47 51.7 53.7 78.1 1990 40.9 71.9 79.7 81.5 81.6 1991 41.1 49.3 63.6 93.2 147 1992 31.4 56.4 76 81.6 83.1 1993 34.3 36.7 52.8 68.8 70.5 1994 23.2 38.7 41.9 43.4 50.8 1995 44.2 62.2 72 72.2 72.4 1996 57 74.8 85.8 86.5 90.4 1997 50 71.1 145.9 182.3 191.3 1998 72.1 94.6 111.9 120.5 120.5 1999 59.3 62.9 82.3 90.7 90.9 2000 62.3 78.3 84.3 84.3 97.2 2001 46.8 70 95.9 95.9 100.8 2003 53.2 86.5 106.1 106.2 106.8 Docsity.com The rainfall intensity (mm/hr) for different durations. 69   Example – 4 (Contd.) Year 1H 2H 6H 12H 24H 1969 44.50 30.80 17.35 9.36 4.83 1970 37.00 24.10 10.42 5.80 3.86 1971 41.00 26.45 13.57 7.24 4.11 1972 30.00 20.00 8.98 4.82 2.72 1973 40.50 26.95 9.25 6.03 3.74 1974 52.40 31.20 13.87 7.78 6.35 1975 59.60 47.00 15.85 7.93 3.97 1976 22.10 21.45 10.27 5.38 2.99 1977 42.20 22.25 7.92 5.00 2.58 1978 35.50 18.40 8.68 4.52 2.40 1979 59.50 58.50 22.08 11.30 5.65 1980 48.20 28.50 13.67 7.23 3.71 1981 41.70 29.30 10.75 5.43 2.85 1982 37.30 21.90 8.42 6.35 3.22 1983 37.00 30.20 11.75 6.00 3.13 1984 60.20 37.05 12.77 10.16 5.10 Year 1H 2H 6H 12H 24H 1986 65.20 36.85 16.32 8.66 4.34 1987 47.00 27.95 10.80 5.47 2.81 1988 148.80 105.40 62.93 36.07 18.70 1989 41.70 23.50 8.62 4.48 3.25 1990 40.90 35.95 13.28 6.79 3.40 1991 41.10 24.65 10.60 7.77 6.13 1992 31.40 28.20 12.67 6.80 3.46 1993 34.30 18.35 8.80 5.73 2.94 1994 23.20 19.35 6.98 3.62 2.12 1995 44.20 31.10 12.00 6.02 3.02 1996 57.00 37.40 14.30 7.21 3.77 1997 50.00 35.55 24.32 15.19 7.97 1998 72.10 47.30 18.65 10.04 5.02 1999 59.30 31.45 13.72 7.56 3.79 2000 62.30 39.15 14.05 7.03 4.05 2001 46.80 35.00 15.98 7.99 4.20 2003 53.20 43.25 17.68 8.85 4.45 Docsity.com The rainfall intensities are calculated using For example, For duration of 2 hour, and 10 year return period, Mean = 33.17 mm/hr, Standard deviation s = 15.9 mm/hr Frequency factor KT = 1.035 72   Example – 4 (Contd.) T Tx x K s= + x Docsity.com xT = 33.17 + 1.035 * 15.9 = 53.9 mm/hr. The values or other durations are tabulated. 73   Example – 4 (Contd.) Duration (hours) Return Period T (Years) 2 5 10 50 100 1H 45.17 64.19 76.79 104.51 116.23 2H 30.55 44.60 53.90 74.36 83.02 6H 12.89 21.36 26.97 39.31 44.53 12H 7.14 12.02 15.25 22.36 25.37 24H 3.91 6.44 8.11 11.79 13.35 Docsity.com 74   Example – 4 (Contd.) 0   20   40   60   80   100   120   140   0H   1H   2H   6H   12H   24H   36H   In te ns ity  (m m /h r)   Dura4on  (Hours)   2Year   5Year   10Year   50Year   100Year   Docsity.com