Quantitative and Graphical Model Selection


Lecture

2023-11-15

Motivation

We have to make choices about which distribution to use, which covariates (if any) to use for nonstationarity, which (if any) parameters to model as nonstationary, how to pool information across space, etc. There is no single “right” answer; how can we proceed in a principled way?

This is a general problem in statistics beyond extreme value analysis.

Vibe checks with plots

Today

  1. Vibe checks with plots

  2. Just Give Me a Number

Data and fit

Code
annmax_precip = CSV.read("data/dur01d_ams_na14v11_houston.csv", DataFrame)
hobby = annmax_precip[annmax_precip.name .== "HOUSTON HOBBY AP", :]
hobby_bayes = gevfitbayes(hobby, :precip_in)
plot(
    hobby.year,
    hobby.precip_in;
    xlabel="Year",
    ylabel="Annual Max Precip (in)",
    label="HOUSTON HOBBY AP",
    marker=:circ,
)

Plot the distribution

  • What we plot: histogram of the data and the probability density function
  • Ideal case: the histogram and the PDF appear to show the same distribution
  • Warnings: systematic deviations
  • Limitations: hard to learn much about the tails of the distribution from this plot
Extremes.histplot(hobby_bayes)
Data 0 10 20 30 40 50 60 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4 3.6 3.8 4.0 4.2 4.4 4.6 4.8 5.0 5.2 5.4 5.6 5.8 6.0 6.2 6.4 6.6 6.8 7.0 7.2 7.4 7.6 7.8 8.0 8.2 8.4 8.6 8.8 9.0 9.2 9.4 9.6 9.8 10.0 10.2 10.4 10.6 10.8 11.0 11.2 11.4 11.6 11.8 12.0 12.2 12.4 12.6 12.8 13.0 13.2 13.4 13.6 13.8 14.0 14.2 14.4 14.6 14.8 15.0 15.2 15.4 15.6 15.8 16.0 16.2 16.4 16.6 16.8 17.0 17.2 17.4 17.6 17.8 18.0 18.2 18.4 18.6 18.8 19.0 19.2 19.4 19.6 19.8 20.0 20.2 20.4 20.6 20.8 21.0 21.2 21.4 21.6 21.8 22.0 22.2 22.4 22.6 22.8 23.0 23.2 23.4 23.6 23.8 24.0 24.2 24.4 24.6 24.8 25.0 25.2 25.4 25.6 25.8 26.0 26.2 26.4 26.6 26.8 27.0 27.2 27.4 27.6 27.8 28.0 28.2 28.4 28.6 28.8 29.0 29.2 29.4 29.6 29.8 30.0 30.2 30.4 30.6 30.8 31.0 31.2 31.4 31.6 31.8 32.0 32.2 32.4 32.6 32.8 33.0 33.2 33.4 33.6 33.8 34.0 34.2 34.4 34.6 34.8 35.0 35.2 35.4 35.6 35.8 36.0 36.2 36.4 36.6 36.8 37.0 37.2 37.4 37.6 37.8 38.0 38.2 38.4 38.6 38.8 39.0 39.2 39.4 39.6 39.8 40.0 40.2 40.4 40.6 40.8 41.0 41.2 41.4 41.6 41.8 42.0 42.2 42.4 42.6 42.8 43.0 43.2 43.4 43.6 43.8 44.0 44.2 44.4 44.6 44.8 45.0 45.2 45.4 45.6 45.8 46.0 46.2 46.4 46.6 46.8 47.0 47.2 47.4 47.6 47.8 48.0 48.2 48.4 48.6 48.8 49.0 49.2 49.4 49.6 49.8 50.0 50.2 50.4 50.6 50.8 51.0 51.2 51.4 51.6 51.8 52.0 52.2 0 100 h,j,k,l,arrows,drag to pan i,o,+,-,scroll,shift-drag to zoom r,dbl-click to reset c for coordinates ? for help ? 0.0 0.1 0.2 0.3 0.00 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.10 0.11 0.12 0.13 0.14 0.15 0.16 0.17 0.18 0.19 0.20 0.21 0.22 0.23 0.24 0.25 0.26 0.27 0.28 0.29 0.30 0.000 0.001 0.002 0.003 0.004 0.005 0.006 0.007 0.008 0.009 0.010 0.011 0.012 0.013 0.014 0.015 0.016 0.017 0.018 0.019 0.020 0.021 0.022 0.023 0.024 0.025 0.026 0.027 0.028 0.029 0.030 0.031 0.032 0.033 0.034 0.035 0.036 0.037 0.038 0.039 0.040 0.041 0.042 0.043 0.044 0.045 0.046 0.047 0.048 0.049 0.050 0.051 0.052 0.053 0.054 0.055 0.056 0.057 0.058 0.059 0.060 0.061 0.062 0.063 0.064 0.065 0.066 0.067 0.068 0.069 0.070 0.071 0.072 0.073 0.074 0.075 0.076 0.077 0.078 0.079 0.080 0.081 0.082 0.083 0.084 0.085 0.086 0.087 0.088 0.089 0.090 0.091 0.092 0.093 0.094 0.095 0.096 0.097 0.098 0.099 0.100 0.101 0.102 0.103 0.104 0.105 0.106 0.107 0.108 0.109 0.110 0.111 0.112 0.113 0.114 0.115 0.116 0.117 0.118 0.119 0.120 0.121 0.122 0.123 0.124 0.125 0.126 0.127 0.128 0.129 0.130 0.131 0.132 0.133 0.134 0.135 0.136 0.137 0.138 0.139 0.140 0.141 0.142 0.143 0.144 0.145 0.146 0.147 0.148 0.149 0.150 0.151 0.152 0.153 0.154 0.155 0.156 0.157 0.158 0.159 0.160 0.161 0.162 0.163 0.164 0.165 0.166 0.167 0.168 0.169 0.170 0.171 0.172 0.173 0.174 0.175 0.176 0.177 0.178 0.179 0.180 0.181 0.182 0.183 0.184 0.185 0.186 0.187 0.188 0.189 0.190 0.191 0.192 0.193 0.194 0.195 0.196 0.197 0.198 0.199 0.200 0.201 0.202 0.203 0.204 0.205 0.206 0.207 0.208 0.209 0.210 0.211 0.212 0.213 0.214 0.215 0.216 0.217 0.218 0.219 0.220 0.221 0.222 0.223 0.224 0.225 0.226 0.227 0.228 0.229 0.230 0.231 0.232 0.233 0.234 0.235 0.236 0.237 0.238 0.239 0.240 0.241 0.242 0.243 0.244 0.245 0.246 0.247 0.248 0.249 0.250 0.251 0.252 0.253 0.254 0.255 0.256 0.257 0.258 0.259 0.260 0.261 0.262 0.263 0.264 0.265 0.266 0.267 0.268 0.269 0.270 0.271 0.272 0.273 0.274 0.275 0.276 0.277 0.278 0.279 0.280 0.281 0.282 0.283 0.284 0.285 0.286 0.287 0.288 0.289 0.290 0.291 0.292 0.293 0.294 0.295 0.296 0.297 0.298 0.299 0.300 0.301 0.0 0.3 Density Density Plot

Probability plot

  • What we plot: empirical CDF (1 - AEP) of against the fitted GEV’s CDF
  • Ideal case: a straight line, indicating perfect agreement between the empirical CDF ann the fitted CDF
  • Warnings: curvature or systematic deviations from the line, especially in the tails
  • Limitations: sampling uncertainty!
Extremes.probplot(hobby_bayes)
Model 0.0 0.5 1.0 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.00 0.000 0.005 0.010 0.015 0.020 0.025 0.030 0.035 0.040 0.045 0.050 0.055 0.060 0.065 0.070 0.075 0.080 0.085 0.090 0.095 0.100 0.105 0.110 0.115 0.120 0.125 0.130 0.135 0.140 0.145 0.150 0.155 0.160 0.165 0.170 0.175 0.180 0.185 0.190 0.195 0.200 0.205 0.210 0.215 0.220 0.225 0.230 0.235 0.240 0.245 0.250 0.255 0.260 0.265 0.270 0.275 0.280 0.285 0.290 0.295 0.300 0.305 0.310 0.315 0.320 0.325 0.330 0.335 0.340 0.345 0.350 0.355 0.360 0.365 0.370 0.375 0.380 0.385 0.390 0.395 0.400 0.405 0.410 0.415 0.420 0.425 0.430 0.435 0.440 0.445 0.450 0.455 0.460 0.465 0.470 0.475 0.480 0.485 0.490 0.495 0.500 0.505 0.510 0.515 0.520 0.525 0.530 0.535 0.540 0.545 0.550 0.555 0.560 0.565 0.570 0.575 0.580 0.585 0.590 0.595 0.600 0.605 0.610 0.615 0.620 0.625 0.630 0.635 0.640 0.645 0.650 0.655 0.660 0.665 0.670 0.675 0.680 0.685 0.690 0.695 0.700 0.705 0.710 0.715 0.720 0.725 0.730 0.735 0.740 0.745 0.750 0.755 0.760 0.765 0.770 0.775 0.780 0.785 0.790 0.795 0.800 0.805 0.810 0.815 0.820 0.825 0.830 0.835 0.840 0.845 0.850 0.855 0.860 0.865 0.870 0.875 0.880 0.885 0.890 0.895 0.900 0.905 0.910 0.915 0.920 0.925 0.930 0.935 0.940 0.945 0.950 0.955 0.960 0.965 0.970 0.975 0.980 0.985 0.990 0.995 1.000 0 1 0.98967559243492530.9883720930232558 0.98407312824275870.9767441860465116 0.96306353740975530.9651162790697675 0.94821716461897480.9534883720930233 0.94009442577372620.9418604651162791 0.92595871243092850.9302325581395349 0.92342041599021730.9186046511627907 0.9194025295916060.9069767441860465 0.9028924369371940.8953488372093024 0.89595419107518030.8837209302325582 0.8595738835795220.872093023255814 0.85678190564000820.8604651162790697 0.84303554372622850.8488372093023255 0.8283683153386910.8372093023255814 0.82336026283044670.8255813953488372 0.81202944964050830.813953488372093 0.8080621189616240.8023255813953488 0.79982656637996330.7906976744186046 0.7695545976690370.7790697674418605 0.75725396811524330.7674418604651163 0.75082259372960060.7558139534883721 0.7441964541730840.7441860465116279 0.7441964541730840.7325581395348837 0.74307276622982820.7209302325581395 0.71938888610203780.7093023255813954 0.68771108355968880.6976744186046512 0.6644441223974680.686046511627907 0.65679961195599890.6744186046511628 0.63933143286382620.6627906976744186 0.6376982483497630.6511627906976745 0.61922464862120740.6395348837209303 0.59433265902445920.627906976744186 0.59433265902445920.6162790697674418 0.59064778237978470.6046511627906976 0.58129269772366590.5930232558139535 0.56589670346352080.5813953488372093 0.5459048233016650.5697674418604651 0.53141443505362980.5581395348837209 0.52719929256446960.5465116279069767 0.52507923593857630.5348837209302325 0.51866920540677320.5232558139534884 0.51218456806076310.5116279069767442 0.510006492650670.5 0.510006492650670.4883720930232558 0.50782017338768830.47674418604651164 0.4922864242587160.46511627906976744 0.464739001972302950.45348837209302323 0.44092954161341380.4418604651162791 0.433642209232127540.43023255813953487 0.40386804952212070.4186046511627907 0.391186708781732230.4069767441860465 0.391186708781732230.3953488372093023 0.375775907958248830.38372093023255816 0.370595418839957560.37209302325581395 0.344404947789251460.36046511627906974 0.344404947789251460.3488372093023256 0.33115536670153840.3372093023255814 0.30979632189817720.32558139534883723 0.29100036627298330.313953488372093 0.28023898899918060.3023255813953488 0.26947745160260350.29069767441860467 0.232002329805032550.27906976744186046 0.229347444821168980.26744186046511625 0.224050461539215120.2558139534883721 0.221408859287325040.2441860465116279 0.218772184677934860.23255813953488372 0.2082798770036840.22093023255813954 0.18506241153164380.20930232558139536 0.172454647678992550.19767441860465115 0.172454647678992550.18604651162790697 0.16009933212260420.1744186046511628 0.148032613714149640.16279069767441862 0.14329490169997590.1511627906976744 0.136290967854083760.13953488372093023 0.129417176970771680.12790697674418605 0.124910802834650590.11627906976744186 0.12046801912526150.10465116279069768 0.113928006070569380.09302325581395349 0.105451343871516240.08139534883720931 0.09928573683295180.06976744186046512 0.097268466371047760.05813953488372093 0.061288035219081530.046511627906976744 0.058179236211390.03488372093023256 0.0144381777016325070.023255813953488372 0.0054201120951420370.011627906976744186 h,j,k,l,arrows,drag to pan i,o,+,-,scroll,shift-drag to zoom r,dbl-click to reset c for coordinates ? for help ? 0.0 0.5 1.0 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.00 0.000 0.005 0.010 0.015 0.020 0.025 0.030 0.035 0.040 0.045 0.050 0.055 0.060 0.065 0.070 0.075 0.080 0.085 0.090 0.095 0.100 0.105 0.110 0.115 0.120 0.125 0.130 0.135 0.140 0.145 0.150 0.155 0.160 0.165 0.170 0.175 0.180 0.185 0.190 0.195 0.200 0.205 0.210 0.215 0.220 0.225 0.230 0.235 0.240 0.245 0.250 0.255 0.260 0.265 0.270 0.275 0.280 0.285 0.290 0.295 0.300 0.305 0.310 0.315 0.320 0.325 0.330 0.335 0.340 0.345 0.350 0.355 0.360 0.365 0.370 0.375 0.380 0.385 0.390 0.395 0.400 0.405 0.410 0.415 0.420 0.425 0.430 0.435 0.440 0.445 0.450 0.455 0.460 0.465 0.470 0.475 0.480 0.485 0.490 0.495 0.500 0.505 0.510 0.515 0.520 0.525 0.530 0.535 0.540 0.545 0.550 0.555 0.560 0.565 0.570 0.575 0.580 0.585 0.590 0.595 0.600 0.605 0.610 0.615 0.620 0.625 0.630 0.635 0.640 0.645 0.650 0.655 0.660 0.665 0.670 0.675 0.680 0.685 0.690 0.695 0.700 0.705 0.710 0.715 0.720 0.725 0.730 0.735 0.740 0.745 0.750 0.755 0.760 0.765 0.770 0.775 0.780 0.785 0.790 0.795 0.800 0.805 0.810 0.815 0.820 0.825 0.830 0.835 0.840 0.845 0.850 0.855 0.860 0.865 0.870 0.875 0.880 0.885 0.890 0.895 0.900 0.905 0.910 0.915 0.920 0.925 0.930 0.935 0.940 0.945 0.950 0.955 0.960 0.965 0.970 0.975 0.980 0.985 0.990 0.995 1.000 0 1 Empirical Probability Plot

QQ plot

  • What we plot: quantiles (i.e., return levels) of the data against quantiles of the fitted GEV
  • Ideal case: a straight line through the data
  • Warnings: curvature or systematic deviations from the line, especially in the tails
  • Limitations: sampling uncertainty!
Extremes.qqplot(hobby_bayes)
Model 0 5 10 15 20 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 3.0 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 4.0 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9 5.0 5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9 6.0 6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8 6.9 7.0 7.1 7.2 7.3 7.4 7.5 7.6 7.7 7.8 7.9 8.0 8.1 8.2 8.3 8.4 8.5 8.6 8.7 8.8 8.9 9.0 9.1 9.2 9.3 9.4 9.5 9.6 9.7 9.8 9.9 10.0 10.1 10.2 10.3 10.4 10.5 10.6 10.7 10.8 10.9 11.0 11.1 11.2 11.3 11.4 11.5 11.6 11.7 11.8 11.9 12.0 12.1 12.2 12.3 12.4 12.5 12.6 12.7 12.8 12.9 13.0 13.1 13.2 13.3 13.4 13.5 13.6 13.7 13.8 13.9 14.0 14.1 14.2 14.3 14.4 14.5 14.6 14.7 14.8 14.9 15.0 15.1 15.2 15.3 15.4 15.5 15.6 15.7 15.8 15.9 16.0 16.1 16.2 16.3 16.4 16.5 16.6 16.7 16.8 16.9 17.0 17.1 17.2 17.3 17.4 17.5 17.6 17.7 17.8 17.9 18.0 18.1 18.2 18.3 18.4 18.5 18.6 18.7 18.8 18.9 19.0 19.1 19.2 19.3 19.4 19.5 19.6 19.7 19.8 19.9 20.0 0 20 16.15348766209949716.86 12.96165458145544314.63 11.37458643221427411.12 10.3546891751633469.95 9.6177831033539829.48 9.0478410273003728.83 8.5868919930657078.73 8.2021092323487138.58 7.873234162100728.05 7.5869509266993247.86 7.3340739926804337.07 7.1080200157450917.02 6.9039218312657696.79 6.7180872456744256.57 6.5476539626639046.5 6.3903615203210796.35 6.2443958885145246.3 6.1082807556092726.2 5.9807997159714145.87 5.8609394432174745.75 5.7478474431558345.69 5.6408001423035825.63 5.539178436665345.63 5.44244871342340455.62 5.3501479469268395.42 5.2618718684761315.18 5.1772654834576245.02 5.0960154011408014.97 5.0178435786414094.86 4.9425021786114784.85 4.8697693117175174.74 4.7994454877155564.6 4.731350638268374.6 4.66532160428186154.58 4.6012100030685914.53 4.5388804079277144.45 4.4782087860935364.35 4.4190811514153534.28 4.3613923962986984.26 4.3050452738861914.25 4.2499495065766074.22 4.1960210010671784.19 4.1431811533792714.18 4.0913562299607164.18 4.0404768130788764.17 3.99047730042627664.1 3.9412954502317783.98 3.89287196426291843.88 3.84515010196351973.85 3.79807531962741153.73 3.7515949289871733.68 3.7056577699107223.68 3.66021389205492963.62 3.61521424032316263.6 3.57061033880314233.5 3.52635396750366863.5 3.48239682563237673.45 3.438690174315923.37 3.39518445049225553.3 3.35182884210833663.26 3.30857081260299963.22 3.2653555597569283.08 3.22212539008204723.07 3.1788189846176113.05 3.13537052473941553.04 3.09170863654505233.03 3.04775509831026662.99 3.0034232355229662.9 2.95861589913922932.85 2.91322288027198262.85 2.86711755086619572.8 2.82015242223872332.75 2.77215315963188052.73 2.7229103419263632.7 2.67216783890205762.67