Retrieve MCMC convergence diagnostics for a rater fit

mcmc_diagnostics(fit, pars = c("pi", "theta"))

Arguments

fit: An rater mcmc_fit object.
pars: A character vector of parameter names to return. By default c("pi", "theta").

Value

A matrix where the columns represent different diagnostics and the rows are different parameters. Currently the first column contains the Rhat statistic and the second bulk effective samples size. The rownames contain the parameter names.

Details

MCMC diagnostics cannot be calculate for the z due to the marginalisation used to fit the models.

These MCMC diagnostics are intended as basic sanity check of the quality of the MCMC samples returned. Users who want more in depth diagnostics should consider using as_mcmc.list() to convert the samples to a coda::mcmc.list() object, or get_stanfit() to extract the underlying stanfit object.

References

Aki Vehtari, Andrew Gelman, Daniel Simpson, Bob Carpenter, and Paul-Christian Bürkner (2019). Rank-normalization, folding, and localization: An improved R-hat for assessing convergence of MCMC. arXiv preprint arXiv:1903.08008.

Examples

# \donttest{

fit <- rater(anesthesia, "dawid_skene")
#> 
#> SAMPLING FOR MODEL 'dawid_skene' NOW (CHAIN 1).
#> Chain 1: 
#> Chain 1: Gradient evaluation took 0.000202 seconds
#> Chain 1: 1000 transitions using 10 leapfrog steps per transition would take 2.02 seconds.
#> Chain 1: Adjust your expectations accordingly!
#> Chain 1: 
#> Chain 1: 
#> Chain 1: Iteration:    1 / 2000 [  0%]  (Warmup)
#> Chain 1: Iteration:  200 / 2000 [ 10%]  (Warmup)
#> Chain 1: Iteration:  400 / 2000 [ 20%]  (Warmup)
#> Chain 1: Iteration:  600 / 2000 [ 30%]  (Warmup)
#> Chain 1: Iteration:  800 / 2000 [ 40%]  (Warmup)
#> Chain 1: Iteration: 1000 / 2000 [ 50%]  (Warmup)
#> Chain 1: Iteration: 1001 / 2000 [ 50%]  (Sampling)
#> Chain 1: Iteration: 1200 / 2000 [ 60%]  (Sampling)
#> Chain 1: Iteration: 1400 / 2000 [ 70%]  (Sampling)
#> Chain 1: Iteration: 1600 / 2000 [ 80%]  (Sampling)
#> Chain 1: Iteration: 1800 / 2000 [ 90%]  (Sampling)
#> Chain 1: Iteration: 2000 / 2000 [100%]  (Sampling)
#> Chain 1: 
#> Chain 1:  Elapsed Time: 2.768 seconds (Warm-up)
#> Chain 1:                2.35 seconds (Sampling)
#> Chain 1:                5.118 seconds (Total)
#> Chain 1: 
#> 
#> SAMPLING FOR MODEL 'dawid_skene' NOW (CHAIN 2).
#> Chain 2: 
#> Chain 2: Gradient evaluation took 0.000184 seconds
#> Chain 2: 1000 transitions using 10 leapfrog steps per transition would take 1.84 seconds.
#> Chain 2: Adjust your expectations accordingly!
#> Chain 2: 
#> Chain 2: 
#> Chain 2: Iteration:    1 / 2000 [  0%]  (Warmup)
#> Chain 2: Iteration:  200 / 2000 [ 10%]  (Warmup)
#> Chain 2: Iteration:  400 / 2000 [ 20%]  (Warmup)
#> Chain 2: Iteration:  600 / 2000 [ 30%]  (Warmup)
#> Chain 2: Iteration:  800 / 2000 [ 40%]  (Warmup)
#> Chain 2: Iteration: 1000 / 2000 [ 50%]  (Warmup)
#> Chain 2: Iteration: 1001 / 2000 [ 50%]  (Sampling)
#> Chain 2: Iteration: 1200 / 2000 [ 60%]  (Sampling)
#> Chain 2: Iteration: 1400 / 2000 [ 70%]  (Sampling)
#> Chain 2: Iteration: 1600 / 2000 [ 80%]  (Sampling)
#> Chain 2: Iteration: 1800 / 2000 [ 90%]  (Sampling)
#> Chain 2: Iteration: 2000 / 2000 [100%]  (Sampling)
#> Chain 2: 
#> Chain 2:  Elapsed Time: 2.559 seconds (Warm-up)
#> Chain 2:                2.11 seconds (Sampling)
#> Chain 2:                4.669 seconds (Total)
#> Chain 2: 
#> 
#> SAMPLING FOR MODEL 'dawid_skene' NOW (CHAIN 3).
#> Chain 3: 
#> Chain 3: Gradient evaluation took 0.00018 seconds
#> Chain 3: 1000 transitions using 10 leapfrog steps per transition would take 1.8 seconds.
#> Chain 3: Adjust your expectations accordingly!
#> Chain 3: 
#> Chain 3: 
#> Chain 3: Iteration:    1 / 2000 [  0%]  (Warmup)
#> Chain 3: Iteration:  200 / 2000 [ 10%]  (Warmup)
#> Chain 3: Iteration:  400 / 2000 [ 20%]  (Warmup)
#> Chain 3: Iteration:  600 / 2000 [ 30%]  (Warmup)
#> Chain 3: Iteration:  800 / 2000 [ 40%]  (Warmup)
#> Chain 3: Iteration: 1000 / 2000 [ 50%]  (Warmup)
#> Chain 3: Iteration: 1001 / 2000 [ 50%]  (Sampling)
#> Chain 3: Iteration: 1200 / 2000 [ 60%]  (Sampling)
#> Chain 3: Iteration: 1400 / 2000 [ 70%]  (Sampling)
#> Chain 3: Iteration: 1600 / 2000 [ 80%]  (Sampling)
#> Chain 3: Iteration: 1800 / 2000 [ 90%]  (Sampling)
#> Chain 3: Iteration: 2000 / 2000 [100%]  (Sampling)
#> Chain 3: 
#> Chain 3:  Elapsed Time: 2.514 seconds (Warm-up)
#> Chain 3:                2.209 seconds (Sampling)
#> Chain 3:                4.723 seconds (Total)
#> Chain 3: 
#> 
#> SAMPLING FOR MODEL 'dawid_skene' NOW (CHAIN 4).
#> Chain 4: 
#> Chain 4: Gradient evaluation took 0.000177 seconds
#> Chain 4: 1000 transitions using 10 leapfrog steps per transition would take 1.77 seconds.
#> Chain 4: Adjust your expectations accordingly!
#> Chain 4: 
#> Chain 4: 
#> Chain 4: Iteration:    1 / 2000 [  0%]  (Warmup)
#> Chain 4: Iteration:  200 / 2000 [ 10%]  (Warmup)
#> Chain 4: Iteration:  400 / 2000 [ 20%]  (Warmup)
#> Chain 4: Iteration:  600 / 2000 [ 30%]  (Warmup)
#> Chain 4: Iteration:  800 / 2000 [ 40%]  (Warmup)
#> Chain 4: Iteration: 1000 / 2000 [ 50%]  (Warmup)
#> Chain 4: Iteration: 1001 / 2000 [ 50%]  (Sampling)
#> Chain 4: Iteration: 1200 / 2000 [ 60%]  (Sampling)
#> Chain 4: Iteration: 1400 / 2000 [ 70%]  (Sampling)
#> Chain 4: Iteration: 1600 / 2000 [ 80%]  (Sampling)
#> Chain 4: Iteration: 1800 / 2000 [ 90%]  (Sampling)
#> Chain 4: Iteration: 2000 / 2000 [100%]  (Sampling)
#> Chain 4: 
#> Chain 4:  Elapsed Time: 2.485 seconds (Warm-up)
#> Chain 4:                2.636 seconds (Sampling)
#> Chain 4:                5.121 seconds (Total)
#> Chain 4: 

# Calculate the diagnostics for all parameters.
mcmc_diagnostics(fit)
#>                     Rhat ess_bulk
#> pi[1]          1.0035320 5498.743
#> pi[2]          1.0026012 5076.069
#> pi[3]          1.0007458 5291.978
#> pi[4]          1.0001892 5033.075
#> theta[1, 1, 1] 1.0009417 5214.032
#> theta[1, 1, 2] 1.0013858 5301.522
#> theta[1, 1, 3] 0.9998773 4936.974
#> theta[1, 1, 4] 1.0011171 5718.578
#> theta[1, 2, 1] 1.0000472 4214.762
#> theta[1, 2, 2] 0.9999385 4782.331
#> theta[1, 2, 3] 1.0004805 4929.371
#> theta[1, 2, 4] 0.9999898 4568.643
#> theta[1, 3, 1] 1.0002313 3301.846
#> theta[1, 3, 2] 0.9998522 3358.121
#> theta[1, 3, 3] 0.9996284 4627.558
#> theta[1, 3, 4] 0.9996716 3895.957
#> theta[1, 4, 1] 1.0004906 4693.281
#> theta[1, 4, 2] 0.9999686 4458.675
#> theta[1, 4, 3] 1.0005123 3336.679
#> theta[1, 4, 4] 0.9999879 4765.514
#> theta[2, 1, 1] 1.0011862 5421.986
#> theta[2, 1, 2] 1.0010528 4818.239
#> theta[2, 1, 3] 1.0004829 4348.149
#> theta[2, 1, 4] 1.0003545 4664.804
#> theta[2, 2, 1] 0.9996384 5363.728
#> theta[2, 2, 2] 1.0017534 6035.110
#> theta[2, 2, 3] 1.0021862 5374.945
#> theta[2, 2, 4] 1.0000688 4826.239
#> theta[2, 3, 1] 1.0005484 4154.808
#> theta[2, 3, 2] 1.0000714 4098.264
#> theta[2, 3, 3] 1.0003610 6007.405
#> theta[2, 3, 4] 1.0005273 5347.180
#> theta[2, 4, 1] 1.0001756 4595.304
#> theta[2, 4, 2] 1.0015818 4789.960
#> theta[2, 4, 3] 1.0012092 4970.000
#> theta[2, 4, 4] 1.0001962 6452.154
#> theta[3, 1, 1] 1.0007405 6153.490
#> theta[3, 1, 2] 1.0004773 4055.643
#> theta[3, 1, 3] 1.0011062 4434.586
#> theta[3, 1, 4] 0.9999839 4380.875
#> theta[3, 2, 1] 1.0003874 4482.381
#> theta[3, 2, 2] 1.0007757 5105.003
#> theta[3, 2, 3] 1.0002928 5194.316
#> theta[3, 2, 4] 0.9999314 4367.095
#> theta[3, 3, 1] 1.0009462 4141.485
#> theta[3, 3, 2] 1.0001679 4320.538
#> theta[3, 3, 3] 1.0012060 4704.920
#> theta[3, 3, 4] 1.0007891 5166.878
#> theta[3, 4, 1] 1.0005473 4656.112
#> theta[3, 4, 2] 1.0001439 4845.291
#> theta[3, 4, 3] 1.0002135 3537.493
#> theta[3, 4, 4] 1.0012423 5365.043
#> theta[4, 1, 1] 1.0028254 4991.455
#> theta[4, 1, 2] 1.0004650 5015.050
#> theta[4, 1, 3] 1.0001970 4593.587
#> theta[4, 1, 4] 1.0010987 4388.465
#> theta[4, 2, 1] 1.0002194 4564.376
#> theta[4, 2, 2] 1.0013287 5668.126
#> theta[4, 2, 3] 1.0007531 5583.316
#> theta[4, 2, 4] 1.0009367 4608.097
#> theta[4, 3, 1] 1.0015779 3801.667
#> theta[4, 3, 2] 0.9997362 4000.457
#> theta[4, 3, 3] 1.0008333 5337.599
#> theta[4, 3, 4] 1.0016975 4333.603
#> theta[4, 4, 1] 1.0001201 5026.147
#> theta[4, 4, 2] 1.0002932 4349.818
#> theta[4, 4, 3] 1.0001225 4283.373
#> theta[4, 4, 4] 1.0005666 5499.146
#> theta[5, 1, 1] 0.9998109 4816.059
#> theta[5, 1, 2] 1.0001387 3779.766
#> theta[5, 1, 3] 1.0001354 4164.362
#> theta[5, 1, 4] 0.9995150 4536.505
#> theta[5, 2, 1] 1.0012484 5114.551
#> theta[5, 2, 2] 1.0014038 5255.300
#> theta[5, 2, 3] 1.0003992 5048.327
#> theta[5, 2, 4] 1.0016844 4015.471
#> theta[5, 3, 1] 1.0020541 4799.471
#> theta[5, 3, 2] 1.0010127 5355.624
#> theta[5, 3, 3] 1.0006313 5768.491
#> theta[5, 3, 4] 1.0002586 4857.757
#> theta[5, 4, 1] 1.0005597 3803.136
#> theta[5, 4, 2] 1.0004343 3939.965
#> theta[5, 4, 3] 1.0010351 4299.237
#> theta[5, 4, 4] 1.0002600 4741.679

# Calculate the diagnostics just for the pi parameter.
mcmc_diagnostics(fit, pars = "pi")
#>           Rhat ess_bulk
#> pi[1] 1.003532 5498.743
#> pi[2] 1.002601 5076.069
#> pi[3] 1.000746 5291.978
#> pi[4] 1.000189 5033.075

# }