Aki Vehtari

@avehtari

Professor in computational Bayesian modeling, Aalto University, Finland. Co-author of Bayesian Data Analysis 3rd ed, Regression and Other Stories, Active Statistics and Bayesian Workflow. #mcmc_stan and #arviz developer.

270 Siguiendo6619 Seguidores353 Publicaciones

Bluesky

¿Es tu perfil?Reclámalo y avehtari.dirty.social es tuyo.Reclamar perfil

Aki Vehtari@avehtari· 28d

Finally, got a copy of Bayesian workflow book in my hand, and it looks awesome! I really like we did get full colors for figures, too

Aki Vehtari@avehtari· Jun 23

New paper "To select or not to select: predictively consistent priors instead of model selection" with Anna Elisabeth Riha, Leevi Lindgren, @davidkohns.bsky.social, @paulbuerkner.com arxiv.org/abs/2606.22850 Model selection is not a substitute for building good models in the first place 1/

To select or not to select: predictively
consistent priors instead of model selection

Anna Elisabeth Riha, Leevi Lindgren, David Kohns, Paul-Christian Bürkner, Aki Vehtari

Bayesian modelling workflows often consider multiple candidate models of varying complexity. Model selection is commonly used to navigate potential trade-offs between model complexity and generalisability to new data. We study when model selection is unnecessary or can even be harmful for predictive performance in finite data regimes and find that the need for selecting simpler models can depend on prior choice. We formalise predictively consistent priors, which keep prior predictive implications stable as model complexity increases. Across examples and numerical experiments, including adding covariates in linear and logistic regression, forward variable selection, and nonlinear modelling, flexible models with predictively consistent priors typically match or outperform selected simpler models in out-of-sample predictive performance. When selection helps, it can indicate poor joint prior implications, such as excessive prior mass on implausible predictive values. Based on our findings, we propose replacing the notion of sparsity or parsimony at the level of model components with specifying priors that remain sensible in predictive space as models become more complex.

Aki Vehtari@avehtari· Jun 8

Two recommendations for cmdstanr in case studies: 1) use sample(..., refresh=0) to avoid 80 lines of MCMC progress reporting 2) cmdstanr::print_stan_file() in Quarto with output asis to get color formatting ``` #| output: asis print_stan_file(stan_file) ``` e.g., avehtari.github.io/Bayesian-Wor...

$print_stan_file(bioassay_stan_file) data { int<lower=0> J; vector[J] x; array[J] int<lower=0> N; array[J] int<lower=0, upper=N> y; } parameters { real a; real<lower=0> b; } model { {a, b} ~ normal(0, 5); y ~ binomial_logit(N, a + b * x); }$

Aki Vehtari@avehtari· May 13

It took quite long time to merge the PR as we wanted to be careful with the design choices as we had to also change from matrix output to dataframe to support text in diagnostic columns. Now the model names are also columns, for example, making it easier to use tinytable 5/

loo_compare(M_1, M_2, M_3) |>
dplyr::select(model, elpd_diff, se_diff, p_worse, diag_diff, diag_elpd) |>
tt() |>
format_tt(j=4, replace = "-") |>
format_tt(i=3, j=4, digits=5)

model elpd_diff se_diff p_worse diag_diff diag_elpd
M_3 0 0 - 3 k_psis > 0.7
M_2 -15 9.3 0.94 1 k_psis > 0.7
M_1 -80 20 0.99996

Aki Vehtari@avehtari· May 13

Here diag_diff indicates that the magnitude of the difference is less than 4 and we can't trust p_worse, but then it really doesn't matter because the performance difference is small anyway 4/

loo_compare(loo(M_3), loo(M_4))

model elpd_diff se_diff p_worse diag_diff diag_elpd
M_4 0.0 0.0 NA
M_3 -2.4 3.2 0.78 |elpd_diff| < 4

Aki Vehtari@avehtari· May 13

diag_elpd is reminding if the single model LOO computation diagnostics are indicating issues, like here some Pareto-k values in PSIS-LOO computation are high and the comparison cannot be trusted 3/

loo_compare(M_1, M_2, M_3)

model elpd_diff se_diff p_worse diag_diff diag_elpd
M_3 0.0 0.0 NA 3 k_psis > 0.7
M_2 -14.7 9.3 0.94 1 k_psis > 0.7
M_1 -79.6 20.2 1.00

Aki Vehtari@avehtari· May 13

This PR adds new columns to the LOO comparison. p_worse is the probability that a model has worse performance than the best model. diag_diff tells whether that probability can be trusted, for example, here due to small number of observations (N<100) 2/

(loo_comp <- loo_compare(M_1, M_2, M_3, M_4))

model elpd_diff se_diff p_worse diag_diff diag_elpd
M_4 0.0 0.0 NA
M_3 -3.1 1.1 1.00 N < 100
M_1 -3.3 2.0 0.95 N < 100
M_2 -3.7 2.1 0.96 N < 100

Aki Vehtari@avehtari· Apr 7

I was charging a rental car and I guess I should have stopped charging at this point for a big win? (Ladattu = Charged, Lopeta lataus = Stop charging)

Screenshot of electric car charging app showing charged -312945,53kWh -109530,86€

Aki Vehtari@avehtari· Mar 13

My wife bought us a new painting by Pia Sinkkonen named "Taasko yövuorossa?" which translates to "On the night shift again?", but in my head I call it "1. Draw some circles. 2. Draw the rest of the funny owls."

A photo of the painting "Taasko yövuorossa?" by Pia Sinkkonen (2026). The painting has a dark reddish background, two golden circles, and four owls staring out of the painting.

Aki Vehtari@avehtari· Mar 12

I've made a geometrical model for disc golf putting with uncertainty in 2D angle and distance control. Based on the model, the putting angle accuracies of top PDGA MPO and FPO players are about 1° and 1.4°, respectively. See more at users.aalto.fi/~ave/casestu...

Figure shows the proportion of successful putts by distance (where we have integrated out the missing distances) and geometrical model 2 (as presented in https://users.aalto.fi/~ave/casestudies/disc_putting/disc_putting.html) based putting probability by distances based on data for top 33 PDGA MPO players.

Aki Vehtari@avehtari· Feb 13

The streets were already too well cleared and sprinkled with gravel yesterday after lunch, so that I had to walk back home, but today the views are even prettier with sunshine

Aki Vehtari@avehtari· Feb 12

Finally enough snow that I could ski from home to campus

Aki Vehtari@avehtari· Feb 11

I felt confused, and it took me a few seconds to realize why

Response scale with options 0 - Not at all likely, 1, 2, 4, 5, 6, 7, 8, 9, 10 Extremely likely

Aki Vehtari@avehtari· Jan 26

Bayesian Workflow by Andrew Gelman, Aki Vehtari, @rmcelreath.bsky.social with @danpsimpson.bsky.social, @charlesm993.bsky.social, @yulingy.bsky.social, Lauren Kennedy, Jonah Gabry, @paulbuerkner.com, @modrakm.bsky.social, @vianeylb.bsky.social (in production, estimated copy-editing time 6 weeks)

**Part 1: From Bayesian inference to Bayesian workflow**

1. Bayesian theory and Bayesian practice
2. Statistical modeling and workflow
3. Computational tools
4. Introduction to workflow: Modeling performance on a multiple choice exam

**Part 2: Statistical workflow**

5. Building statistical models
6. Using simulations to capture uncertainty
7. Prediction, generalization, and causal inference
8. Visualizing and checking fitted models
9. Comparing and improving models
10. Statistical inference and scientific inference

**Part 3: Computational workflow**

11. Fitting statistical models
12. Diagnosing and fixing problems with fitting
13. Approximate algorithms and approximate models
14. Simulation-based calibration checking
15. Statistical modeling as software development

**4. Case studies**

16. Coding a series of models: Simulated data of movie ratings
17. Prior specification for regression models: Reanalysis of a sleep study
18. Predictive model checking and comparison: Clinical trial
19. Building up to a hierarchical model: Coronavirus testing
20. Using a fitted model for decision analysis: Mixture model for time series competition
21. Posterior predictive checking: Stochastic learning in dogs
22. Incremental development and testing: Black cat adoptions
23. Debugging a model: World Cup football
24. Leave-one-out cross validation model checking and comparison: Roaches
25. Model building and expansion: Golf putting
26. Model building with latent variables: Markov models for animal movement
27. Model building: Time-series decomposition for birthdays
28. Models for regression coefficients and variable selection: Student grades
29. Sampling problems with latent variables: No vehicles in the park
30. Challenge of multimodality: Differential equation for planetary motion
31. Simulation-based calibration checking in model development workflow

**Appendices**

A. Statistical and computational workflow for Bayesians and non-Bayesians
B. How to get the most out of Bayesian Data Analysis

Aki Vehtari@avehtari· Jan 23

Juho Timonen defended his thesis succesfully 🎉 (the thesis includes two papers I helped with: speeding up ODEs with importance sampling dx.doi.org/10.1002/sta4... and longitudinal GPs doi.org/10.1093/bioi...). Photo includes supervisor Harri Lähdesmäki and the opponent @mrattray.bsky.social

Aki Vehtari@avehtari· Jan 5

Sunshine, smoking ice and ice flowers along our walk today (photos by my wife)

Photo of clear sky and frozen sea with wisps of fog on the surface

Aki Vehtari@avehtari· Nov 11

Sea view today close to my home

Photo of slightly foggy calm sea and islands

Aki Vehtari@avehtari· Oct 29

5 mins ago

Aki Vehtari@avehtari· Oct 21

Aki Vehtari@avehtari· Oct 6

I was two weeks on vacation in sunny and warm Sardinia, Italy, played in a beach ultimate tournament (this year we were the 4th best team in the world, and the best non-USA team), learned to kite surf, snorkeled, and ate lots of delicious food and gelato

Photo of many kite surfers at Punta Trettu, Sardinia

Aki Vehtari@avehtari· Sep 3

And the discrete rootogram with white background

Aki Vehtari@avehtari· Sep 3

bayesplot 1.14.0 CRAN release mc-stan.org/bayesplot/ with contributions from @tjmahr.com, Behram Ulukır, and @teemusailynoja.bsky.social My favorite new feature is the discrete style ppc_rootogram() as proposed in teemusailynoja.github.io/visual-predi... and shown below 1/3

A discrete rootgram plot. The unmodified counts are displayed on the vertical axis, while using square root scaling for the axis, thus enabling direct reading of the count frequencies. The predictions are displayed as light blue points and interval lines. The observations are overlaid with darker points, except the observations outside the predictive credible intervals are highlighted in red.

Aki Vehtari@avehtari· Aug 21

We tested the accuracy of the MCSE with 41 posteriordb posteriors of varying complexity, plus with one Birthdays posterior. MCSE matches well the variation in repeated runs of MCMC and bridge sampling. Most of the variation in bridge sampling accuracy is explained by the number of dimensions.

posteriordb posteriors + Birthdays: The left plot shows estimated MCSE vs standard deviation of log marginal likelihood estimates from repeated MCMC runs. MCSE estimate has small bias except for a few posteriors with the highest variability.

posteriordb posteriors + Birthdays: The number of posterior dimensions vs standard deviation of log
marginal likelihood estimates from repeated MCMC runs. The estimate variability tends to increase with the
number of dimensions, which is natural due to the curse of dimensionality. In addition, there is variation depending
on how non-normal the posterior distribution is.

Aki Vehtari@avehtari· Aug 13

For categorical and ordinal data a series of calibration plots can be used. The plots below show one of these calibration plots for Model 1 and Model 2 (the same as in the first post in this thread). The red line going most time outside the blue envelope indicates that Model 1 is misspecifed. 3/4

PAV-adjusted calibration plots for models 1 and 1

Aki Vehtari@avehtari· Aug 13

Instead of PPC bar graphs, it is better to look at the calibration of the predictive probabilities with binned calibration plots or even better with PAV-adjusted calibration plot. 2/4

Binned and PAV-adjusted calibration plot

Aki Vehtari@avehtari· Aug 13

Posterior predictive checking of binary, categorical and many ordinal models with bar graphs is useless. Even the simplest models without covariates usually have such intercept terms that category specific probabilities are learned perfectly. Can you guess which model, 1 or 2, is misspecifed? 1/4

Useless posterior predictive checking bar graphs for Models 1 and 2

Aki Vehtari@avehtari· Aug 4

It's sometimes difficult to get the focus needed for book writing, but this place was perfect for me

A man sitting on a chair and writing with a laptop next to a Finnish lake on a sunny summer day

Aki Vehtari@avehtari· Jul 31

Based on this photo from 1920's at Helsinki University of Technology (which was later merged to Aalto University), they were also teaching how to draw an owl! (cc @rmcelreath.bsky.social)

Photo of an ornamental painting class at Helsinki University of Technology in the 1920s with a stuffed owl on a table

Aki Vehtari@avehtari· Jun 23

A new revised version of "Uncertainty in Bayesian leave-one-out cross-validation based model comparison" with Sivula, @mansmag.bsky.social, and Matamoros. We have clarified the goal of the paper, made more clear that the uncertainty is described by the posterior of unknown elpd difference, 1/4

Aki Vehtari@avehtari· Jun 4

The best gelato in Finland