Bayesian Model Averaging under Predictor Redundancy via Density-Ratio Posterior Compression
About
Bayesian model averaging in support-indexed regression induces a posterior distribution over active predictor supports. Under predictor redundancy, posterior mass can spread across many nearly interchangeable supports, making exact-support summaries unstable or hard to interpret even when prediction is stable. We study how to report an already fitted Bayesian model averaging posterior without changing the Bayesian target. A report uses hard or soft regions of support space, and its compressed reporting law is compared with the reference posterior through an explicit density ratio. This ratio gives computable total-variation and Kullback--Leibler distortion, bounds for bounded predictive summaries, retained-mass diagnostics, and fallback-weight diagnostics. The framework covers fixed hard regions, metric-ball regions, posterior-cluster regions, and pooled-pruned region dictionaries. We prove exact error formulas and validation bounds for these region reports, and give conditions under which a few regions can replace a long list of individual supports. In simulations, our region reports often give shorter and clearer summaries while preserving the main posterior information, and the density-ratio diagnostics show when too much information has been lost.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Posterior Compression | Compact exact benchmark (seven regimes (two correlations, five replications)) | Total Variation (TV)1.4 | 7 | |
| Posterior Compression | large p = 100 benchmark | TV0.049 | 7 | |
| Real-response spectroscopy diagnostics | Gasoline | Total Variation (TV)0.056 | 7 | |
| Real-response spectroscopy diagnostics | tecator | TV0.053 | 7 | |
| Support-level compression | Semi-synthetic real-X Tecator | TV0.058 | 3 |