PosteriorBench: New Benchmark for Generative Inverse Solvers

What changed

Researchers have introduced PosteriorBench, a novel benchmark specifically designed to evaluate the distributional accuracy of generative inverse solvers. Traditional evaluation methods for these solvers often concentrate on whether a model can produce a single, plausible reconstruction. However, this approach is insufficient for ill-posed problems, where multiple valid solutions can exist and be consistent with the same sparse or noisy observations. In such scenarios, a solver might achieve high pointwise accuracy but still fail to represent the true posterior distribution due to issues like mode collapse, overconfident uncertainty estimates, or averaging incompatible solutions. PosteriorBench aims to rectify this by evaluating how well solvers recover the full set of possible solutions, not just the single best sample.

The benchmark comprises four distinct physics-based inverse problems: Darcy flow inversion, Poisson source recovery, carbon capture and storage (CCS), and light transport material inference. For each task, high-fidelity reference posteriors are constructed using computationally intensive but established procedures such as rejection sampling and Markov chain Monte Carlo (MCMC). These references serve as ground truth for assessing distributional accuracy. PosteriorBench employs a suite of five metrics to evaluate these posteriors: posterior-mean error, posterior-standard-deviation error, maximum mean discrepancy (MMD), sliced Wasserstein distance (SWD), and radially averaged power-spectrum error. These metrics collectively assess pointwise accuracy, marginal uncertainty, distributional alignment, and global frequency fidelity.

The benchmark is designed to cover a wide range of challenging conditions, including sparse sensing, low-resolution observations, nonlinear forward models, varying noise levels, and multimodal priors. It also features a unified pipeline for distribution matching and uncertainty quantification. Initial experiments using PosteriorBench have revealed significant distribution-matching gaps across current generative solvers. The findings also indicate that neural operators can enhance resolution robustness, and that guidance weights and generation noise play a key role in calibrating posterior variance. The code for PosteriorBench is publicly available on GitHub.

Why it matters for builders

For AI builders working with generative models for scientific applications, PosteriorBench offers a more rigorous and realistic evaluation framework. It shifts the focus from simple reconstruction accuracy to a deeper understanding of how well models capture the inherent uncertainty and multi-modal nature of solutions in inverse problems. This is critical for developing trustworthy AI systems that can provide reliable insights in fields like geophysics, medical imaging, and materials science, where understanding the range of possibilities is as important as finding a single answer.

Practical impact

Builders can leverage PosteriorBench to benchmark their own generative inverse solvers and identify specific weaknesses in distributional accuracy. The benchmark's diverse set of tasks and evaluation metrics allows for a comprehensive assessment of model performance across various scientific domains and problem complexities. The insights gained from PosteriorBench experiments, such as the importance of guidance weights and generation noise for posterior variance calibration, can directly inform the design and tuning of future generative models. Developers can use the provided code to integrate these evaluation methods into their development pipelines, ensuring their models are not only accurate but also provide meaningful uncertainty quantification.

Caveats and source limits

The source material indicates that the reference posteriors are constructed using computationally heavy procedures, which might limit the scale of evaluations. While the benchmark covers four diverse tasks, it does not explicitly detail the exact number of samples or the computational resources required for each task. The findings regarding neural operators and the role of guidance weights and generation noise are based on the authors' experiments within the benchmark, and further independent validation across a broader range of solvers and tasks would be beneficial. The source does not provide specific performance numbers for existing solvers on PosteriorBench, only general observations about distribution-matching gaps and calibration issues.