<ul data-eligibleForWebStory="true"><li>Prior-data fitted networks (PFNs) are transformers pre-trained on synthetic data to enable Bayesian inference through in-context learning.</li><li>CausalFM is introduced as a framework for training PFN-based foundation models in causal inference settings.</li><li>It formalizes Bayesian priors for causal inference based on structural causal models (SCMs) and derives criteria for valid priors.</li><li>A new family of prior distributions using causality-inspired Bayesian neural networks is proposed in CausalFM.</li><li>CausalFM supports Bayesian causal inference in back-door, front-door, and instrumental variable adjustment settings.</li><li>A foundation model for estimating conditional average treatment effects (CATEs) using back-door adjustment is trained explicitly in CausalFM.</li><li>CausalFM shows competitive performance for CATE estimation with various benchmarks.</li><li>The framework can serve as a general recipe for training foundation models in causal inference across disciplines.</li><li>CausalFM presents a new paradigm that could potentially revolutionize how causal inference is conducted in fields like medicine and economics.</li></ul>

Foundation Models for Causal Inference via Prior-Data Fitted Networks

Discover more