Spurious correlations can hinder robust decision-making and generalization out-of-distribution (OOD).Naive in-distribution empirical risk minimizers often achieve the best OOD accuracy.Many popular benchmarks evaluating robustness to spurious correlations are misspecified.The need to reassess how robustness to spurious correlations is evaluated is highlighted.