Graph Neural Networks (GNNs) need reliable tools for explaining their predictions.Existing faithfulness metrics are not interchangeable.Optimizing for faithfulness may not always be a sensible design goal for regular GNN architectures.Faithfulness is tightly linked to out-of-distribution generalization.