Differential privacy (DP) is important for protecting privacy in machine learning applications.A new scalable DP mechanism, SWAG-PPM, is proposed for deep learning models.SWAG-PPM outperforms the industry standard DP-SGD for a similar privacy budget.SWAG-PPM demonstrates its effectiveness on a workplace injury text classification task.