A naukri.com initiative
Arxiv
2d
296
Image Credit: Arxiv
Generalization Error Analysis for Attack-Free and Byzantine-Resilient Decentralized Learning with Data Heterogeneity
- Decentralized learning, allowing model training across scattered agents, is being focused on in signal and information processing.
- Generalization errors of decentralized learning algorithms are less explored despite scrutiny on optimization errors.
- Understanding generalization errors is vital for assessing model performance on new data for real-world applications.
- The paper conducts a detailed analysis of generalization errors in attack-free and Byzantine-resilient decentralized learning with heterogeneous data.
- This analysis is carried out under mild assumptions, unlike previous studies focusing on homogeneous data or strict bounded stochastic gradient assumptions.
- Results emphasize the impact of data heterogeneity, model initialization, and stochastic gradient noise on decentralized learning's generalization error.
- Byzantine attacks by malicious agents notably affect generalization error, primarily tied to data heterogeneity rather than sample size.
- Numerical experiments verify the theoretical results on both convex and non-convex tasks.
Read Full Article
17 Likes
For uninterrupted reading, download the app