<ul data-eligibleForWebStory="true"><li>Self-supervised learning (SSL) has become essential in machine learning for training data representations without a supervised signal.</li><li>Most SSL methods use the cosine similarity between embedding vectors, embedding data effectively on a hypersphere.</li><li>Recent works suggest that embedding norms play a role in SSL, contrary to previous beliefs.</li><li>This paper resolves the contradiction and establishes the role of embedding norms in SSL training.</li><li>Theoretical analysis, simulations, and experiments show that embedding norms affect SSL convergence rates and network confidence.</li><li>Smaller embedding norms correspond to unexpected samples in the network.</li><li>Manipulating embedding norms can significantly impact convergence speed in SSL.</li><li>The study highlights the importance of embedding norms in understanding and optimizing network behavior in SSL.</li></ul>

On the Importance of Embedding Norms in Self-Supervised Learning

Discover more