<ul><li>SparseJEPA is an extension of Joint Embedding Predictive Architectures (JEPA) that integrates sparse representation learning to enhance the quality of learned representations.</li><li>SparseJEPA encourages shared latent space variables among data features with strong semantic relationships while maintaining predictive performance.</li><li>The architecture was tested on the CIFAR-100 dataset and a lightweight Vision Transformer for image classification and low-level tasks using transfer learning.</li><li>Incorporating sparsity improves the latent space, leading to more meaningful and interpretable representations.</li></ul>

SparseJEPA: Sparse Representation Learning of Joint Embedding Predictive Architectures

Discover more