A study demonstrates the breakthrough performance of Mamba in autoregressive speech generation.Mamba outperforms the SaShiMi architecture in audio modeling and generation tasks.Mamba shows consistent improvement with longer context lengths.A small Mamba model surpasses state-of-the-art GAN- and diffusion-based models in speech generation.