<ul><li>Recurrent neural networks (RNNs) have shown strong performance and faster inference compared to Transformers.</li><li>A novel method is proposed to replace the computationally expensive backpropagation through time (BPTT) algorithm with a fixed gradient feedback mechanism.</li><li>The method leverages state-space model (SSM) principles to directly propagate gradients from future time steps, reducing training overhead.</li><li>Experiments on language modeling benchmarks demonstrate competitive perplexity scores while significantly reducing training costs.</li></ul>

Fast Training of Recurrent Neural Networks with Stationary State Feedbacks

Discover more