Real-time reinforcement learning (RL) involves challenges such as limited actions per second and observational delay.Pipelining can address the limited actions issue, improving throughput and potential policy quality.To tackle observational delay, a solution that leverages temporal skip connections and history-augmented observations is proposed.Architectures with temporal skip connections achieve strong performance and parallel neuron computation can accelerate inference.