Rethinking Memory: A Unified Linear Approach for Mindful Agents
Published:
In reinforcement learning (RL), memory isn’t just a bonus—it’s a necessity. When agents operate in environments where they can’t directly see everything they need (think navigating a maze), they must rely on memory to make decisions. This is where things get tricky: most current memory models fail under the weight of complex, long-term tasks where agents must selectively retain and erase memories based on relevance.
👀 In this setting, it is surprising that simple recurrent memory such as RNN, LSTM and GRU outperforms powerful memory models such as Transformer and attention-based memories.
Enter the 👉Stable Hadamard Memory (SHM [1]). This new memory framework promises to revolutionize how RL agents manage their memories by solving longstanding problems with stability, flexibility, and scalability. Let’s dive into how it works, why it matters, and what the experiments reveal.
