Brain

Curious Agents Saga (Part 2), Novelty or Surprise: How to Make Your Deep Reinforcement Learning Agents Curious?

less than 1 minute read

Published:

Novelty or Surprise: How to Make Your Deep Reinforcement Learning Agents Curious?

Table of Content

  • Challenges in Exploration
  • Novelty
    • State Counting
    • Change Counting
    • Novelty through Reachability
    • Novelty via Reconstruction
  • Surprise
    • Predictive Surprise
    • Bayesian Surprise
  • Hybrid Approaches
    • Surprise + Novelty
    • Novelty of Surprise

Challenges in Exploration

In the preceding post, we studied classical exploration strategies primarily grounded in theoretical justifications under naïve assumptions, challenging to scale in complex scenarios. 🧠 Why is scaling a big problem? Practical environments often involve huge continuous state and action spaces, as exemplified below:

Exploration in Doom

Read the full article

Check our papers:

You May Also Enjoy

The Best of Time-Series Forecasting (Part II): Advancements in Time Series Modeling Through Large Language Models

less than 1 minute read

Published:

Part 1 of my blog looked at how time-series forecasting has evolved—from traditional models like ARIMA to deep learning methods like Transformers. These approaches brought big improvements, especially in handling complex and long-range patterns. However, they also have limits, especially when it comes to adapting to new data or working well across very different domains. Read more

Rethinking Memory: A Unified Linear Approach for Mindful Agents

less than 1 minute read

Published:

In reinforcement learning (RL), memory isn’t just a bonus—it’s a necessity. When agents operate in environments where they can’t directly see everything they need (think navigating a maze), they must rely on memory to make decisions. This is where things get tricky: most current memory models fail under the weight of complex, long-term tasks where agents must selectively retain and erase memories based on relevance. Read more