SoftwareApplication
name: SMZ
description: Code underlying the publication: VariBASed: Variational Bayes-Adaptive Sequential Monte-Carlo Planning for Deep Reinforcement Learning
description: VariBASeD: Variational Bayes-Adaptive Sequential Monte-Carlo Planning — JAX implementation accompanying "VariBASed: Variational Bayes-Adaptive Sequential Monte-Carlo Planning for Deep Reinforcement Learning" (De Vries, He, Oren, van der Vaart, de Weerdt, Spaan, 2026). Targets approximate Bayes-optimal exploration-exploitation by combining amortized variational belief learning (via S5 state-space models), SMC planning from the trtpi codebase, and meta-RL into a unified EM framework. The E-step uses a particle-filter planner with nested importance-sampling weights correcting for mismatch between variational and true beliefs; the M-step distills SMC-improved policies and fits the belief ELBO jointly. Designed for single-GPU throughput with fixed-size batches and hidden-state caching. Preliminary results on gridworld and continuous function optimization show favorable sample- and runtime-efficiency scaling with planning budget over a recurrent PPO (RL²) baseline.
code repository: https://data.4tu.nl/v3/datasets/8d23ea64-a4fa-4702-92c2-8bb9ca6fb2d7.git
author: https://orcid.org/0009-0008-2927-7448
author: https://orcid.org/0000-0002-8528-8650
author: Person
  given name: Yaniv
  family name: Oren
  name: Yaniv Oren
author: https://orcid.org/0009-0004-0867-9289
author: https://orcid.org/0000-0002-0470-6241
author: Person
  given name: Matthijs
  family name: Spaan
  name: Matthijs Spaan
keywords: Model predictive control
keywords: Autonomous Path-Planning
keywords: Partially observable Markov decision process
keywords: Bayesian optimization
keywords: Reinforcement learning
keywords: deep learning
programming language: Jupyter Notebook
programming language: Markdown
programming language: Other
programming language: Python
programming language: Shell
programming language: YAML
programming language: large-text-file
license:
  creative work: creativeWork
    name: MIT
reference publication: arxiv.2602.18857

https://orcid.org/0009-0008-2927-7448
type: Person
given name: Joery
family name: de Vries
name: Joery de Vries

https://orcid.org/0000-0002-8528-8650
type: Person
given name: Jinke
family name: He
name: Jinke He

https://orcid.org/0009-0004-0867-9289
type: Person
given name: Pascal
family name: van der Vaart
name: Pascal van der Vaart

https://orcid.org/0000-0002-0470-6241
type: Person
given name: M.M. (Mathijs)
family name: de Weerdt
name: M.M. (Mathijs) de Weerdt

arxiv.2602.18857
type: ScholarlyArticle
name: VariBASed: Variational Bayes-Adaptive Sequential Monte-Carlo Planning for Deep Reinforcement Learning
url: https://doi.org/10.48550%2Farxiv.2602.18857