Paper: https://arxiv.org/abs/2511.14759
Slide 2
Pi0.6* is the latest iteration from Physical Intelligence. Note the star – they did train a Pi0.6 but didn’t write paper about it, just providing a basic model card. The main improvements from 0.5 to 0.6 is just some small architectural changes and training data. They then advance from 0.6 to 0.6* using reinforcement learning.
Slide 3
Reinforcement learning is not really an area I know a ton about so while reading the paper I had to do some reading up on some of the RL details, so I’ve included a quick primer here of RL in the context of the paper.
Slide 4
That being said, I’m sure we’ve at least all seen this before. Reinforcement learning at its core involves this feedback loop where the agent takes and action that interacts with the environment in some way, a reward is calculated based on the current state and action, and we try to maximize the total rewards.
Slide 5
And we’ve seen this be an incredibly powerful tool, a lot of the big breakthroughs in AI have been because of RL.