index

A robot making dinner must remember which ingredient it already added. It must also notice that the pan moved a moment ago. Those are different memory requirements, operating on different timescales.

Physical Intelligence’s recent work makes both requirements visible. Its π0.7 release studies a policy conditioned on richer task and behavior context. Its Multi-scale Embodied Memory work addresses recent observations and longer-term task history.¹² Together, these projects suggest a research agenda in which context design matters alongside model capacity.

A policy needs to know which behavior the data asks for

A dataset can contain several reasonable actions for a superficially similar scene. A slow careful placement and a quick transfer can both be valid, but averaging their motions may produce neither.

The π0.7 release describes conditioning on language, visual subgoals, episode-level quality and speed information, and control conventions, within a broader training mixture.¹ These signals help specify which behavior an example represents. The release’s demonstrations and evaluations remain evidence about the tested tasks and conditions.

For replication, preserve the conditioning fields with the trajectories. Dropping them changes the learning problem. A model then has to explain behavioral variation that the original training setup labeled explicitly.

At evaluation time, state which fields are supplied and how they are obtained. A desired visual subgoal generated by another system adds a component whose errors belong in the end-to-end result.

Action chunks trade planning frequency against feedback

Predicting several actions together can reduce inference overhead and represent a coherent short motion. Executing the whole chunk without a fresh observation can delay correction after an unexpected event.

A 50 Hz first-order controller follows a target that moves after one second. It refreshes its target only between chunks.

Steps between observations

8

Observation delay

80 ms

Solid: controller. Dashed: target.

Tracking RMSE

0.239

Observation calls

19

Chunk duration

160 ms

x[t+1] = x[t] + 0.15 × (cached target − x[t]). This fixture exposes observation cadence; it does not implement a learned robot policy.

The target moves after one second. A controller observes it only at chunk boundaries, with an additional input delay. Longer chunks reduce the number of observation calls but can increase tracking error.

The controller is an authored first-order system. It does not implement π0.7, and it excludes chunk replacement or overlapping inference. Its purpose is to identify the timing variables an experiment should report.

A learned policy evaluated at a different observation cadence is operating in a different closed loop. Report observation capture time, inference completion, action issue time, and the number of actions executed before replanning. An average model latency alone cannot describe that loop.

A useful memory must preserve the right fact

MEM combines short-horizon observation history with a longer-term language-based memory mechanism. The work describes retaining detailed recent information while actively maintaining abstract task history.²

This is a response to a budget problem. Keeping every camera frame at full detail becomes costly over a long task. Compressing history can reduce that cost while discarding exactly the detail needed for a later decision.

Dense frames cost 256 tokens at 2 frames/s. Event summaries cost 32 tokens each. Summary content quality is outside this calculation.

Recent frames retained

10 s

Seconds per older summary

20 s

Compact total

6,080

Dense / compact

50.53×

Recent frames and older summaries cover disjoint time spans. A smaller token budget says nothing about whether a summary retained the fact the next action needs.

The ten-minute fixture counts recent frame tokens and older summaries separately. Increase the summary interval and the token count drops. Nothing in the calculation proves that the coarser summaries remember which cupboard contained the salt.

That is the central evaluation problem. A memory system should be tested on facts that matter later, including facts absent from the current view. Token savings and task success need separate measurements.

The fixture is not MEM’s implementation. Its fixed rates simply make the accounting inspectable before asking the harder question about retained information.

History can introduce a shortcut too

A policy can correlate an earlier observation with an action even when that relationship is accidental. More history gives it more opportunities to use such correlations.

Construct paired episodes that end in similar current observations but require different actions because of an earlier event. Also construct episodes with different irrelevant histories but the same correct next action. The first pair tests whether memory helps; the second tests whether it distracts.

Keep success definitions tied to the task. A policy that repeatedly revisits an already cleaned surface may look active while failing to make progress. Log repeated subtasks, recovery attempts, and interventions alongside completion time.

Physical Intelligence’s Reinforcement Learning Tokens work separately studies rapid online adaptation through an additional learning mechanism.³ Do not attribute those results to the base policy or to MEM without reproducing the relevant setup. Model conditioning, memory, and adaptation are distinct experimental changes.

A small but discriminating research setup

Choose a task with a delayed dependency, such as remembering which of several containers received an object. Randomize the relevant event and delay, while keeping the final visible arrangement controlled.

Compare a current-observation policy, a bounded recent-history policy, and a policy with an explicit longer-term memory. Match the action execution loop and report the memory construction cost. Evaluate several delays and introduce a controlled observation interruption.

Freeze the task split before tuning summaries. Save memory contents at each decision so failures can be traced to missing information, incorrect recall, or incorrect action selection despite correct recall.

This kind of experiment can be informative without a huge robot fleet. It asks exactly when context changes the decision and whether the system uses it reliably. That is stronger evidence about long-horizon competence than a longer uninterrupted demonstration by itself.

References

Footnotes

1. Physical Intelligence, “π0.7,” April 16, 2026. Conditioning, training mixture, and evaluations. Read the research post. ↩ ↩²

2. Physical Intelligence, “VLAs with Long and Short-Term Memory,” March 3, 2026. Multi-scale Embodied Memory. Read the research post. ↩ ↩²

3. Physical Intelligence, “Reinforcement Learning Tokens,” March 19, 2026. Read the research post. ↩