Temporal Gradient Inversion for Private Trajectory Reconstruction in Embodied Reinforcement Learning

Abstract

Trajectory leakage

Why a stream leaks more than a frame

Gradient-inversion attacks have mostly been studied on a single batch: one gradient in, one image out. An embodied agent is different. It shares a gradient at every step of a trajectory, and consecutive observations come from the same room, seen from nearly the same pose.

A single-frame attacker throws that redundancy away. TRACE keeps it: each reconstruction becomes context for the next, so the attacker gets a better prior with every gradient it reads. Pixel error falls and fidelity rises as the trajectory gets longer. We call this the temporal reconstruction gain.

At a glance

• 18.8dB

  PSNR on held-out scenes

• 100%

  Action accuracy, PPO and A2C

• 3–4.5ms

  Per reconstructed frame

• ~6,900×

  Minimum speedup over optimization attacks

Amortized and autoregressive

A trained inverter conditions on the previous reconstruction and the current gradient to emit the next observation and action. No per-sample optimization, so a frame takes milliseconds rather than minutes.

Two structural signals

Cross-time correlation between successive gradients, bounded via conditional mutual information, and closed-form action recovery from the policy head, proven exact when entropy regularization is small.

Beyond the base setting

Holds up across recurrent, residual, and compact transformer victims, multi-modal inputs, larger discrete action spaces, and aggregated gradients.

What it means for defenses

Per-step perturbation is not enough. Protecting a gradient stream calls for sequence-aware privacy mechanisms.

Attack pipeline

The attacker sits at the server and receives one policy-learning gradient per environment step; it never sees the agent's camera. Because TRACE is amortized, there is no per-sample optimization: attacking a new trajectory is a single forward pass per step.

1. 01

   Gradient encoder

   Compresses each per-step policy gradient into a compact latent token.

2. 02

   Image encoder

   Embeds the reconstructions TRACE has already produced for earlier steps, so the model carries a visual memory of the scene.

3. 03

   Causal temporal transformer

   Reads gradient tokens and image tokens interleaved in time. Causal masking means step t only sees steps up to t, matching what an online attacker would have.

4. 04

   Residual decoder and action head

   Reconstructs the observation and predicts the action at every step. The action is also exposed in closed form by the policy-head gradient.

Because the inverter conditions on its own earlier reconstructions, small errors could snowball over a long rollout. Training is exposure-aware, so errors do not compound as the trajectory grows.

Results

We evaluate on point-goal navigation in AI2-THOR. The victim sees 84×84 RGB frames and picks one of five discrete actions; the attacker sees only the per-step policy gradients. Below is one held-out trajectory and what TRACE recovers from its gradient stream.

t = 0MoveAheadt = 1LookUpt = 2RotateLeftt = 3LookUpt = 4LookUpt = 5RotateLeftt = 6LookUpt = 7MoveAhead

Ground truth

TRACE

Comparison with gradient-inversion baselines

DLG and Inverting Gradients optimize a dummy input per sample; Learning to Invert is a learned single-frame inverter. All three are adapted from supervised classification with an actor-critic surrogate loss. PSNR in dB; means over held-out sequences, standard deviations are in the paper. Bold marks the best value per column.

The optimization attacks spend 30–75 seconds per frame and still land near noise. Learning to Invert is fast and gets the action right, but inverting each gradient in isolation caps its image quality. TRACE adds about 2 dB PSNR over it, raises SSIM by roughly 0.1, and nearly halves LPIPS, the perceptual distance, while staying within a few milliseconds per frame.

Generality across victims

The leakage is not an artifact of one small CNN. We retrain a separate attacker for each PPO victim below, with no data augmentation, covering a larger action space (K = 10), a compact vision transformer, residual IMPALA-style networks, a recurrent policy, and a victim with multi-modal inputs.

K is the number of discrete actions. Bold marks the best reconstruction.

More capacity does not protect the victim. The wider IMPALA network leaks the most, reaching 22.4 dB PSNR. The Tiny ViT is the hardest case for image reconstruction, yet its actions are still recovered 99% of the time. Across every architecture, action accuracy stays above 97%.

Defenses

We test the standard per-step defenses a deployment might reach for: quantizing the gradient, pruning gradient entries, adding Gaussian noise, and DP-SGD with per-sample clipping. The question is how strong each one has to be before TRACE stops working.

t = 1t = 2t = 3t = 4t = 5t = 6t = 7t = 8

Ground truth

No defense

Pruning 50%

Noise σ = 0.01

DP-SGD

Attack quality under each defense

Highlighted cells mark settings where the attack clearly degrades. Pruning settings give the fraction of gradient entries kept.

Pruning does almost nothing: keeping only 10% of the PPO gradient leaves PSNR at 18.8 dB and action accuracy at 99.9%. 8-bit quantization and σ ≤ 0.01 noise are similarly harmless. The attack only breaks under 4-bit or coarser quantization, σ = 0.1 noise, or DP-SGD, where action accuracy falls toward the 20% chance level for five actions. Protecting a gradient stream likely needs sequence-aware privacy mechanisms rather than per-step perturbation.

Citation

@inproceedings{bhujel2026trace,
  title         = {Temporal Gradient Inversion for Private Trajectory Reconstruction in Embodied Reinforcement Learning},
  author        = {Bhujel, Sudip and Shi, Shanghao and Huang, Ruiquan and Zhang, Ning and Xiao, Yang},
  booktitle     = {Advances in Neural Information Processing Systems (NeurIPS)},
  year          = {2026},
  eprint        = {2609.30258},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  url           = {https://trace-gia.github.io/}
}