ISSN 3083-4471 API Registry JSON Sitemap
TIRAS Transactions on Intelligent Robotics and Autonomous Systems An open access journal of Kaggleyes Academy
TIRAS > Vol. 1, No. 2 (August 2026) > KAGG:2608.00002
Vol. 1, No. 2 (August 2026) — Article KAGG:2608.00002

Latent Motion Tokens for Future-Range Prediction in Quadruped Obstacle Avoidance

Authors: Marcus Hill, Ivan Sokolov, Yating Feng, Rachel Kim

Robotics Institute, Georgia Institute of Technology, Atlanta, GA, USA

[Received 2026-07-21; accepted 2026-08-01; published online 2026-08-04]

Abstract:

Learned range predictors for legged-robot navigation usually regress the free distance along a fixed set of rays. This output format is difficult to condition on the motion the robot intends to execute, and it forces the planner and the reactive safety layer to share a single ray resolution. This paper encodes the predicted future range field of a quadruped as a small set of latent motion tokens, one per cluster of control-aligned directions. A transformer generates the tokens from recent range history and the planned motion, and a lightweight query decoder returns the future free range for any ray, horizon, and candidate motion. Because queries are resolved against the tokens rather than a fixed output layer, the same predictor serves a 2 Hz planner issuing coarse queries and a 50 Hz reactive layer issuing dense queries. In Isaac Lab experiments with ANYmal in 20 cluttered obstacle-avoidance scenes, with 5 seeds and 30 trials per scene and seed, the token predictor attains a range prediction error of 0.87 m, a 95th-percentile query latency of 13.6 ms on the onboard Orin computer, and an avoidance success rate of 91.8 percent, compared with 0.91 m, 14.4 ms, and 79.8 percent for a FutureRay-style direct-regression baseline, while using 1.2 million instead of 1.3 million parameters. Ablations indicate that motion-plan conditioning and the number of tokens matter most, and the recorded failure cases identify occluded turns and planner stalls as remaining weaknesses.

Journal: Transactions on Intelligent Robotics and Autonomous Systems (TIRAS)
Volume / Issue: Vol. 1, No. 2 (August 2026)
Article ID: KAGG:2608.00002
DOI: 10.5803/kagg.2608.00002
Received: 2026-07-21
Accepted: 2026-08-01
Published online: 2026-08-04
License: CC BY 4.0

References (45)

  1. J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V. Tsounis, V. Koltun, and M. Hutter. Learning Agile and Dynamic Motor Skills for Legged Robots. arXiv:1901.08652, 2019. arXiv:1901.08652
  2. J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter. Learning Quadrupedal Locomotion over Challenging Terrain. arXiv:2010.11251, 2020. arXiv:2010.11251
  3. T. Miki, J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter. Learning Robust Perceptive Locomotion for Quadrupedal Robots in the Wild. arXiv:2201.08117, 2022. arXiv:2201.08117
  4. N. Rudin, D. Hoeller, P. Reist, and M. Hutter. Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning. arXiv:2109.11978, 2021. arXiv:2109.11978
  5. T. Zang, S. Wang, H. Huang, and S. Cheng. FutureRay: Control-Aligned Future Range for Agile Quadruped Navigation. arXiv:2608.01197, 2026. arXiv:2608.01197
  6. V. Makoviychuk et al. Isaac Gym: High Performance GPU-Based Physics Simulation for Robot Learning. arXiv:2108.10470, 2021. arXiv:2108.10470
  7. M. Mittal et al. Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments. arXiv:2301.04195, 2023. arXiv:2301.04195
  8. A. Kumar, Z. Fu, D. Pathak, and J. Malik. RMA: Rapid Motor Adaptation for Legged Robots. arXiv:2107.04034, 2021. arXiv:2107.04034
  9. R. Yang, M. Zhang, N. Hansen, H. Xu, and X. Wang. Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers. arXiv:2107.03996, 2021. arXiv:2107.03996
  10. A. Agarwal, A. Kumar, J. Malik, and D. Pathak. Legged Locomotion in Challenging Terrains using Egocentric Vision. arXiv:2211.07638, 2022. arXiv:2211.07638
  11. X. Cheng, K. Shi, A. Agarwal, and D. Pathak. Extreme Parkour with Legged Robots. arXiv:2309.14341, 2023. arXiv:2309.14341
  12. Z. Zhuang, Z. Fu, J. Wang, C. Atkeson, S. Schwertfeger, C. Finn, and H. Zhao. Robot Parkour Learning. arXiv:2309.05665, 2023. arXiv:2309.05665
  13. D. Hoeller, N. Rudin, D. Sako, and M. Hutter. ANYmal Parkour: Learning Agile Navigation for Quadrupedal Robots. arXiv:2306.14874, 2023. arXiv:2306.14874
  14. N. Rudin, D. Hoeller, M. Bjelonic, and M. Hutter. Advanced Skills by Learning Locomotion and Local Navigation End-to-End. arXiv:2209.12827, 2022. arXiv:2209.12827
  15. D. Hoeller, L. Wellhausen, F. Farshidian, and M. Hutter. Learning a State Representation and Navigation in Cluttered and Dynamic Environments. arXiv:2103.04351, 2021. arXiv:2103.04351
  16. J. Truong, D. Yarats, T. Li, F. Meier, S. Chernova, D. Batra, and A. Rai. Learning Navigation Skills for Legged Robots with Learned Robot Embeddings. arXiv:2011.12255, 2020. arXiv:2011.12255
  17. Y. Kim, C. Kim, and J. Hwangbo. Learning Forward Dynamics Model and Informed Trajectory Sampler for Safe Quadruped Navigation. arXiv:2204.08647, 2022. arXiv:2204.08647
  18. J. Frey, M. Mattamala, N. Chebrolu, C. Cadena, M. Fallon, and M. Hutter. Fast Traversability Estimation for Wild Visual Navigation. arXiv:2305.08510, 2023. arXiv:2305.08510
  19. R. Grandia, F. Jenelten, S. Yang, F. Farshidian, and M. Hutter. Perceptive Locomotion through Nonlinear Model Predictive Control. arXiv:2208.08373, 2022. arXiv:2208.08373
  20. M. Pfeiffer, M. Schaeuble, J. Nieto, R. Siegwart, and C. Cadena. From Perception to Decision: A Data-driven Approach to End-to-end Motion Planning for Autonomous Ground Robots. arXiv:1609.07910, 2016. arXiv:1609.07910
  21. L. Tai, G. Paolo, and M. Liu. Virtual-to-real Deep Reinforcement Learning: Continuous Control of Mobile Robots for Mapless Navigation. arXiv:1703.00420, 2017. arXiv:1703.00420
  22. H.-T. L. Chiang, A. Faust, M. Fiser, and A. Francis. Learning Navigation Behaviors End-to-End with AutoRL. arXiv:1809.10124, 2018. arXiv:1809.10124
  23. G. Kahn, A. Villaflor, B. Ding, P. Abbeel, and S. Levine. Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation. arXiv:1709.10489, 2017. arXiv:1709.10489
  24. G. Kahn, P. Abbeel, and S. Levine. BADGR: An Autonomous Self-Supervised Learning-Based Navigation System. arXiv:2002.05700, 2020. arXiv:2002.05700
  25. D. Shah, A. Sridhar, A. Bhorkar, N. Hirose, and S. Levine. GNM: A General Navigation Model to Drive Any Robot. arXiv:2210.03370, 2022. arXiv:2210.03370
  26. D. Shah, A. Sridhar, N. Dashora, K. Stachowicz, K. Black, N. Hirose, and S. Levine. ViNT: A Foundation Model for Visual Navigation. arXiv:2306.14846, 2023. arXiv:2306.14846
  27. A. Sridhar, D. Shah, C. Glossop, and S. Levine. NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration. arXiv:2310.07896, 2023. arXiv:2310.07896
  28. A. Loquercio, E. Kaufmann, R. Ranftl, M. Müller, V. Koltun, and D. Scaramuzza. Learning High-Speed Flight in the Wild. arXiv:2110.05113, 2021. arXiv:2110.05113
  29. N. Mohajerin and M. Rohani. Multi-Step Prediction of Occupancy Grid Maps with Recurrent Neural Networks. arXiv:1812.09395, 2018. arXiv:1812.09395
  30. M. Itkina, K. Driggs-Campbell, and M. J. Kochenderfer. Dynamic Environment Prediction in Urban Scenes using Recurrent Representation Learning. arXiv:1904.12374, 2019. arXiv:1904.12374
  31. X. Weng, J. Wang, S. Levine, K. Kitani, and N. Rhinehart. Inverting the Pose Forecasting Pipeline with SPF2: Sequential Pointcloud Forecasting for Sequential Pose Forecasting. arXiv:2003.08376, 2020. arXiv:2003.08376
  32. B. Mersch, X. Chen, J. Behley, and C. Stachniss. Self-supervised Point Cloud Prediction Using 3D Spatio-temporal Convolutional Networks. arXiv:2110.04076, 2021. arXiv:2110.04076
  33. M. Bansal, A. Krizhevsky, and A. Ogale. ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst. arXiv:1812.03079, 2018. arXiv:1812.03079
  34. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention Is All You Need. arXiv:1706.03762, 2017. arXiv:1706.03762
  35. J. Lee, Y. Lee, J. Kim, A. R. Kosiorek, S. Choi, and Y. W. Teh. Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks. arXiv:1810.00825, 2018. arXiv:1810.00825
  36. A. Jaegle, F. Gimeno, A. Brock, A. Zisserman, O. Vinyals, and J. Carreira. Perceiver: General Perception with Iterative Attention. arXiv:2103.03206, 2021. arXiv:2103.03206
  37. A. Jaegle et al. Perceiver IO: A General Architecture for Structured Inputs and Outputs. arXiv:2107.14795, 2021. arXiv:2107.14795
  38. N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko. End-to-End Object Detection with Transformers. arXiv:2005.12872, 2020. arXiv:2005.12872
  39. L. Mescheder, M. Oechsle, M. Niemeyer, S. Nowozin, and A. Geiger. Occupancy Networks: Learning 3D Reconstruction in Function Space. arXiv:1812.03828, 2018. arXiv:1812.03828
  40. J. J. Park, P. Florence, J. Straub, R. Newcombe, and S. Lovegrove. DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation. arXiv:1901.05103, 2019. arXiv:1901.05103
  41. B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. arXiv:2003.08934, 2020. arXiv:2003.08934
  42. M. Tancik, P. P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. T. Barron, and R. Ng. Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains. arXiv:2006.10739, 2020. arXiv:2006.10739
  43. D. Hafner, J. Pasukonis, J. Ba, and T. Lillicrap. Mastering Diverse Domains through World Models. arXiv:2301.04104, 2023. arXiv:2301.04104
  44. A. D. Ames, S. Coogan, M. Egerstedt, G. Notomista, K. Sreenath, and P. Tabuada. Control Barrier Functions: Theory and Applications. arXiv:1903.11199, 2019. arXiv:1903.11199
  45. A. Kendall and Y. Gal. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? arXiv:1703.04977, 2017. arXiv:1703.04977
Cite this article Marcus Hill, Ivan Sokolov, Yating Feng, Rachel Kim, "Latent Motion Tokens for Future-Range Prediction in Quadruped Obstacle Avoidance," Transactions on Intelligent Robotics and Autonomous Systems, 2026, vol. 1, no. 2, KAGG:2608.00002. DOI: 10.5803/kagg.2608.00002.