We introduce a new method that extracts knowledge from a large language model (LLM) to produce object-level plans, which describe high-level changes to object state, and uses them to bootstrap task and motion planning (TAMP). Existing work uses LLMs to directly output task plans or generate goals in representations like PDDL. However, these methods fall short because they rely on the LLM to do the actual planning or output a hard-to-satisfy goal. Our approach instead extracts knowledge from an LLM in the form of plan schemas as an object-level representation called functional object-oriented networks (FOON), from which we automatically generate PDDL subgoals. Our method markedly outperforms alternative planning strategies across several pick-place tasks in simulation.
We provide a Jupyter Notebook file in our GitHub repository, which you can use to test our object-level planning method as well as implementations of the baselines implemented in our work. You will however need to have a valid OpenAI API key on your machine to access GPT-4.0!
In order to see robot execution, you will also need to set up CoppeliaSim and its corresponding Python API. Read more here. If you are interested in using the OMPL method implemented for this work, please refer to this repository.
@inproceedings{paulius2025olp,
title={\href{https://arxiv.org/abs/2409.12262}{Bootstrapping Object-Level Planning with Large Language Models}},
author={Paulius, David and Agostini, Alejandro and Quartey, Benedict and Konidaris, George},
booktitle={Proceedings of the 2025 IEEE International Conference on Robotics and Automation (ICRA)},
year={2025}
}