Following work on joint object-action representations, functional object-oriented networks (FOON) were introduced as a knowledge graph representation for robots. A FOON contains symbolic concepts useful to a robot's understanding of tasks and its environment for object-level planning.
Prior to this work, little has been done to show how plans acquired from FOON can be executed by a robot, as the concepts in a FOON are too abstract for execution. We thereby introduce the idea of exploiting object-level knowledge as a FOON for task planning and execution.
Our approach automatically transforms FOON into PDDL and leverages off-the-shelf planners, action contexts, and robot skills in a hierarchical planning pipeline to generate executable task plans. We demonstrate our entire approach on long-horizon tasks in CoppeliaSim and show how learned action contexts can be extended to never-before-seen scenarios.
Below you will find some videos for experimental results from Section V of our paper.
Other materials can be found in our Google Drive.
In Section V-B, our goal is to show how our approach produces micro-plans that adapt to the configuration of the environment. It is important to note here that these plans are derived directly from the Fast-Downward planner, and we do not handcraft these plans as would be needed for hierarchical task networks (HTN).
Below are videos corresponding to Figure 7 in the paper. Each video is a solution to a single functional unit or macro-planning operator. Notably, each execution varies due to the state of the robot's environment.
We show some examples of whole recipe execution and partial recipe execution for the Bloody Mary cocktail and Greek salad recipes. A simulated robot uses a total of 703 action contexts (635 from the cocktail scenario + 68 from the salad scenario) learned from demonstration.
In the cocktail task, robot execution was 96% successful for whole execution and 92% for partial execution; in the salad task, robot execution was 80% successful for whole execution and 84% for partial execution.
Note: Micro-plans can be opened with a text editor.
@article{paulius2023_foonlhpe,
title={{Long-Horizon Planning and Execution with Functional Object-Oriented Networks}},
author={Paulius*, David and Agostini*, Alejandro and Lee, Dongheui},
journal={IEEE Robotics and Automation Letters},
year={2023},
volume={8},
number={8},
pages={4513-4520},
doi={10.1109/LRA.2023.3285510}
}