CAPE: Corrective Actions from Precondition Errors using Large Language Models

TL;DR – In this paper, we introduce CAPE: an approach to correct errors encountered during robot plan execution. We exploit the ability of large language models to generate high-level plans and to reason about causes of errors.

January 2024 · Shreyas Sundara Raman, Vanya Cohen, Ifrah Idrees, Eric Rosen, Ray Mooney, Stefanie Tellex, David Paulius

Approximate Task Tree Retrieval in a Knowledge Network for Robotic Cooking

TL;DR – In this paper, we introduce the idea of connecting FOONs to robotic task and motion planning. We automatically transform a FOON graph, which exists at the object level (i.e., it is a representation that uses meaningful labels or expressions close to human language), into task planning specifications written in PDDL (not a very intuitive way to communicate about tasks).

July 2022 · Md Sadman Sakib, David Paulius, Yu Sun

Robot Learning of Assembly Tasks from Non-expert Demonstrations using Functional Object-Oriented Network

TL;DR – This was a collaboration with Clemson University’s Yunyi Jia and Yi Chen, who were interested in using FOONs for representing assembly tasks. They successfully utilized and adapted a FOON to robotic assembly execution.

July 2022 · Yi Chen, David Paulius, Yu Sun, Yunyi Jia

Task Planning with a Weighted Functional Object-Oriented Network

TL;DR – In this paper, we attempt to execute task plan sequences extracted from FOONs. However, these sequences may contain actions that are not executable by a robot. Therefore, a human is introduced in the planning and execution loop, and both the robot and human assistant work together to solve the task.

May 2021 · David Paulius, Kelvin Sheng Pei Dong, Yu Sun

Functional Object-Oriented Network: Construction & Expansion

TL;DR – In this paper, we explore methods in natural language processing (NLP) – specifically semantic similarity – for expanding or generalizing knowledge contained in a FOON. This alleviates the need for demonstrating and annotating graphs by other means.

May 2018 · David Paulius, Ahmad Babaeian Jelodar, Yu Sun

Functional Object-Oriented Network for Manipulation Learning

TL;DR – This was the very first paper on FOON: the functional object-oriented network. Here, we introduced what they are and how they can be used for task planning. They are advantageous for their flexibility and human interpretability.

October 2016 · David Paulius, Yongqiang Huang, Roger Milton, William David Buchanan, Jeanine Sam, Yu Sun