Cooking Robotics: Perception and motion planning

ICRA 2024 Workshop @Yokohama, Japan

17th May 2024,

Annex Hall F205-206 in Pacifico Yokohama

Abstract

In 2023-24, automation in the food industry and daily cooking assistants have gained increasing importance due to labor shortages, economic disparities, and increased hygiene emphasis in the post-COVID world. Additionally, today's diverse society demands personalized gastronomy services.

Recent technological advancements, such as large language models for control, big data for learning, and highly precise sensors, offer promising solutions to these challenges. However, there are limited opportunities to propose practical applications for these technologies or to provide comprehensive overviews of the associated issues.

This workshop invites world leading research scientists and practitioners to explore the new frontiers of `robots in cooking’, addressing various scientific research questions, including hardware considerations, key challenges in multimodal perception, motion planning and control, experimental methodologies, and benchmarking approaches. Our workshop will provide a platform for exchanging insights and addressing the challenges of these technologies in real-world settings through the lens of cooking robotics.

Furthermore, our aim is to bridge the gap between academia, industry, and professionals by inviting chefs, collaborating with a competition, Food Topping Challenge and showcasing tangible robot demonstrations. Participants will engage in interactive sessions to share state-of-the-art technologies and promote further research in the field of cooking robotics.

Topics of Interest

Unique elements of the workshop

Industry engagement to understand demand
Online demonstrations to understand real-world challenges

Topics

Perception
- Computer vision
- Haptics, tactile and proprioceptive sensing
- Taste, smell, appearance
- Ingredient / food characteristics and dynamics
Motion planning and control
- Learning skills (pouring, stirring, cutting, transporting, etc)
- Deformable object manipulation
- Manipulation with tools
Task Planning
- Task planning with recipe / LLM
- Long-horizontal manipulation
- Planning in the cooking domain (recipe and dish creation, manipulation in cluttered space, etc)
Hardware
- Soft robotics
- Hands for food manipulation
- Haptics, tactile and proprioceptive sensors
Application
- Human robot interaction
- 3D food printing and designing
- Safety, reliability and hygiene
- Needs and seeds for society, industry, and home

Invited Speakers

Tamim Asfour

Karlsruhe Institute of Technology

Fumiya Iida

University of Cambridge

Josie Hughes

EPFL

Mako Miyatake

Freelancer chef of "designing the future of the food"

Priya Sundaresan / Dorsa Sadigh

Stanford University

Akihiko Yamaguchi

Tohoku University / Finger Vision

Kevin Zhang / Oliver Kroemer

Carnegie Mellon University

Panel discussion: Industry × Academia

Bridge the gap among academia, industry, and professionals, and discuss the future of cooking robotics.

Let's share needs and seeds, challenges, and ambitions!!

Lead panelist

Dr. Michael Spranger COO of Sony AI Inc.

"To enhance chefs’ creativity, we’re looking to apply AI and robotics throughout the creation process. We’re using AI systems to put vast amounts of food data into the hands of chefs, supporting their knowledge and skills when it comes to ingredient selection and pairing. And we’re building robots that can manipulate various food items and cooking utensils with high precision, dexterity and speed, to assist chefs in their cooking and plating processes. " (from SonyAI website)

Special guest panelist

Dr. Hiroya Kawasaki

Executive Specialist at Institute of Food Sciences and Technologies, Ajinomoto Co., Inc. Board of Directors of the NPO, Japanese Culinary Academy.

His research interests include the culinary science of professional cooking techniques and the sensory evaluation techniques.

Panelists from academia

Prof. Tamim Asfour (KIT)
Prof. Fumiya Iida (Cambridge)
Prof. Sethu Vijayakumar (Edinburgh)

Collaboration with "Food Topping Challenge" Competition

https://icra2024.rt-net.jp/

Yuki Nakagawa et al, RT Corporation

In their competition, some teams will challenge two cooking tasks: "A fried chicken pick and place to tray on conveyors" and "Serving Bowl meal, the Ikura Don." They will compete for speed, accuracy, visual quality, and the amount of food loss and cost.

We invite awarded teams to present and demonstrate their splendid works!

Make a revolution in food automation!!

Real robot demonstrations

Kawada Robotics Coporation

Live tele-operation of Nextage

Juchheim Co., Ltd.

THEO, the Baumkuchen cake baking robot

Waseda Uni., JST Moonshot Program goal 3

Table wiping demonstration with Dry-AIREC

XELA Robotics

3 axis tactile sensors

Tsukuba Uni.

Making hamburgers by CRANE-X7

Important Dates

Paper and Video submission deadline: 31st March 2024 7th April 2024
Notification of acceptance: 14th April 2024 16th April 2024.
Workshop date: 17th May 2024

Call for Participation (Closed)

We encourage participants to submit their research in the form of a video (180 sec) and a short (1-4 pages, excluding references) paper. We welcome contributors from academia, industry and professionals!!

Please check the detail in Call for contributions page. (Closed)

Please check the lists of accepted papers in Accepted works.

Award sponsored by Sony Research

We awarded 3 splendid works with the prize (20,000 JPY each).

Best Paper Award:

Amisha Bhaskar, Rui Liu, Guangyao Shi, and Pratap Tokekar, “LAVA: Long-horizon Visual Action based Food Acquisition”

Best Video Award:

Jeremy Siburian, Cristian C. Beltran-Hernandez, and Masashi Hamaya, “Integrated Task and Motion Planning for Real-World Cooking Tasks”

Best Poster Award:

Cristian C. Beltran-Hernandez, Nicolas Erbetti, Masashi Hamaya, “SliceIt!: Simulation-Based Reinforcement Learning for Compliant Robotic Food Slicing”