Embedded Files
6D robotic grasping beyond top-down bin-picking scenarios is a challenging task. Previous solutions based on 6D grasp synthesis with robot motion planning usually operate in an open-loop setting, which are sensitive to grasp synthesis errors. In this work, we propose a new method for learning closed-loop control policies for 6D grasping. Our policy takes a segmented point cloud of an object from an egocentric camera as input, and outputs continuous 6D control actions of the robot gripper for grasping the object. We combine imitation learning and reinforcement learning and introduce a goal-auxiliary actor-critic algorithm for policy learning. We demonstrate that our learned policy can be integrated into a tabletop 6D grasping system and a human-robot handover system to improve the grasping performance of unseen objects.
Overview
Overview
submit_video.mp4Ablation Studies in Simulation
Ablation Studies in Simulation
We ablate our methods on simulation from several perspectives. We show expert demonstrations and object repositories as our data. We compare different policy models and with an open-loop system. We test on moving objects and show simple extensions of our method to cluttered-scene and placing. Finally, we show that our trained policy in simulation does have limitations where the same grasp can work in simulation but fail in real-world tests due to contact modeling.
We ablate our methods on simulation from several perspectives. We show expert demonstrations and object repositories as our data. We compare different policy models and with an open-loop system. We test on moving objects and show simple extensions of our method to cluttered-scene and placing. Finally, we show that our trained policy in simulation does have limitations where the same grasp can work in simulation but fail in real-world tests due to contact modeling.
Object Repository
Object Repository
object_repo.mp4Expert Demonstration
Expert Demonstration
expert_demonstration.mp4Model Comparison
Model Comparison
model_comparison.mp4Open-Loop Comparison
Open-Loop Comparison
dynamic_scene_grasp.mp4Tabletop 6D Grasping in the Real World
Tabletop 6D Grasping in the Real World
We combine unseen object instance segmentation and GA-DDPG for 6D grasping of unseen objects.
We combine unseen object instance segmentation and GA-DDPG for 6D grasping of unseen objects.
unseen_objects_4x.mp4We compare with a state-of-the-art open-loop method and show our policy can be used to improve the system for tabletop 6D grasping. We tested the methods on fixed (left) and varying (right) robot initial states on 9 YCB objects and 10 unseen objects.
We compare with a state-of-the-art open-loop method and show our policy can be used to improve the system for tabletop 6D grasping. We tested the methods on fixed (left) and varying (right) robot initial states on 9 YCB objects and 10 unseen objects.
Human-to-robot Handover in the Real World
Human-to-robot Handover in the Real World
We combine human body tracking and object segmentation with GA-DDPG for human-to-robot handover.
We combine human body tracking and object segmentation with GA-DDPG for human-to-robot handover.
handover.mp4We conduct experiments with different handover locations and object motion.
We conduct experiments with different handover locations and object motion.
Page updated
Google Sites
Report abuse