Soft Body Manipulation with Differentiable Physics

Dynamic state representation learning that can accurately describe dynamics can significantly accelerate reinforcement learning training. However, deformable objects have very complicated dynamics and high DoFs. We propose DiffSRL, an end-to-end dynamic state representation learning pipeline that uses a differentiable physics engine to learn the representation of deformable objects. Our specially designed loss function can guide neural networks to be aware of dynamics and constraints. We benchmark the performance of our methods as well as other state representation algorithms with downstream tasks on PlasticineLab. Our model demonstrates superior performance most of the time on all tasks. We also demonstrate our model’s performance in a real-world setting with two manipulation tasks on a UR-5 robot arm.