Soft Body Manipulation with Differentiable Physics

Projects from courses/Final Year Project

Supervisor: Dr. PAN Jia (Department of Computer Science)

Project information

Project descriptions

Dynamic state representation learning that can accurately describe dynamics can significantly accelerate reinforcement learning training. However, deformable objects have very complicated dynamics and high DoFs. We propose DiffSRL, an end-to-end dynamic state representation learning pipeline that uses a differentiable physics engine to learn the representation of deformable objects. Our specially designed loss function can guide neural networks to be aware of dynamics and constraints. We benchmark the performance of our methods as well as other state representation algorithms with downstream tasks on PlasticineLab. Our model demonstrates superior performance most of the time on all tasks. We also demonstrate our model’s performance in a real-world setting with two manipulation tasks on a UR-5 robot arm.

Team information

Project leader: YAO Shang Wen, BEng(CompSc)

Team member(s): CHEN Sirui, BEng(CompSc); LIU Yunhao, BEng(CompSc)

Project poster
Project video
Project images

Winner of the Inno Show Award @ The 5th Inno Show

This project team was selected for the Inno Show award at the 5th Inno Show. 

This team has received HK$20,000 sponsorship to participate in international design competition(s) and consumables expenses to support further development of the project.

The best project award - COMP3329 Computer Game Design and Programming @ The 1st Engineering InnoShow

This project team was selected for the best project award – COMP3329 Computer Game Design and Programming at the 1st Engineering InnoShow.

Members of the Club Grenade project team in HKUEAA Annual Dinner
Photos in the Engineering Inno Show