Human Activity Detection in VR Scenario

Projects supported by Funding Scheme for student projects/activities by Tam Wing Fan Innovation Fund and Philomathia Foundation Innovation Fund

Supervisor: Prof. Y.H. KUO (Department of Industrial and Manufacturing Systems Engineering)

Project information

Project descriptions

Develop a VR-Based Motion Capture System: Create a robust and accurate motion capture system to track and record users’ movements in real-time within the VR environment. To collect data for the later machine learning model training and testing.
Machine Learning Model and Algorithm design for Behavior Detection: Construct a combined system of suitable computer vision models to detect and analysis human activity in VR Scenario. Multi-models will be involved in this system, and an algorithm will be designed to help the system predict activities based on the predictions from those models, e.g. Voting Classifiers.
Real-Time Feedback and User Interaction: Integrate the behavior detection model with the VR environment to offer real-time feedback to users based on their actions and interactions, enhancing immersion and user engagement.

Project innovation

  • Machine Learning approach to achieve human activity detection in VR scenario
  • Transfer Learning based on SlowFast Model but adding VR scenario data
  • Human detection based on Yolov8


Team information

Project leader: HE Zixuan, BEng(CompSc)

Team member(s): GONG Nanqi, BEng(CE)

Project poster
Project video
Project images

Winner of the Inno Show Award @ The 6th Inno Show

This project team was selected for the Inno Show award at the 6th Inno Show. 

This team has received HK$20,000 sponsorship to participate in international design competition(s) and consumables expenses to support further development of the project.

The best project award - COMP3329 Computer Game Design and Programming @ The 1st Engineering InnoShow

This project team was selected for the best project award – COMP3329 Computer Game Design and Programming at the 1st Engineering InnoShow.

PERfECT Wearables: for Minimally-Invasive and Continuous Blood-Glucose Monitoring was selected for the Inno Show Award
Photos in the Engineering Inno Show