Skeleton Detection Using MediaPipe as a Tool for Musculoskeletal Disorders Analysis

Abstract

Skeleton detection, also known as human pose estimation (HPE), is becoming more and more popular as it can be applied in a range of applications such as game entertainment, human-machine interaction, VR-based projects, medical rehabilitation, etc. Thanks to the booming development of deep learning, HPE solutions can be implemented using deep learning methods which require standard 2D RGB images or video sequences as input. That is, technology nowadays is making HPE solutions more and more lightweight and fast which is possible to run on mobile devices for the daily use of skeleton detection. This article covers a brief survey of current deep learning-based human pose estimation approaches in the first place. Then, a lightweight deep learning model – MediaPipe – will be illustrated from all the perspectives of its structure, working flow, strengths & weaknesses and the more concerned compatibility in platforms and programming languages. As a result, a multi-platform application for collecting movement data from patients suffering from musculoskeletal diseases relying on MediaPipe is introduced. Finally, there is a summary of achievements and obstacles of application development, which is significant as it can be a signpost for teams who are doing or about to do an application based on the MediaPipe library.