Within the SimGait project we seek to develop machine learning methods for the modelling of pathological human locomotion. This is a collaborative Sinergia project funded by the Swiss National Science Foundation. The overall goal of the project is to develop, through a combination of machine learning and neuromechanical simulation, accurate models of human locomotion. In addition to the DMML group, the project partners are also the Kinesiology Lab of the University of Geneva, led by Stéphane Armand, and the Biorobotics (BioRob) Lab of EPFL, led by Auke Ijspeert. The project will (1) model pathological gaits resulting from motor impairments, such as cerebral palsy, and (2) compare and combine neuromechanical simulation and machine learning approaches for gait analysis. It brings together expertise on pathological gait, neuromechanical simulation models, machine learning, coupled with a unique collection of relevant, real world data.
Motivation: Nowadays, a doctor uses two combined sources of information for assessing a patient's gait state:
- Information that stems from the physical examination of a patient (e.g. range of motion or min/max angles of joints, spasticity or "stiffness" of the muscles, selectivity or controlability over individual muscles and anthropomorphy, i.e. the dimensions of the patient's body), which is usually gathered by physiotherapists.
- Information that stems from clinical acquisition of the patient's gait, which comes through carefully inspecting the gait of a patient through multiple-view video recordings.
Ideally, this information would suffice for a surgeon to anticipate the outcome of a given surgical operation on the patient's gait. However, it seems to be a matter of debate whether this is possible, due to the complexity of such examinations and data.
An interesting path towards strengthening a doctor's understanding about a patient's gait inefficiencies and what causes them along with alleviating the need of a series of dubious surgical operations comes through the combination of machine learning and neuromechanical simulations techniques. Focusing on human and pathological locomotion modeling, could provide doctors with more intuition about the effect that different types of surgical operations would have on a patient's gait, thus limiting the amount of interaction between a patient and a surgeon while also increasing the percentage of effective surgical gait operations.
Challenges: Human locomotion is a highly complex dynamical system. Rendering the learning procedure of locomotion models is a challenging task due to high variability, both in measurements (extrinsic) and among patient physical characteristics (intrinsic). Moreover, pathological locomotion modeling is even more challenging due to the limited availability of training data (numerous variables need to be tuned under severe scarcity of patient data).
Goals: The aims of this project is to address the above discussed challenges, by bringing together machine learning and neuromechanical simulation techniques. The efforts of our group are focused on three distinct axes:
- learning dynamical systems using conditional generative models
- learning with simulated data
- learning, tuning and improving simulators.
In general, we are particularly interested in the tight integration of simulation and learning and their interaction in a virtuous cycle with each one improving the other.
- Collected at the Willy Taillard Kinesiology lad of the University Hospitals of Geneva.
- Human gait kinematic data samples corresponding to the full gait cycle (i.e. the period of time between two identical points in the human gait process). The kinematic data provides assistance at analyzing the movements of the human body during the gait by including important information about positions, angles, velocities and accelerations of the different body segments and joints.
- Data characterizing different pathologies such as cerebral palsy, spastic paraplegia, idiopathic toe walkers (ITW), stroke/trauma, etc.
Learning models of pathological gait - Approach:
- We undertake two modeling exercises:
- A generative one, in which we only seek to model and generate gait curves (no clinical data involved).
- A predictive one, in which we seek to predict (generate) gait given a patient's clinical data.
- We explore two modeling paradigms:
- Our variant of GRUs which exploits stochastic serializations (SRNN) for data augmentation, as well as state regularizers.
- Graph networks, which aim to bring in information on the skeletal structure.
Learning models of pathological gait - Results:
- SRNN Gait generations: In the following examples we present the real gait signal graphs (left column) and their generated examples using SRNN (right column) during a full gait cycle for the right ankle and the right hip angles (top row and bottom row, respectively).
- NRI Gait Generations: In the following examples we present the real gait signal graphs (left column) and their generated examples using NRI (right column) during a full gait cycle for the right ankle and the right hip angles (top row and bottom row, respectively).
Learning using simulated data - Approach
Another promising research direction we seek to follow is applying imitation learning on simulated gait data in order to learn a gait locomotion model. Since there is an apparent data scarcity issue when we try to learn gait models, we hope to alleviate that through the usage of the BioRob simulator. The goal is to use imitation learning to learn a policy (controller) for the simulator that will try to reproduce pathological gait.
In Imitation Learning we learn a policy to make decisions from expert demonstrations. Well known use cases of imitation learning are:
- Video-game playing algorithms, that imitate human game-play in order to become more efficient at their own play, sometimes even surpassing human performance.
- Self-driving cars, where an agent learns to drive by imitation the driving behavior of humans.
- The GAIL (Generative Adversarial Imitation Learning) algorithm, where Imitation Learning techniques are applied to solve similar test cases to this project, although in less complex environments (MuJoCo environments). The following video presents an expert policy (left) and the corresponding learned policy (right) for the Walker2d-v2 MuJoCo task. The expert policy was among the experimental results of the SAM paper, a direct extension to GAIL, by Lionel Blondé and Alexandros Kalousis, more details about which can be found in this earlier blog post.
The next steps towards learning from simulated data is to exploit the recent success of Imitation Learning and Reinforcement Learning in similar settings in order to learn robust human gait locomotion models by applying them to our, more complex, case.
Results at a glance:
- Gait generation signal using Structural Recurrent Neural Network (SRNN) and Neural Relational Inference for Interacting Systems (NRI).
- Creation of two learning models for reinforcement learning agents: MLP-based and GNN-based. GNNs allow us to represent an agent in a structured manner improving robustness to environment changes.
- Learn patient specific agents using (1) reinforcement learning (RL) approach with a manually designed imitation learning reward and (2) Imitation learning approach retrieving the policy and reward function from the expert demonstration.
- Implementation of Generative Adversarial Imitation Learning (GAIL) MLP-based work with: state-action, state-only and state-state demos. State-action (SA) pairs and state-state (SS) pairs have a similar performance. State-only (SO) however seems to have more difficulty reaching expert performance.
References Structural Recurrent Neural Network (SRNN) for Group Activity Analysis, Biswas and Gall, 2018.  Neural Relational Inference for Interacting Systems, Kipf et al, 2018.  Generative Adversarial Imitation Learning, Ho and Ermon, 2016.