User-based tailoring with Reinforcement Learning for an mHealth, COPD-focused intervention to increase physical activity
Sandra Straková MSc. (EEMCS-BSS), dr. Mannes Poel (EEMCS-DMB), dr. Anouk Middelweerd (EEMCS-BSS), dr. ir. Monique Tabak (EEMCS-BSS), dr. ir. Wendy d’Hollosy (EEMCS-BSS), dr. Tessa C. Beinema (EEMCS-HMI)
Abstract
Introduction
For people with chronic obstructive pulmonary disease (COPD), adherence to physical activity (PA) guidelines is lacking, despite proven benefits. Previous studies addressed this problem via mobile health (mHealth) and Just-in-Time adaptive interventions (JITAI), which can provide frequent reminders and motivational cues to users to stimulate PA. In order to create an effective solution, such interventions necessitate the use of behavioural theories and user-based tailoring to increase the relevance of the coaching to each user. However, approaches that have made use of these concepts tend to rely on inflexible, rule-based methods.
Objectives
The main objective of this study was to create an algorithm using Reinforcement Learning (RL), a type of Machine Learning, that could automatically adapt the intervention content to stimulate users with COPD to engage in PA, by learning which messages are relevant to each user’s current context and behaviour.
Methods
The created RL-based algorithm was trained and tested within a simulation, with created user profiles that have differing needs related their behavioural state and previous PA. At given times of the day within the simulation, the algorithm selected messages assessed by the algorithm to be relevant to the current user context, and the user’s PA was monitored.
Results
The results have shown a superior performance of the RL-based algorithm over an algorithm that selects the coaching messages at random, in delivering more relevant content that helped the simulated users increase their engagement in PA. Using this approach, we have shown the potential of RL in creating a JITAI tailored to users with COPD.