Development of a machine-learning workflow to detect fallen persons on images made by a vacuum-cleaning robot

Aim and Research Question(s)

The aim is the development and implementation of a machine learning algorithm to detect a fallen person on images made by a vacuum cleaning robot with a camera mounted on it.

Background

In the elderly care, fall issues are a big problem (1). In Western Europe, nearly 50.000 older adults died in the year 2017 due to falls and 71.9% of injuries requiring medical treatment in people over 70 years are due to falls (2).

Methods

Two pre-trained Convolutional Neural Network models (CNN) were set up and fine-tuned: InceptionResnetV2 (IR, classic CNN) and MobileNetV2 (MN, lightweight CNN). A dataset of 785 images was created (394 images of partially visible persons in their apartments (Figure 1), 391 images without persons). Images were taken from a vacuum robot's perspective, with a camera lens height of 16 cm. Figure 1: Original training image

Results and Discussion

The workflow consists of a preparation step that ensures that the incoming images can be appropriately processed, a preprocessing step to adapt the images to the specifics needed of the model, the setup of the individual pre-trained model, and the final prediction step. The models can state if the image belongs to the category "person" if even a part like a hand or a knee is visible (Figure 2). Figure 2: Processed image with predicted class as title

The training metrics show better values than the final prediction metrics (Table 1). The differences may be caused by overfitting, although steps have been taken to prevent it. Table 1: Performance metrics during training and prediction of unseen images

Conclusion

The machine-learning workflow is functional and works well. Further work must aim for larger datasets and retrain the models considering overfitting. IR reaches better results, but it might be indicated to choose MN if the computational power is low. Whether the accuracy rate can be weighed against the resource requirements must be determined by further research.

References

(1) Montero-Odasso, M., et al. (2022). World guidelines for falls prevention and management for older adults: A global initiative. Age and Ageing, 51(9), afac205. https://doi.org/10.1093/ageing/afac205 (2) Haagsma, J. A., et al. (2020). Falls in older aged adults in 22 European countries: Incidence, mortality and burden of disease from 1990 to 2017. Injury Prevention, 26(Suppl 1), i67–i74. https://doi.org/10.1136/injuryprev-2019-043347