Remote heart rate monitoring - Assessment of the Facereader rPPg by Noldus


Autoři: Simone Benedetto aff001;  Christian Caldato aff001;  Darren C. Greenwood aff002;  Nicola Bartoli aff001;  Virginia Pensabene aff004;  Paolo Actis aff004
Působiště autorů: TSW XP Lab, Via Terraglio, Treviso, Italy aff001;  Leeds Institute for Cardiovascular and Metabolic Medicine, University of Leeds, Leeds, United Kingdom aff002;  Leeds Institute for Data Analytics, University of Leeds, Leeds, United Kingdom aff003;  School of Electronic and Electrical Engineering, University of Leeds, Leeds, West Yorkshire, United Kingdom aff004;  School of Medicine, Leeds Institute of Biomedical and Clinical Sciences, University of Leeds, Leeds, West Yorkshire, United Kingdom aff005
Vyšlo v časopise: PLoS ONE 14(11)
Kategorie: Research Article
doi: 10.1371/journal.pone.0225592

Souhrn

Remote photoplethysmography (rPPG) allows contactless monitoring of human cardiac activity through a video camera. In this study, we assessed the accuracy and precision for heart rate measurements of the only consumer product available on the market, namely the FacereaderTM rPPG by Noldus, with respect to a gold standard electrocardiograph. Twenty-four healthy participants were asked to sit in front of a computer screen and alternate two periods of rest with two stress tests (i.e. Go/No-Go task), while their heart rate was simultaneously acquired for 20 minutes using the ECG criterion measure and the FacereaderTM rPPG. Results show that the FacereaderTM rPPG tends to overestimate lower heart rates and underestimate higher heart rates compared to the ECG. The Facereader rPPG revealed a mean bias of 9.8 bpm, the 95% limits of agreement (LoA) ranged from almost -30 up to +50 bpm. These results suggest that whilst the rPPG FacereaderTM technology has potential for contactless heart rate monitoring, its predictions are inaccurate for higher heart rates, with unacceptable precision across the entire range, rendering its estimates unreliable for monitoring individuals.

Klíčová slova:

Cameras – Electrocardiography – Face – Heart rate – Imaging techniques – Light – Sensory physiology – Skin physiology


Zdroje

1. Mittelstadt B, Fairweather B, Shaw M, Mcbride N. The Ethical Implications of Personal Health Monitoring. International Journal of Technoethics. 2014;5: 37–60. doi: 10.4018/ijt.2014070104

2. Lauriks S, Reinersmann A, Roest HGVD, Meiland F, Davies R, Moelaert F, et al. Review of ICT-Based Services for Identified Unmet Needs in People with Dementia. Advanced Information and Knowledge Processing Supporting People with Dementia Using Pervasive Health Technologies. 2010; 37–61. doi: 10.1007/978-1-84882-551-2_4

3. Pantelopoulos A, Bourbakis NG. A survey on wearable sensor-based systems for health monitoring and prognosis. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews). 2010; 40(1): 1–12. doi: 10.1109/TSMCC.2009.2032660

4. Majumder S, Mondal T, Deen M. Wearable Sensors for Remote Health Monitoring. Sensors. 2017;17: 130. doi: 10.3390/s17010130 28085085

5. Nedungadi P, Jayakumar A, Raman R. Personalized Health Monitoring System for Managing Well-Being in Rural Areas. Journal of Medical Systems. 2017;42. doi: 10.1007/s10916-017-0854-9 29242996

6. Tang PC, Ash JS, Bates DW, Overhage JM, Sands DZ. Personal Health Records: Definitions, Benefits, and Strategies for Overcoming Barriers to Adoption. Journal of the American Medical Informatics Association. 2006;13: 121–126. doi: 10.1197/jamia.M2025 16357345

7. Agree EM, Freedman VA, Cornman JC, Wolf DA, Marcotte JE. Reconsidering Substitution in Long-Term Care: When Does Assistive Technology Take the Place of Personal Care? The Journals of Gerontology Series B: Psychological Sciences and Social Sciences. 2005;60. doi: 10.1093/geronb/60.5.s272 16131628

8. Kaelber DC, Jha AK, Johnston D, Middleton B, Bates DW. A research agenda for personal health records (PHRs). Journal of the American Medical Informatics Association. 2008; 15(6): 729–736. doi: 10.1197/jamia.M2547 18756002

9. Perednia DA. Telemedicine Technology and Clinical Applications. JAMA: The Journal of the American Medical Association. 1995;273: 483. doi: 10.1001/jama.1995.03520300057037 7837367

10. Hu PJ, Chau PY, Sheng ORL, Tam KY. Examining the Technology Acceptance Model Using Physician Acceptance of Telemedicine Technology. Journal of Management Information Systems. 1999;16: 91–112. doi: 10.1080/07421222.1999.11518247

11. Norris AC. Essentials of Telemedicine and Telecare. 2001; doi: 10.1002/0470846348

12. Martinez AW, Phillips ST, Carrilho E, Thomas SW, Sindi H, Whitesides GM. Simple Telemedicine for Developing Regions: Camera Phones and Paper-Based Microfluidic Devices for Real-Time, Off-Site Diagnosis. Analytical Chemistry. 2008;80: 3699–3707. doi: 10.1021/ac800112r 18407617

13. Zhao F, Li M, Qian Y, Tsien JZ. Remote Measurements of Heart and Respiration Rates for Telemedicine. PLoS ONE. 2013;8. doi: 10.1371/journal.pone.0071384 24115996

14. Zhang Z. Heart rate monitoring from wrist-type photoplethysmographic (PPG) signals during intensive physical exercise. 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP). 2014; doi: 10.1109/globalsip.2014.7032208

15. Wallen MP, Gomersall SR, Keating SE, Wisløff U, Coombes JS. Accuracy of Heart Rate Watches: Implications for Weight Management. Plos One. 2016;11. doi: 10.1371/journal.pone.0154420 27232714

16. Shcherbina A, Mattsson C, Waggott D, Salisbury H, Christle J, Hastie T, et al. Accuracy in Wrist-Worn, Sensor-Based Measurements of Heart Rate and Energy Expenditure in a Diverse Cohort. Journal of Personalized Medicine. 2017;7: 3. doi: 10.3390/jpm7020003 28538708

17. Benedetto S, Caldato C, Bazzan E, Greenwood DC, Pensabene V, Actis P. Assessment of the Fitbit Charge 2 for monitoring heart rate. Plos One. 2018;13. doi: 10.1371/journal.pone.0192691 29489850

18. Adam MT, Krämer J, Müller MB. Auction Fever! How Time Pressure and Social Competition Affect Bidders’ Arousal and Bids in Retail Auctions. Journal of Retailing. 2015;91: 468–485. doi: 10.1016/j.jretai.2015.01.003

19. Astor PJ, Adam MTP, Jerčić P, Schaaff K, Weinhardt C. Integrating Biosignals into Information Systems: A NeuroIS Tool for Improving Emotion Regulation. Journal of Management Information Systems. 2013;30: 247–278. doi: 10.2753/mis0742-1222300309

20. Teubner T, Adam M, Riordan R. The Impact of Computerized Agents on Immediate Emotions, Overall Arousal and Bidding Behavior in Electronic Auctions. Journal of the Association for Information Systems. 2015;16: 838–879. doi: 10.17705/1jais.00412

21. Arnetz BB. Techno-Stress. Journal of Occupational & Environmental Medicine. 1996;38: 53–65. doi: 10.1097/00043764-199601000-00017 8871332

22. Riedl R. On the biology of technostress. ACM SIGMIS Database. 2012;44: 18. doi: 10.1145/2436239.2436242

23. Riedl R, Kindermann H, Auinger A, Javor A. Technostress from a Neurobiological Perspective. Business & Information Systems Engineering. 2012;4: 61–69. doi: 10.1007/s12599-012-0207-7

24. Fischer T, Halmerbauer G, Meyr E, Riedl R. Blood Pressure Measurement: A Classic of Stress Measurement and Its Role in Technostress Research. Information Systems and Neuroscience Lecture Notes in Information Systems and Organisation. 2017;: 25–35. doi: 10.1007/978-3-319-67431-5_4

25. Rouast PV, Adam MTP, Chiong R, Cornforth D, Lux E. Remote heart rate measurement using low-cost RGB face video: a technical literature review. Frontiers of Computer Science. 2018;12: 858–872. doi: 10.1007/s11704-016-6243-6

26. Rouast PV, Adam MTP, Cornforth DJ, Lux E, Weinhardt C. Using Contactless Heart Rate Measurements for Real-Time Assessment of Affective States. Information Systems and Neuroscience Lecture Notes in Information Systems and Organisation. 2016;: 157–163. doi: 10.1007/978-3-319-41402-7_20

27. Wang YJ, Minor MS. Validity, reliability, and applicability of psychophysiological techniques in marketing research. Psychology and Marketing. 2008;25: 197–232. doi: 10.1002/mar.20206

28. Souiden N, Ladhari R, Chiadmi N-E. New trends in retailing and services. Journal of Retailing and Consumer Services. 2018; doi: 10.1016/j.jretconser.2018.07.023

29. Karmarkar U. R., & Plassmann H. (2019). Consumer neuroscience: Past, present, and future. Organizational Research Methods, 22(1), 174–195.

30. Shaw SD, Bagozzi RP. The neuropsychology of consumer behavior and marketing. Consumer Psychology Review. 2017;1: 22–40. doi: 10.1002/arcp.1006

31. Rouast P. V., Adam M., & Chiong R. (2019). Deep learning for human affect recognition: insights and new developments. IEEE Transactions on Affective Computing. doi: 10.1109/TAFFC.2017.2678472

32. Castellini P, Martarelli M, Tomasini E. Laser Doppler Vibrometry: Development of advanced solutions answering to technologys needs. Mechanical Systems and Signal Processing. 2006;20: 1265–1285. doi: 10.1016/j.ymssp.2005.11.015

33. Melis MD, Morbiducci U, Scalise L, Tomasini E, Delbeke D, Baets R, et al. A preliminary study for the evaluation of large artery stiffness: a non contact approach. Artery Research. 2008;2: 100–101. doi: 10.1016/j.artres.2008.08.343

34. Nam Y, Kong Y, Reyes B, Reljin N, Chon KH. Monitoring of Heart and Breathing Rates Using Dual Cameras on a Smartphone. Plos One. 2016;11. doi: 10.1371/journal.pone.0151013 26963390

35. Wieringa FP, Mastik F, Van Der Steen A. F. W. Contactless Multiple Wavelength Photoplethysmographic Imaging: A First Step Toward “SpO2 Camera” Technology. Annals of Biomedical Engineering. 2005;33: 1034–1041. doi: 10.1007/s10439-005-5763-2 16133912

36. Shaffer F, Ginsberg JP. An Overview of Heart Rate Variability Metrics and Norms. Frontiers in Public Health. 2017;5. doi: 10.3389/fpubh.2017.00258 29034226

37. Zangróniz R, Martínez-Rodrigo A, López M, Pastor J, Fernández-Caballero A. Estimation of Mental Distress from Photoplethysmography. Applied Sciences. 2018;8: 69. doi: 10.3390/app8010069

38. Li G, Chung W-Y. Detection of Driver Drowsiness Using Wavelet Analysis of Heart Rate Variability and a Support Vector Machine Classifier. Sensors. 2013;13: 16494–16511. doi: 10.3390/s131216494 24316564

39. Mcduff D, Hurter C, Gonzalez-Franco M. Pulse and vital sign measurement in mixed reality using a HoloLens. Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology—VRST 17. 2017; doi: 10.1145/3139131.3139134

40. Allen J. Photoplethysmography and its application in clinical physiological measurement. Physiological Measurement. 2007;28. doi: 10.1088/0967-3334/28/3/r01 17322588

41. Lindberg L-G. Optical properties of blood in motion. Optical Engineering. 1993;32: 253. doi: 10.1117/12.60688

42. Verkruysse W, Svaasand LO, Nelson JS. Remote plethysmographic imaging using ambient light. Optics Express. 2008;16: 21434. doi: 10.1364/oe.16.021434 19104573

43. Gastel MV, Stuijk S, Haan GD. Motion Robust Remote-PPG in Infrared. IEEE Transactions on Biomedical Engineering. 2015;62: 1425–1433. doi: 10.1109/TBME.2015.2390261 25585411

44. Mitsuhashi R, Okada G, Kurita K, Kagawa K, Kawahito S, Koopipat C, et al. Noncontact pulse wave detection by two-band infrared video-based measurement on face without visible lighting. Artificial Life and Robotics. 2018;23: 345–352. doi: 10.1007/s10015-018-0430-5

45. Poh M-Z, Mcduff DJ, Picard RW. Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Optics Express. 2010;18: 10762. doi: 10.1364/OE.18.010762 20588929

46. Lewandowska M, Nowak J. Measuring Pulse Rate with a Webcam. Journal of Medical Imaging and Health Informatics. 2012;2: 87–92. doi: 10.1166/jmihi.2012.1064

47. Li X, Chen J, Zhao G, Pietikainen M. Remote Heart Rate Measurement from Face Videos under Realistic Situations. 2014 IEEE Conference on Computer Vision and Pattern Recognition. 2014; doi: 10.1109/cvpr.2014.543

48. Sviridova N, Zhao T, Aihara K, Nakamura K, Nakano A. Photoplethysmogram at green light: Where does chaos arise from? Chaos, Solitons & Fractals. 2018;116: 157–165. doi: 10.1016/j.chaos.2018.09.016

49. Wang W, Stuijk S, Haan GD. Exploiting Spatial Redundancy of Image Sensor for Motion Robust rPPG. IEEE Transactions on Biomedical Engineering. 2015;62: 415–425. doi: 10.1109/TBME.2014.2356291 25216474

50. Takano C, Ohta Y. Heart rate measurement based on a time-lapse image. Medical Engineering & Physics. 2007;29: 853–857. doi: 10.1016/j.medengphy.2006.09.006

51. Tang C, Lu J, Liu J. Non-contact heart rate monitoring by combining convolutional neural network skin detection and remote photoplethysmography via a low-cost camera. IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2018. 1309–1315.

52. Chen, W., & McDuff, D. (2018). Deepphys: Video-based physiological measurement using convolutional attention networks. In Proceedings of the European Conference on Computer Vision (ECCV) (pp. 349–365)

53. Tasli HE, Gudi A, Ivan P, den Uyl M. European Patent Application No. 2960862A1. 2015

54. Gonzalez Viejo C., Fuentes S., Torrico D., & Dunshea F. (2018). Non-Contact Heart Rate and Blood Pressure Estimations from Video Analysis and Machine Learning Modelling Applied to Food Sensory Responses: A Case Study for Chocolate. Sensors, 18(6), 1802.

55. Wu H, Wang T, Dai T, Lin Y, Wang Y. A Real-Time Vision-Based Heart Rate Measurement Framework for Home Nursing Assistance. Proceedings of the 3rd International Conference on Crowd Science and Engineering—ICCSE18. 2018; doi: 10.1145/3265689.3265718

56. Wang W, Brinker ACD, Haan GD. Full video pulse extraction. Biomedical Optics Express. 2018;9: 3898. doi: 10.1364/BOE.9.003898 30338163

57. Wilson F. N., Johnston F. D., Rosenbaum F. F., & Barker P. S. (1946). On Einthoven's triangle, the theory of unipolar electrocardiographic leads, and the interpretation of the precordial electrocardiogram

58. Cootes TF, Edwards GJ, Taylor CJ. Active appearance models. IEEE Transactions on Pattern Analysis & Machine Intelligence. 2001; 6: 681–685. doi: 10.1109/34.927467

59. Poh MZ, McDuff DJ, Picard RW. U.S. Patent Application. 2011: No. 13/048,965

60. Tasli HE, Gudi A, Uyl MD. Remote PPG based vital sign measurement using adaptive facial regions. 2014 IEEE International Conference on Image Processing (ICIP). 2014; doi: 10.1109/icip.2014.7025282

61. Fillmore M. T., Rush C. R., & Hays L. (2006). Acute effects of cocaine in two models of inhibitory control: implications of non‐linear dose effects. Addiction, 101(9), 1323–1332 doi: 10.1111/j.1360-0443.2006.01522.x 16911732

62. Bland JM, Altman DG. Measuring agreement in method comparison studies. Statistical Methods in Medical Research. 1999;8: 135–160. doi: 10.1177/096228029900800204 10501650

63. Bland JM, Altman DG. Agreement Between Methods of Measurement with Multiple Observations Per Individual. Journal of Biopharmaceutical Statistics. 2007;17: 571–582. doi: 10.1080/10543400701329422 17613642

64. van Gastel MV, Balmaekers B, Verkruysse W, Oetomo SB. Near-continuous non-contact cardiac pulse monitoring in a neonatal intensive care unit in near darkness. Optical Diagnostics and Sensing XVIII: Toward Point-of-Care Diagnostics. 2018; doi: 10.1117/12.2293521

65. Fukunishi M, Kurita K, Yamamoto S, Tsumura N. Video Based Measurement of Heart Rate and Heart Rate Variability Spectrogram from Estimated Hemoglobin Information. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 2018; doi: 10.1109/cvprw.2018.00180

66. Green G, Chaichulee S, Villarroel M, Jorge J, Arteta C, Zisserman A et al. Localised photoplethysmography imaging for heart rate estimation of pre-term infants in the clinic. Optical Diagnostics and Sensing XVIII: Toward Point-of-Care Diagnostics. 2018; https://doi.org/10.1117/12.2289759


Článek vyšel v časopise

PLOS One


2019 Číslo 11