A system for bedside assistance that integrates a robotic bed and a mobile manipulator

Autoři: Ariel S. Kapusta aff001;  Phillip M. Grice aff001;  Henry M. Clever aff001;  Yash Chitalia aff001;  Daehyung Park aff001;  Charles C. Kemp aff001
Působiště autorů: Healthcare Robotics Lab, Institute for Robotics and Intelligent Machines, Georgia Institute of Technology, Atlanta, GA, United States of America aff001
Vyšlo v časopise: PLoS ONE 14(10)
Kategorie: Research Article
doi: 10.1371/journal.pone.0221854


Various situations, such as injuries or long-term disabilities, can result in people receiving physical assistance while in bed. We present a robotic system for bedside assistance that consists of a robotic bed and a mobile manipulator (i.e., a wheeled robot with arms) that work together to provide better assistance. Many assistive tasks depend on moving with respect to the person’s body, and the complementary physical and perceptual capabilities of the two robots help with respect to this general goal. The system provides autonomy for common tasks, as well as an interface for direct teleoperation of the two robots. Autonomy handles coarse motions of the robots by estimating the person’s pose using a pressure sensing mat and then moving the robots to configurations optimized for the task. After completing these motions, the user is given fine control of the robots to complete the task. In an evaluation using a medical mannequin, we found that the robotic bed’s motion and perception each improved the assistive robotic system’s performance. The system achieved 100% success over 9 trials involving 3 tasks. Using the system with the bed movement or the body pose estimation capabilities turned off resulted in success in only 33% or 78% of the trials, respectively. We also evaluated our system with Henry Evans, a person with severe quadriplegia, in his home. In a formal test, Henry successfully used the bedside-assistance system to perform 3 different tasks, 5 times each, without any failures. Henry’s feedback on the system was positive regarding usefulness and ease of use, and he noted benefits of using our system over fully manual teleoperation. Overall, our results suggest that a robotic bed and a mobile manipulator can work collaboratively to provide effective personal assistance and that the combination of the two robots is beneficial.

Klíčová slova:

Arms – Hygiene – Knees – Legs – Medical devices and equipment – Robotics – Robots – Web-based applications


1. KUKA Robots and Automation. Rehab Robot Gives New Hope for Bedridden Patients;. Available from: https://youtu.be/qotp80vfteM [cited 2019-05-25].

2. Grice PM, Killpack MD, Jain A, Vaish S, Hawke J, Kemp CC. Whole-arm tactile sensing for beneficial and acceptable contact during robotic assistance. In: Rehabilitation Robotics (ICORR), 2013 IEEE International Conference on. IEEE; 2013. p. 1–8.

3. Kapusta A, Kemp CC. Task-centric optimization of configurations for assistive robots. Autonomous Robots. 2019. doi: 10.1007/s10514-019-09847-2

4. Cavallo F, Aquilano M, Bonaccorsi M, Limosani R, Manzi A, Carrozza MC, et al. Improving domiciliary robotic services by integrating the ASTRO robot in an AmI infrastructure. In: Gearing up and accelerating cross-fertilization between academic and industrial robotics research in Europe: Springer; 2014. p. 267–282.

5. Kapusta A, Chitalia Y, Park D, Kemp CC. Collaboration Between a Robotic Bed and a Mobile Manipulator May Improve Physical Assistance for People with Disabilities. In: RO-MAN 2016 Workshop on behavior adaptation, interaction and learning for assistive robots (BAILAR 2016); 2016.

6. Grice P, Kemp CC. Assistive Mobile Manipulation: Designing for Operators with Motor Impairments. In: Proceedings of Robotics: Science and Systems (RSS 2016) Workshop on Socially and Physically Assistive Robotics for Humanity,; 2016.

7. Topping M, Smith J. The development of Handy 1, a rehabilitation robotic system to assist the severely disabled. Industrial Robot: An International Journal. 1998;25(5):316–320. doi: 10.1108/01439919810232459

8. Brose SW, Weber DJ, Salatin BA, Grindle GG, Wang H, Vazquez JJ, et al. The role of assistive robotics in the lives of persons with disability. American Journal of Physical Medicine & Rehabilitation. 2010;89(6):509–521. doi: 10.1097/PHM.0b013e3181cf569b

9. Graeser A, Heyer T, Fotoohi L, Lange U, Kampe H, Enjarini B, et al. A supportive friend at work: Robotic workplace assistance for the disabled. IEEE Robotics & Automation Magazine. 2013;20(4):148–159. doi: 10.1109/MRA.2013.2275695

10. Argall BD. Turning assistive machines into assistive robots. In: Quantum Sensing and Nanophotonic Devices XII. vol. 9370. International Society for Optics and Photonics; 2015. p. 93701Y.

11. Bilyea A, Seth N, Nesathurai S, Abdullah H. Robotic assistants in personal care: A scoping review. Medical engineering & physics. 2017;49:1–6.

12. Dario P, Guglielmelli E, Laschi C, Teti G. MOVAID: a personal robot in everyday life of disabled and elderly people. Technology and Disability. 1999;10(2):77–93. doi: 10.3233/TAD-1999-10202

13. Schaeffer C, May T. Care-o-bot-a system for assisting elderly or disabled persons in home environments. Assistive technology on the threshold of the new millenium. 1999.

14. Bien Z, Chung MJ, Chang PH, Kwon DS, Kim DJ, Han JS, et al. Integration of a rehabilitation robotic system (KARES II) with human-friendly man-machine interaction units. Autonomous robots. 2004;16(2):165–191. doi: 10.1023/B:AURO.0000016864.12513.77

15. Kargov A, Asfour T, Pylatiuk C, Oberle R, Klosek H, Schulz S, et al. Development of an anthropomorphic hand for a mobile assistive robot. In: Rehabilitation Robotics, 2005. ICORR 2005. 9th International Conference on. IEEE; 2005. p. 182–186.

16. Graf B, Reiser U, Hägele M, Mauz K, Klein P. Robotic home assistant Care-O-bot® 3-product vision and innovation platform. In: Advanced Robotics and its Social Impacts (ARSO), 2009 IEEE Workshop on. IEEE; 2009. p. 139–144.

17. Jain A, Kemp CC. EL-E: an assistive mobile manipulator that autonomously fetches objects from flat surfaces. Autonomous Robots. 2010;28(1):45–64. doi: 10.1007/s10514-009-9148-5

18. Yamamoto T, Nishino T, Kajima H, Ohta M, Ikeda K. Human support robot (HSR). In: ACM SIGGRAPH 2018 Emerging Technologies. ACM; 2018. p. 11.

19. Martinez-Martin E, del Pobil AP. Personal robot assistants for elderly care: an overview. In: Personal Assistants: Emerging Computational Technologies. Springer; 2018. p. 77–91.

20. Graeser A. Ambient intelligence and rehabilitation robots—A necessary symbiosis for robust operation in unstructured environments. In: 2010 9th International Symposium on Electronics and Telecommunications. IEEE; 2010. p. 9–16.

21. Verma S, Gonthina P, Hawks Z, Nahar D, Brooks JO, Walker ID, et al. Design and Evaluation of Two Robotic Furnishings Partnering with Each Other and Their Users to Enable Independent Living. In: Proceedings of the 12th EAI International Conference on Pervasive Computing Technologies for Healthcare. ACM; 2018. p. 35–44.

22. De Aguiar CH, Fateminasab R, Frazelle CG, Scott R, Wang Y, Wooten MB, et al. The networked, robotic home+ furniture suite: a distributed, assistive technology facilitating aging in place. In: 2016 IEEE International Conference on Automation Science and Engineering (CASE). IEEE; 2016. p. 1067–1072.

23. Park KH, Bien Z, Lee JJ, Kim BK, Lim JT, Kim JO, et al. Robotic smart house to assist people with movement disabilities. Autonomous Robots. 2007;22(2):183–198. doi: 10.1007/s10514-006-9012-9

24. Chen TL, Ciocarlie M, Cousins S, Grice PM, Hawkins K, et al. Robots for humanity: A case study in assistive mobile manipulation. 2013.

25. Mohammed S, Park HW, Park CH, Amirat Y, Argall B. Special Issue on Assistive and Rehabilitation Robotics. Autonomous Robots. 2017;41(3):513–517. doi: 10.1007/s10514-017-9627-z

26. Yamazaki K, Ueda R, Nozawa S, Mori Y, Maki T, Hatao N, et al. A demonstrative research for daily assistive robots on tasks of cleaning and tidying up rooms. In: Proceedings of the 14th Robotics Symposia; 2009. p. 522–527.

27. Ciocarlie M, Hsiao K, Leeper A, Gossow D. Mobile manipulation through an assistive home robot. In: Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on. IEEE; 2012. p. 5313–5320.

28. Grice PM, Kemp CC. In-home and remote use of robotic body surrogates by people with profound motor deficits. PLOS ONE. 2019;14(3):1–28. doi: 10.1371/journal.pone.0212904

29. Hawkins KP, Grice PM, Chen TL, King CH, Kemp CC. Assistive mobile manipulation for self-care tasks around the head. In: Computational Intelligence in Robotic Rehabilitation and Assistive Technologies (CIR2AT), 2014 IEEE Symposium on. IEEE; 2014.

30. Park D, Hoshi Y, Mahajan HP, Rogers WA, Kemp CC. Toward Active Robot-Assisted Feeding with a General-Purpose Mobile Manipulator: Design, Evaluation, and Lessons Learned. arXiv preprint arXiv:190403568. 2019;.

31. Nguyen H, Anderson C, Trevor A, Jain A, Xu Z, Kemp CC. El-e: An assistive robot that fetches objects from flat surfaces. In: Robotic helpers, int. conf. on human-robot interaction; 2008.

32. Mukai T, Hirano S, Nakashima H, Kato Y, Sakaida Y, Guo S, et al. Development of a nursing-care assistant robot RIBA that can lift a human in its arms. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE; 2010. p. 5996–6001.

33. Mosteo AR, Montano L. A survey of multi-robot task allocation. Instituto de Investigación en Ingenierıa de Aragón, University of Zaragoza, Technical Report No AMI-009-10-TEC. 2010.

34. Yan Z, Jouandeau N, Cherif AA. A survey and analysis of multi-robot coordination. International Journal of Advanced Robotic Systems. 2013;10. doi: 10.5772/57313

35. Doriya R, Mishra S, Gupta S. A brief survey and analysis of multi-robot communication and coordination. In: International Conference on Computing, Communication & Automation. IEEE; 2015. p. 1014–1021.

36. Cortés J, Egerstedt M. Coordinated control of multi-robot systems: A survey. SICE Journal of Control, Measurement, and System Integration. 2017;10(6):495–503. doi: 10.9746/jcmsi.10.495

37. Dias MB, Zlot R, Kalra N, Stentz A. Market-based multirobot coordination: A survey and analysis. Proceedings of the IEEE. 2006;94(7):1257–1270. doi: 10.1109/JPROC.2006.876939

38. Parker LE. Multiple mobile robot systems. In: Springer Handbook of Robotics. Springer; 2008. p. 921–941.

39. RIZK Y, AWAD M, TUNSTEL EW. Cooperative Heterogeneous Multi-Robot Systems: A Survey. 2019.

40. Trumpf Medical. TruSystem 7000dV OR Table Redefining Patient Positioning in da Vinci Surgery;. Available from: https://www.trumpfmedical.com/en/solutions/robotic-or/trusystem-7000dv/ [cited 2019-05-25].

41. Accuray. Cyberknife System robotic radiosurgery system;. Available from: https://www.cyberknife.com/ [cited 2019-05-25].

42. Roy B, Basmajian A, Asada HH. Maneuvering a bed sheet for repositioning a bedridden patient. In: Robotics and Automation, 2003. Proceedings. ICRA’03. IEEE International Conference on. vol. 2. IEEE; 2003. p. 2224–2229.

43. Van Der Loos HM, Ullrich N, Kobayashi H. Development of sensate and robotic bed technologies for vital signs monitoring and sleep quality improvement. Autonomous Robots 15 1. 2003. doi: 10.1023/A:1024444917917

44. Peng SW, Lian FL, Fu LC. Mechanism design and mechatronic control of a multifunctional test bed for bedridden healthcare. Mechatronics, IEEE/ASME Transactions on. 2010;15(2):234–241. doi: 10.1109/TMECH.2009.2021470

45. Portillo-Velez RD, Vázquez-Santacruz E, Morales-Cruz C, Gamboa-Zúñiga M. Mechatronic design and manufacturing of an affordable healthcare robotic bed. Journal of Rehabilitation and Assistive Technologies Engineering. 2016;3:1–13. doi: 10.1177/2055668316668792

46. Seo KH, Choi TY, Oh C. Development of a robotic system for the bed-ridden. Mechatronics. 2011;21(1):227–238. doi: 10.1016/j.mechatronics.2010.10.011

47. Panasonic. “Resyone Plus” Robotic Care Bed/Wheelchair;. Available from: https://www.panasonic.oa.hk/english/products/age-free-product/resyone-plus/xpn-s10601hk.aspx [cited 2019-05-25].

48. Orun B, Martins D, Roesler C. Review of assistive technologies for bedridden persons; 2015.

49. Grice P, Chitalia Y, Rich M, et al. Autobed: Open Hardware for Accessible Web-Based Control of an Electric Bed. RESNA; 2016.

50. Chen K, Gabriel P, Alasfour A, Gong C, Doyle WK, Devinsky O, et al. Patient-Specific Pose Estimation in Clinical Environments. IEEE Journal of Translational Engineering in Health and Medicine. 2018;6:1–11.

51. Achilles F, Ichim AE, Coskun H, Tombari F, Noachtar S, Navab N. Patient MoCap: Human pose estimation under blanket occlusion for hospital monitoring applications. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2016. p. 491–499.

52. Harada T, Sato T, Mori T. Pressure distribution image based human motion tracking system using skeleton and surface integration model. In: Robotics and Automation, 2001. Proceedings 2001 ICRA. IEEE International Conference on. vol. 4. IEEE; 2001. p. 3201–3207.

53. Grimm R, Sukkau J, Hornegger J, Greiner G. Automatic patient pose estimation using pressure sensing mattresses. In: Bildverarbeitung für die Medizin 2011. Springer; 2011. p. 409–413.

54. Farshbaf M, Yousefi R, Pouyan MB, Ostadabbas S, Nourani M, Pompeo M. Detecting high-risk regions for pressure ulcer risk assessment. In: Bioinformatics and Biomedicine (BIBM), 2013 IEEE International Conference on. IEEE; 2013. p. 255–260.

55. Ostadabbas S, Pouyan MB, Nourani M, Kehtarnavaz N. In-bed posture classification and limb identification. In: 2014 IEEE Biomedical Circuits and Systems Conference (BioCAS) Proceedings. IEEE; 2014. p. 133–136.

56. Casas L, Navab N, Demirci S. Patient 3D body pose estimation from pressure imaging. International Journal of Computer Assisted Radiology and Surgery. 2019;14(3):517–524. doi: 10.1007/s11548-018-1895-3 30552647

57. Clever HM, Kapusta A, Park D, Erickson Z, Chitalia Y, Kemp CC. 3D Human Pose Estimation on a Configurable Bed from a Pressure Image. Intelligent Robots and Systems (IROS), 2018 IEEE/RSJ International Conference on. 2018;.

58. Diankov R. Automated construction of robotic manipulation programs. Carnegie Mellon University; 2010.

59. Smits R. KDL: Kinematics and Dynamics Library;. http://www.orocos.org/kdl.

60. Elbanhawi M, Simic M. Sampling-based robot motion planning: A review. IEEE Access. 2014;2:56–77. doi: 10.1109/ACCESS.2014.2302442

61. Stilman M, Kuffner JJ. Navigation among movable obstacles: Real-time reasoning in complex environments. International Journal of Humanoid Robotics. 2005;2(04):479–503. doi: 10.1142/S0219843605000545

62. Garrett CR, Lozano-Pérez T, Kaelbling LP. Backward-forward search for manipulation planning. In: Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on. IEEE; 2015. p. 6366–6373.

63. Diankov R, Ratliff N, Ferguson D, Srinivasa S, Kuffner J. Bispace planning: Concurrent multi-space exploration. Proceedings of Robotics: Science and Systems IV. 2008;63.

64. Zacharias F, Borst C, Hirzinger G. Capturing robot workspace structure: representing robot capabilities. In: Intelligent Robots and Systems, 2007. IROS 2007. IEEE/RSJ International Conference on. Ieee; 2007. p. 3229–3236.

65. Zacharias F, Sepp W, Borst C, Hirzinger G. Using a model of the reachable workspace to position mobile manipulators for 3-d trajectories. In: Humanoid Robots, 2009. Humanoids 2009. 9th IEEE-RAS International Conference on. IEEE; 2009. p. 55–61.

66. Porges O, Stouraitis T, Borst C, Roa MA. Reachability and capability analysis for manipulation tasks. In: ROBOT2013: First Iberian Robotics Conference. Springer; 2014. p. 703–718.

67. Leidner D, Dietrich A, Schmidt F, Borst C, Albu-Schaffer A. Object-centered hybrid reasoning for whole-body mobile manipulation. In: Robotics and Automation (ICRA), 2014 IEEE International Conference on. IEEE; 2014. p. 1828–1835.

68. Hsu D, Latcombe JC, Sorkin S. Placing a robot manipulator amid obstacles for optimized execution. In: Assembly and Task Planning, 1999.(ISATP’99) Proceedings of the 1999 IEEE International Symposium on. IEEE; 1999. p. 280–285.

69. Stulp F, Fedrizzi A, Beetz M. Learning and performing place-based mobile manipulation. In: Development and Learning, 2009. ICDL 2009. IEEE 8th International Conference on. IEEE; 2009. p. 1–7.

70. Diankov R, Kuffner J. Openrave: A planning architecture for autonomous robotics. Robotics Institute, Pittsburgh, PA, Tech Rep CMU-RI-TR-08-34. 2008;79.

71. Burget F, Bennewitz M. Stance selection for humanoid grasping tasks by inverse reachability maps. In: Robotics and Automation (ICRA), 2015 IEEE International Conference on. IEEE; 2015. p. 5669–5674.

72. Vahrenkamp N, Asfour T, Dillmann R. Robot placement based on reachability inversion. In: Robotics and Automation (ICRA), 2013 IEEE International Conference on. IEEE; 2013. p. 1970–1975.

73. McDermott D, Ghallab M, Howe A, Knoblock C, Ram A, Veloso M, et al. PDDL-the planning domain definition language. 1998.

74. Hansen N, Ostermeier A. Completely derandomized self-adaptation in evolution strategies. Evolutionary computation. 2001;9(2):159–195. doi: 10.1162/106365601750190398 11382355

75. Hansen N, Akimoto Y, Baudis P. CMA-ES/pycma on Github; 2019. Zenodo, Available from: https://doi.org/10.5281/zenodo.2559634.

76. Wiener JM, Hanley RJ, Clark R, Nostrand JFV. Measuring the Activities of Daily Living: Comparisons Across National Surveys. Journal of Gerontology: SOCIAL SCIENCES. 1990;45(6):S229–237. doi: 10.1093/geronj/45.6.S229

77. Andersen CK, Wittrup-Jensen KU, Lolk A, Andersen K, Kragh-Sørensen P. Ability to perform activities of daily living is the main factor affecting quality of life in patients with dementia. Health and quality of life outcomes. 2004;2(1):52. doi: 10.1186/1477-7525-2-52 15383148

78. Vest M, Murphy T, Araujo K, Pisani M, et al. Disability in activities of daily living, depression, and quality of life among older medical ICU survivors: a prospective cohort study. 2011.

Článek vyšel v časopise


2019 Číslo 10