[Abstract+References] Impact of commercial sensors in human computer interaction: a review

Abstract

Nowadays, the communication gap between humans and computers might be reduced due to multimodal sensors available in the market. Therefore, it is important to know the specifications of these sensors and how they are being used in order to create human computer interfaces, which tackle complex tasks. The purpose of this paper is to review recent research regarding the up-to-date application areas of the following sensors:

(1) Emotiv sensor, which identifies emotions, facial expressions, thoughts, and head movements from users through electroencephalography signals,

(2) Leap motion controller, which recognizes hand and arm movements via vision techniques,

(3) Myo armband, which identifies hand and arm movements using electromyography signals and inertial sensors, and

(4) Oculus rift, which provides immersion into virtual reality to users.

The application areas discussed in this manuscript go from assistive technology to virtual tours. Finally, a brief discussion regarding advantages and shortcomings of each sensor is presented.

References

  1. Abreu JG, Teixeira JM, Figueiredo LS, Teichrieb V (2016) Evaluating sign language recognition using the myo armband. In: Virtual and augmented reality (SVR), 2016 XVIII symposium on, IEEE, pp 64–70Google Scholar
  2. Bassily D, Georgoulas C, Guettler J, Linner T, Bock T (2014) Intuitive and adaptive robotic arm manipulation using the Leap motion controller. In: ISR/Robotik 2014; 41st international symposium on robotics; proceedings of, VDE, pp 1–7Google Scholar
  3. Bernardos AM, Sánchez JM, Portillo JI, Wang X, Besada JA, Casar JR (2016) Design and deployment of a contactless hand-shape identification system for smart spaces. J Ambient Intell Humaniz Comput 7(3):357–370CrossRefGoogle Scholar
  4. Blaha J, Gupta M (2014) Diplopia: A virtual reality game designed to help amblyopics. In: Virtual reality (VR), 2014 iEEE, IEEE, pp 163–164Google Scholar
  5. Boschmann A, Dosen S, Werner A, Raies A, Farina D (2016) A novel immersive augmented reality system for prosthesis training and assessment. In: Biomedical and health informatics (BHI), 2016 IEEE-EMBS international conference on, IEEE, pp 280–283Google Scholar
  6. Brennan CP, McCullagh PJ, Galway L, Lightbody G (2015) Promoting autonomy in a smart home environment with a smarter interface. In: Engineering in medicine and biology society (EMBC), 2015 37th annual international conference of the IEEE, IEEE, pp 5032–5035Google Scholar
  7. Cacace J, Finzi A, Lippiello V, Furci M, Mimmo N, Marconi L (2016) A control architecture for multiple drones operated via multimodal interaction in search & rescue mission. In: Safety, security, and rescue robotics (SSRR), 2016 IEEE international symposium on, IEEE, pp 233–239Google Scholar
  8. Carrino F, Tscherrig J, Mugellini E, Khaled OA, Ingold R (2011) Head-computer interface: a multimodal approach to navigate through real and virtual worlds. In: International conference on human-computer interaction, Springer, pp 222–230Google Scholar
  9. Charles D, Pedlow K, McDonough S, Shek K, Charles T (2014) Close range depth sensing cameras for virtual reality based hand rehabilitation. J Assist Technol 8(3):138–149CrossRefGoogle Scholar
  10. Chuan CH, Regina E, Guardino C (2014) American sign language recognition using Leap motion sensor. In: Machine learning and applications (ICMLA), 2014 13th international conference on, IEEE, pp 541–544Google Scholar
  11. Ciolan IM, Buraga SC, Dafinoiu I (2016) Oculus rift 3D interaction and nicotine craving: results from a pilot study. In: ROCHI–international conference on human-computer interaction, p 58Google Scholar
  12. Da Gama A, Fallavollita P, Teichrieb V, Navab N (2015) Motor rehabilitation using Kinect: a systematic review. Games Health J 4(2):123–135CrossRefGoogle Scholar
  13. dos Reis Alves SF, Uribe-Quevedo AJ, da Silva IN, Ferasoli Filho H (2014) Pomodoro, a mobile robot platform for hand motion exercising. In: Biomedical robotics and biomechatronics 2014 5th IEEE RAS & EMBS international conference on, IEEE, pp 970–974Google Scholar
  14. Duvinage M, Castermans T, Petieau M, Hoellinger T, Cheron G, Dutoit T (2013) Performance of the emotiv epoc headset for P300-based applications. Biomed Eng Online 12(1):56CrossRefGoogle Scholar
  15. Farahani N, Post R, Duboy J, Ahmed I, Kolowitz BJ, Krinchai T, Monaco SE, Fine JL, Hartman DJ, Pantanowitz L (2016) Exploring virtual reality technology and the Oculus rift for the examination of digital pathology slides. J Pathol Inform 7Google Scholar
  16. Fiałek S, Liarokapis F (2016) Comparing two commercial brain computer interfaces for serious games and virtual environments. In: Karpouzis K, Yannakakis GN (eds) Emotion in games, Springer, Switzerland, pp 103–117Google Scholar
  17. Funasaka M, Ishikawa Y, Takata M, Joe K (2015) Sign language recognition using Leap motion controller. In: Proceedings of the international conference on parallel and distributed processing techniques and applications (PDPTA), the steering committee of the world congress in computer science, computer engineering and applied computing (WorldComp), p 263Google Scholar
  18. Gándara CV, Bauza CG (2015) Intellihome: a framework for the development of ambient assisted living applications based in low-cost technology. In: Proceedings of the Latin American conference on human computer interaction, ACM, p 18Google Scholar
  19. Gomez-Gil J, San-Jose-Gonzalez I, Nicolas-Alonso LF, Alonso-Garcia S (2011) Steering a tractor by means of an EMG-based human-machine interface. Sensors 11(7):7110–7126CrossRefGoogle Scholar
  20. Gonzalez-Sanchez J, Chavez-Echeagaray ME, Atkinson R, Burleson W (2011) Abe: an agent-based software architecture for a multimodal emotion recognition framework. In: Software architecture (WICSA), 2011 9th working IEEE/IFIP conference on, IEEE, pp 187–193Google Scholar
  21. Grubišić I, Skala Kavanagh H, Grazio S (2015) Novel approaches in hand rehabilitation. Period Biol 117(1):139–145Google Scholar
  22. Guna J, Jakus G, Pogačnik M, Tomažič S, Sodnik J (2014) An analysis of the precision and reliability of the Leap motion sensor and its suitability for static and dynamic tracking. Sensors 14(2):3702–3720CrossRefGoogle Scholar
  23. Gunasekera WL, Bendall J (2005) Rehabilitation of neurologically injured patients. In: Moore AJ, Newell DW (eds) Neurosurgery, Springer, London, pp 407–421Google Scholar
  24. Güttler J, Shah R, Georgoulas C, Bock T (2015) Unobtrusive tremor detection and measurement via human-machine interaction. Proced Comput Sci 63:467–474CrossRefGoogle Scholar
  25. Han J, Shao L, Xu D, Shotton J (2013) Enhanced computer vision with Microsoft Kinect sensor: a review. IEEE Trans Cybern 43(5):1318–1334CrossRefGoogle Scholar
  26. Hettig J, Mewes A, Riabikin O, Skalej M, Preim B, Hansen C (2015) Exploration of 3D medical image data for interventional radiology using myoelectric gesture control. In: Proceedings of the eurographics workshop on visual computing for biology and medicine, eurographics association, pp 177–185Google Scholar
  27. Ijjada MS, Thapliyal H, Caban-Holt A, Arabnia HR (2015) Evaluation of wearable head set devices in older adult populations for research. In: Computational science and computational intelligence (CSCI), 2015 international conference on, IEEE, pp 810–811Google Scholar
  28. Jurcak V, Tsuzuki D, Dan I (2007) 10/20, 10/10, and 10/5 systems revisited: their validity as relative head-surface-based positioning systems. Neuroimage 34(4):1600–1611CrossRefGoogle Scholar
  29. Kefer K, Holzmann C, Findling RD (2016) Comparing the placement of two arm-worn devices for recognizing dynamic hand gestures. In: Proceedings of the 14th international conference on advances in mobile computing and multi media, ACM, pp 99–104Google Scholar
  30. Khademi M, Mousavi Hondori H, McKenzie A, Dodakian L, Lopes CV, Cramer SC (2014) Free-hand interaction with Leap motion controller for stroke rehabilitation. In: Proceedings of the extended abstracts of the 32nd annual ACM conference on human factors in computing systems, ACM, pp 1663–1668Google Scholar
  31. Khan FR, Ong HF, Bahar N (2016) A sign language to text converter using Leap motion. Int J Adv Sci Eng Inf Technol 6(6):1089–1095Google Scholar
  32. Kim SY, Kim YY (2012) Mirror therapy for phantom limb pain. Korean J Pain 25(4):272–274CrossRefGoogle Scholar
  33. Kiorpes L, McKeet SP (1999) Neural mechanisms underlying amblyopia. Curr Opin Neurobiol 9(4):480–486CrossRefGoogle Scholar
  34. Kleven NF, Prasolova-Førland E, Fominykh M, Hansen A, Rasmussen G, Sagberg LM, Lindseth F (2014) Training nurses and educating the public using a virtual operating room with Oculus rift. In: Virtual systems & multimedia (VSMM), 2014 international conference on, IEEE, pp 206–213Google Scholar
  35. Kutafina E, Laukamp D, Bettermann R, Schroeder U, Jonas SM (2016) Wearable sensors for elearning of manual tasks: Using forearm emg in hand hygiene training. Sensors 16(8):1221CrossRefGoogle Scholar
  36. Li C, Rusak Z, Horvath I, Kooijman A, Ji L (2016) Implementation and validation of engagement monitoring in an engagement enhancing rehabilitation system. IEEE Trans Neural Syst Rehabil Eng 25(6):726–738CrossRefGoogle Scholar
  37. Li C, Yang C, Wan J, Annamalai AS, Cangelosi A (2017) Teleoperation control of baxter robot using kalman filter-based sensor fusion. Syst Sci Control Eng 5(1):156–167CrossRefGoogle Scholar
  38. Liarokapis F, Debattista K, Vourvopoulos A, Petridis P, Ene A (2014) Comparing interaction techniques for serious games through brain-computer interfaces: a user perception evaluation study. Entertain Comput 5(4):391–399CrossRefGoogle Scholar
  39. Lupu RG, Ungureanu F, Stan A (2016) A virtual reality system for post stroke recovery. In: System theory, control and computing (ICSTCC), 2016 20th international conference on, IEEE, pp 300–305Google Scholar
  40. Marin G, Dominio F, Zanuttigh P (2014) Hand gesture recognition with Leap motion and Kinect devices. In: Image processing (ICIP), 2014 IEEE international conference on, IEEE, pp 1565–1569Google Scholar
  41. McCullough M, Xu H, Michelson J, Jackoski M, Pease W, Cobb W, Kalescky W, Ladd J, Williams B (2015) Myo arm: swinging to explore a VE. In: Proceedings of the ACM SIGGRAPH symposium on applied perception, ACM, pp 107–113Google Scholar
  42. Mewes A, Saalfeld P, Riabikin O, Skalej M, Hansen C (2016) A gesture-controlled projection display for CT-guided interventions. Int J Comput Assist Radiol Surg 11(1):157–164CrossRefGoogle Scholar
  43. Mousavi Hondori H, Khademi M (2014) A review on technical and clinical impact of Microsoft Kinect on physical therapy and rehabilitation. J Med Eng 2014. doi:10.1155/2014/846514
  44. Nicola Bizzotto M, Alessandro Costanzo M, Leonardo Bizzotto M (2014) Leap motion gesture control with osirix in the operating room to control imaging: first experiences during live surgery. Surg Innov 1:2Google Scholar
  45. Nugraha BT, Sarno R, Asfani DA, Igasaki T, Munawar MN (2016) Classification of driver fatigue state based on EEG using Emotiv EPOC+. J Theor Appl Inf Technol 86(3):347Google Scholar
  46. Oskoei MA, Hu H (2007) Myoelectric control systems: a survey. Biomed Sign Process Control 2(4):275–294CrossRefGoogle Scholar
  47. Palmisano S, Mursic R, Kim J (2017) Vection and cybersickness generated by head-and-display motion in the Oculus rift. Displays 46:1–8CrossRefGoogle Scholar
  48. Phelan I, Arden M, Garcia C, Roast C (2015) Exploring virtual reality and prosthetic training. In: Virtual reality (VR), 2015 IEEE, IEEE, pp 353–354Google Scholar
  49. Powell C, Hatt SR (2009) Vision screening for amblyopia in childhood. Cochrane Database Syst Rev. doi:10.1002/14651858.CD005020.pub3
  50. Qamar A, Rahman MA, Basalamah S (2014) Adding inverse kinematics for providing live feedback in a serious game-based rehabilitation system. In: Intelligent systems, modelling and simulation (ISMS), 2014 5th international conference on, IEEE, pp 215–220Google Scholar
  51. Qamar AM, Khan AR, Husain SO, Rahman MA, Baslamah S (2015) A multi-sensory gesture-based occupational therapy environment for controlling home appliances. In: Proceedings of the 5th ACM on international conference on multimedia retrieval, ACM, pp 671–674Google Scholar
  52. Quesada L, López G, Guerrero L (2017) Automatic recognition of the american sign language fingerspelling alphabet to assist people living with speech or hearing impairments. J Ambient Intell Humaniz Comput 8(4):625–635Google Scholar
  53. Ramachandran VS, Rogers-Ramachandran D (2008) Sensations referred to a patient’s phantom arm from another subjects intact arm: perceptual correlates of mirror neurons. Med Hypotheses 70(6):1233–1234CrossRefGoogle Scholar
  54. Ranky G, Adamovich S (2010) Analysis of a commercial EEG device for the control of a robot arm. In: Bioengineering conference, proceedings of the 2010 IEEE 36th annual northeast, IEEE, pp 1–2Google Scholar
  55. Rautaray SS, Agrawal A (2015) Vision based hand gesture recognition for human computer interaction: a survey. Artif Intell Rev 43(1):1–54CrossRefGoogle Scholar
  56. Rechy-Ramirez EJ, Hu H (2014) A flexible bio-signal based HMI for hands-free control of an electric powered wheelchair. Int J Artif Life Res (IJALR) 4(1):59–76CrossRefGoogle Scholar
  57. Simoens P, De Coninck E, Vervust T, Van Wijmeersch JF, Ingelbinck T, Verbelen T, Op de Beeck M, Dhoedt B (2014) Vision: smart home control with head-mounted sensors for vision and brain activity. In: Proceedings of the fifth international workshop on Mobile cloud computing & services, ACM, pp 29–33Google Scholar
  58. Snow PW, Loureiro RC, Comley R (2014) Design of a robotic sensorimotor system for phantom limb pain rehabilitation. In: Biomedical robotics and biomechatronics 2014 5th IEEE RAS & EMBS international conference on, IEEE, pp 120–125Google Scholar
  59. Sonntag D, Orlosky J, Weber M, Gu Y, Sosnovsky S, Toyama T, Toosi EN (2015) Cognitive monitoring via eye tracking in virtual reality pedestrian environments. In: Proceedings of the 4th international symposium on pervasive displays, ACM, pp 269–270Google Scholar
  60. Subha DP, Joseph PK, Acharya R, Lim CM (2010) EEG signal analysis: a survey. J Med Syst 34(2):195–212CrossRefGoogle Scholar
  61. Toutountzi T, Collander C, Phan S, Makedon F (2016) Eyeon: An activity recognition system using myo armband. In: Proceedings of the 9th ACM international conference on PErvasive technologies related to assistive environments, ACM, p 82Google Scholar
  62. Verkijika SF, De Wet L (2015) Using a brain-computer interface (BCI) in reducing math anxiety: evidence from South Africa. Comput Educ 81:113–122CrossRefGoogle Scholar
  63. Vikram S, Li L, Russell S (2013) Handwriting and gestures in the air, recognizing on the fly. Proc CHI 13:1179–1184Google Scholar
  64. Villagrasa S, Fonseca D, Durán J (2014) Teaching case: applying gamification techniques and virtual reality for learning building engineering 3D arts. In: Proceedings of the second international conference on technological ecosystems for enhancing multiculturality, ACM, pp 171–177Google Scholar
  65. Wake N, Sano Y, Oya R, Sumitani M, Kumagaya Si, Kuniyoshi Y (2015) Multimodal virtual reality platform for the rehabilitation of phantom limb pain. In: Neural engineering (NER), 2015 7th international IEEE/EMBS conference on, IEEE, pp 787–790Google Scholar
  66. Webel S, Olbrich M, Franke T, Keil J (2013) Immersive experience of current and ancient reconstructed cultural attractions. In: Digital heritage international congress (DigitalHeritage), 2013, IEEE, vol 1, pp 395–398Google Scholar
  67. Webster D, Celik O (2014) Systematic review of Kinect applications in elderly care and stroke rehabilitation. J Neuroeng Rehabil 11(1):108CrossRefGoogle Scholar
  68. Weichert F, Bachmann D, Rudak B, Fisseler D (2013) Analysis of the accuracy and robustness of the Leap motion controller. Sensors 13(5):6380–6393CrossRefGoogle Scholar
  69. Weisz J, Shababo B, Dong L, Allen PK (2013) Grasping with your face. In: Desai JP, Dudek G, Khatib O, Kumar V (eds) Experimental robotics, Springer, Heidelberg, pp 435–448Google Scholar
  70. Yu N, Xu C, Wang K, Yang Z, Liu J (2015) Gesture-based telemanipulation of a humanoid robot for home service tasks. In: Cyber technology in automation, control, and intelligent systems (CYBER), 2015 IEEE international conference on, IEEE, pp 1923–1927Google Scholar
  71. Zecca M, Micera S, Carrozza MC, Dario P (2002) Control of multifunctional prosthetic hands by processing the electromyographic signal. Crit Rev Biomed Eng 30:4–6CrossRefGoogle Scholar
  72. Zyda M (2005) From visual simulation to virtual reality to games. Computer 38(9):25–32CrossRefGoogle Scholar

Source: Impact of commercial sensors in human computer interaction: a review | SpringerLink

, , , , , ,

  1. Leave a comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: