Skip to main content
SpringerLink
Log in

IntraVox: A Personalized Human Voice to Support Users with Complex Needs in Smart Homes

  • Conference paper
  • First Online:
Human-Computer Interaction – INTERACT 2021 (INTERACT 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12932))

Included in the following conference series:

Abstract

We present IntraVox, a novel voice-based interaction system that supports users with complex needs by introducing a highly personalized, human voice command between smart home devices and sensors. Voice-enabled personal assistants (e.g., Amazon Alexa, Google Home) can be used to control smart home devices and increase the quality of life. However, people suffering from conditions such as dementia or learning disabilities and autism (LDA) can encounter verbal and memory problems in communicating with the assistants. IntraVox supports these users by verbally sending smart home control commands, based on the sensor data collected, to a voice-enabled personal assistant using a human voice. Over 12 months, we conducted 7 workshops and 9 interviews with 79 participants from multiple stakeholder categories - people with LDA, older and frail people, social and health care organisations. Results show that our solution has the potential to support people with complex needs and increase independence. Moreover, IntraVox could have the added value of reinforcing the learning of specific commands and making visible the inner workings of a home automation system, by giving users a clear cause-and-effect explanation of why certain events occur.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adib, F., Mao, H., Kabelac, Z., Katabi, D., Miller, R.: Smart homes that monitor breathing and heart rate. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (2015)

    Google Scholar 

  2. Amazon Skills Kit. https://amzn.to/363uZUU, Accessed 23 Jan 2021

  3. Arik, S.- Ö., et al.: Deep voice: real-time neural text-to-speech. In: Proceedings of the 34th International Conference on Machine Learning, (ICML’17), vol. 70, pp. 195–204. JMLR.org (2017)

    Google Scholar 

  4. Atefi, S., Truelove, A., Rheinschmitt, M., Almeida, E., Ahmed, I., Alipour, A.: Examining user reviews of conversational systems: a case study of Alexa skills. arXiv preprint arXiv:2003.00919 (2020)

  5. Audacity. https://www.audacityteam.org/

  6. Bakhai, A., Constantin, A., Alexandru, C.A.: Motivate me! an alexa skill to support higher education students with autism. In: International Conferences Interfaces and Human Computer Interaction and Game and Entertainment Technologies (2020)

    Google Scholar 

  7. Barnes, J., Richie, E., Lin, Q., Jeon, M., Park, C.-H.: Emotive voice acceptance in human-robot interaction. In: Proceedings of the 24th International Conference on Auditory Display (2018)

    Google Scholar 

  8. Bespoken. https://bit.ly/2LO9rVp, Accessed 23 Jan 2021

  9. Brewer, R.-N., Morris, M.-R., Lindley, S.-E.: How to remember what to remember: exploring possibilities for digital reminder systems. In: Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies, vol. 1, no. 3, pp. 1–20 (2017)

    Google Scholar 

  10. Burgoon, J.-K., Bonito, J.-A., Bengtsson, B., Cederberg, C., Lundeberg, M., Allspach, L.: Interactivity in human-computer interaction: a study of credibility, under-standing, and influence. Comput. Human Behav. 16(6), 553–574 (2000)

    Article  Google Scholar 

  11. Cho, M., Lee, S.-S., Lee, K.-P.: Once a kind friend is now a thing: understanding how conversational agents at home are forgotten. In: Proceedings of the 2019 on Designing Interactive Systems Conference (2019)

    Google Scholar 

  12. Colley, M.: Use of frequency volume charts and voiding diaries. Nurs. Times 111(5)(2015)

    Google Scholar 

  13. Corbett, E., Weber, A.: What can I say? addressing user experience challenges of a mobile voice user interface for accessibility. In: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services, pp. 72–82 (2016)

    Google Scholar 

  14. Cruz-Sandoval, D., Favela, J.: Co-designing ambient-assisted interventions using digital interlocutors for people with dementia. In: Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers, pp. 813–821 (2017)

    Google Scholar 

  15. Cruz-Sandoval, D., Penaloza, C.-I., Favela, J., Castro-Coronel, A.-P.: Towards social robots that support exercise therapies for persons with dementia. In: Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers, pp. 1729–1734 (2018)

    Google Scholar 

  16. Daher, M., El Najjar, M., Diab, A., Khalil, M., Charpillet, F.: Multi-sensory assistive living system for elderly in-home staying, pp. 168–171. IEEE (2018)

    Google Scholar 

  17. Dahmen, D., Minor, B., Cook, D., Vo, T., Schmitter-Edgecombe, M.: Smart home-driven digital memory notebook support of activity self-management for older adults. Gerontechnology 17, 113–125 (2018)

    Article  Google Scholar 

  18. Geeng, C., Franziska Roesner, F.: Who’s in control? interactions in multi-user smart homes. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–13 (2019)

    Google Scholar 

  19. Home Assistant. https://www.home-assistant.io/, Accessed 23 Jan 2021

  20. Hsieh, H.-F., Shannon, S.: Three approaches to qualitative content analysis. Qual. Health Res. 15(9), 1277–1288 (2005)

    Article  Google Scholar 

  21. Igoe, T.: Making Things Talk: Practical Methods for Connecting Physical Objects, 3rd edn. O Reilly, Sebastopol (2017)

    Google Scholar 

  22. Juniper Research. https://bit.ly/3iGO8Ri, Accessed 23 Jan 2021

  23. Kanda, T., Ishiguro, H., Ono, T., Imai, M., Mase, K.: Multi-robot cooperation for human-robot communication. In: Proceedings 11th IEEE International Workshop on Robot and Human Interactive Communication, pp. 271–276. IEEE (2002)

    Google Scholar 

  24. Kim, D., Park, K., Park, Y., Ju, J., Ahn, J.-H.: Alexa: tell me more: the effect of advertisements on memory accuracy from smart speakers. In: PACIS (2018)

    Google Scholar 

  25. Kiseleva, J., et al.: Understanding user satisfaction with intelligent assistants. In: Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval, pp. 121–130 (2016)

    Google Scholar 

  26. Kowalski, J., et al.: Older adults and voice interaction: a pilot study with google home. In: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (2019)

    Google Scholar 

  27. Large, D.R., Burnett, G.E.: The effect of different navigation voices on trust and attention while using in-vehicle navigation systems. J. Saf. Res. 49, 69-e1 (2014)

    Article  Google Scholar 

  28. Lee, K.-M., Nass, C.: Social-psychological origins of feelings of presence: creating social presence with machine-generated voices. Media Psychol. 7(1) (2005)

    Google Scholar 

  29. Li, X., et al.: IndoTrack: device-free indoor human tracking with commodity Wi-Fi. In: Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 1, no. 3, pp. 1–22 (2017)

    Google Scholar 

  30. Lombard, M., Ditton, T.: At the heart of it all: the concept of presence. J. Comput.-Mediated Commun. 3(2), JCMC321 (1997)

    Google Scholar 

  31. Luger, E., Sellen, A.: “Like Having a Really Bad PA” The Gulf between User Expectation and Experience of Conversational Agents. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. (2016)

    Google Scholar 

  32. McGee-Lennon, M.R., Gray, P.D.: including stakeholders in the design of home care systems: identification and categorisation of complex user requirements. In: INCLUDE Conference (2007)

    Google Scholar 

  33. Mennicken, S., Zihler, O., Juldaschewa, F., Molnar, V., Aggeler, D., Huang, E.M.:" It’s like living with a friendly stranger" perceptions of personality traits in a smart home. In: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 120–131 (2016)

    Google Scholar 

  34. Milani, A., Lorusso, M.L., Molteni, M.: The effects of audiobooks on the psychosocial adjustment of pre-adolescents and adolescents with dyslexia. Dyslexia 16(1), 87–97 (2010)

    Article  Google Scholar 

  35. Nass, C.I., Brave, S.: Wired for Speech: How Voice Activates and Advances the Human-Computer Relationship. MIT press, Cambridge (2005)

    Google Scholar 

  36. Nass, C.I., Foehr, U., Brave, S., Somoza, M.: The effects of emotion of voice in synthesized and recorded speech. In: Proceedings of the AAAI Symposium Emotional and Intelligent II: The Tangled Knot of Social Cognition. North Falmouth (2001)

    Google Scholar 

  37. Nass, C.I., Steuer, J., Tauber, E.R.: Computers are social actors. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 72–78 (1994)

    Google Scholar 

  38. Oppenheimer, A.: From experience: products talking to people-conversation closes the gap between products and consumers. J. Prod. Innov. Manag. 22(1), 82–91 (2005)

    Article  Google Scholar 

  39. Perelmutter, B., McGregor, K.K., Gordon, K.R.: Assistive technology interventions for adolescents and adults with learning disabilities: an evidence-based systematic review and meta-analysis. Comput. Educ. 114, 139–163 (2017)

    Article  Google Scholar 

  40. Plivo. https://www.plivo.com/, Accessed 23 Jan 2021

  41. Porcheron, M., Fischer, J.E., Reeves, S., Sharples, S.: Voice interfaces in everyday life. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1–12 (2018)

    Google Scholar 

  42. Pradhan, A., Mehta, K., Findlater, L.: “Accessibility Came by Accident" use of voice-controlled intelligent personal assistants by people with disabilities. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1–13 (2018)

    Google Scholar 

  43. Purington, A., Taft, J.G., Sannon, S., Bazarova, N.N., Taylor, S.H.: “Alexa is my new BFF” social roles, user satisfaction, and personification of the amazon echo. In: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems, pp. 2853–2859 (2017)

    Google Scholar 

  44. Rafferty, J., Nugent, C.-D., Liu, J., Chen, L.: From activity recognition to intention recognition for assisted living within smart homes. IEEE Trans. Human-Mach. Syst. 47(3) (2017)

    Google Scholar 

  45. RPi Text to Speech. https://bit.ly/3poEAxf, Accessed 23 Jan 2021

  46. Sayago, S., Neves, B.B., Cowan, B.R.: Voice assistants and older people: some open issues. In: Proceedings of the 1st International Conference on Conversational User Interfaces, pp. 1–3 (2019)

    Google Scholar 

  47. Shneiderman, B., Maes, P.: Direct manipulation vs. interface agents. Interactions 4(6), 42–61 (1997)

    Article  Google Scholar 

  48. Simpson, J., Gaiser, F., Macík, M., Breßgott, T.: Daisy: a friendly conversational agent for older adults. In: Proceedings of the 2nd Conference on Conversational User Interfaces, pp. 1–3 (2020)

    Google Scholar 

  49. Søndergaard, M.L.K., Hansen, L.K. : Intimate futures: staying with the trouble of digital personal assistants through design fiction (2018)

    Google Scholar 

  50. Suzuki, Y., Kato, K., Naomi Furui, N., Sakamoto, D., Sugiura, Y.: Cushion interface for smart home control (2020)

    Google Scholar 

  51. Tan, X.Z., Reig, S., Carter, E.J., Steinfeld, A.: From one to another: how robot-robot interaction affects users’ perceptions following a transition between robots, pp. 114–122. IEEE (2019)

    Google Scholar 

  52. Twilio. https://www.twilio.com/, Accessed 23 Jan 2021

  53. Vacher, M., et al.: Evaluation of a context-aware voice interface for ambient assisted living: qualitative user study vs. quantitative system evaluation. ACM Trans. Accessible Comput. (TACCESS) 7(2), 1–36 (2015)

    Article  Google Scholar 

  54. Vines, J., et al.: Making family care work: dependence, privacy and remote home monitoring telecare systems. In: Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing (2013)

    Google Scholar 

  55. Williams, T., Briggs, P., Scheutz, M.: Covert robot-robot communication: human perceptions and implications for human-robot interaction. J. Human-Rob. Interact. 4(2), 24–49 (2015)

    Google Scholar 

  56. Yonan, C.A., Sommers, M.S.: The effects of talker familiarity on spoken word identification in younger and older listeners. Psychol. Aging 15(1), 88 (2000)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Design, Northumbria University, Newcastle upon Tyne, UK

    Ana-Maria Salai & Lars Erik Holmquist

  2. Nursing, Midwifery and Health, Northumbria University, Newcastle upon Tyne, UK

    Glenda Cook

Authors
  1. Ana-Maria Salai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Glenda Cook
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lars Erik Holmquist
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ana-Maria Salai .

Editor information

Editors and Affiliations

  1. Department of Electrical and Information Engineering, Polytechnic University of Bari, Bari, Italy

    Carmelo Ardito

  2. Computer Science Department, University of Bari Aldo Moro, Bari, Italy

    Rosa Lanzilotti

  3. Computer Science Department, University of Pisa, Pisa, Italy

    Alessio Malizia

  4. Department of Computer Science, University of York, York, UK

    Helen Petrie

  5. Computer Science Department, University of Bari Aldo Moro, Bari, Italy

    Antonio Piccinno

  6. Computer Science Department, University of Bari Aldo Moro, Bari, Italy

    Giuseppe Desolda

  7. Microsoft Research, Redmond, WA, USA

    Kori Inkpen

Rights and permissions

Reprints and permissions

Copyright information

© 2021 IFIP International Federation for Information Processing

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Salai, AM., Cook, G., Holmquist, L.E. (2021). IntraVox: A Personalized Human Voice to Support Users with Complex Needs in Smart Homes. In: Ardito, C., et al. Human-Computer Interaction – INTERACT 2021. INTERACT 2021. Lecture Notes in Computer Science(), vol 12932. Springer, Cham. https://doi.org/10.1007/978-3-030-85623-6_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-85623-6_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-85622-9

  • Online ISBN: 978-3-030-85623-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation