An Enhanced Intelligent Agent with Image Description Generation

Ben Fielding, Philip Kinghorn, Kamlesh Mistry, Li Zhang

Research output: Chapter in Book/Report/Conference proceedingChapter

5 Citations (Scopus)


In this paper, we present an Embodied Conversational Agent (ECA) enriched with automatic image understanding, using vision data derived from state-of-the-art machine learning techniques for the advancement of autonomous interaction with the elderly or infirm. The agent is developed to conduct health and emotion well-being monitoring for the elderly. It is not only able to conduct question-answering via speech-based interaction, but also able to provide analysis of the user’s surroundings, company, emotional states, hazards and fall actions via visual data using deep learning techniques. The agent is accessible from a web browser and can be communicated with via voice means, with a webcam required for the visual analysis functionality. The system has been evaluated with diverse real-life images to prove its efficiency.
Original languageEnglish
Title of host publicationIntelligent Virtual Agents
Place of PublicationLondon
ISBN (Print)978-3-319-47664-3
Publication statusPublished - 23 Nov 2016

Publication series

NameLecture Notes in Computer Science
ISSN (Electronic)0302-9743


Dive into the research topics of 'An Enhanced Intelligent Agent with Image Description Generation'. Together they form a unique fingerprint.

Cite this