Spotted: Designing engagement-aware agents for multiparty conversations

Using machine learning to sense when users are engaged and act on it. 

// published on Conference on Human Factors in Computing Systems-Latest Proceeding Volume // visit site

Designing engagement-aware agents for multiparty conversations
Qianli Xu, Liyuan Li, Gang Wang

Recognizing users' engagement state and intentions is a pressing task for computational agents to facilitate fluid conversations in situated interactions. We investigate how to quantitatively evaluate high-level user engagement and intentions based on low-level visual cues, and how to design engagement-aware behaviors for the conversational agents to behave in a sociable manner. Drawing on machine learning techniques, we propose two computational models to quantify users' attention saliency and engagement intentions. Their performances are validated by a close match between the predicted values and the ground truth annotation data. Next, we design a novel engagement-aware behavior model for the agent to adjust its direction of attention and manage the conversational floor based on the estimated users' engagement.