Senior Researcher at Inria Paris, chaire PR[AI]RIE Institute of Interdisciplinary research in AI, Dean's Professor in Language Technologies at Carnegie Mellon University.
As part of our work on socially capable conversational agents, we launched Son of SARA, an embodied agent designed to foster natural, effective human–agent interaction by combining rapport with task effectiveness. The project aims to equip agents with essential verbal and non-verbal interactional skills, building on our lab’s tradition of socially aware ECAs and research on human communication and collaboration. By leveraging Large Language Models and cutting-edge deep learning, we seek to enhance their naturalness, adaptability, and overall interactive capacities.
How do children connect with each other while learning together? This project examines collaborative learning in school settings by combining brain imaging, behavioral observation, and fine-grained interaction analysis. Working directly with schools, we study pairs of children as they engage in structured collaborative problem-solving tasks. Using fNIRS hyperscanning, we simultaneously measure neural activity from both partners to examine how brain dynamics align during moments of coordination, explanation, and joint reasoning.
To better understand the natural human communication and create conversational agents that adapt to us rather than forcing us to adapt to them, in this project, we work on the phenomenon of conversational grounding. Conversational grounding is the essential, yet often invisible, “glue” of human interaction. This project bridges the gap between the theoretical foundations of cognitive science and the probabilistic nature of deep learning to move beyond simple intent classification.
Interpersonality is a collaborative research project between the Articulab in Inria Paris and the Korea Electronics Technology Institute (KETI). It investigates how personality is constructed, expressed, perceived, and adapted in human interactions, and how these processes can be computationally modeled to design socially intelligent artificial agents. To this end, we analyze small-group collaborative interactions between humans and examine how rapport is built and maintained in these settings.
In this project, our objective is to design socially-aware conversational models that can build and maintain rapport with humans over time, as a way of improving human-agent collaboration. By integrating theories of interpersonal communication with state-of-the-art conversational models like Large Language Models, we work on exploring how virtual agents powered by these models can foster meaningful social connection while improving task outcomes, particularly in educational contexts.
(Thanks to the NSF for generous funding)
The RAPT project is currently working along two fronts: developing a theory of how rapport is built, maintained, and destroyed among teens, and developing a computational architecture and system implementation that allows a virtual peer to build, maintain (and if necessary respond to destroying) rapport in the context of math tutoring.
SARA is a Socially-Aware Robot Assistant that interacts with people in a whole new way, personalizing the interaction and improving task performance by relying on information about the relationship between the human user and virtual assistant.
(Thanks to the Heinz Family Foundation for generous funding)
Curiosity is essential in science learning as it motivates students to explore and produce knowledge by assuming the role of a scientist. Unfortunately, curiosity is often de-emphasized in traditional teacher-led classes where students are held to the role of knowledge consumer rather than independent and self-motivated groups of knowledge seekers and creators. In this project, we aim to foster curiosity, exploration, and self-efficacy as a way to improve science learning for elementary school students by incorporating a role-playing game, virtual peer, multimodal sensing and mixed-reality technology.
(Thanks to the Yahoo InMind project for generous funding)
The Articulab is now a part of the agreement between Yahoo! and CMU, called InMind project, where we’re trying to innovate the next next next generation of personal assistant agent.
(Thanks to the Heinz Family Foundation for generous funding)
In this project, we aim to address the systematically-reduced standardized test scores of African American students compared to their Euro-American peers by using virtual peer technology to understand the role of dialect, and more broadly, cultural congruence, on students' performance, and to help students achieve in the classroom.