Ying Xu, Ph.D.

Assistant Professor
Graduate School of Education
Harvard University

Two children reading with conversational agents

Storybook reading by family members provides a stimulating environment for children’s development of their language and communication skills. The benefits of storybook reading are amplified by engaging children in dialogic interactions that revolve around story narratives. By increasing opportunities for dialogic reading, we can ensure that more children develop the language and literacy skills necessary for school success.

To enrich children’s home literacy environments, I carried out a series of studies to design and investigate the impact of conversational agents in the form of smart speakers to support children’s language learning by engaging them in storybook reading.

Highlights

  1. My study suggested having dialogue with an agent reading partner promoted children’s story comprehension and engagement equally as well as a human reading partner did (paper in Child Development).

  2. Children’s communication patterns differed when they interacted with a human vs. agent:

    • The agent partner promoted the audible intelligibility of children’s responses
    • Reading with a human partner elicited longer, more lexically diverse, and more relevant responses from children (paper in Computers & Education).

  3. Children had varying patterns of communicating with the agents based on their age and language proficiency (paper in IDC2020).

  4. This series of studies has produced practical design heuristics that inform the development of conversational technologies for young users who are still developing their language skills and cognitive functioning (paper in CHI2021; paper in IDC2020).

Future Work

Sesame workshop ebook and outcomes

To extend this project, I have initiated a collaboration with Sesame Workshop to develop conversational e-books featuring a popular Sesame character as children’s dialogic partners. This work is innovative in three important ways:

  1. The conversational agents are designed to promote parent-children interaction during shared story reading, allowing me to examine the benefits of “trialogue” among an agent, parent, and the child for language development.

  2. The conversational e-books are intended to be culturally relevant to Latino communities. I will engage in a series of co-design processes with families in a low-income Spanish-speaking Latino community.

  3. The conversational agents will be capable of code-switching, comprehending and responding to multilingual children’s mixed language use. This will provide groundbreaking findings for conversational systems to move beyond monolingual interactions.

Publications

  • Xu, Y., Aubele, J., Vigil, V., Bustamante, A., Kim, Y-S., & Warschauer, M. (2021). Dialogue with a conversational agent promotes children’s story comprehension via enhanced engagement. Child Development. [DOI]

  • Xu, Y., Wang, D., Collins, P., Lee, H., & Warschauer, M. (2021). Same benefits, different communication patterns: Comparing children’s reading with a conversational agent vs. a human partner. Computers & Education. 161. 104059. [DOI]

  • Xu, Y., Branham, S., Collins, P., Deng, X., Warschauer, M. (2021). Are current voice interfaces designed to support children’s language development? In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. [DOI]

  • Xu, Y., & Warschauer, M. (2020). Exploring young children’s engagement in joint reading with a conversational agent. In Proceedings of the 19th ACM International Conference on Interaction Design and Children (IDC ’20). Association for Computing Machinery, New York, NY, USA. [DOI]

  • Xu, Y., & Warschauer, M. (2020). A content analysis of voice-based apps on the market for early literacy development. In Proceedings of the 19th ACM International Conference on Interaction Design and Children (IDC ’20). Association for Computing Machinery, New York, NY, USA. [DOI] (Best Paper Award Honorable Mention)

  • Xu, Y., & Warschauer, M. (2019). Young children’s reading and learning with conversational agents. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. May 4–9, 2019, Glasgow, Scotland, UK. ACM. [DOI]

Dataset

  • Xu, Y., & Warschauer, M. (2021), Conversational Agents for Dialogic Reading, Dryad, Dataset. [DOI]

Updated: