Ying Xu, Ph.D.

Assistant Professor
Graduate School of Education
Harvard University

Two children reading with conversational agents

Storybook reading by family members provides a stimulating environment for children’s development of their language and communication skills. The benefits of storybook reading are amplified by engaging children in dialogic interactions that revolve around story narratives. By increasing opportunities for dialogic reading, we can ensure that more children develop the language and literacy skills necessary for school success.

To enrich children’s home literacy environments, I carried out a series of studies to design and investigate the impact of conversational agents in the form of smart speakers to support children’s language learning by engaging them in storybook reading.

Highlights

  1. Having dialogue with an agent reading partner promoted children’s story comprehension and engagement equally as well as a human reading partner did(paper in Child Development).

  2. Children’s communication patterns differed when they interacted with a human vs. agent:

    • The agent partner promoted the audible intelligibility of children’s responses
    • Reading with a human partner elicited longer, more lexically diverse, and more relevant responses from children (paper in Computers & Education).

  3. Children had varying patterns of communicating with the agents based on their age and language proficiency (paper in IDC2020).

  4. This series of studies has produced practical design heuristics that inform the development of conversational technologies for young users who are still developing their language skills and cognitive functioning (paper in CHI2021; paper in IDC2020).

  5. When designed properly, conversational AI can promote parent-children interaction during shared story reading, allowing me to examine the benefits of “trialogue” among an agent, parent, and the child for language development (paper in IDC 2023).

  6. Teachers and parents are generally enthusiastic about using AI to support joint reading activities at home and at school (paper in IDC 2023).

Upcoming Studies

Sesame workshop ebook and outcomes

  1. Carrying out co-design studies with Latine communities in California and Michigan to develop a series of 24 science-focused storybooks for families with children aged 4 to 6.

  2. Testing the effects of access to these culturally responsive storybooks on children’s English and home language development, science learning, and attitudes towards heritage languages and cultures.

  3. Exploring different approaches such as theory-driven prompt engineering, retrieval-augmented generation (RAG), and fine-tuning, to safeguard the AI-generated dialogue and make it more educationally beneficial for children.

Selected Publications

  • Zhang, C., Liu, X., Ziska, K., Jeon, S., Yu, C.L., & Xu, Y. (2024). Mathemyths: Leveraging large language models to teach mathematical language through child-AI co-creative storytelling. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. [DOI]

  • Xu, Y., He, K., Vigil, V., Ojeda-Ramirez, S., Liu, X., Levine, J., Cervera, K., & Warschauer, M. (2023). “Rosita Reads With My Family”: Developing A Bilingual Conversational Agent to Support Parent-Child Shared Reading. In Proceedings of the ACM International Conference on Interaction Design and Children (IDC ’23). Association for Computing Machinery, New York, NY, USA. [DOI]

  • Xu, Y., Aubele, J., Vigil, V., Bustamante, A., Kim, Y-S., & Warschauer, M. (2022). Dialogue with a conversational agent promotes children’s story comprehension via enhanced engagement. Child Development. [DOI]

  • Xu, Y., Wang, D., Collins, P., Lee, H., & Warschauer, M. (2021). Same benefits, different communication patterns: Comparing children’s reading with a conversational agent vs. a human partner. Computers & Education. 161. 104059. [DOI]

  • Xu, Y., Branham, S., Collins, P., Deng, X., Warschauer, M. (2021). Are current voice interfaces designed to support children’s language development? In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. [DOI]

  • Xu, Y., & Warschauer, M. (2020). Exploring young children’s engagement in joint reading with a conversational agent. In Proceedings of the 19th ACM International Conference on Interaction Design and Children (IDC ’20). Association for Computing Machinery, New York, NY, USA. [DOI]

  • Xu, Y., & Warschauer, M. (2020). A content analysis of voice-based apps on the market for early literacy development. In Proceedings of the 19th ACM International Conference on Interaction Design and Children (IDC ’20). Association for Computing Machinery, New York, NY, USA. [DOI] (Best Paper Award Honorable Mention)

  • Xu, Y., & Warschauer, M. (2019). Young children’s reading and learning with conversational agents. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. May 4–9, 2019, Glasgow, Scotland, UK. ACM. [DOI]

Dataset

  • Xu, Y., & Warschauer, M. (2021), Conversational Agents for Dialogic Reading, Dryad, Dataset. [DOI]

Updated: