Ying Xu, Ph.D.

Assistant Professor
Graduate School of Education
Harvard University

Elinor asks questions

Children develop their skills and interests in science, technology, engineering, and mathematics (STEM) through informal learning experiences, which, ideally, should be rich with conversation and social interaction (e.g., asking and answering questions). While STEM-oriented television programming has become an important learning resource for many children, its benefits could be amplified by fostering children’s dialogic engagement.

I have been partnering with PBS KIDS to carry out pioneering research that aims to integrate conversational agents into children’s television shows so that children can have contingent interaction with media characters, with the goal of supporting active engagement and learning. I have developed such “conversational videos” for a popular PBS KIDS animated science show, Elinor Wonders Why, and conducted studies involving children aged four to six years.

Highlights

  1. My studies consistently found that the conversational videos elicited children’s active verbal engagement during video watching (paper in CHI2020; paper in CHI2022).

  2. These conversational videos improved children’s understanding of STEM concepts, enhanced children’s engagement, and stimulated more science discussion among parents and children compared to videos currently aired on PBS KIDS without conversational characters (paper in Journal of Applied Developmental Psychology ).

  3. The conversational videos helped children form a positive relationship with the media characters, which translated into improved learning (paper Journal of Applied Developmental Psychology).

Future Work

We have been awarded a $3 million NSF grant to extend this project along three directions:

  1. We will improve the design of the conversational videos/characters to increase their potential communicative and educational benefits.

    • The media characters will be designed to process multilingual speech input so that children may respond not only in English but also in their home language.
    • We will enhance the media characters’ capability to engage in in-depth back and forth conversation so that children can have multiple exchanges on a topic.
    • We will provide multimedia scaffolding during the conversation to facilitate children’s verbal interaction with the characters.

  2. We will conduct large scale longitudinal studies to examine how children and their families interact with and learn from conversational videos naturalistically in their homes, focusing on Latino children in under-resourced households.

  3. At the end of this project, the conversational videos produced from this project will be distributed via PBS KIDS platforms to millions of children across the country.

Publications

  • Xu, Y., Vigil, V., Bustamante, & Warschauer, M. (conditionally accepted). Contingent interaction with a television character promotes children’s science learning and engagement  . In Journal of Applied Developmental Psychology.

  • Xu, Y., Vigil, V., Bustamante, & Warschauer, M. (2022). “Elinor’s talking to me!”: Integrating conversational AI into children’s science narrative programming. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. [DOI]

  • Xu, Y., & Warschauer, M. (2020). Using conversational agents to foster young children’s science learning from screen media. In Proceedings of the 19th ACM International Conference on Interaction Design and Children (IDC ’20). [DOI]

  • Xu, Y., & Warschauer, M. (2020). Wonder with Elinor: Designing a socially contingent video viewing experience. In Proceedings of the 19th ACM International Conference on Interaction Design and Children (IDC ’20). Association for Computing Machinery, New York, NY, USA. [DOI] (Research and Design Competition Honorable Mention)

  • Xu, Y., & Warschauer, M. (2020). “Elinor is talking to me on the screen!” Integrating conversational agents into children’s television programming. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. April 25-30, 2020, Honolulu, HI. ACM. [DOI]

Updated: