Elinor asks questions

Children develop their skills and interests in science, technology, engineering, and mathematics (STEM) through informal learning experiences, which, ideally, should be rich with conversation and social interaction (e.g., asking and answering questions). While STEM-oriented television programming has become an important learning resource for many children, its benefits could be amplified by fostering children’s dialogic engagement.

I have been partnering with PBS KIDS to carry out pioneering research that aims to integrate conversational agents into children’s television shows so that children can have contingent interaction with media characters, with the goal of supporting active engagement and learning. I have developed such “conversational videos” for a popular PBS KIDS animated science show, Elinor Wonders Why, and conducted studies involving children aged four to six years.

Highlights

  1. My studies consistently found that the conversational videos elicited children’s active verbal engagement during video watching [paper in CHI2020; paper under review].

  2. These conversational videos improved children’s understanding of STEM concepts, enhanced children’s engagement, and stimulated more science discussion among parents and children compared to videos currently aired on PBS KIDS without conversational characters [paper under review].

  3. The conversational videos helped children form a positive relationship with the media characters, which translated into improved learning [paper under review].

Future Work

We have been awarded a $3 million NSF grant to extend this project along three directions:

  1. We will improve the design of the conversational videos/characters to increase their potential communicative and educational benefits.

    • The media characters will be designed to process multilingual speech input so that children may respond not only in English but also in their home language.
    • We will enhance the media characters’ capability to engage in in-depth back and forth conversation so that children can have multiple exchanges on a topic.
    • We will provide multimedia scaffolding during the conversation to facilitate children’s verbal interaction with the characters.
  2. We will conduct large scale longitudinal studies to examine how children and their families interact with and learn from conversational videos naturalistically in their homes, focusing on Latino children in under-resourced households.

  3. At the end of this project, the conversational videos produced from this project will be distributed via PBS KIDS platforms to millions of children across the country.

Publications

  • Xu, Y., & Warschauer, M. (2020). Using conversational agents to foster young children’s science learning from screen media. In Proceedings of the 19th ACM International Conference on Interaction Design and Children (IDC ’20). Association for Computing Machinery, New York, NY, USA. [DOI]

  • Xu, Y., & Warschauer, M. (2020). Wonder with Elinor: Designing a socially contingent video viewing experience. In Proceedings of the 19th ACM International Conference on Interaction Design and Children (IDC ’20). Association for Computing Machinery, New York, NY, USA. [DOI] (Research and Design Competition Honorable Mention)

  • Xu, Y., & Warschauer, M. (2020). “Elinor is talking to me on the screen!” Integrating conversational agents into children’s television programming. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. April 25-30, 2020, Honolulu, HI. ACM. [DOI]

  • Xu, Y., Vigil, V., Bustamante, & Warschauer, M. (under review). “Elinor’s talking to me!”: Integrating conversational technologies into children’s science narrative programming.

  • Xu, Y., Vigil, V., Bustamante, A., & Warschauer, M. (under review). Contingent interaction with a television character enabled by artificial intelligence promotes children’s science learning and engagement.