Role: Study co-lead
Funder: Corporation for Public Broadcasting, U.S. Department of Education (Ready to Learn)
Team: Cross-functional team of education researchers, statisticians, and media producers
Timeline: 2019–2023
Quick Impact Snapshot:
- Two 9-week RCTs with 263 first-grade children
- Demonstrated g = .25 effect on real-world informational text use
- Results presented to Congress and used to support CPB funding
- Findings published in AERJ and disseminated to PBS, GBH, CPB
- Adapted mid-study from in-person to remote implementation due to COVID-19
The Need
The Corporation for Public Broadcasting (CPB) commissioned a rigorous evaluation of Molly of Denali, a PBS KIDS multimedia property designed to support children's ability to use informational text. This work was part of the U.S. Department of Education’s Ready to Learn (RTL) initiative.
Why This Study Was Essential
- Federal Requirement: Congress mandates that RTL projects include at least one rigorous study meeting What Works Clearinghouse (WWC) standards. These studies must also contribute to broader insights on effective educational media and disseminate findings to audiences such as researchers, media producers, and the public.
2. GPRA Compliance: CPB and PBS are required to conduct rigorous evaluation research under the Government Performance and Results Act (GPRA).
Planning the Research
Study Design
I co-led this pre-registered efficacy trial in partnership with EDC. We designed the study to meet WWC standards without reservations, using a randomized controlled design to evaluate the impact of Molly of Denali on children's ability to use informational text. We sought feedback from key constituents—Alaska Native community members, CPB, PBS, and the production team at GBH—to inform the study design, and used a pilot phase to refine our methods.
Cultural Responsiveness
To ensure cultural sensitivity and relevance, we collaborated with a Native Alaskan advisory board. I also attended the Alaska Federation of Natives Convention to speak directly with attendees about the show’s meaning and role in their communities.
Pre-registration & Measure Development
To support scientific transparency, I pre-registered the study in the Registry of Efficacy and Effectiveness Studies (REES).
Because there were no existing measures of children's ability to use informational text, our team, led by my collaborator at EDC, developed an assessment to measure these skills.
I led the development of parent pre- and post-test surveys capturing home media use, implementation, and perceptions of Molly of Denali.
Conducting the Research
Our sample included 263 first-grade children. We initially designed the study for in-person implementation across early childhood education settings. Midway through data collection, the COVID-19 pandemic required a rapid pivot to virtual data collection. We successfully adapted our protocols to maintain rigor and consistency across remote and in-person contexts, ensuring fidelity to the study design.
Both studies were nine-week randomized controlled trials. All participants received data-enabled tablets, and we randomly assigned participants to one of two groups:
- Treatment group: Tablets were pre-loaded with curated Molly of Denali videos, games, and related resources.
- Control group: Tablets were configured to block access to all PBS KIDS content as well as other apps focused on informational text.
We administered pre- and post-assessments aligned with the study’s research questions, measured informational text comprehension and use, and analyzed implementation fidelity through back-end data and parent surveys. For more on the study design, see our AERJ article.
Key Findings
Access to Molly of Denali resources improved children’s ability to use informational text to solve real-world problems. Across both studies, children in the treatment group outperformed those in the control group on assessments of informational text use, with statistically significant effect sizes of g = .25 (Study 1) and g = .27 (Replication
Study).
Younger children and those who spent more time with the Molly of Denali content benefited the most. On average, every additional hour spent engaging with the materials was associated with a measurable increase in post-test scores.
These promising results—achieved through a light-touch, nine-week intervention involving about an hour of weekly media use demonstrate that well-designed educational media can support early learning at home, even in times of disrupted in-school education.
Dissemination & Impact
Public-Facing Outputs
We achieved the CPB's federal mandate to disseminate findings with wide audiences of researchers, practitioners, and caregivers. Our dissemination outlets included:
- Peer-reviewed publication in American Educational Research Journal
- Blog post summarizing study insights
- Conference presentations (e.g., AERA, NAEYC, SRCD)
- Short-form social media video sharing findings
- Short report sharing findings with PBS member stations and the pubic
Policy Relevance
Findings were cited in testimony before the U.S. Congress as evidence of Ready to Learn’s educational impact.
My Role in the Project
- Co-led study design and execution
- Pre-registered the study to ensure rigor and transparency
- Designed and led parent survey development
- Partnered with CPB, PBS KIDS, EDC, and local stakeholders
- Ensured culturally responsive methodologies
- Disseminated findings to academic, public, and policy audiences