Providing Usage Recommendations and
Predicting Student Achievement
for an EdTech Product

(project in progress)
MY ROLE: Learning Analyst
MY TEAM: Learning Team
(Lead Learning Analyst, Director of Impact Research)

BACKGROUND

EdTech products have gained immense popularity in schools over the past decade. Research till date has not been able to provide specific recommendations for usage for this particular company’s product.

As part of initial discovery, the team learned that data from prior years was insufficient to provide concrete usage recommendations.

As a solution, the learning team provided a novel statistical analysis technique to understand underlying patterns in the data to provide concrete usage recommendations.

PROJECT OVERVIEW

RESEARCH OBJECTIVE

The two goals below aimed to improve clarity on product and marketing strategies and enable company teams and the product users to focus on specific aspects of the product to see growth.

Provide Product Usage Recommendations

How much should students and teachers use the EdTech Product to see the best student English Language Arts (ELA) outcomes?

Predict Factors that Influence Student Achievement

Does usage (behaviors) or performance (scores) on the EdTech Product relate to external student ELA outcomes?

MY ROLE

Taking the lead on the project, my role involved the following components:

  • Collaborate with team members to define research objectives

  • Using R packages and pipelines, obtain data by using SQL from a server

  • Conducting data cleaning and data merging processes

  • Completing data analysis (including linear regressions, mixed effect regression models, ANOVA, t-test, Exploratory and Confirmatory Factor Analyses)

  • Data visualizations (using R’s ‘ggplot’ package)

  • Communicating results and recommendations clearly through external reports and sharing findings in 2 presentations (one that was internal to the Learning Team and a second that was for Product, Tech, and Learning teams)

This project was completed on a part-time schedule between June to September 2023.

Presentations and Reports are available upon request.

RESEARCH METHOD

This project involved analyzing data that was obtained from the EdTech platform usage and student achievement tests. The collected data was queried using SQL, cleaned and analyzed using R and R Studio and several relevant statistical packages like tidyverse, lme4, nfactors.

KEY LEARNINGS

This study provided concrete student and teacher product usage recommendations.

It also highlighted various successful product features in predicting student achievement and provided a low-stakes measure of student achievement on the platform for classroom users.

RESEARCH IMPACT

This study provided the Product Team with information to further improve aspects of the product depending on the type of user (students or teachers).

PRODUCT TEAM

The Marketing Team is now able to provide concrete usage recommendations for the user and tailor their message for specific users (students or teachers).

MARKETING TEAM

For students and teachers in the classroom, the findings from this study will help them understand how much to use the product, focus teaching and learning efforts, and provide low-stakes measures of achievement.

CLASSROOM USERS