Saajha is on a mission to eradicate foundational literacy and math gaps in K-5 students studying in India’s low-income public schools
Foundational research
Assisted the Product Manager in developing a foundational research plan for understanding the impact of sharing digital rewards on the user retention.
Project in a nutshell
In a 3 month long research experiment that aimed to study the impact of digital rewards on the retention of parents in Saajha’s e-learning programs, I was responsible for the following:
Foundational research
Assisted the Product Manager in developing a foundational research plan for understanding the impact of sharing digital rewards on the user retention levels.
Design system
Creating design assets and custom illustrations, such as badges, stars, and progress journey posters for digital rewards to support the research experiments.
Making recommendations
Formulating recommendations for future experiments to improve the product based on data collected during the study.
The Product: WhatsApp chatbot
Saajha delivers Math and Hindi learning activities to children aged 5-11 via a WhatsApp chatbot. After a phone assessment of the child's skill levels by a Relationship Manager, parents receive targeted activities through the Saajha Bot. The assessment data is managed on the "Saajha for Saajhedar" app, and activities are automated via Turn.io.
Challenge: Low user retention rates
Saajha administers assessments of the child’s learning levels to share targeted learning activities over 6 phone calls. However, on analyzing the calling data it was discovered that there was a 96% drop-out rate in the parents who joined the program v/s the parents who finished the third assessment call.
Research experiment
Can sharing digital rewards improve retention rates?
The goal of the research experiment was to investigate if there would be an uptick in the percentage of parents who complete the third call for assessment after receiving rewards as compared to parents who do not receive any rewards.
Description
1 hour-long phone interviews with 10 participants.
Research study design
Participant criteria
5 participants who had opted out of the program after the first or second call.
Research questions
What motivated parents to stay long in the program?
What would have helped parents stay longer in the program?
Recruitment
Parents were recruited from the existing database that had information about their status in the program.
User stories
Experiment structure
A database of 13,500 parents was divided into three groups of 4,500 each. The first two groups received the treatment, while the third group served as the control group. An A/B test was conducted to compare the results of the first treatment group with the control group, and the second treatment group with the control group. The goal was to calculate the percentage change in the number of parents who completed the third assessment call, comparing those who received rewards versus those who did not. Calling data was collected to provide the necessary evidence.
Implementation plan
Experiment results
The percentage of parents who completed the learning assessments till the third round were compared. There was a 2 % increase in retention in Treatment 1 as in comparison to the control group. No change was found in Treatment 2 that had more reward messages.
