Login to view full entry:
Revisions to the Profile Reset Process (ATA Experiment 107)
ETH -19 -1460Last modified on January 27th, 2026 at 9:01 am
-
Abstract
PxD is partnering with Ethiopia’s Agricultural Transformation Agency (ATA) to help improve the effectiveness of their voice-based mobile advisory service, the 8028 hotline, by conducting continuous iterations and experiments, as well as by making suggestions for improvements to and customization of the service. The service has millions of registered farmers and represents the first in Africa to be maintained by a government entity at such a large scale.
Around 23% of users reset their profile at least once, which we hypothesized was often accidental. These resets caused significant friction, including wasted user time and effort in re-entering registration information more than once. The objective of this experiment was to reduce the number of unnecessary profile resets. This experiment tested whether providing users with clearer explanations of the reset process and reminders of the information they have already submitted can reduce unnecessary profile resets. The results suggest that, while the treatments encouraged more users to explore the reset menu, the treatments reduced the likelihood of triggering a full reset; this behavior is consistent with users making more informed or deliberate choices. -
Status
Completed
-
Start date
Q2 Apr 2019
-
End date
Q4 Nov 2019
-
Experiment Location
Ethiopia
-
Partner Organization
Ethiopian ATA
-
Agricultural season
_N/A
-
Experiment type
A/B test
-
Sample frame / target population
8028 hotline Amharic language users
-
Sample size
240,823
-
Outcome type
Platform engagement
-
Mode of data collection
PxD administrative data
-
Research question(s)
Does providing clearer explanations about user profile resets reduce accidental resets?
-
Research theme
Communication technology
-
Research Design
This was a multi-arm A/B test conducted among Amharic-speaking 8028 users. Farmers were randomly assigned to control and treatment groups. Treatment interventions varied the framing of the reset prompt and accompanying information:
- Control (n = 120,232): Standard message with no additional explanation. Profile reset prompted by pressing *.
- Treatment Arm 1, Reset (n = 30,196): “…To erase the information you have entered about yourself and re-enter it, press 9.”
- Treatment Arm 2, Reset + information (n = 30,247):
- “…To erase the information you have entered about yourself and re-enter it, press 9.”
- “You filled in some information about yourself like your region, zone, woreda, etc. This information is required so we can send you customized information. Once you fill in the correct information, you are never required to change this information. If you decide to erase all the information you have entered about yourself, you will be asked to enter this information again. You may decide to do this if you want to correct any information about yourself that is incorrect, or use the system in a different language, or if you are receiving push alerts for the wrong location. If you do not decide to change the information about yourself, you will still be able to use the 8028 system as much as you want, even if some of the information about yourself is incorrect.”
- Treatment Arm 3, Reset + information + existing user information (n = 29,944):
- “…To hear the information you have entered about yourself, or to erase the information you have entered about yourself and re-enter it, press 9.”
- [Same explanation as Treatment Arm 2.]
- “You have entered the following information about yourself: Language—XX; Region—YY; Zone—ZZ; Woreda—AA.”
- Treatment Arm 4, Partial reset + information + existing user information (n = 30,204):
- “…To hear the information you have entered about yourself, or to erase the information you have entered about yourself and re-enter it, press 9.”
- [Same explanation as Treatment Arm 2.]
- “You have entered the following information about yourself: Language—XX; Region—YY; Zone—ZZ; Woreda—AA.”
- “If you would like to enter or change your language, press 1; if you would like to enter or change your region, zone, or woreda, press 2; to return back to the main menu, press *; to hear these menu options again, press #.”
The analysis for this experiment used administrative data on calls over a 2.5-month period from April 9 to June 24, 2019, with the main outcomes of interest being access to the reset menu, and profile-reset actions.
-
Results
Contrary to expectations, mapping the menu to “9” and revising its description led more instead of fewer users to access the reset menu. All treatments increased the share of users accessing the reset menu in comparison to the control-group mean of 22.9%, with statistically significant effects ranging from 1.7 percentage points (pp) in Treatment group 1 and 2 pp in Treatment group 2 to 5.6 pp in Treatment groups 3 and 4 (all p < 0.001).
However, all treatments significantly reduced profile resets as expected. Compared to 17.8% of control group users who reset their profile, resets fell by 8.0, 10.9, 11.4, and 15.3 pp respectively in Treatment groups 1, 2, 3, and 4 (all p < 0.001). Treatment 4 was the most effective at reducing profile resets, likely because it also introduced a partial reset option.
Taken together, these results suggest that, while the treatments encouraged more users to explore the reset menu, the treatments reduced the likelihood of triggering a full reset; this behavior is consistent with users making more informed or deliberate choices.