The PxD Experiment Registry documents our design experiments and impact evaluations—from simple A/B tests to large-scale randomized impact evaluations. It captures experiments we've conducted on our own services and with partners, measuring how specific service design changes affect outcomes and the overall impact of digital agriculture services. The Registry is searchable, filterable, and exportable, and is designed as an open resource to share learnings with others building digital agricultural services. For questions or publishing inquiries, contact info@precisiondev.org.

not-registered PxD staff internal login:

Follow-up Revisions to the Profile Reset Process (ATA Experiment 110)

ETH -19 -1463

Last modified on January 27th, 2026 at 9:00 am

    Basic Information

  • Abstract
    PxD is partnering with Ethiopia’s Agricultural Transformation Agency (ATA) to help improve the effectiveness of their voice-based mobile advisory service, the 8028 hotline, by conducting continuous iterations and experiments, as well as by making suggestions for improvements to and customization of the service. The service has millions of registered farmers and represents the first in Africa to be maintained by a government entity at such a large scale.

    This experiment follows up on the results from “Revisions to the Profile Reset Process (ATA Experiment 107)”. The previous experiment showed that treatment interventions that varied the framing of the reset prompt and accompanying information encouraged more users to explore the reset menu, but they reduced the likelihood of triggering a full reset; this behavior is consistent with users making more informed or deliberate choices.

    This experiment explored whether a more integrated menu design could further improve user experience by placing reset options under a clearly labeled menu item, and by offering separate actions for reviewing versus updating profile elements. The number of treatment-group farmers who chose to reset their profile (fully or partially) was significantly lower than control-group farmers. The results indicate that the treatment encouraged greater engagement with the reset menu and related information while reducing the incidence of reset requests.
  • Status
    Completed
  • Start date
    Q4 Oct 2019
  • Experiment Location
    Ethiopia
  • Partner Organization
    Ethiopian ATA
  • Agricultural season
    _N/A

    • Research Design

  • Experiment type
    A/B test
  • Sample frame / target population
    8028 hotline Amharic language users who had registered their profile
  • Sample size
    51,815
  • Outcome type
    Knowledge, Service engagement, Information access
  • Mode of data collection
    PxD administrative data
  • Research question(s)
    Does placing profile reset options under a clearly labeled menu item, with separate actions for reviewing or updating profile elements:
    1. change how often users access the reset menu, and
    2. decrease unintentional or unnecessary resets?
  • Research theme
    Communication technology
  • Research Design

    8028 hotline Amharic language users who had registered their profile were randomly assigned to two experimental groups:

    Control Group (n = 25,610): Users received the standard main menu, which included a direct option to reset their profile.

    Treatment Group (n = 26,205): Users received a modified main menu that included: “Press 9 to hear your current profile settings or reset your profile.” Users who selected this option were taken to a new reset sub-menu with the following options:

    • Press 1 to hear your current profile settings (then return to the main menu)
    • Press 2 to reset your language (then return to the main menu)
    • Press 3 to reset your location (then return to the main menu)
    • Press 4 to go back to the main menu

    The analysis for this experiment used administrative data on calls over a five-week period from February 1 to March 8, 2020, with the main outcomes of interest being access to the reset menu, and profile-reset actions.

    For more information on the previous experiment, see Revisions to Profile Reset Process (ATA Experiment 107)

    • Results

  • Results
    The treatment intervention of modified menu options reduced the likelihood of users resetting their profile (fully or partially) by 3.7 percentage points (pp) compared to the control-group mean of 13.1% (p < 0.001). This comparison is between full resets in the control group and any reset (full or partial) in the treatment group.

    The treatment significantly increased the share of users who accessed the reset menu, by 4.5 pp compared to the control-group mean of 18.2% (p < 0.001). During the experiment period, 11.2% of treatment-group users chose to listen to their current profile; this behavior was enabled by the new feature that was not available to control-group users.

    Overall, these results indicate that the treatment encouraged greater engagement with the reset menu and related information, while reducing the incidence of reset requests.

You have successfully submitted or edited this experiment.

Date Result Anticipated
Add reminder to Google Calendar to update your entry on: 31 Dec 2020

Contact us for assistance.