The PxD Experiment Registry documents our design experiments and impact evaluations—from simple A/B tests to large-scale randomized impact evaluations. It captures experiments we've conducted on our own services and with partners, measuring how specific service design changes affect outcomes and the overall impact of digital agriculture services. The Registry is searchable, filterable, and exportable, and is designed as an open resource to share learnings with others building digital agricultural services. For questions or publishing inquiries, contact info@precisiondev.org.

not-registered PxD staff internal login:

Rotating Menu for Seasonally-Relevant Content (ATA Experiment 108)

ETH -18 -1533

Last modified on May 5th, 2026 at 3:42 pm

    Basic Information

  • Abstract
    PxD is partnering with Ethiopia’s Agricultural Transformation Agency (ATA) to help improve the effectiveness of their voice-based mobile advisory service, the 8028 hotline, by conducting continuous iterations and experiments, as well as by making suggestions for improvements to and customization of the service. The service has millions of registered farmers and represents the first in Africa to be maintained by a government entity at such a large scale.

    Review of 8028 hotline usage showed minimal variation in users’ menu selection patterns throughout the seasons. Across the year, a large majority of callers consistently selected the "pre-planting" option, which was listed first in the menu. This behavior raised two possibilities: either farmers place a high value on pre-planting content throughout the year, or their selection is driven by the option’s menu position. To better understand this behavior, we implemented a menu rotation experiment that reordered options according to the crop calendar.

    Across two calendar periods analyzed, promoting a topic to the featured position for selecting Option 1 drove selection rates by treatment farmers roughly 10–14 times higher than control group rates, demonstrating a strong and consistent effect of topic placement on user selection. By rotating the order of options based on the crop calendar, the experiment likely led treatment group users to receive content that was more aligned with seasonal relevance, improving the targeting of advisory information.
  • Status
    Completed
  • Start date
    Q2 Jun 2018
  • End date
    Q1 Feb 2020
  • Experiment Location
    Ethiopia
  • Partner Organization
    Ethiopian ATA
  • Agricultural season
    _Multiple seasons

    • Research Design

  • Experiment type
    A/B test
  • Sample frame / target population
    8028 hotline Amharic language users
  • Sample size
    50,000
  • Outcome type
    Information access, Platform engagement
  • Mode of data collection
    PxD administrative data
  • Research question(s)
    Does rotating the order of IVR advisory menu options to align with the agricultural calendar increase access to seasonally relevant content among 8028 hotline users?

  • Research theme
    Communication technology
  • Research Design

    8028 users were randomly assigned to control or treatment groups. The control group continued to receive the default fixed menu order, with “Pre-planting” listed first throughout the entire period.

    Users in the treatment group received a rotated menu, with the order of five advisory categories changing throughout the agricultural season based on the crop calendar. The five categories included Pre-planting, Planting, Crop Protection, Fertilizer Side Dressing, and Post-harvest Management.

    The menu order was rotated across multiple periods throughout the season. We analyzed administrative data for two calendar periods during which menu rotation occurred.

    • Results

  • Results
    The intervention produced strong evidence that menu order significantly influences user selection behavior. Across calendar rotation, the majority of users (~60%) selected the first menu option regardless of topic, followed by second option. This pattern suggests that user choices are primarily driven by menu placement rather than content relevance.

    We analyzed data for experiment implementation across two calendar periods with different promoted topics. Across both periods, promoting a topic to the featured position for selecting Option 1 drove selection rates by treatment farmers roughly 10–14 times higher than control group rates, demonstrating a strong and consistent effect of topic placement on user selection. Calendar period 2 (Aug 14 – Sep 10) promoted Fertilizer Side Dressing as Option 1. Treatment users selected this topic at a rate of 60.7%, compared to just 5.9% among control users — a 54.8 percentage point increase. Calendar period 3 (Sep 17 – Oct 4) promoted Post Harvest as Option 1. Treatment users selected this topic at a rate of 60.6%, compared to 4.4% among control users — a 56.2 percentage point lift. Both results were highly statistically significant (p-values close to 0).

    By rotating the order of options based on the crop calendar, the experiment likely led treatment group users to receive content that was more aligned with seasonal relevance, improving the targeting of advisory information.

You have successfully submitted or edited this experiment.

Contact us for assistance.