We selected the sample of rice farmers in Odisha from a list of phone numbers provided by our government partner. We set aside a batch of 10,000 valid numbers that are not on the Indian “do not disturb” list, and randomly selected waves of 600 phone numbers for testing. We designed six treatments that combined automated voice calls in the morning (10AM) or evening (6:30PM) with text message alerts sent one hour or 24 hours ahead (or not at all).
 
The outcome is a binary variable describing call completion: It equals 1 if the call recipient answered the five questions asked during the call to complete enrollment.
 
Starting on June 3, 2019, a new experimental wave began every other day and was completed the following day for a total of 17 waves. For each call strategy, we tracked how many farmers successfully completed the enrollment call. We began by assuming all six strategies were equally likely to work, then updated our estimates as real data came in from each wave. Based on these updated success rates, we adjusted how many farmers would be assigned to each strategy in the next round—gradually shifting more farmers toward the strategies that were working best.
 
For more information, see Kasy and Sautmann (2021).

Across 12 villages in two blocks of Gujarat where the Krishi Tarang service was operational, we selected approximately 600 farmers to be assigned at random to one of four conditions in which farmers received:

• Control (C): an SHC only
• Treatment 1 (T1): an SHC and an audio recording
• Treatment 2 (T2): an SHC and a video clip
• Treatment 3 (T3): an SHC and a visit by an agronomist

We also randomly varied whether the SHC recommended higher fertilizer use than is typically practiced by farmers in the area or lower fertilizer use than is typically practiced. Each set of recommendations was plausible, given the soil composition in the area. This was done to understand whether trust in information is driven by a bias towards believing that more fertilizer is always better than less. Half of the participants in groups T1, T2, T3, and C received high-recommendation SHCs and the other half received low-recommendation SHCs. Farmers in the three treatment groups were also provided with a written supplement to convert fertilizer recommendations from the kg/hectare unit of area used by the SHC to the kg/bigha unit commonly used in this setting.

We first assessed farmer beliefs without providing any soil nutrient information, by asking the farmer to provide fertilizer recommendations to a hypothetical friend or cousin cultivating irrigated cotton. Second, we showed each farmer an SHC and asked the farmer to answer: (1) factual questions about the SHC’s recommendations about three specific types of fertilizer, and (2) opinion questions, to gauge the farmer’s perception of the trustworthiness of these recommendations. Finally, as per the farmer’s assignment to a treatment group, the SHC’s recommendations were also explained through an audio recording, video clip, or visit by an agronomist. Farmers were then asked to answer the questions again. At this time, they were also asked other questions regarding their knowledge of soil fertility, trust in recommendations under different scenarios, and willingness to participate in lotteries whereby, if chosen, they would have to pay INR 250 (or 200, or 150) to have a soil test worth INR 250 performed for their field.

For more information see Cole and Sharma (2017).

The A/B test will run for seven weeks, during which coffee farmers will receive one advisory per week based on the coffee crop calendar.

Farmers will be randomly assigned with equal probability to one of three groups, stratified by district and gender. The groups are:

• T0: human-generated instructional advisories delivered with a human voice (status quo).
• T1: AI-generated instructional advisories delivered with an AI-generated voice.
• T2: AI-generated instructional advisories delivered with a human voice.

We will collect data for seven rounds (R1–R7). We will use PxD administrative platform data to assess user engagement, measured as:

• Listening rates: The proportion of the advisory call that the farmer listens to.
• Pick-up rates: The proportion of advisory calls that are picked up in Round 2 to Round 7 of calls.
• 80% listening rates: The proportion of the advisory calls when farmers listen to at least 80% of the content.

We randomly assigned four types of LCC distribution methods at the village level, stratified by district and whether PxD’s service penetration rate in the village was above or below the median. We assigned treatment randomly to infer results clearly and at the village level, as some distribution types had a higher risk of spillovers than others did. We designed the pilot to provide preliminary insights on the efficacy of the different distribution channels, in order to rule out mechanisms that are unlikely to work and to gather qualitative insights on whether and how farmers use the tool. With only 32 villages and 830 farmers in the pilot, the study was not designed to detect significant differences in distribution channels or causal impacts of receiving LCCs on farmer outcomes.

We facilitated the distribution of LCCs to 418 farmers, provided them with a context-specific instruction booklet, and delivered advisory content over the phone with behavioral nudges on how and when to use LCCs. We sent these messages every week from the start of the sowing period in Gujarat, for three weeks.

During May 2022, we tested the following LCC distribution mechanisms:

1. By visiting farmers door-to-door, PxD’s staff distributed LCCs directly to 231 farmers and trained the farmers on the correct use of the LCC.
2. PxD partnered with an NGO local foundation that regularly interacts with an extensive network of cotton farmers in-person. PxD used a train-the-trainers model to ensure the NGO’s staff could properly train farmers on the use of the LCC. The NGO foundation distributed LCCs to 115 farmers and trained them during their village meetings.
3. PxD’s staff trained four agro-dealers to distribute LCCs to their customers, since we had confirmed through our scoping activities that agro-dealers are a primary source for farmers of information about the application of nitrogen fertilizer. Twenty-two farmers in total received LCCs from these four agro-dealers.
4. PxD distributed LCCs to 19 farmers and asked each of these farmers to distribute two additional LCCs to their peers. Thirty-one farmers in these villages received an LCC from their peers in the village.

Throughout the pilot, we also followed up with 412 control farmers who did not receive the LCC.

We conducted a phone midline survey during August–September 2022, in-person focus group discussions and qualitative interviews during October 2022, and an in-person end-of-season survey during November 2022–January 2023, to understand whether LCC recipients changed their nitrogen-use patterns and the impact of LCCs on other farming outcomes.

We used the cost of cotton production in Gujarat, as described in Cole and Fernando (2021), to estimate potential cost reductions for farmers who receive an LCC.

Using a random-walk sampling method, we recruited 1,253 cotton farmers across 60 villages in six blocks in three eastern districts of Maharashtra.

Eligible participants were the primary agricultural decision-makers in their household, aged 18–75. Their primary crop for Kharif 2024 (the intervention year) was cotton, grown on land they own (not solely rented or sharecropped). They applied urea to cotton in the Kharif 2023 season, possessed a verifiable 10-digit mobile number, and had at least one plot that was under cotton cultivation in each of the past two Kharif seasons and would be under cotton again in Kharif 2024. Eligible participants were willing to accompany the enumerator for GPS area measurement and soil sampling for this plot.

We stratified recruited farmers by block of residence, and randomly assigned the farmers to one of five experimental arms:

1. Treatment 1 (T1, n = 208 farmers): LCC distribution and in-person training, plus digital extension support via Interactive Voice Response (IVR) and WhatsApp.
2. Treatment 2 (T2, n = 208 farmers): Same as T1, plus a conditional payment of ~$5 per acre at the end of the season for verified LCC use.
3. Treatment 3 (T3, n = 209 farmers): Same as T1, plus an unconditional upfront payment of ~$5 per acre (no usage conditions).
4. Treatment 4 (T4, n = 207 farmers): Same as T1, plus an upfront payment of ~$5 per acre for LCC use, with the provision that the payment might be rescinded if the farmer does not use the LCC.
5. Control (n = 421 farmers): No intervention.

We collected baseline survey data, soil samples, and GPS coordinates of the monitored plots prior to randomization. We collected chlorophyll meter and LCC readings two times during the cotton growing season at approximately 55 and 75 days after sowing (DAS). The endline survey measured self-reported fertilizer quantities and timing, LCC use and knowledge, yields, and production costs. We used administrative data to measure user engagement with the digital advisory services (IVR/WhatsApp).

The integrated data collection and intervention timeline was as follows:
1. Baseline—Survey, soil samples, GPS (March 29–May 2, 2024).
2. Training and distribution—LCC training, LCC distribution, contract offer (May 21–June 6, 2024).
3. Ex-ante payments—Groups paid before June 30, 2024.
4. Digital advisory—IVR/WhatsApp messages (June–July 2024).
5. Sowing/resowing phone checks—Scheduled monitored-plot field visits (July–August 2024).
6. Monitored plot visit 1 (~55 DAS)—Chlorophyll meter and LCC readings (July 17–October 16, 2024).
7. Monitored plot visit 2 (~75 DAS)—Chlorophyll meter and LCC readings (August 14–October 29, 2024).
8. Phone survey on LCC use; ex-post conditional payments (September 18–October 31, 2024).
9. In-person endline survey A—Fertilizer use, LCC use (December 14, 2024–January 10, 2025).
10. In-person endline survey B—Yield, production costs, contract recall (April 28–June 3, 2025).

Primary Outcomes of Interest:
1. LCC panel reading.
2. Chlorophyll meter reading.
3. Self-reported urea use (kg/acre).
4. Self-reported nitrogen use (kg/acre).

Secondary outcomes of interest:
1. Pick-up and listening rates of voice push messages over the phone.
2. Index of Correct LCC-Use Knowledge.
3. Self-reported cotton fiber yield (kg/acre).
4. Self-reported fertilizer cost (INR/acre).
5. Self-reported production cost (INR/acre).
6. Self-reported profit (INR/acre).

See the American Economic Association’s registry for randomized controlled trials for this study: https://www.socialscienceregistry.org/trials/14033

Sample frame and selection criteria:
We selected 12 groundnut-growing blocks that had active Krushisharang-affiliated dealers.
For each block, we recruited three additional agro-dealers by randomly sorting agro-dealers from publicly available rosters and closing recruitment once we successfully recruited three agro-dealers.

Randomization protocol:
We randomly assigned the 12 blocks in equal numbers to one of three treatment groups (four blocks per group):
Group 1: Direct delivery model.
Group 2: Hub-and-spoke model.
Group 3: Krushisharang-dealer-directed model.
We randomized at the block level, stratified by whether the Krushisharang-registered agro-dealer in the block had received a free trial-seed bag during the previous Rabi 2024 pilot.

Intervention details:
Group 1—Direct-delivery model: Krushisharang’s transport service directly delivered bags to all four agro-dealers in each block (one Krushisharang-affiliated dealer and three recruited agro-dealers).
Group 2—Hub-and-spoke model: Krushisharang’s transport service delivered all four bags per block to the Krushisharang-affiliated dealer; we asked this dealer to retain one bag and distribute the remaining three to the recruited agro-dealers in the block; we facilitated these linkages.
Group 3—Dealer-directed model: Krushisharang’s transport service delivered all four bags per block to the Krushisharang-affiliated dealer; we asked this dealer to retain one bag, independently select three agro-dealers in their block, and deliver the remaining bags to these selected agro-dealers.

Data collection and measurement:
We conducted phone surveys to track the delivery to the intended agro-dealers.

For more information on underlying evidence for this study, see Dar et al. (2024) or the working paper version.

Intervention details and treatment arms:
We sent monsoon onset forecasts via SMS to approximately 38 million farmers across 13 states (Uttar Pradesh, Madhya Pradesh, Haryana, Punjab, Himachal Pradesh, Jharkhand, Bihar, West Bengal, Odisha, Rajasthan, Maharashtra, Chhattisgarh, and Uttarakhand). In Odisha, we randomized the rollout of forecast delivery such that farmers in 57 blocks received forecasts via both IVR calls and SMS messages, while farmers in 61 blocks received forecasts only via SMS messages, thus allowing for a comparison between IVR+SMS and SMS-only blocks.

Treatment group (n = 1,007): Received forecasts via IVR calls and SMS messages.
Comparison group (n = 1,051): Received forecasts via SMS messages only.

Data collection and measurement methods:
Using administrative and platform data for all 38 million recipients, we gathered information on the delivery of and engagement with forecast messages. We conducted monitoring surveys via telephone, and collected farmer feedback on their message recall, message comprehension, and use of forecasts for agricultural decision-making. We conducted experimental surveys via telephone calls with treatment and comparison groups, to estimate the causal impact of monsoon onset forecasts on farmer behavior and crop outcomes, thereby leveraging the randomized rollout across selected blocks.

Sample frame and selection criteria:
Monitoring surveys: In Odisha, farmers were sampled from 57 blocks in the state where IVR-based forecasts had been disseminated to them. A total of 15,000 farmers were randomly selected for surveys from those registered under the Krishi Samruddhi program, which is a free IVR service providing agricultural advisory to farmers. In Bihar and Madhya Pradesh, farmers were sampled from 203 sub-districts in Bihar and 391 sub-districts in Madhya Pradesh, where SMS-based forecasts had been disseminated to them. A total of 4,050 farmers were randomly selected in Bihar, and 7,275 farmers in Madhya Pradesh; these farmers were registered with the Pradhan Mantri–Kisan scheme, which provides financial support to landholding farmers.

Experimental survey: In Odisha, a total of 2,058 farmers were randomly sampled from 118 blocks. Of these, 57 blocks had received forecast calls while 61 blocks had not.

Randomization protocol with clustering or stratification:
Monitoring survey: In Odisha, the sample was stratified by block, forecast grid box, and historical engagement with voice-call forecast dissemination in 2024. For the first survey round, we classified farmers as high-engagement if their pick-up and listening rates (conditional on pickup) exceeded the average for the dissemination sample during May and June 2024. We oversampled high-engagement farmers using a 70:30 allocation between the high- and low-engagement categories. For the second survey round, we applied the same classification only to farmers who had picked up the forecast calls in round 3 of the dissemination cycle. In Madhya Pradesh and Bihar, farmers were equally allocated within each stratum, where strata were defined by the combination of state, forecast grid box, and district. We determined the number of farmers selected per stratum within each state based on the operational capacity of the respective Kisan Call Centers that conducted the phone surveys.

Experimental survey: In Odisha, blocks were randomly assigned to the treatment or control groups in each forecast grid box stratum. Within each block, we used simple random sampling to select respondents for the survey.

For more information on the previous trial, see “Weather Forecast Dissemination 2024”.

The push and invitation messages were randomized as the messages were sent, and the randomizations were not blocked or stratified in any way.

The main outcomes of interest are whether specific platform components (the FAW menu of informational topics, the FAW monitoring tool, and the FAW misconception quiz) were accessed or completed. Accessed means the users started the tool, either by replying to the invitation message or sending in a keyword (such as CHECK) to activate the tool. Completion means they finished using the tool. In the case of the FAW monitoring tool, users entered five values for their FAW infestation and the height of their maize to complete using the tool; the farmers then received a recommendation customized to the information they had shared.

For further information on the trial that these experiments were part of, see Kenya Fall Armyworm Trial 2019

Sample frame and selection criteria:
The study sampled PM Kisan beneficiaries for forecast dissemination. For MO forecasts, 21 districts in Telangana were selected where high-accuracy forecasts were available. For TR forecasts, 39 districts across Uttar Pradesh, Madhya Pradesh, Rajasthan, Telangana, and Chhattisgarh were selected, based on a high correlation with IMD’s all-India total rainfall forecasts.

Randomization protocol with clustering or stratification:
Message dissemination for MO employed a randomized saturation design. Sub-districts were randomly assigned to receive forecasts at different coverage levels (30% vs. 90% of farmers). For TR, 1,000–2,000 farmers per district were randomly assigned as comparison farmers and did not receive messages.

Intervention:
TR forecasts were sent to 8.6 million farmers, while MO forecasts were sent to 0.85 million farmers in Telangana, with 0.4 million of these farmers also receiving TR forecasts. The first round of forecast SMS messages (both MO and TR) was sent on May 11, 2024, followed by a second round of updated messages sent from June 5 to June 8, 2024.

Data collection and measurement methods:

• May 2024, Round 1: A telephonic Kisan Call Center (KCC) survey of 19,000 base sample farmers across five states; captured farmers’ beliefs about MO and TR.
• May–June 2024, Round 2: A telephonic KCC survey of 1,000 base sample farmers after message dissemination; focused on forecast use, satisfaction, and trust.
• August–October 2024, Round 3: A telephonic KCC survey of 23,000 base sample farmers; gathered insights into farmers’ beliefs about the 2024 monsoon and their related agricultural decisions for the Kharif season.
• Around harvest time, Round 4: An in-person survey of approximately 2,500 of the farmers in Telangana who received MO forecasts; gathered detailed information on agricultural activity, income, consumption, and household finances.

We conducted LIF and real-world IVR experiments to expose farmers to weather forecasts and light-touch informational treatments. We first conducted the LIF experiment before rolling out the IVR forecast service experiment. Phase 0 of the forecast experiment included farmers who had participated in the LIF and had opted to receive forecasts, thereby creating overlap between the two study samples. We combined data from both components in some analyses, to link farmers’ responses and behaviors in the LIF with their subsequent engagement with the forecast service.

• Lab-in-the-field experiment
  • Sample frame and selection criteria:
    We randomly selected 21 gram panchayats (GPs) across two blocks in Chikmagalur and Kodagu districts in Karnataka. Within these GPs, we randomly sampled 1,212 farmers from the rosters, maintained by the Coffee Board of India, of small- and medium-holder coffee farmers and from existing users of CKT.
  • Randomization protocol with clustering or stratification:
    We conducted randomization on the spot at the GP level. We stratified farmers by GP and assigned them to one of three groups:
    (1) climate change salience treatment (T1),
    (2) climate change salience and probability training treatment (T1 + T2), or
    (3) control group (C).
  • Intervention details:
    T1: We showed farmers a short video on the effects of climate change on coffee cultivation in Karnataka, India.
    T1 + T2: We showed farmers a video that combined the climate change content with an explanation of probability concepts.
    The control group did not receive any video information before participating in the experimental games.
  • Data collection and measurement methods:
    The experiment involved two experimental games designed to assess how farmers interpreted, updated, and acted upon probabilistic rainfall forecasts. The first, a “market-choice” game, measured how accurately farmers interpreted probabilistic forecasts. The second, an “agricultural decision-making” game, measured how farmers used probabilistic forecasts to update their beliefs and make agricultural decisions. In addition to game outcomes, we collected in-person data on farmer characteristics, farm characteristics, risk preferences, and understanding of probabilities.

• IVR Weather Service Experiment via CKT: We included a total of 27,120 farmers across 21 forecast grid cells in 11 blocks in Karnataka in the study sample. We conducted randomization at the village level and stratified it by forecast grid cell. Farmers received an onboarding call explaining the service, and then, according to their treatment status, five-day cumulative rainfall forecasts via voice calls every five days. We made up to three call attempts per scheduled forecast.

• • Phase 0: We first offered the forecast service in April 2024 to farmers who had participated in the LIF experiment and had opted to receive forecasts. We assigned villages with at least five farmers to one of two experimental arms that received: (1) probabilistic forecasts only, and (2) probabilistic forecasts with additional forecast interpretation voice calls.
  • Phase 1: We assigned farmers in five blocks—Somwarpet, Mudigere, Sakleshpura, Belur, and Alur—to Phase 1 and gave them access to the forecast service, starting in July 2024. We assigned villages to one of four arms that received: (1) probabilistic forecasts only; (2) probabilistic forecasts with forecast interpretation voice calls; (3) deterministic forecasts only; and (4, the control group) standard advisory voice calls only.
  • Phase 2: We assigned farmers in six additional blocks—Chikmagalur, Koppa, Narasimharajapura, Madikeri, Arkalgud, and Sringeri—to Phase 2 and gave them access to the forecast service, starting in August 2024. We assigned villages to one of five arms that received: (1) probabilistic forecasts only; (2) probabilistic forecasts with a recommended action or advisory; (3) deterministic forecasts only; (4) deterministic forecasts with a recommended action or advisory; and (5, the control group) standard advisory voice calls only.

For further information see working paper: Surendra, Cole, and Harigaya (2025)

We evaluated the impact of ACLs for water tanks using a stratified, matched-octad randomized controlled trial with 1,512 smallholder dairy farmers in Kenya. To increase power, we first screened out farmers with no interest in a tank loan. Eligible farmers were stratified by their dairy cooperative, access to piped water, and the presence of young children in the household, then grouped into octads using the Mahalanobis distance. Four farmers from each octad were randomly offered an ACL and four were assigned to the control group.

For further information on the previous RCT, see Jack et al., 2023.

See the American Economic Association’s (AEA) registry for randomized controlled trials for this study: RCT ID AEARCTR-0009662

Sample Frame and Selection Criteria:
The sample comprised 2,457 farmers in the blocks of Ananthagiri, Chinthapalli, Araku Valley, Paderu, G. Madugula, and Hukumpeta. All farmers who were onboarded to the CKT service during or after January 2024 were included in the sample frame. The onboarding process to CKT involves an in-person profiling survey after which they are given the choice to opt-in to the service. Farmers who were onboarded between July 2023 and January 2024 in the blocks of interest were included only if they had not received or engaged with the relevant advisories on the CKT service.

Randomization Protocol:
Farmers were stratified by block and by prior CKT experience (i.e., enrollment before or after January 2024) and then randomized into two groups.

Intervention Details:
Time frame: August 2024 to October 2024.
The control group was provided access to the CKT hotline only, while the treatment group was provided access to the full outbound voice-call advisories and SMS dissemination in addition to the hotline.

• Control Group: Onboarding calls, Farmer testimonial calls, Hotline nudge calls, and Hotline access.

• Treatment Group: Same interventions as Control Group plus composting advisory calls, WSB advisory calls, and SMS reminders for specific advisories.

Data Collection and Measurement Methods:
We used in-person surveys by field agents from January to February 2025 to collect the following endline data: demographic details, self-reported adoption of composting, WSB management, harvesting practices, information sources utilized for coffee practices, and CKT service insights. Additionally, 20% of profiled farmers in each village, in equal numbers for treatment and control group farmers, were randomly assigned to an objective measurement survey. Field staff assessed tree shade and compost pits as additional measurements to supplement farmer self-reported outcomes. A back-check telephonic survey was conducted with 10% of farmers who consented, via the Coffee Board call center, for quality assurance.

A total of 1,547 farmers were randomly assigned to the treatment group or the control group. Both groups received the same core GeoPotato service, with the key difference being whether the fungicide advice included a brand reference.

Farmers in the control group (n = 839) received standard alerts that recommended only the chemical composition of fungicides, without reference to any specific brand. Farmers in the treatment group (n = 708) received branded alerts that explicitly recommended fungicides from Bayer.

OAF had phone numbers for approximately 13,000 of the approximately 14,500 FPs in Rwanda. To recruit participants, enumerators from OAF phoned FPs and asked if they consented to responding to a baseline survey. From June to July 2021, OAF and PxD conducted a 25-minute baseline phone survey to collect data on FPs’ demographic characteristics, motivations, personalities, performance in the previous agricultural season, and goals for the upcoming agricultural season. The 10,187 FPs who completed the baseline survey became the study sample.

The intervention took place in three settings: (i) a subsidized input promotion campaign, (ii) a demonstration plot, and farmer training campaign, and (iii) an agroforestry trees campaign. FPs received SMS messages from August to November 2021 shortly before and at the beginning of the 2022A season, which ran from September 2021 to January 2022. Endline survey data collection was completed in April 2022.

We randomized participants into one of four main groups with stratification by the district and the quartile level of personality traits variation in baseline responses:

• Control: FPs did not receive any motivational messages.
• Random Message (T1): FPs received motivational messages matching personality traits that were randomly assigned to them.
• Matching Message (T2): FPs received motivational messages matching their personality traits.
• Mismatching Message (T3): FPs received motivational messages mismatching their personality traits.

Additionally, FPs in each of the four groups (C, T1, T2, and T3) were subdivided to either (a) receive a goal reminder based on their response in the baseline survey, or (b) not receive a goal reminder. The goal reminders were implemented for two settings: (i) the subsidized input promotion campaign, and (ii) the demonstration plot and the farmer training campaign.

See Working Paper: Martin Abel et al., 2025 “The Effect of Reminders for Self-Set Goals on Productivity”

See previous trial: Motivational Messages for Extension Workers in Rwanda Tree Program 2018

The sample comprised all 8028 hotline users who selected the Livestock menu. Farmers were randomly assigned to the control group with no changes to the Livestock menu options or to the treatment group with the two poultry menu options combined under a new Poultry menu.

• Control group: Five livestock menu options for Dairy, Fattening, Small-Scale Poultry, Improved Household Poultry, Apiculture.
• Treatment group: Four livestock menu options for Dairy, Fattening, Poultry, Apiculture. If users selected the Poultry option, then they could press 1 for Small-Scale Poultry or press 2 for Improved Household Poultry.

We used administrative data to measure the key outcomes of interest, namely access to content, and distribution of accessed content.

This trial was conducted in 173 constituencies that had a minimum number of 30 medium or late planters who had opted-in to receive the cropping series (CS) SMS advisory for maize, beans, or tomatoes from the MoA-INFO platform in the LR 2021. Users could opt-in to the CS for a maximum of two crops at a time. Farmers were randomly assigned to the treatment or control groups, stratified at the constituency level. Farmers assigned to the treatment group for one crop were also assigned to the treatment group for any other crops they had opted-in to at the time of randomization. The crop-wise randomization order was maize first, followed by beans, then tomatoes.

The number of users who opted-in to maize at the beginning of the LR 2021 in the relevant constituencies was the highest (57,691 users), followed by bean (37,325), then tomato (17,707). The trial was designed to result in 10,000 farmers in the treatment group for each crop. Therefore the proportion of treated versus control farmers was 17.3% for maize, 26.8% for bean, and 56.7% for tomato. The proportion of medium and late planters in the treatment groups was determined based on the number of medium- and late-planting MoA-INFO farmers from qualifying constituencies. The three treatment groups were composed of 63% medium planters and 37% late planters.

Maize, bean, and tomato treatment farmers received fortnightly push messages throughout the LR 2021, with recommendations for the best time to spray with pesticides. Treatment farmers were also invited to browse certain sections of the MoA-INFO content—the Fall armyworm (FAW), bean fly, and tomato leafminer (Tuta absoluta) menus, and the FAW monitoring tool—by sending specific keywords (i.e., FAW, BEAN, TOMATO, CHECK). Control group farmers received standard CS messages for the crops they had opted-in to.

During the months of May and June, PxD conducted two rounds of monitoring surveys to gather feedback on the PRISE model.

The outcomes for engagement with the MoA-INFO platform were: (1) opting-out of the CS, and (2) browsing the relevant MoA-INFO platform menus as encouraged by the treatment messages. We measured the opt-out outcome as a dummy variable if a user opted-out at any point in the LR 2021 season. We measured menu browsing as a dummy variable if a user browsed the menu at least one time during the same period, and we counted the total number of times the user browsed during that period. Users could still browse menus by sending the keyword, even if they opted-out of receiving CS push messages.

For further information on the previous trial, see PRISE Pesticide Application Campaign SR 2020

This project ran over a period of six weeks, from October 5 to November 23, 2021. The first three weeks were pilot weeks, when we observed farmers’ responses to randomly scheduled IVR calls. We trained a machine learning model to predict the best timing of advisory calls for individual farmers based on their engagement patterns and characteristics. The last three weeks of the study were evaluation weeks, when we evaluated a targeted call-time policy.

The sample size for the intervention was about 1.3 million farmers, but we restricted the analysis to a subsample of about 900,000 farmers whose data we had for all the covariates.

During the evaluation weeks, we separated the Ama Krushi service operation time into 91 time bins: 7 days (Monday to Sunday) X 13 hour-long blocks per day (8AM–9PM). We randomized farmers at the individual level. Control group farmers received IVR calls at a random time—a randomly chosen specific minute in the 8AM to 9PM period for 7 days per week. Treatment group farmers received the IVR calls during their predicted optimal time bin. The specific minute within the bin was randomly chosen. As there were technical constraints on the number of calls that could be sent per hour, some treatment farmers were assigned to their second-best time bin.

There were three call attempts per message when farmers did not pick up the first attempts. Re-tries happened 24 hours after the previous attempt. Messages were prioritized in the following order: paddy advisories, non-paddy crop advisories, and livestock advisories. We use administrative data to measure pick-up rates as a binary outcome of interest.

For further information see Athey et al. (2024)

The sample comprised 69,701 users in the control group and 69,990 in the treatment group:

• Control group (“name first”) received the standard registration process: Name messages → Location messages → Crop opt-in messages
• Treatment group (“crops first”) received the redesigned registration process: Crop opt-in messages → Name messages → Location messages

Randomization was stratified to ensure baseline comparability between treatment and control groups, which was confirmed by balance checks. Key outcomes that were measured were: response to registration prompts, provision of profile information, crop opt-ins, platform engagement (e.g., menu use), and total SMS interactions.

The sample comprised farmers from Karnataka who were growing Arabica coffee and who had picked up at least one CKT call in the previous six months. We randomized farmers at the individual level and stratified them by district, smartphone user, gender, and whether their listening rate in the previous six months was above the median. Farmers were randomly assigned to the following groups:

1. Treatment 1 (T1)—Push-call reminders: Received the standard advisory message and 4 summary reminders via push calls.
2. Treatment 2 (T2)—SMS reminders: Received the standard advisory message and 4 summary reminders via SMS messages.
3. Treatment 3 (T3)—Push-call + SMS reminders: Received the standard advisory message and 4 summary reminders, 2 via push calls and 2 via SMS messages.
4. Control: Received the standard advisory messages only.

Messages were sent over seven weeks from September 8 to October 27, 2021.

The sources of data for this experiment were the endline survey and the usage data (inbound, outbound, Q&A). The endline survey covered farmers’ knowledge of key practices and adoption of key practices, farmers’ interest in receiving promotional messages in the future, and farmers’ feedback on the content and timing of messages.

Primary outcomes of interest were the proportion of farmers with knowledge of one or more recommended WSB practices, and the proportion of farmers reporting adoption of one or more recommended WSB practices.

We randomly selected four out of the seven established OAF Duka shops. The random selection of Duka shops was done with county stratification, which meant choosing one shop per county in Bungoma, Nandi, and Kakamega, and the shop of Yala in Siaya. The Duka shops that participated in the EDB trial were: Bungoma, Kabiyet, Malava, and Yala. The three out-of-sample Duka shops (Malakisi, Kapsabet, Ingotse), as well as other less established OAF Duka shops, were instead eligible for the SSD program that offered farmers slightly smaller discounts than the EDB discount. The EDB and the SSD discounts were for the purchase of a bundle of hybrid maize seeds and fertilizers; the bundles were designed with an optimal input ratio.

In January 2021, OAF identified a list of potential clients of the seven Duka shops. The list included two types of farmers: (a) past clients of the Duka shops, and (b) previous OAF core program members from sites that are near to those Duka shops.

The EDB study sample consisted of 16,170 potential clients of the four Duka shops: 13,040 were past clients of the four Duka shops, and 3,130 were former OAF core program members from sites that are near to the four Duka shops. The list of potential clients of the three out-of-sample shops comprised 12,159 past clients and 1,761 past core program members.

The farmers in the EDB study sample were randomized with stratification at the Duka shop level and the farmer type level (past clients or former core program members), and were randomly assigned to one of the following groups:

• Control group (C): 1/3 probability.
• Time limited discount group (T1): 1/3 probability:
  • Standard message sub-group (T1-A): 1/6 probability;
  • Rationale explained sub-group (T1-B): 1/6 probability.
• Long period discount group (T2): 1/3 probability.

Treatment farmers received 7 to 9 SMS messages explaining the agronomic benefits of adopting planting fertilizer DAP and topdressing fertilizer CAN in combination with hybrid maize seeds. Treatment farmers were also offered a small, time-limited digital coupon to purchase the recommended inputs in a bundle designed with optimal quantities; this offer nudged farmers to visit the OAF Duka shops. T1 farmers were given a coupon with an expiration date of February 15, while T2 farmers were given a longer period to use the coupon until April 16. T1 farmers were further divided into two subgroups with different message framing. T1-A farmers received the standard messaging offering the coupon discount and nudging them to purchase the bundle of inputs. Messages sent to T1-B farmers also included the rationale for the recommendation to make purchases just after the short rainy season’s harvest, which was to encourage farmers to invest their income in agricultural inputs: “Invest in farm inputs when you have money now!” Control farmers were not sent any SMS message and did not receive a discount coupon to buy the bundle.

In the last two weeks of July, we conducted a qualitative phone survey about farmers’ input purchases and preferences. The survey gathered the feedback of 83 farmers, of whom 64 were part of the treatment group (77%), and 19 were part of the control group (23%).

For reference see Duflo, Kremer, and Robinson (2011).

The sample consisted of 600 farmers who were part of the Safaricom blast and had initiated registration on MoA-INFO during the LR 2021 season. Farmers were randomly assigned to a control group or to two treatment groups that received either an automated or a live human follow-up call. Randomization was conducted at the individual farmer level and stratified by county.

Farmers were allocated into six groups, based on random assignment and whether they had opted-in to the LR 2021 cropping series (CS):

• Control group (n = 200)—No call:
  • C1 (n = 100): Started registration but did not opt-in to the CS and were not assigned to receive CS or high-priority messages (in another A/B test).
  • C2 (n = 100): Completed registration and opted-in to the CS.

• Treatment 1—Received an automated IVR call (n = 200)
  • T1a (n = 100): Opted-in to the CS.
  • T1b (n = 100): Did not opt-in to the CS.

• Treatment 2—Received a live human call from an enumerator (n = 200)
  • T2a (n = 100): Opted-in to the CS.
  • T2b (n = 100): Did not opt-in to the CS.

We used administrative platform data for menu browsing and content access to measure the farmers’ engagement with the MoA-INFO service after they had received the calls.

Over 1 million paddy farmers were randomly assigned at the individual level to a control group and two treatment groups, with equal probabilities. Randomization was stratified by gender, smartphone ownership, and whether the farmer had actively rated an IVR message in the six months prior to the experiment. The three groups received the service rating prompt as follows:

Control group (T0): Received the standard Likert scale rating prompt ordered from 5 (very useful) to 1 (not useful).

Treatment 1 (T1): Received a reversed-order scale so that farmers heard “1—Not Useful” first.

Treatment 2 (T2): Received a reversed-designation scale, so that “5—Not Useful” was presented first, followed by the rest of the options.

The experiment ran for two weeks.

The experiment randomized sites into control and treatment arms, with 50% of sites allocated to each arm. Farmer groups at the control sites received no messages.

At treatment sites, 70% of the farmer groups received one of four variations of SMS-based interventions:
1. Basic SMS messages.
2. Basic SMS messages with FO endorsement messages.
3. Basic SMS messages combined with messages correcting common misconceptions.
4. A combination of all three—basic SMS, FO endorsement, and misconception correction messages.

Each variant was evenly distributed across 17.5% of treatment groups. The remaining 30% of groups at treatment sites were designated as spillover groups and did not receive any messages directly.

Randomization was stratified by district and by an indicator capturing whether a group included farmers who were part of OAF’s Monitoring, Evaluation, and Learning (MEL) survey activities.

We selected approximately 32,000 farmers who were active Ama Krishi users as the sample frame. We then randomized farmers at the individual level into three groups:

1. Treatment 1: Received a heads-up SMS message 10 minutes before a push call.
2. Treatment 2: Received a heads-up SMS 1 hour before the push call.
3. Control group: Received no heads-up SMS message.

We stratified the randomization by prior outbound engagement (above or below the median listening rate in the previous season), smartphone ownership, and gender.

We designed the intervention to test whether sending a heads-up SMS shortly before a scheduled advisory call increases the likelihood that farmers pick up and listen to the call. The experiment ran over four weeks to account for week-to-week variation and to estimate an average treatment effect.

We used administrative data to measure call pick-up and listening rates as primary outcomes of interest. Based on previous benchmarks, the experiment is powered to identify a Minimum Detectable Effect (MDE) of 1.9–2 percentage points (pp) in pick-up and listening rates, using an average baseline listening rate of 51.4% and a pick-up rate of 57.0%. Due to SMS delivery constraints, the sample size was adjusted to maintain statistical power.

The sample for this experiment comprised all MoA-INFO users who had been active in the previous 12 months before the LR 2021 when the experiment was implemented. All 160,000 farmers in the sample received an SMS invitation to browse the new filter of the SST. The SMS invitation included the label of the new filter. The treatment groups received different labels:

1. T1: “top-yielding seed varieties”.
2. T2: “expert recommended”.

The sample frame comprised 12,738 farmers who had not yet opted-in to the LR 2021 CS and had not provided location information. Farmers were individually randomized into three treatment arms with approximately equal probability. No stratification was applied during randomization.

The intervention tested the effect of message framing on opt-in rates. Farmers were assigned to receive one of three SMS invitation messages. Two versions of invitation messages had an element of “potential benefits” structured as loss framed and gain framed, while the third version was neutrally framed:

Neutral-framed group (n = 4,291): “Select up to two crops to receive advice about this season. Reply A to choose crops to receive regular messages.”

Gain-framed group (n = 4,209): “Improve your harvests this season by getting advice about two crops. Reply A to choose crops to receive regular messages.”

Loss-framed group (n = 4,238): “You risk suffering crop loss this season if you do not cultivate them with appropriate techniques. Reply A to choose crops to receive expert cropping advice.”

We used administrative data of opt-in responses to measure whether the different framings of the opt-in invitation affect farmers’ likelihood of subscribing to advisory content.

Inactive MoA-INFO farmers (n = 2,664) who had previously initiated the registration process were randomly assigned to two groups.

Treatment group 1, demographic information request (50% of sample): Message requested users to indicate the type of phone they use,
“Congratulations! You’ve been a great MoA-INFO farmer for a year. To serve you better, tell us, what is your phone type? a) Smartphone b) Brick phone.”

Treatment group 2, agronomic information request (50% of sample): Message requested users to indicate the size of their farm,
“Congratulations! You’ve been a great MoA-INFO farmer for a year. To serve you better, tell us, what is the size of your farm in acres? a) 0–2 b) 2–5 c) Above 5.”

No group received the questions without the anniversary messages.

Our outcome of interest was SMS response rates to the information request message, which we measured with PxD administrative platform data.

The sample comprised 25,357 farmers registered for the 8028 service. Each farmer was randomly allocated to one of the four treatment arms, and received a series of push calls to provide information relevant to the farmer’s crop and location.

The four experimental arms for varying the voice of the narrator were: female journalist (control group), male journalist, female agronomist, and male agronomist.

We measured engagement outcomes using administrative data. The pick-up rate was defined as the proportion of farmers who picked up the phone, and the listening rate was defined as the proportion of the call duration to the total length of the content.

This experiment ran from June to August 2019, and April 2021 to May 2023 with around 20 calls per farmer scheduled during this time period. During this time period, 424,964 calls were used for the analysis.

The study randomized 4,518 farmers into two groups to evaluate the effect of language on the farmers’ engagement with voice message advisory services. The treatment group (n = 2,360) received messages in the local dialect, while the control group (n = 2,158) received messages in standard Bangla. Randomization was stratified by farmer union, crop type (mungbean or watermelon), and planting date, to ensure balance in key characteristics across treatment arms.

Because watermelon farmers joined the service later, they received fewer messages on average than the mungbean farmers.

We sent an SMS survey to 5,000 randomly selected users between July 16 and July 23, 2020. Respondents were asked who is the primary phone user (self or other person) and to provide the gender of the primary phone user. The survey incorporated three randomized elements:

1. Answer Response Option Format: 50% of users received answer choices in the form of letters (A/B/C). The other 50% received numeric options (1/2/3).
2. Introductory Message Framing: 50% of users received an initial message asking, “Would you like to start?” The other 50% received, “Can we borrow one minute of your time?”
3. Gender Answer Order: 50% of users who reached the gender question received the answer options in the sequence: I don’t know; Female; Male. The other 50% received the sequence: I don’t know; Male; Female.

Randomization was done at the individual level with each user randomly assigned to one combination of the above elements.

The measured outcomes were:

• Primary: Response rates to the initial message.
• Secondary: Time spent on surveys, completion rates, and gender response patterns.

We randomly assigned FPs to treatment and control experimental arms. FPs in treatment arms received motivational messages whose content was designed to reinforce the content of a national radio campaign that was ongoing at the time. FPs in the control arm did not receive the messages. The experiment was implemented over 40 days.

We randomly assigned our total sample of 3,000 FPs into one of five experimental arms of equal size (n = 600):

Group 0: Control—Received no messages.

Group 1: Basic message (BM)—Received one message every 10 days for a total of four messages over the 40 days. The message contained basic information about fertilizer use and was derived from the radio campaign during that 10-day period.

Group 2: BM + nudge—Received two messages every 10 days for a total of eight messages over the 40 days. The first message was the same BM that Group 1 received. The second message was sent later in the 10-day period as a simple “nudge” message to remind the FPs about the content of the BM.

Group 3: BM + “be a visible leader”—Received two messages every 10 days for a total of eight messages over the 40 days. First, the FPs received the same BM, and subsequently they received a message that encouraged them to “be a visible leader” in their community.

Group 4: BM + “farmer appreciation”—Received two messages every 10 days for a total of eight messages over the 40 days. First, the FPs received the same BM, and subsequently they received a message of encouragement that emphasized that other farmers appreciate the FPs’ work and the role they play in the community.

By comparing Group 0 to Group 1, we estimated the causal effect of receiving a single message on our outcomes of interest: (i) FP adoption of best practices, and (ii) the number of farmers the FP taught to adopt similar practices. By comparing Group 1 to Group 2, we estimated the effect of receiving a single message versus two messages, when the second message was a nudge reminder about the content of the first message. Finally, by comparing groups 3 and 4 to Group 2, we identified whether specific message-framing mattered.

We used a follow-up survey to collect data on FPs’ farming practices and performance as volunteer extension workers, and 1,828 FPs completed the survey.

Our behavior index comprised six outcomes measures:
1. Whether the FP in question used fertilizer on their demonstration plot,
2. Whether the FP in question used fertilizer on their personal plot,
3. Whether the FP in question used both DAP (diammonium phosphate) and urea fertilizers as recommended,
4. The number of farmers the FP trained on the use of fertilizer,
5. The number of farmers the FP trained on their demonstration plot, and
6. Whether the FP correctly answered a question testing their knowledge about fertilizer.

Farmers were individually randomized into three groups: a control group, a treatment group, and a preference group.

Farmers in the control group continued to receive advisory calls once per week, with each call maintaining the standard pre-treatment length. The treatment group received calls twice per week, but each call was half the standard length. The preference group was given the opportunity to choose the frequency of their advisory calls, which allowed personalized scheduling based on their preferences.

After randomization, the baseline characteristics of the preference and control groups were found to be unbalanced. To account for this imbalance, we applied a difference-in-differences analysis to estimate the effect of the preference-based treatment on the outcome measures. The experiment ran for ten weeks.

By the beginning of Kharif 2019, 4,425 farmers had registered for the service. These farmers grew one of or a combination of the following crops: paddy (T-Aman variety), chili, and brinjal. The Agro360 recommendations were sent to all farmers from July 16 to December 30, 2019. Recommendations were customized according to crop and sowing data, using the information collected at registration. The messages covered the entire crop cycle: pre-planting, sowing, seedling stage, transplanting, tillering, reproduction, ripening, and harvesting. Paddy farmers received a total of 16 messages, chili farmers received 14 messages, and brinjal farmers received 12 messages.

Farmers were randomly assigned to four groups:
Group 1 (control group; n = 1,093): Received standard voice messages.
Group 2 (n = 1,102): Received voice messages in a conversational style.
Group 3 (n = 1,108): Received voice messages that began with a trivia fact.
Group 4 (n = 1,122): Received voice messages that began with a trivia fact and continued in a conversational style. However, this group was dropped from the analysis because of implementation challenges.

​Sample messages:

Group 1, Standard message: “For aman rice cultivation keep a drain filled with water for 5–7 days after sowing. After 5–7 days of sprouting, 2–3 cm water in the seedbed helps to control the rice field from weeds and birds.”

Group 2, Conversational message (after the characters Ramjan and Kashem had been introduced in the first message, to set the context): “Ramjan: Brother Kashem, I’ve completed sowing seedlings in the seedbed. I have heard there should be space for drainage in the seedbed, shouldn’t it? Kashem: Yes, you’re right. Ramjan, when rice seedlings become 5–7 days old, keep some water storage in the drain near the seedbed; it will help to grow seedlings properly.”

Group 3, Trivia message: “Did you know—there are more than 40,000 varieties of paddy in the world” + standard message.

Group 4 Conversational + trivia message: “Did you know—there are more than 40,000 varieties of paddy in the world” + conversational message.

In order to ensure that key characteristics were balanced between the different treatment groups, randomization was stratified by crop and by planting date.

The delivery reports of the voice messages were used to measure engagement outcomes for pick-up and listening rates.

This trial was conducted during Kenya’s 2020 short rainy season (SR 2020) across all 189 constituencies with at least 30 late or very late planters who had opted-in to receive the cropping series (CS) SMS advisory for maize, beans, or tomatoes from the MoA-INFO platform. Users could opt-in to the CS for a maximum of two crops at a time. Farmers were randomly assigned to the treatment or control groups. Farmers assigned to the treatment group for one crop were also assigned to the treatment group for any other crops they had opted-in to at the time of randomization. The crop-wise randomization order was maize first, followed by beans, then tomatoes.

Treatment group farmers received fortnightly SMS messages based on CABI’s PRISE model. The messages offered time-sensitive recommendations on optimal pesticide application tailored by crop, location, and planting date. The treatment group farmers were also encouraged to access relevant MoA-INFO platform menus—such as the FAW menu for maize, the bean fly menu, the tomato leafminer (Tuta absoluta) menu, and the FAW monitoring tool—by texting specific keywords (e.g., FAW, BEAN, TOMATO, CHECK). Control group farmers continued to receive standard CS messages without PRISE alerts or nudges to targeted platform tools.

Approximately 18,431 maize farmers and 10,000 farmers each for bean and tomato totalling 29,554 farmers were allocated to the treatment group and 19,872 farmers were in the control group. In each crop’s treatment group, 40% of farmers were late planters and 60% were very late planters, which reflected the distribution of the planting timelines of MoA-INFO users.

The outcomes for engagement with the MoA-INFO platform were: (1) opting-out of the CS, and (2) browsing the relevant MoA-INFO platform menus as recommended by the treatment messages. We measured the opt-out outcome as a dummy variable if a user opted-out at any point in the season over a 4-month period from September 29, 2020 to January 31, 2021. We measured menu browsing as a dummy variable if a user browsed the menu at least one time during the same period, and we counted the total number of times the user browsed during that period. Users could still browse menus by sending the keyword, even if they opted-out of receiving CS push messages.

The pilot targeted MoA-INFO users who had opted-in to banana content during the long rainy 2020 season (LR 2020) and had not opted-out of the IVR invitation. Not opting-out was defined as either responding with the “A” key (indicating “No” to the question “Do you want to discontinue this service?”) or not responding to the invitation message. The rate for not opting-out in this group was 86%. Banana farmers who were MoA-INFO users in the short rainy 2019 season (SR 2019) were also included in the pilot to see whether the IVR technology could attract these users back to the platform.

The core analysis focused on the 7,500 eligible LR 2020 banana farmers. Randomization was conducted at the individual level. Stratification for randomization was based on county and the types of crops each farmer had opted-in to on MoA-INFO.

From this group, 202 farmers were randomly selected to participate in a “Pilot of the Pilot”—a short feedback survey run immediately after one IVR message was delivered and listened to.

The remaining 7,298 farmers were randomized into three treatment arms:

• T1 (n = 2,424): Control group; received regular SMS messages.
• T2 (n = 2,430): Treatment group; received weekly IVR messages.
• T3 (n = 2,444): Treatment group; received the same weekly IVR messages as T2, plus follow-up SMS reminders summarizing the key points of each IVR message.

Each group received seven weekly push messages pertaining to the topic of banana pests and diseases. A follow-up phone survey (via voice call) of 204 farmers across all experimental groups (T1, T2, T3) assessed their knowledge of banana pests and diseases.

The trial sample consisted of all 11,336 MoA-INFO farmers from early planting constituencies who had opted-in to both maize and bean CS, had not opted-out of these CS at the time of randomization, had self-reported their ward location, and were in a ward with at least one other farmer who met these criteria (otherwise the ward would be dropped due to ward-level fixed effects).

The sample was stratified at the ward level and randomized at the individual level because farming recommendations vary by the predominant agro-ecological zone (AEZ) at the ward level. Out of the 11,336 farmers in the sample, 1,417 (12.5%) were randomized to be part of the control group that did not receive the selected seven CS topic invitations: maize seed, maize fertilizer, maize topdressing, maize post-harvest storage, bean seed and fertilizers, bean planting, and bean post-harvest storage. The remaining 9,919 treatment farmers received all seven of the selected CS invitation messages (unless they opted-out).

A phone survey of a subsample of 2,939 farmers gathered information on their farming practices, yield, and demographics.

Given that the farmers had opted-in to receive CS recommendations, we wanted to send CS invitations to as many farmers as possible. Therefore, we chose to have a small control group and measure a limited set of outcomes on behavior change.

For further information on a previous trial, see “Kenya Fall Armyworm Trial 2019”.

Half of the 35,573 “active” Ama Krushi users in the Balasore or Bargarh district that reported planting Rabi paddy were randomly allocated to receive modified paddy advisory messages. The randomization was stratified by district, gender, and farmers’ Ama Krushi engagement during the last Kharif season (2020).

To measure the effect of modified messages on farmer engagement, we used the administrative data on engagement outcomes collected by the Ama Krushi system. We collected no baseline survey data; however, we collected survey data at the endline on farmers’ adoption of priority practices for a small share of our study sample to estimate the effect of the treatment on the adoption of inputs recommended in the paddy advisory. We attempted to interview 1,000 farmers about their agricultural practices, but only 556 completed the survey. There was no indication of differential attrition, but the small sample size reduces the experiment’s power to detect adoption effects.

Treatment effects were estimated using linear and logistic models for binary variables and linear models for continuous variables. We used robust standard errors and cluster standard errors where appropriate.

During the long rainy season 2016 (LR 2016), a total of 32,659 OAF clients in six districts were randomized into the control group or one of the four treatment groups:

• Control (n = 8,344):
  •Core program clients in the designated sites.

• Treatment 1 (n = 5,570)—Complete Package:
  • Clients received 25 kg lime for free during the LR 2016;
  • FOs received 100 kg lime for free during the LR 2016;
  • FOs attended a one-day intensive Lime Bootcamp;
  • FOs received a lime sales incentive (50 KES per client buying lime, up to 6,000 KES total).

• Treatment 2 (n = 6,281)—FO Focused:
  • FOs received 100 kg lime for free during the LR 2016;
  • FOs attended a one-day intensive Lime Bootcamp;
  • FOs received a lime sales incentive (50 KES per client buying lime, up to 6,000 KES total).

• Treatment 3 (n = 6,737)—Client Focused:
  • Clients received 25kg lime for free during the LR 2016.

• Treatment 4 (n = 5,727)—Minimal Package:
  • FOs attended a one day intensive Lime Bootcamp;
  • FOs received a lime sales incentive (50 KES per client buying lime, up to 6,000 KES total).

The treatments were designed to give clients and FOs varying levels of experience with lime—both directly and through training. The effect of each of these treatments on subsequent lime adoption was measured during the long rainy season 2017 enrollment.

The project had three main components.

First, farmers from the same five districts as used in the Kenya Agricultural Lime Messaging Trial 2017 were randomized at the individual level, stratified by district, to either receive (treatment) or not receive (control) two SMS message streams. The first message stream, as in the 2017 trial, encouraged farmers to adopt lime. The other message stream emphasized the adoption of extra CAN fertilizer.

Second, a subset of farmers who had not previously adopted lime were randomly assigned to receive social learning nudges with information about “resource farmers”, who were farmers from their site who purchased lime from OAF for the 2017 season. Farmers were provided with the resource farmer’s name and phone number.

Third, approximately 180,000 OAF farmers in other districts outside the original five were sent randomly assigned SMS lime promotion messages encouraging the farmers to adopt lime; the messages were varied in framing (basic, highlighting yield increases, encouraging experimentation, making social comparisons, and promoting self-efficacy) and repetitions (1 to 5 messages).

For further information on the previous trial, see Kenya Agricultural Lime Messaging Trial 2017

For further information on PxD’s lime work , see Fabregas et al., 2024.

OAF and PxD collaborated to test the effectiveness of SMS-based extension messages that promoted the adoption of lime by smallholder farmers in western Kenya. The intervention was designed as a randomized controlled trial with two treatment arms and one control group:

• Control group: Farmers did not receive any messages.
• Broad message treatment group: Farmers received simple SMS messages encouraging them to use lime, and providing a toll-free number for more information. The message read: “Hello [name], Your soil is acidic: Use lime to reduce acidity and increase yields. Call xxx-xxxx.”
• Specific message treatment group: Farmers received more detailed, customized messages. These included locally relevant information, such as the level of soil acidity measured in their area, the recommended quantity of lime, the estimated total cost, and the expected yield improvement. The message read: “Hello [name], Your soil is [highly/moderately] acidic. We recommend [amount] kg of lime per acre at [total cost] Ksh. Use lime to reduce acidity and increase yields by [percentage]%. Call xxx-xxxx.”

Customized messages were based on soil tests that had previously been conducted in the region.

A total of 4,884 farmers were included in the study. Of these, 3,325 were randomly assigned to one of the two treatment groups and received SMS messages, while 1,559 farmers served as the control group and did not receive any messages. The same SMS messages were delivered six times between August and September 2016—before the OAF input contract signing period—at a time when farmers had to decide whether to request inputs from OAF for the following 2017 season.

We measured the primary outcome of lime adoption with 1) administrative data from input discount coupons redeemed at participating agro-dealer shops and 2) endline phone survey data.

For further information, see Fabregas et al., 2024.

We selected 48 constituencies out of 290 in Kenya and sent PRISE messages to inform maize farmers about the optimal pesticide spraying time for their area, based on the PRISE model. We randomly assigned farmers to two groups:

Control (n = 2,576): The farmers received PRISE messages about the optimal pesticide spraying time for maize in their area, based on the PRISE model. They did not receive any push messages about the FAW information available on the MoA-INFO platform, but they had access to all FAW content on the platform through the menu or by sending keywords.

Treatment (n = 6,024): The farmers received several series of push messages throughout the season; the messages were about FAW and encouraged the farmers to access sections of MoA-INFO content, specifically the FAW menu and the monitoring tool.

The monitoring tool is a decision-support tool that sends farmers to their farm to measure their FAW infestation rate. The tool provides farmers with recommendations (whether to spray pesticides) based on their recorded infestation rate and the size of their maize. The FAW menu is a directory of information about FAW detection, management, origins, pesticides, and misconceptions. Users can select the FAW topics that they are interested in learning about.

The push messages to treatment farmers corresponded to the five stages of the PRISE model and were adjusted by constituency, depending on the rate of FAW infestation throughout the season:

• Stage 1: An introductory message asking if farmers had seen FAW, plus messages informing farmers that FAW was in their area, then messages about managing FAW with cultural control methods (natural solutions like applying ash or pepper).
• Stage 2: An introductory message asking if farmers had seen FAW, then two messages informing farmers about the importance of good pesticide timing and the expected “Best Pesticide Date”. If a user replied to this “Best Pesticide Date” within 2 hours, they were sent to the FAW Menu; otherwise, no additional messages were sent.
• Stage 3: An introductory message asking if farmers had seen FAW. If a user replied within 7 hours, they were referred to the FAW menu; otherwise, no additional messages were sent.
• Stage 4: Two messages, one directing farmers to use the monitoring tool and the other providing information about the best time to apply pesticides.
• Stage 5: One message to remind users how to access the monitoring tool.

We conducted a follow-up phone survey to measure the effects on knowledge, practices, and the extent of FAW damage.

A sample of 50,000 platform users were randomized to receive one of the following messages:

V1 Information: “Hello from MoA-INFO the Ministry of Agriculture information service. Send MENU to 40130 now for free information on maize, beans, potatoes and Fall Armyworm.”

V2 Make it easy: “Hello from MoA-INFO. This service is free. Please select a topic you would like to know about: A. Maize; B. Beans; C. Irish Potato; D. Fall Armyworm.”

V3 Targeted/Reaching out: “Hi. We have noticed that you have not used the Ministry of Agriculture’s free information service in a while. Send MENU to 40130 to start now!”

V4 Eye catching: “Do you want to improve your crop yields? Send MENU to 40130 now to access the Ministry of Agriculture’s free information service.”

V5 New content: “Hi. MoA-INFO has just added new information on pests and diseases for maize, beans and Irish potatoes. Send MENU to 40130 now to learn more!”

We measured the propensity of these users to access content using the MoA-INFO menu.

Note on sleeper users:
All the sleeper users had completed the registration survey. Before February 2019, by sending the keyword “FARM” or “SHAMBA”, the user auto-progressed through the registration. After February 2019, a user was still considered registered even if they only sent in the keyword. The user would have received a number of invitations to join the platform’s cropping series (CS) messages; the number depended on when the user registered. Users who joined before February 2019 received maize CS messages for the short rainy season 2018. Almost every registered platform user (99.03% of the registered users) received a message from the platform in the month before 23 April 2019. This would most likely have been an invitation to receive a CS.

Our sample comprised two groups of people whose phone numbers we had access to and who had not yet registered on the MoA-INFO platform:

1. Default Group: ~8,350 people who had sent MoA-INFO a message that was not correctly formatted, meaning their registrations were unsuccessful.
2. Referral Group: ~3,000 people who had been invited to join the service by a fellow farmer but had not yet registered.

In this experiment, these unregistered users were invited to join the platform. They were randomly selected to receive either a control message (one used in the previous nationwide campaign) or one of two improved treatment messages:

• Control: “The Ministry of Agriculture is launching a free SMS information service! Send the word FARM to 40130 to learn about Fall Armyworm.”
• Treatment 1: “Do you worry about pests attacking your crops? Send the word FARM to 40130 to learn about Fall Armyworm for free from the Ministry of Agriculture.”
• Treatment 2: “Send the word FARM to 40130 to learn about Fall Armyworm for free from the Ministry of Agriculture. Over 100,000 Kenyans already have. Don’t miss out!”

Follow-up messages were sent after 3 days, as follows:

• Follow-up 1: “Don’t forget to send the word FARM to 40130 to learn how to fight Fall Armyworm for free.”
• Follow-up 2: “Don’t forget to send the word FARM to 40130 to learn how to fight Fall Armyworm for free. Over 100,000 Kenyans already have. Don’t miss out!”

8028 hotline Amharic language users who had registered their profile were randomly assigned to two experimental groups:

Control Group (n = 25,610): Users received the standard main menu, which included a direct option to reset their profile.

Treatment Group (n = 26,205): Users received a modified main menu that included: “Press 9 to hear your current profile settings or reset your profile.” Users who selected this option were taken to a new reset sub-menu with the following options:

• Press 1 to hear your current profile settings (then return to the main menu)
• Press 2 to reset your language (then return to the main menu)
• Press 3 to reset your location (then return to the main menu)
• Press 4 to go back to the main menu

The analysis for this experiment used administrative data on calls over a five-week period from February 1 to March 8, 2020, with the main outcomes of interest being access to the reset menu, and profile-reset actions.

For more information on the previous experiment, see Revisions to Profile Reset Process (ATA Experiment 107)

Users were randomly assigned to the control group or one of two treatment arms:

Control group (n = 88,672): Menu options remained technical or scientific terminology (e.g., “Pre-planting”).

Treatment Arm 1 (T1; n = 44,518): Menu options were reworded to plain-language descriptions of the content category (e.g., “Things you should do before you plant”).

Treatment Arm 2 (T2; n = 44,551): Menu options were restructured to give examples of the subtopics available under each menu (e.g., “For insect control, disease control, or weed control, press 3”).

This experiment was conducted from July 2019 to February 2020. We used administrative platform data to measure the main outcome of interest, menu selection.

For more information on a previous trial, see Rotate Menu Seasonally (ATA Experiment 108).

Beginning in March 2019, when users registered for the MoA-INFO platform, we randomized them into two groups. The “opt-in” treatment group was selected with a 20% probability and was asked to opt-in to the maize CS during the registration survey. The control group was asked if they ever farm maize. Farmers from this group were later invited to opt-in to the maize CS, but only after they completed the registration survey. The messages in the registration survey were as follows:

Opt-in Treatment: “Would you like to receive regular messages giving you advice on how to grow better maize? A. Yes B. No.”
Control: “To give you this advice we just need to know a little about your farming practices. Do you ever grow maize on your farm? A. Yes B. No.”

After registration, the control group was asked to opt-in to the maize CS messages. Both groups were asked to opt-in to bean and Irish potato CS messages.

This was a multi-arm A/B test conducted among Amharic-speaking 8028 users. Farmers were randomly assigned to control and treatment groups. Treatment interventions varied the framing of the reset prompt and accompanying information:

• Control (n = 120,232): Standard message with no additional explanation. Profile reset prompted by pressing *.
• Treatment Arm 1, Reset (n = 30,196): “…To erase the information you have entered about yourself and re-enter it, press 9.”
• Treatment Arm 2, Reset + information (n = 30,247):
  • “…To erase the information you have entered about yourself and re-enter it, press 9.”
  • “You filled in some information about yourself like your region, zone, woreda, etc. This information is required so we can send you customized information. Once you fill in the correct information, you are never required to change this information. If you decide to erase all the information you have entered about yourself, you will be asked to enter this information again. You may decide to do this if you want to correct any information about yourself that is incorrect, or use the system in a different language, or if you are receiving push alerts for the wrong location. If you do not decide to change the information about yourself, you will still be able to use the 8028 system as much as you want, even if some of the information about yourself is incorrect.”
• Treatment Arm 3, Reset + information + existing user information (n = 29,944):
  • “…To hear the information you have entered about yourself, or to erase the information you have entered about yourself and re-enter it, press 9.”
  • [Same explanation as Treatment Arm 2.]
  • “You have entered the following information about yourself: Language—XX; Region—YY; Zone—ZZ; Woreda—AA.”
• Treatment Arm 4, Partial reset + information + existing user information (n = 30,204):
  • “…To hear the information you have entered about yourself, or to erase the information you have entered about yourself and re-enter it, press 9.”
  • [Same explanation as Treatment Arm 2.]
  • “You have entered the following information about yourself: Language—XX; Region—YY; Zone—ZZ; Woreda—AA.”
  • “If you would like to enter or change your language, press 1; if you would like to enter or change your region, zone, or woreda, press 2; to return back to the main menu, press *; to hear these menu options again, press #.”

The analysis for this experiment used administrative data on calls over a 2.5-month period from April 9 to June 24, 2019, with the main outcomes of interest being access to the reset menu, and profile-reset actions.

Users of the 8028 hotline service were randomly assigned to one of two groups:

Treatment Group: Were not prompted to save their selected crop to their profile; were directed to the content menu immediately after making a selection.

Control Group: Received the standard prompt asking them to save their selected crop to their profile before proceeding to access the content.

We used administrative platform data to measure the number of content items accessed.

Study publication forthcoming.

New users of the 8028 hotline service were randomly assigned to one of two groups:

• Control group: Received the standard language menu with existing pause intervals.
• Treatment group: Experienced a longer pause (2 seconds longer) between each language option.

Randomization occurred at the individual user level upon first contact with the 8028 hotline system.

We used administrative platform data to measure whether new users made a language selection and accessed content in their first call.

Study publication forthcoming.

We randomly assigned new users who called in to the 8028 hotline service for the first time to the treatment or control group:

• Treatment group: New users were directed straight to the menu to access agricultural content immediately after selecting their language.
• Control group: New users were required to complete the full registration process (indicating region, zone, woreda) before accessing any agricultural content.

We used administrative platform data to measure whether new users accessed any agricultural content and the number of topics they accessed.

Study publication forthcoming.

The sample consisted of new farmers who registered on the MoA-INFO platform following referrals by existing MoA-INFO users (see Methods of Inviting Farmers to Refer Fellow Farmers). Each newly registered farmer was randomly assigned to a treatment group. In one group, farmers were asked their name at the beginning of the registration survey, and in another group, farmers were asked whether they farm potatoes and beans, in addition to all farmers being asked whether they farm maize.

The main outcome of interest is the number of farmers who successfully completed the registration survey within 24 hours. This indicates whether users have actively completed the registration instead of simply auto-progressing through the survey.

This experiment builds on previous trials, which found that improved invitation messages could increase registration completion rates. For more information, see: Improving Invitation Messaging to Increase Registration Rates.

The sample frame included only farmers who were receiving wheat and cumin advisory during the ongoing Rabi season, and were not part of other experiments at the time.

A total of 4,999 farmers were randomized into two groups:

• Treatment group (N = 2,499): Farmers received push calls without the introductory jingle.
• Control group (N = 2,500): Farmers received push calls with the standard jingle (status quo).

The experiment employed individual-level randomization, stratified by whether a farmer consistently listened to fewer than 8 seconds per push call prior to the experiment (3.67% of the overall sample). This stratification ensured balance across groups based on the farmers’ prior call engagement.

The random assignment of Ama Krushi users in three lowland districts was stratified by district. Following the randomization, 187 farmers were dropped from the sample because their names were missing from the profiling dataset, and 15 were dropped because they were inactive (they had neither picked up any outbound call nor used the inbound service). A balance check confirmed that the remaining sample of 9,798 farmers was balanced across the control and treatment arms.

The control group received the regular weekly seed advisory, which provided information on several locally appropriate seed varieties. The treatment group received a message highlighting the benefits of the FT seeds (Swarna Sub-1 and CR 1009 Sub-1) for two consecutive weeks. In the treatment group, all farmers with high land and some farmers with medium and low land also received the regular weekly seed advisory.

Farmers were able to refer fellow farmers to the MoA-INFO platform by texting phone numbers to shortcode 40130. Referred individuals who were not already registered received a welcome message prompting them to opt-in to the service.

MoA-INFO users were randomly selected to receive one of three referral treatments:

• SMS invitation only (n = 2,735): Farmers received a one-time SMS message inviting them to refer fellow farmers.
• Live phone call only (n = 747): During a live phone call that took place as part of a survey, farmers were encouraged to refer fellow farmers.
• SMS invitation + live phone call (n = 247): Farmers received both the SMS invitation and phone encouragement.

Referred farmers who were not already registered on the platform received the following message: “Welcome! A friend invited you to MoA-INFO, the Ministry of Agriculture’s free information service! Would you like to learn about Fall Armyworm? A. Yes B. No.” Farmers who replied “Yes” were directed to the registration survey; those who replied “No” received a message saying that, if they changed their mind, they could receive the information by texting “FARM” or “SHAMBA” at any time.

Half of the farmers invited to refer were selected to receive a set of messages addressing the most common misconceptions about FAW. The first message asked: “There are many lies and rumors being told about Fall Armyworm (FAW)! Do you want to learn the truth? A.Yes B. No”. Farmers who texted “Yes” received five facts to dispel misconceptions about FAW. Farmers who replied “No” received a message saying that the information could be accessed at any time by texting “TRUTH”. The objective was to test whether these messages increase the propensity to refer fellow farmers to the MoA-INFO platform.

Pre-experiment:
An indicative phone survey was conducted to understand farmers’ demand for information about cumin. Blocks, with a majority of cumin-growing farmers, were chosen from the PxD database. Some farmers were randomly selected and asked whether they would be interested in growing cumin if PxD provided them with information on cumin. In three blocks, 45% of the farmers who were asked responded positively, and the experiment proceeded.

Sampling frame and selection:
We defined the population of farmers as those who didn’t grow cumin but grew other crops at the time of profiling. The total sample size was 9,000 (4,500 control and 4,500 treatment). The final randomization stratified farmers by whether they were surveyed at baseline. After attrition, removal of phone numbers designated “do-not-disturb” with the telecom operators, and removal of farmers who declined to receive information, the endline sample consisted of 5,133 farmers; a total of 3,630 of these farmers were surveyed for the endline.

Intervention:
The treatment group received digital technical advice on growing cumin, such as atmospheric conditions suited for sowing and the method of sowing. We did not provide any persuasive content, such as the benefits of growing cumin. We sent a total of five automated calls: three on sowing, one with a weather alert, and one on protecting the crop against pests during rainfall. The control group received no information.

Weather and pest alert messages were not part of the initial design, but were sent as the last two messages for the last week (fourth week) of information.

The study sample was drawn from the UCAT impact evaluation standalone treatment group. All farmers in this group, except for those who had never picked up any UCAT calls, were included in the sample. The resulting 1,570 coffee farmers were receiving standalone digital advisory treatment and were not receiving in-person training. The intervention was randomized at the individual level into two treatment groups, stratified by villages, the implementer location (HRNS or TNS), and the language the farmer speaks. The groups received reminders as follows:

1. T1 (n = 790): Received an SMS reminder one hour before the weekly push call.
2. T2 (n = 780): Received an SMS reminder 24 hours before the weekly push call.

The content of the SMS reminder was identical for the T1 and T2 groups. The study ran for four weeks (from August 12, 2020 to September 15, 2020) with each farmer in the study receiving four reminders in total. Due to implementation issues, reminders were not sent during the week of September 2, so we extended the study for another week to complete the four weeks of implementation.

The sample included farmers who were: (1) “active” on the outbound service, meaning they had received and picked up at least one call in the previous six months; and (2) growing crops in the Kharif 2020 season. The different groups in this experiment were:

1. Control: Farmers received no reminder messages.
2. Standard: Farmers received reminders and instructional voice messages with information about the inbound service.
3. Conversational: Farmers received standard messages framed as a conversation between two farmers; the conversation reminded farmers about the inbound service and provided instructions on how to call in.
4. Farmer testimonial: Farmers received standard messages and a recorded testimonial from an Ama Krushi inbound-service user about their experience using the service.
5. Social nudge: Farmers received a short trivia-style announcement that provided statistics about how their peers use the inbound service, followed by standard messages.

We sent treatment farmers one message framed in the style of the assigned treatment group every two weeks for a total of five messages over ten weeks. Control group farmers received no reminder messages.

Using the same sample of 15,914 farmers from a previous experiment The Effect of Reminder and Instructional Messages on Inbound Engagement, we maintained the control group and randomly assigned treatment farmers from the previous experiment to three new treatment arms. Farmer randomization was stratified along the original treatment groups for the type of reminder messages that farmers previously received, geography, and their engagement with the inbound service during the 10 weeks of the previous experiment. This previous engagement was categorized as: (i) they didn’t call in at all, (ii) they called in but were unsuccessful, meaning they made blank calls or calls under 60 seconds or recorded invalid questions, and (iii) they called and were successful, meaning they accessed a feature for over 60 seconds or recorded a valid question, which was defined as a farmer asking a query related to agriculture.

The groups for this study received the following interventions:

1. Reminder only (n = 5,017): Received a series of five messages from June to August 2020.
2. Reminder + Remote Training (n = 4,502): Received a series of five messages from June to August 2020, and were offered remote training from October to December 2020 to use the inbound service.
3. Reminder + Pest message (n = 2,395): Received a series of five messages from June to August 2020, and received an additional reminder message in October 2020 requesting farmers to share a question or a report of pest incidence on their field via the record-a-question feature.
4. Control (n = 4,000): Did not receive any treatment.

Remote training was conducted via live phone calls from trained call center surveyors. Surveyors explained to farmers how to access and use the IVR service, with a focus on how to record a question.

We measured outcome indicators with PxD administrative data, which included the proportion of farmers who:

• called the inbound service,
• called and accessed a feature,
• called and accessed a feature and listened to at least 60 seconds, and
• called and asked a valid question.

The total sample for this experiment was 40,000 MoA-INFO users, who were selected randomly from users that opted-in to receive maize messages in the 2019 long rainy season.

First, we randomized the message type between a regular message that gave the user a keyword to access the monitoring tool, and an easy message that bypassed the first question and took users directly into the monitoring tool. The chance of receiving the regular message was equal to the chance of receiving the easy message. Second, we randomized the time of day that the messages were sent. Each user had a 25% chance of being sent their message at 7AM, midday, 3PM and 6PM.

The messages were as follows:

• Regular message: “Hi. Now is a great time to scout your maize for Fall Armyworm! Send CHECK to 40130 for help scouting your maize.”
• Easy message: “Are you ready to assess the extent of Fall Armyworm damage on your farm and receive advice? You will need to go to your shamba. A. Yes, B. No

To determine the effect of matching the narrator’s gender to that of the user on the rates of push-call pick-up and completion, we used content recordings of both male and female narrators in four languages, and randomly assigned users to the following three groups:

• Control (n = 2,002): No change in the narrator’s voice gender
  
• Treatment 1 (T1, n = 854): Switch IVR recording voice from not matching to matching the user’s gender
• Treatment 2 (T2, n = 1,146): Switch IVR recording voice from matching to not matching the user’s gender

The sample included 4,002 farmers in total (1,576 female and 2,426 male participants). Randomization was stratified by gender to ensure balanced representation across conditions. For the control group, the gender of the narrator’s voice remained the same before and after the introduction of the intervention. Each treatment group received weekly IVR content tailored to different implementer (HRNS or TNS) categories. T1 changed the narrator’s voice to match the gender of the user. Female (male) farmers who previously received IVR content voiced by a male (female) narrator began receiving content voiced by a female (male) narrator. T2 changed the narrator’s voice to no longer match the gender of the user. The matching or mismatching of the narrator’s and user’s genders allowed us to study the impact of the perceived expertise and/or socio-cultural dynamics on the outcomes of interest.

The content was delivered in four different languages—English, Luganda, Runyankore, and Rutooro—using both male and female narrators. Farmers could call back and still access the advice for free if they missed the initial calls or wished to listen to previous messages. The duration of the experiment was eight weeks, after which the narrator’s gender was no longer based on the user’s gender.

Treatment 1—(n = 1,999): Each recipient was asked first to enter their county as text, then to choose their constituency and ward from filtered lists. If their county was not classified, then they were asked to enter constituency and ward as text.

Treatment 2—starting with county (n = 1,999): Each recipient received text questions about county, constituency, and ward.

Treatment 3—starting with district, excluding ward (n = 2,000): Each recipient received text questions about district, division, location, and sublocation.

Treatment 4—starting with district, including ward (n = 2,000): Each recipient received text questions about district, division, ward, location, and sublocation.

The sample was stratified by treatment status and language of the original registration message, forming four groups of roughly equal size:

• Control & Swahili (n = 248)
• Control & English (n = 249)
• Treatment & Swahili (n = 248)
• Treatment & English (n = 249)

The treatment group received, as their first message, a question about their preferred language with a choice between English and Swahili. The control group was sent directly to the first question of the survey.

The quiz contained four questions covering key topics related to FAW: recognizing signs of FAW, using early planting as a preventive measure, destroying FAW caterpillars and egg masses, and using pesticides. This design enabled testing for both the overall impact of the language-selection prompt and any differential effects by language.

See more information on the experiment “The Effect of a Language Prompt on Farmers’ Platform Engagement” that this experiment builds on.

The first group (n = 1,538) was sent the following message, which implied that the location data would allow the service to send improved recommendations: “In order to make better recommendations, we would like to know where your farm is. What county is your farm in?”

The second group (n = 1,349) was sent the following message, which focused on the management of FAW: “To better fight Fall Armyworm, we would like to know where your farm is. What county is your farm in?”

The sample consisted of 17,873 MoA-INFO users, with randomization conducted at the individual level. Users were divided into a control group (N = 11,920), which received no reminder message, and a treatment group (N = 5,953), which was split into four sub-groups.

One of the treatment sub-groups (n = 2,982) received a general reminder, without specific content, encouraging the users to access the FAW menu. The other three sub-groups received topic-specific reminders about: detecting FAW (n = 994), managing FAW (n = 994), and the origins of FAW (n = 993).

This experiment tested whether prompting users to select a preferred language at the beginning of an Interactive Voice Response (IVR) call affects engagement. Participants were randomized into two groups:

• Treatment group (n = 48,312): Received a first message asking for their preferred language before proceeding to the content questions.
• Control group (n = 48,779): Did not receive the first message of language selection and proceeded directly to the registration survey content questions.

Randomization was conducted at the individual level. The final sample comprised 97,091 participants.

When users opted-in to a specific menu option of Pesticides, Managing FAW, Detecting FAW, or FAW Origins and Lifecycle, they were automatically enrolled in the experimental sample and had a 25% probability of being prompted to take a brief satisfaction survey. This survey asked the farmer two questions: (1) How useful is the information? (on a 1–5 scale, where 1 = not useful and 5 = very useful), and (2) Do you plan to follow the recommendations? (with response options of Yes, No, or Maybe).

To avoid survey fatigue and repeated exposure, a label was assigned to the user’s profile when a user completed the survey for a given topic, to prevent future survey prompts in the same topic. Users not selected into the survey prompt formed the control group, while those prompted formed the treatment group. Random assignment was implemented at the individual level upon topic selection.

OAF farmers were invited to register on the MoA-INFO platform in March 2019 at the beginning of the long rainy season in western Kenya. The invitations were sent to 296,252 farmers in 18,471 groups. Invited farmers were randomized into a control group and three treatment groups:

• C—No Push (Control): On completing the registration, farmers were introduced to the keywords MENU and CHECK; farmers were neither reminded of these keywords nor introduced to the FAW misconception messages (Keyword: TRUTH) and the FAW quiz (Keyword: QUIZ).
• T1—Info Push: Farmers received the control keywords after registration; farmers received push message reminders to access the menu, the misconception messages, and the FAW quiz.
• T2—Tool Push: Farmers received the control keywords after registration; farmers received push message reminders to monitor crops using the interactive monitoring tool (Keyword: CHECK).
• T3—All Push: Farmers received the control keywords after registration; farmers received all the push message reminders.

The randomization was conducted at the OAF farmer-group level. Farmers in the treatment groups received the various push messages from March to July 2019. After the SMS campaign was completed, OAF conducted a phone survey to gather information and feedback from a sub-sample of farmers, from July to September 2019. The survey sample was randomly drawn from farmers in the no-push group (C), info-push group (T1), and all-push group (T3).

The trial also included a series of four smaller A/B tests implemented during the season referred to as “Blast experiments” which can be viewed: Blast Experiments: Names, Active Language, Proverbs, & Quiz Scores

This experiment was designed as a follow-up to the 2018 Rwanda Lime Trial. The tentative finding from that trial was that the diverse-message groups were more likely to adopt lime than the same-message groups. In this follow-up experiment, we aimed to validate the 2018 result with greater statistical power. To do this, we randomized all OAF farmer groups for all the farmers in a group either to receive the same message (same-message group), or to receive different messages (diverse-message group). The randomization was stratified by district, and by whether the group had any 2019 lime adopters. In diverse-message groups, we randomized individuals to receive different messages, which were stratified by farmer group to ensure maximum diversity in each group. The messages were taken from the 2019 Rwanda Soil Health Trial, since these messages had been more thoroughly field tested than the 2018 messages had been. The messages were as follows:

M1: Basic message “Hi [Name], use travertine, fertilizer and compost on your fields this season to get a better harvest.”
M2: Feed your family “Hi [Name], use travertine, fertilizer and compost on your field this season. You’ll get better harvests to feed your family.”
M3: Social comparison “Hi [Name], some Tubura farmers have doubled their harvest by using travertine with fertilizer and compost.”
M4: House metaphor “Hi [Name], to get great harvests, you build your soils strength like you build a house. Compost is the foundation, travertine is the strong frame, and fertilizer is the roof.”

All messages ended with “Buy travertine and fertilizer from Tubura!”

During the OAF enrollment period for the first agricultural season of 2019 in Rwanda (“A season”), we implemented an SMS campaign encouraging the adoption of recommended soil-health inputs. To test aspects of the campaign, over 80,000 farmers were assigned treatments at both the group and individual levels.

First, farmer groups were randomly assigned at equal probability to one of the following arms:

• Same message within a group (G1): All message recipients in the group received identical messages; the message type was randomly assigned at the group level.
• Diverse messages within a group (G2): Farmers were randomly assigned a message type at the individual level.
• Control group (G0): No farmers received SMS messages.

Four types of messages were used:

• M1: Feed your family
• M2: Social comparison
• M3: Basic
• M4: House metaphor

Second, individual farmers in G1 and G2 were randomly assigned to one of the following groups:

• Low message volume (25%): Farmers received two messages in a six-day time window.
• High message volume (25%): Farmers received four messages in a six-day time window.
• Control (50%): No messages.

Note that this message campaign design is different from that of the 2018 trial, in which messages were separated by three to four days over a two-week period.

For further information on the previous trial, see Rwanda Agricultural Lime Trial 2018

To evaluate the effect of the outreach calls, we ran a clustered randomized control trial in which farmer groups were randomly assigned to one of the experimental arms in three stages. First, 75% of farmer groups were assigned to receive calls to farmers reminding them that they had not yet completed their loan repayment (treated farmer groups) and 25% were assigned to the control group. Second, half of the treated farmer groups were assigned to also receive calls to the GLs, with information about how many group members had not yet completed their loans. Third, farmer groups were independently randomized for sharing a farmer’s pledge with the OAF field team; 10% of groups were randomly selected to not have their farmers’ pledge information shared. This third component allowed us to isolate the impacts of the calls themselves from the possible impact of additional focus on treated clients by the OAF field team that might result from sharing the pledge.

Treatment groups were randomly ordered. Each week, the OAF team drew groups from the randomly ordered list, and farmers from these groups who fell below the healthy repayment path that week were called. In addition, after the initial deadline, GLs who had completed their own loan were also called if they had struggling clients in their group.

For further information on the previous trial, see Kenya Repayment 2018 Trial

We developed three different messages to encourage farmers to make the down payment:
M1: A “Pay soon” reminder to avoid being dropped from the program.
M2: A “Social” message that many other farmers had already qualified.
M3: A “Benefits” message to avoid missing out on OAF benefits.

Farmers were randomly assigned into a control group, which received one of three messages sent three times, and a treatment group, which received all three messages in random order.

Treatment Arms:
T0(a): Receive same message (M1); Share: 1/6; n (estimated) ≈ 40,911
T0(b): Receive same message (M2); Share: 1/6; n (estimated) ≈ 40,911
T0(c): Receive same message (M3); Share: 1/6; n (estimated) ≈ 40,911
T1: Receive all three messages in random order (M1–M3); Share: 1/2; n (estimated) ≈ 122,734

We randomized a list of farmers who had not prepaid by December 7, 2018. The sample included all farmers with unique global client IDs and phone numbers on the OAF database. Duplicates of these variables were dropped randomly, and group leaders were excluded from the sample. The total sample was 245,467 farmers.

For further information on the previous trial, see Kenya Down Payment Trial 2018

In February 2018, 291,000 farmers in over 40,000 farmer groups located across 1,500 sites (a site is a geographic division supervised by one program field officer) were block randomized into three equally sized experimental groups based on group size, site, and the percentage of group members that had already paid at the time of randomization. Groups were assigned to receive no reminders (C), reminders about individual repayment (T1), or reminders about individual and group repayment (T2). Of the farmers in the groups assigned to receive reminder messages (i.e., T1 and T2), two-thirds were randomly selected to receive individual treatment with direct messages (treated farmers).

OAF implemented two waves of message campaigns. In the first campaign, starting on July 5, messages were sent three times a week to treated farmers, to encourage full repayment by the early date of July 22. In the second campaign, weekly reminders about repayment were sent to treated farmers in the month preceding the official deadline, which was September 2. Throughout the experiment, only treated farmers received individual reminders. The only message untreated farmers could ever receive was a group reminder, and they would only receive one if they had completed repayment of their own loan and still had members in their group that had yet to finish repayment.

In order to estimate the effect of SMS messages on travertine (agricultural lime) adoption, we used a randomized controlled trial. The randomization had different treatment arms and they varied along different lines.

Group Randomization:
Farmer groups were randomized into four different categories:
1. A control group, G0: Farmers did not receive any text messages.
2. The same-message group, G1: Farmers belonging to the same group all received the same SMS messages with the same content.
3. The different-message group, G2: Farmers belonging to the same group all received messages with different content.
4. The half-variation group, G3: Half of the farmers of each group were randomly selected to receive messages that differed from individual to individual while the other half did not receive any messages. We used this treatment arm to measure spillovers between farmers who possess a phone and belong to the same treatment group.

SMS Variation:
Seven different SMS messages were designed and randomized at the group level in the case of G1 and at the individual level in the cases of G2 and G3. The messages were the following:

T1 Is a basic message: Many fields in Rwanda have acidic soil and need TRAVERTINE to increase yields. Order from TUBURA now.
T2 Gives specific information on dosage and impact: Many fields in Rwanda have acidic soil. Applying 25 kg/acre of TRAVERTINE will increase yields by 20%. Order from TUBURA now.
T3 Helps to self-diagnose the need for lime: Do you have fields with poor harvests even when you use fertilizer? You probably have acidity and need TRAVERTINE to increase yields. Order from TUBURA now.
T4 Helps to diagnose via a soil test: Ask your Field Officer for a free soil test to learn if your fields are acidic and you need to order TRAVERTINE to increase yields.
T5 Explains how lime works: Many fields in Rwanda have acidity, which blocks fertilizer uptake. Applying TRAVERTINE solves the problem, increasing crop yields. Order from TUBURA now.
T6 Promotes urgency and need: Many fields in Rwanda have acidic soil and need TRAVERTINE to increase yields. Order it immediately, when signing your TUBURA order form.
T7 Addresses location-targeted acidity: In your site the soil is acidic. If you apply 25 kg/are of TRAVERTINE you can boost yields by 20%. Order from TUBURA now.

Framing:
Furthermore, the framing of the message introduced additional variation. Individuals and groups were randomly assigned to receive a loss-framed or gain-framed message. The former highlighted that not using travertine amounted to loss, while the latter explained the possible gains derived from using travertine.

Frequency:
We also varied the frequency with which farmers received an SMS message. Farmers received a maximum of four messages. Each farmer always received the same SMS message.

Social Nudge:
Some individuals were assigned to receive an additional social nudge, such as an additional message encouraging the recipient to share the message with other farmers, especially those who do not have phones.

For further information, see Fabregas et al., 2024.

Farmers received either no message or one of six main treatments. There were several sub-treatments in several of these treatment arms.

Experimental Design:
Control: No SMS message; no framing; n = 90,000.
T1 (Basic): Basic SMS message; no framing; n = 22,500.
T2 (Benefits): Message mentions OAF benefits; framing in terms of general gain, general loss, family gain, or family loss; n = 11,250 for each type of framing (45,000 in total).
T3 (Early): Message to prepay early; framing in terms of gain, or loss; n = 11,250 for each type of framing (22,500 in total).
T4 (Small Amounts): Message to pay in small amounts; no framing; n = 22,500.
T5 (Social): Message with information on peers; framing in terms of basic information, OAF, site, or group; n = 11,250 for each type of framing (45,000 in total).
T6 (GL): Message to talk to group leader (GL); no framing; n = 22,500.

Three cross-randomizations tested the message framing:
1. Deadline: Farmers received messages about the deadline after December 15.
2. Amount Remaining: Farmers received information about the amount they had left to pay out of the total down payment amount.
3. Last Reminder: Farmers received a last message that informed them that the message was the final reminder.

The timing of the messages was randomized independently.

There were four main treatments for message timing:
1. Messages every week for 6 weeks.
2. Messages in weeks 1, 3, and 6.
3. Messages in weeks 4, 5, and 6.
4. Messages in weeks 1, 5, and 6.

During the campaign registration period (August–October 2018) FPs were tasked with both mobilizing farmers in their village to register for the tree-distribution campaign and ensuring that registered farmers arrived on the distribution day of the campaign. OAF, in collaboration with PxD, implemented an SMS campaign to nudge FPs and SEDOs to register farmers for this campaign. This experiment tested the relative effects of messaging FPs directly to remind them about the campaign, messaging SEDOs about the campaign, or messaging both the relevant SEDO and the FP for the village in question. Using the randomly assigned variation in which the FPs and SEDOs received the messages, we identified the effects of SMS nudges on FP performance by the share of their target number of farmers who arrived on the tree-distribution day.

Our sample was drawn from a subset of the villages receiving trees during the 2018 tree-distribution campaign. We randomly drew 522 of the cells in the government database of SEDOs to form the sample frame. There were a total of 4,590 villages in the chosen cells and 1,418 of these villages had FPs whose phone numbers were not in the database. We designated these FPs as “no phone FPs” and included them in all appropriate analyses. The remaining 3,172 villages with FPs whose phone numbers were in the government database formed the sample of FPs we randomized.

We randomly assigned cells (and therefore SEDOs) into one of three experimental arms:
1. Control arm: The SEDO responsible for the cell did not receive any messages.
2. Nudge arm: The SEDO received reminder messages to get FPs in their cell to register farmers for the tree campaign.
3. Nudge + target arm: The SEDO received reminder messages to get FPs in their cell to register farmers for the tree campaign and these messages also included explicit mention of the target number of farmers to be registered by FPs in that SEDO’s cell.

We cross-randomized the 3,172 FPs who had phone numbers in the government database into one of two groups.
1. Treatment group: Received motivational messages that encouraged the FP to register farmers for the campaign.
2. Control group: Did not receive those motivational messages.

Randomization was not stratified by cell treatment status.
Message recipients received a single message per week during the mobilization phase of the campaign until their distribution day.

Camps were randomly assigned to either treatment (n = 105) or control (n = 68) groups from a list of high-FAW-risk areas, stratified by province and camp size. We sent messages to all the farmers in the treatment camps who were registered as FISP beneficiaries for the 2019–2020 agricultural season.

From December 2019 to February 2020, we sent a total of 39 messages to 86,244 farmers in treatment camps across four provinces (Central, Eastern, Southern, and Luapula). Messages covered FAW monitoring, cultural pest control practices, preventative methods, and correct use of pesticides and fertilizers, and were delivered in Bemba, Nyanja, or Tonga languages, based on camp-level language assignments that the Ministry of Agriculture provided.

The survey sample comprised the 3,028 FISP-registered farmers on the ZIAMIS platform who were reached via phone and consented to participate in the survey. We asked farmers a maximum of four questions about their actions, depending on whether farmers spotted FAW and used pesticides.

The sample of 386,467 farmers, from 53,891 OAF farmer groups, was randomized into treatment (80%) or control (20%). Treated farmers received two messages, one about zinc-fortified beans (KMR 13) and one about vegetables, each week for three weeks. Messages were varied in two dimensions.

First, we randomly assigned farmers to receive either identical messages about each crop over three weeks or differently framed messages about each crop emphasizing, for example, anemia prevention, availability of training, or benefits for children during the message campaign. Second, we randomized message variation at the group level: In some groups, each farmer in a group received the identical message in any given week; in other farmer groups, the message content was randomized across members. This created four experimental groups.

At the individual level, we tested which is more effective: sending different messages or focusing on reiterating and reinforcing the same content by repeating the same message. Additionally, we used a phone survey to measure whether message variation within groups led to information-sharing and discussion about the recommended products between group members, and ultimately to adoption of input from OAF. Lastly, in order to increase conversation within groups and increase the likelihood that farmers in different-message groups would realize that their group members had received different content, we sent a social-nudge message to half of all treated farmer groups. The message encouraged farmers to share the messages with their group members, and we evaluated its effect.

The message texts were as follows:

Beans 1 Nutrition: “Hello [Name]! FAIDA (NUTRITIOUS) BEANS have important vitamins and minerals for everyone’s health. Try 1/4 acre from OAF this season for only KES 1250!”
Beans 2 Trainings: “Hello [Name]! Order FAIDA (NUTRITIOUS) BEANS from OAF, and we will train you how to get good yields! Try 1/4 acre this season for only KES 1250!”
Beans 3 Agronomic: “Hello [Name]! FAIDA (NUTRITIOUS) BEANS are suitable for your area, mature in 80–85 days and yield 6–13 bags per acre! Try 1/4 acre from OAF for KES 1250!

Vegetables 1 Nutrition: “Hi [Name]! Many Kenyans are at risk of anemia! OAF is offering sukuma, spinach, saga, and managu, which prevent anemia and add strength, for a low price.”
Vegetables 2 Trainings: “Hi [Name]! OAF is offering sukuma, spinach, saga, and managu. We have certified seeds and will train you on these crops so you get good yields!”
Vegetables 3 Children: “Hi [Name]! Children who eat leafy vegetables grow strong and perform better at school! OAF is offering sukuma, spinach, saga, and managu for a low price!”

Farmers were randomly assigned to three treatment arms at the WUA level:

T1 “Inbound only” (n = 99 WUAs): Farmers had access to the inbound service only, meaning that they could call into the Krishi Katha service, but did not receive the weekly advisory push calls. This group was the control group.

T2 “Inbound and outbound” (n = 175 WUAs): Farmers had access to the inbound and outbound services, meaning they could call into the Krishi Katha service and also received weekly advisory push calls.

T3 “Inbound and outbound + Intensive training” (n = 178 WUAs): In addition to the inbound and outbound services, farmers received a bundle of intensive training and encouragement interventions including:

• In-person training from PxD field staff to demonstrate the features of Krishi Katha service and to train farmers to use it. (107 WUAs)
• Remote training from PxD call center staff to explain the features of Krishi Katha service and how to use it. (178 WUAs)
• Training reinforcement SMS message two to three days after field and remote training, to remind farmers about the training and to encourage them to use the Krishi Katha service. (178 WUAs)
• Important questions asked by farmers, curated by PxD agronomists and broadcast as additional voice calls to all T3 farmers in their respective blocks every week for 13 weeks. (178 WUAs)
• Endorsement messages recorded by Presidents/Vice Presidents/Secretaries of participating WUAs to build the farmers’ trust in the Krishi Katha service. (87 WUAs)

We used PxD administrative data to measure farmers’ pick-up and listening rates for the outbound service and farmers’ access to features in the inbound service from July 1, 2019 to November 27, 2019. We conducted a baseline survey of 1,067 farmers prior to the intervention to measure their initial knowledge and adoption levels, and then an endline survey of 1,739 farmers in December 2019 to measure their knowledge and adoption rates.

In 2021, we used a random walk approach to recruit 13,675 rice farmers, comprising 5,204 Cohort 1 farmers and 8,471 Cohort 2 farmers, and conducted an in-person baseline survey with them. Recruited farmers were randomly assigned to the treatment or control group with equal probability. Farmers in the treatment group were provided access to the Ama Krushi service; farmers in the control group were not. Cohort 1 and 2 farmers were invited to register for the service at the beginning of the Kharif season of 2021 and 2022, respectively. Cohort 1 farmers were followed up twice, at the end of the Kharif seasons of 2021 and 2022, to collect farmers’ self-reported data on agricultural knowledge, practice adoption, rice yield, harvest sales, cultivation costs, and crop losses.

We used VIs to construct an objective yield measure for all farmers in the study sample.

Read full working paper: Cole S., Goldberg J., Harigaya T., Zhu J. (2025)
See also: PxD 2024 Annual Report feature

8028 hotline users (n = 160,000) were randomly assigned to either the control group or the treatment group. If a user did not make a selection within 10 seconds of the menu options being played, then:

Control group: The system automatically hung up (status quo).
Treatment group: The menu was automatically replayed.

We used administrative platform data to measure the user-engagement outcomes of language selection and content access.

The experiment was implemented for one month from November 29 to December 31, 2017.

Study publication forthcoming.