| Experiment (select + to view abstract) | Abstract | Start date | Location | Research tags | Status | Experiment Type | ||
|---|---|---|---|---|---|---|---|---|
| PaddyAI–Artificial Intelligence versus Human Engagement: A Comparison for IVR Advisories | PxD operates the Coffee Krishi Taranga (CKT) platform in collaboration with the Coffee Board of India to provide a voice-based advisory service for coffee farmers through a two-way Interactive Voice Response (IVR) system. We have developed an AI-based chatbot feature, PaddyAI, using large language models (LLMs) to generate deeply customized and localized advisory on demand. This A/B test is the first in a series of service improvement experiments as PxD integrates AI-enabled IVR advisories through PaddyAI. The objectives of this A/B test are to understand (1) whether AI-generated advisory content has similar user engagement compared to human-generated advisory content, and (2) whether an AI voice has similar user engagement compared to a human voice delivering the content. Findings from this A/B test will help improve the generation of advisory and the text-to-speech processes of PaddyAI. | 2025 | India, Karnataka, India | Coffee Board of India | Agricultural management advice, Artificial intelligence (AI), Message narration, Service design | Ongoing | _N/A | A/B test |
| Leaf Color Chart Distribution Pilot | Farmers in low and middle-income countries face multiple barriers to optimizing agricultural decisions and often don’t have access to actionable information to adapt their practices. These challenges are particularly salient in fertilizer application: Returns to fertilizer are highly heterogeneous, which makes it difficult for farmers to observe the effectiveness of their actions. With large fertilizer subsidies in South Asia, many farmers overuse nitrogen fertilizers. A Leaf Color Chart (LCC), a simple plastic strip with graduated green shades, provides both real-time estimates of a crop’s nitrogen status and rule-of-thumb guidance on correct quantities of urea fertilizer to apply to address the crop’s need for nitrogen. We piloted a light-touch intervention to encourage the use of LCCs by cotton farmers in Gujarat, India. The pilot experiment was designed to generate suggestive insights on farmers’ interests and the barriers to using LCCs, and to elicit the farmers’ feedback. We randomly assigned 32 villages to four distribution channels, namely PxD, a non-governmental organization (NGO) partner, agro-dealers, and peer farmers, and distributed LCCs to a total of 418 randomly selected farmers, who all received a context-specific instruction booklet and digital advisory. We saw wide use of the tool: More than half of the cotton farmers who received an LCC reported using the tool. Distribution by agro-dealers and by peer farmers were the most promising distribution channels for LCC adoption and accurate recall of LCC usage instructions, relative to the benchmark in-person distribution by PxD staff. Preliminary findings suggest that, on average, the farmers who received LCCs reported applying 35% less nitrogen fertilizer and harvesting 11% more than those who did not. | 2022 | Gujarat, India, India | _N/A | Agricultural management advice, Input recommendations, Service design, Soil fertility | Completed | Kharif | Other |
| Assessing the Impact of Leaf Color Charts on Cotton Farmers | Farmers in low- and middle-income countries face multiple barriers to optimizing agricultural decisions, and often lack access to actionable information to adapt their practices. These challenges are particularly salient in fertilizer application: Returns to fertilizer application are highly heterogeneous, which makes it difficult for farmers to observe the effectiveness of their actions. With large fertilizer subsidies in South Asia, many farmers overuse nitrogen fertilizers. A Leaf Color Chart (LCC), a simple plastic strip with graduated green shades, provides both real-time estimates of a crop’s nitrogen status and rule-of-thumb guidance on the correct timing and quantities of urea fertilizer application. We evaluate whether access to LCCs, paired with in-person training and digital advisory messages throughout the season, can improve nitrogen management for cotton farmers in Maharashtra, India, and examine how alternative financial incentive designs affect LCC adoption. Using a random-walk sampling method, we recruited 1,253 cotton farmers across 60 villages during Kharif 2024 and individually randomized them (stratified by block) into five arms: control; LCC + training + digital advisory; and three variants that add small financial incentives (~$5/acre) delivered either ex-post conditional on use, ex-ante unconditional, or ex-ante conditional on use. To evaluate impacts, we collected both objective nitrogen-use-efficiency measures using mid-season chlorophyll meter checks, and farmers’ self-reported use of fertilizer and cotton yield using an endline survey, to analyze the outcomes of LCC adoption, nitrogen application behavior, and cotton yield. | 2024 | India, Maharashtra | J-PAL South Asia | Agricultural management advice, Input recommendations, Service design, Soil fertility | Completed | Kharif | Impact Evaluation |
| Piloting the Distribution of Stress-tolerant Seeds to Agro-dealers | Climate change is increasing smallholder exposure to weather shocks, which makes scalable delivery of climate-adaptation technologies a policy priority. Input suppliers, who are farmers’ trusted source of information, often lack incentives to introduce new technologies rapidly, or they avoid taking this risk, without intervention. Building on a study (Dar et al., 2024), which found over 50% higher adoption rates from in-person free trial-seed delivery to private agro-dealers when compared to spreading information from government agents to farmers, our objective was to refine a free trial-seed distribution model to use fewer in-person components than Dar et al. used, so that policymakers can facilitate seed distribution at scale using existing seed distribution logistics. Building on work in a pilot in Rabi 2024, PxD partnered with Krushisharang, a private seed company in Gujarat, to pilot low-cost approaches to distributing free trial-seed bags for a high-yielding groundnut variety resistant to pests and diseases, Girnar 4, in Kharif 2025. We randomized 12 blocks into three groups that varied the delivery models of free trial-seed bags to local agro-dealers: (1) direct delivery with implementer-led recruitment of local recipient dealers; (2) collection-based hub-and-spoke distribution via the distributor-affiliated dealer, also with implementer-led recruitment; and (3) dealer-directed distribution at the affiliated dealer’s discretion. Results indicate that implementer-led recruitment (direct-delivery and hub-and-spoke models) achieved full or near-full distribution to targeted agro-dealers. By contrast, dealer-directed distribution produced minimal reach beyond distributor-affiliated dealers. | 2025 | Gujarat, India | Krushisharang Agriclinic Pvt. Ltd. & Fule Seeds | Input markets, Input recommendations, Service design | Completed | Kharif | Other |
| Monsoon Onset Forecast Dissemination to Farmers in India–Kharif 2025 | Climate change is increasing variability in the timing of agricultural growing seasons, thus creating significant challenges for farmers planning their agricultural decisions. Building on a successful pilot “Weather Forecast Dissemination 2024” in 2024 when PxD sent monsoon onset and total rainfall forecasts to 9.45 million farmers across 5 states, in 2025 PxD partnered with the Ministry of Agriculture and Farmers Welfare (MoAFW), the Department of Agriculture and Farmers’ Empowerment of Odisha (DAFE), Human-Centered Weather Forecasts Initiative (HCWF), and the Development Innovation Lab India (DIL–India) at the University of Chicago to scale up this initiative and deliver agriculturally relevant monsoon-onset forecasts to the mobile phones of farmers across India. Approximately 38 million farmers across 13 states received AI-driven monsoon onset forecasts, produced by the HCWF team and disseminated by MoAFW via SMS, two to four weeks in advance of the monsoon onset in May and June 2025, to support agricultural decision-making for the Kharif season. In Odisha, we randomized the roll-out of forecast delivery such that farmers in some blocks received forecasts via both Interactive Voice Response (IVR), disseminated by DAFE through the Krushi Samruddhi program, and SMS, while farmers in other blocks received forecasts only via SMS. This allowed for a comparison of outcomes between IVR+SMS and SMS-only blocks. We examined whether farmers recalled receiving the forecasts, listened to IVR messages, comprehended the forecast content, utilized the forecast information in agricultural decision-making, and adjusted planting choices (including crop/variety selection, area, timing, or replanting), and assessed the impact on crop outcomes. | 2025 | India, Odisha, India | Development Innovation Lab - University of Chicago, Government of Odisha, Human Centered Weather Forecasts Initiative, India Ministry of Agriculture | Message framing, Service design, Weather information | Completed | Kharif | Other |
| Blast Experiments: Names, Active Language, Proverbs, & Quiz Scores | One Acre Fund (OAF) is an agricultural service provider that provides support for smallholder farmers in Africa in accessing agricultural inputs, training, and markets, to help the farmers increase their harvests and income. PxD and OAF began collaborating in 2016 on efforts to increase adoption of agricultural inputs and improve OAF operations in Kenya and Rwanda. PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. During the long rainy season 2019 (LR 2019), we sent regular push messages to OAF farmers encouraging them to access the Fall Armyworm (FAW) components of the MoA-INFO platform, namely the FAW menu of informational topics, the FAW monitoring tool, and the FAW misconception quiz. While sending these messages in the “OAF Kenya Fall Armyworm 2019” trial, we implemented a series of four A/B tests we called “Blast Experiments” to test the effects of message design tweaks on farmer access to the FAW components of the MoA-INFO platform. 1. Names: We tested the effect of including users’ first names in invitation messages on farmers’ access to the FAW components. The intervention increased access to both the FAW menu and the FAW misconception quiz by 3 percentage points (pp) over 19% in the control group, but did not have a statistically significant effect on access to or completion of the FAW monitoring tool. 2. Active language: We tested the relative effects of language, with active “now” framing compared to “at any time” framing, on farmer access to the FAW components. Using “send MENU… now” messages increased access to the menu by 3 pp over the 12% access rate of the “send MENU… at any time” messages. 3. Proverbs: We tested the effect of including one of various Swahili proverbs at the beginning or at the end of an invitation message on farmer access to the FAW components. Inserting a Swahili proverb at the start of an invitation message increased access to the FAW menu, but had no effect when the same proverb was placed at the end of the message. 4. Quiz scores: We tested the effect of including farmers’ prior best FAW-quiz score in invitation messages on their access to the FAW misconception quiz. The intervention increased access rates by 11 pp over 57% in the control group, which indicates that performance-based framing can meaningfully boost re-engagement. | 2019 | Kenya | Kenya Ministry of Agriculture, One Acre Fund (OAF) | Communication technology | Completed | Long Rains | A/B test |
| Lab-in-the-field and Weather Forecast Service Experiment with Coffee Farmers | PxD operates the Coffee Krishi Taranga (CKT) platform in collaboration with the Coffee Board of India to provide a voice-based advisory service for coffee farmers through a two-way Interactive Voice Response (IVR) system. We partnered with the Climate Forecast Applications Network (CFAN) to develop highly localized, 5-day rainfall forecasts tailored to the contexts of coffee farmers in Karnataka. Farmers face significant productivity risks from weather variability, and these risks are amplified by climate change. We conducted an experiment with coffee farmers registered with CKT in Karnataka to examine how short-to-medium-range rainfall forecasts could support farmers’ decision-making, conditional on their ability to accurately interpret, trust, and act on forecasts. The study examined: how farmers form beliefs about weather and forecast accuracy as they observe repeated forecasts and outcomes; whether light-touch informational treatments improve farmers’ understanding of probabilities; and how climate vulnerability influences farmers’ beliefs about weather and forecast accuracy. Using lab-in-the-field (LIF) and real-world IVR experiments, we exposed farmers to weather forecasts and light-touch informational treatments. While farmers had a high demand for forecast services, their trust in forecasts decreased after they received erroneous forecast predictions, which led to a decrease in the frequency of farmers’ use of the service. Accuracy in initial forecast delivery mitigated this effect, which highlights the importance of early successes for building long-term trust in a new service feature. When climate change was made salient, farmers were more likely to use forecasts and were more tolerant of forecast errors, which underscores the value of forecasts in climate adaptation. | 2023 | India, Karnataka, India | CFAN, Coffee Board of India | Agricultural management advice, Message framing, Service design, Weather information | Completed | _Multiple seasons | Other |
| Asset Collateralized Loans: Non-linear Repayments | Rainfall and temperature variability induced by climate change pose a substantial economic risk to smallholder dairy farmers in developing countries. Rainwater harvesting tanks may help farmers adapt to climate uncertainty. Previous work has found that Asset Collateralized Loans (ACLs) helped farmers purchase water tanks in Kenya (Jack et al., 2023). This study builds on previous work and evaluates the impact of ACLs for water tanks on economic and household outcomes with two dairy cooperatives in Rift Valley, Kenya. The study also tested an alternative ACL design in which required monthly payments were a function of milk income, which may be better adapted to farmers with seasonally variable income. | 2022 | Kenya | Development Innovation Lab - University of Chicago, LEPESA SACCO, Lessos Dairy Cooperative, Sirikwa Dairy Cooperative | Agricultural management advice, Input recommendations, Service design, Social learning | Ongoing | _Multiple seasons | Impact Evaluation |
| The Effect of the Full Coffee Krishi Taranga Outbound Advisory Service on the Adoption of Recommended Practices | PxD operates the Coffee Krishi Taranga (CKT) platform in collaboration with the Coffee Board of India to provide a voice-based advisory service for coffee farmers through a two-way Interactive Voice Response (IVR) system. Coffee has been increasingly affected by climate variability; this trend emphasizes the need for timely agronomic information to support farmers’ decision-making. We evaluated the impact of CKT advisories on the adoption of priority coffee management practices, namely composting and white stem borer (WSB) management, by smallholder farmers in Andhra Pradesh. We examined whether access to the full CKT outbound service leads to an increase in the adoption of recommended coffee practices. All 2,457 registered farmers in the blocks of Ananthagiri, Chinthapalli, Araku Valley, Paderu, G. Madugula, and Hukumpeta were included in the sample, stratified by block and prior CKT experience, and then randomized into two groups. The control group was provided access to the CKT hotline only, while the treatment group was provided access to outbound voice-call advisories and SMS dissemination, in addition to the hotline. Results are awaited. | 2024 | India | Coffee Board of India | Agricultural management advice, Service design | Ongoing | _Multiple seasons | A/B test |
| The Influence of Bayer-branded GeoPotato Alerts on Farmers’ Behavior and Perceptions | mPower is a Bangladesh-based social enterprise that specializes in technology-based development solutions. As part of its mobile agriculture program, Agro360, mPower sends text messages with crop management recommendations and weather alerts to farmers to improve the efficiency of their crops’ cultivation and promote practices that mitigate the impact of climate change on their crops. PxD and mPower partnered with Bayer, a commercial input supplier, to test branded SMS alerts in the GeoPotato advisory service provided to potato farmers in Bangladesh. The partnership aimed to explore sustainable models for delivering digital advisory services, by testing how branded content affects farmers’ behavior and perceptions. This trial assessed whether Bayer-branded GeoPotato alerts influence farmers’ recognition of the brand, use of fungicide, and trust in the GeoPotato service, as well as farmers’ willingness to adopt recommendations. Farmers were randomly assigned to receive either standard alerts or alerts that included Bayer branding. The study measured outcomes related to brand recall, fungicide use, and trust in the service. Branded messages led to an 18% increase in the use of Bayer products, without undermining farmers’ trust in the GeoPotato service or their willingness to adopt its recommendations. However, the branded alerts did not have a significant impact on farmers’ ability to recall the Bayer brand or recognize its products. These findings support the potential of NGO partnerships with commercial input suppliers to sustain digital advisory services while boosting their product-uptake. | 2018 | Bangladesh | mPower | Extension agents, Input markets, Social learning | Completed | _N/A | A/B test |
| Motivational Messages for Extension Workers in Rwanda Tree Program 2022 | One Acre Fund (OAF) is an agricultural service provider that provides support for smallholder farmers in Africa in accessing agricultural inputs, training, and markets, to help the farmers increase their harvests and income. PxD and OAF began collaborating in 2016 on efforts to increase adoption of agricultural inputs and improve OAF operations in Kenya and Rwanda. During the 2022A agricultural season, OAF and PxD collaborated on a digital messaging project that targeted volunteer extension agents known as farmer promoters (FPs) in Rwanda, following a previous project “Motivational Messages for Extension Workers in Rwanda Tree Program 2018”. We examined whether SMS messages could improve FP performance, using two approaches: goal reminders, and personality-tailored motivational messages. Using a randomized controlled trial with 10,187 FP volunteer extension workers, we found that goal reminders sent during the regular working period increased a productivity index by 0.08 standard deviations, while reminders sent late had no effect. By contrast, personality-tailored motivational messages showed no performance improvements. These results suggest that workers who set ambitious yet realistic goals are more likely to respond to reminders and increase productivity, which is consistent with prior evidence on externally set goals. Given the extremely low cost of sending SMS reminders (US$0.005 per message sent), digital goal reminders can be a highly cost-effective way to strengthen extension services. | 2021 | Rwanda | One Acre Fund (OAF) | Communication technology, Extension agents, Input recommendations, Message framing | Completed | Season A | Impact Evaluation |
| PRISE Pesticide Application Campaign LR 2021 | The Pest Risk Information Service (PRISE) is an early warning model, run by the Centre for Agriculture and Bioscience International (CABI), that predicts the optimal timing of pesticide application. PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. CABI partnered with PxD to send SMS messages to farmers to help them make better decisions about pesticide application when confronted with pests in their fields. Linking this information to their planting date gives farmers early warning to act against the pests. This trial is a replication of a trial run in Kenya’s 2020 short rainy season (SR 2020) “PRISE Pesticide Application Campaign SR 2020”. The intervention in this experiment targeted maize, bean, and tomato farmers. The timing of fortnightly SMS messages was linked to pest-risk forecasts and planting dates during the long rainy season 2021 (LR 2021). The objective was to disseminate timely advice on pesticide application and evaluate the effect of the advice on farmers’ engagement with the MoA-INFO platform and on their pest management decisions. Consistent with the results from the SR 2020 trial, we find that the PRISE message campaign increased MoA-INFO menu browsing rates across all crops, thus confirming that push messages drive platform engagement. However, we observe slightly higher opt-out rates in treatment groups, which suggests that some users may experience SMS fatigue. Farmers generally perceived the PRISE recommendations as accurate, but cited their limited resources as a barrier to acting on the advice. | 2021 | Kenya | CABI, Kenya Ministry of Agriculture | Pest management | Completed | Long Rains | A/B test |
| Odisha Ama Krushi Call-time Customization | PxD operates Ama Krushi, a free agriculture information service delivered over mobile phones, in collaboration with the State Government of Odisha Department of Agriculture, using a two-way Interactive Voice Response (IVR) platform with “outbound” push calls and an “inbound” hotline service. This study evaluated whether personalizing the timing of information delivery can increase user engagement with IVR content. Specifically, we tested a targeted delivery strategy where the timing of advisory calls was personalized based on a machine learning model trained on farmers’ past engagement patterns and characteristics; the targeted delivery was compared to a random delivery schedule. By combining predictive modeling with a randomized controlled trial, this study aimed to generate actionable insights on the role of service customization in improving farmer engagement with digital extension services. We find that the targeted timing strategy increased pick-up rates by 1.7 percentage points (pp) overall, with a larger 3.2 pp gain by low-engagement users. These results highlight the potential of data-driven customization to enhance farmer engagement with digital advisory services. | 2021 | Odisha, India | Golub Capital Social Impact Lab, Government of Odisha | Communication technology, Message timing and frequency | Completed | Kharif | A/B test |
| MoA-INFO LR 2021 Registration: Crops First or Names First? | PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. In this A/B test, we tested whether a redesigned registration process inviting users to opt-in to the crop advisory message series first, before asking registration questions, increased farmers’ registration completion and platform-engagement behavior. The treatment group received crop opt-in messages first; the control group received name registration messages first. Compared with the control group, the treatment group had a higher rate of crop series opt-ins but provided less profile information (e.g., name, location) and engaged slightly less with the platform after registration. Overall, switching the registration order improved specific registration metrics but reduced others, which indicates that there are trade-offs in registration design choices. | 2021 | Kenya | Kenya Ministry of Agriculture | Message framing | Completed | Long Rains | A/B test |
| Kenya Early Discounted Bundle Trial | One Acre Fund (OAF) is an agricultural service provider that provides support for smallholder farmers in Africa in accessing agricultural inputs, training, and markets, to help the farmers increase their harvests and income. PxD and OAF began collaborating in 2016 on efforts to increase adoption of agricultural inputs and improve OAF operations in Kenya and Rwanda. We conducted an Early Discounted Bundle (EDB) trial with OAF in Kenya during the 2021 long rainy season to test whether digital interventions alone could replicate the input adoption effects found by Duflo et al. (2011), particularly in contexts where farmers already had high familiarity with recommended inputs. The intervention targeted past OAF Duka shop clients and former core program members with an SMS campaign promoting hybrid maize seeds combined with fertilizers (DAP or CAN); the campaign offered time-limited digital discount coupons for bundled purchases shortly after the 2020 short rainy season harvest. A bundled input package is designed to ensure that farmers purchase the three complementary inputs in appropriate quantities for their land size. While the trial did not increase actual bundle sales, it significantly boosted individual farmers’ input purchases and Duka shop engagement (they were 11.5–16.4% more likely to visit the shops) and moved purchase timing 2–6 days earlier. Treatment effects were strongest for maize hybrid seeds (16.2–19.1% increase) and DAP planting fertilizer (15.7–18% increase). We compared these findings to OAF's other existing discount program, Super Saver Discount (SSD), and found that in-person promotional strategies achieved higher bundle uptake than the digital-only approach. This finding suggests that, while digital-only campaigns are more cost-effective, in-person extension plays an important role in closing the adoption gap. | 2021 | Kenya | One Acre Fund (OAF) | Communication technology, Input recommendations, Message framing | Completed | Long Rains | Impact Evaluation |
| Automated versus Live Human Follow-up Calls | PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. To identify strategies for boosting farmers’ engagement with the service, we tested the effectiveness of different follow-up modalities (receiving an automated call or a live human call) for users who initiated the registration process. Farmers who received blast invitations from Safaricom (a large Kenyan phone company) and started registration on the MoA-INFO platform in the long rainy season 2021 (LR 2021) were randomly assigned to a control group, or to two treatment groups that received either an automated follow-up call or a live human follow-up call. Receiving an automated Interactive Voice Response (IVR) call is associated with a 7.6 percentage point (pp) increase in the probability of accessing the menu compared to control farmers. However, live human calls seem to have an impact on more measures of platform engagement than automated calls do. Therefore, having an initial human contact following registration may drive more engagement by farmers than having an automated call does. | 2021 | Kenya | Kenya Ministry of Agriculture | Communication technology | Completed | Short Rains | A/B test |
| The Order and Designation of the Prompts of a Rating Scale | PxD operates Ama Krushi, a free agriculture information service delivered over mobile phones, in collaboration with the State Government of Odisha Department of Agriculture using a two-way Interactive Voice Response (IVR) platform with “outbound” push calls and an “inbound” hotline service. The outbound service is the primary way in which PxD provides farmers with timely, customized agricultural information. Farmers’ ratings of how useful they find the information provide important feedback for PxD to implement improvements to the service. Different monitoring and evaluation efforts, such as polling surveys and rating scales, have yielded different results regarding the usefulness of the advisory. This variation may reflect the differences in the feedback channels. Of farmers who received advisory messages in a previous six-month period, 26% rated at least one message on a Likert scale ordered from 5 (very useful) to 1 (not useful), which was played at the end of the weekly push call. The average rating of the service during that six-month period was 4.2. We conducted a test involving individual-level randomization of over 1 million paddy farmers. Different versions of an IVR-message rating scale were tested by comparing reversed-order and reversed-designation prompts to the default prompts. The results indicate that the ratings farmers provide are substantially influenced by the order the rating options are presented and the values assigned to them. | 2021 | India, Odisha, India | Government of Odisha | Communication technology, Message framing | Completed | Kharif | A/B test |
| Narrator’s Gender A/B Test, Nigeria | PxD delivers free agricultural advisory services to Rural Poor Stimulus Facility (RPSF) recipients in Nigeria through automated voice calls, in partnership with the Federal Ministry of Agriculture & Rural Development (FMARD). The project provided agronomic advice in the 2021 dry and wet seasons to 107,542 smallholder farmers, in the form of push calls on their mobile phones. With the aim of improving pick-up and call completion rates, we conducted an A/B test to identify the gender of the narrator that is most effective in driving farmer engagement with the push-call content. The messages were recorded by both female and male narrators. Farmers were randomly assigned to two treatment groups (T1 and T2). T1 received advisory messages recorded by a male narrator, while T2 received messages recorded by a female narrator. Narrator gender influenced listening behavior. In week 2, female farmers were 3.8 percentage points (pp) more likely to pick up calls when messages were voiced by a female narrator (over a 46.5% control mean), while no such effect was observed in male farmers. Conversely, male farmers who received messages from a female narrator listened to 1.39% more of the content on average (over a 51.8% control mean), with statistically significant improvements observed in four of the seven weeks. However, narrator gender had no effect on how much of the content female farmers listened to. | 2021 | Nigeria | International Fund for Agricultural Development | Communication technology, Gender, Message narration | Completed | Wet Season | A/B test |
| Kenya Planting Dates Trial | One Acre Fund (OAF) is an agricultural service provider that provides support for smallholder farmers in Africa in accessing agricultural inputs, training, and markets, to help the farmers increase their harvests and income. PxD and OAF began collaborating in 2016 on efforts to increase adoption of agricultural inputs and improve OAF operations in Kenya and Rwanda. Planting at the optimal time has the potential to increase a farmer’s yield, especially for late planters, who plant after the optimal date. This trial tested whether sending farmers an SMS message prompting them to plant at the optimal time increases their likelihood of planting at this time. We also tested whether sending an additional SMS message with a personal endorsement from a trusted source and/or addressing farmers’ misconceptions about the optimal planting times increases the effectiveness of the basic SMS message. Sending farmers planting date recommendations via SMS brings their planting dates closer, on average, to the optimal date. The effect is higher for late planters; late planters in the treatment groups planted on average four days closer to the optimal date compared to late planters in the control group. While we do not find significant effects on the proportion of farmers who planted in the optimal time window, we do find some evidence that the intervention increases the likelihood of late planters planting within five days of the optimal date. Additional SMS messages intended to boost effectiveness of the basic message did not have the intended effect. We do not find any presence of spillover effects. | 2021 | Kenya | One Acre Fund (OAF) | Agricultural management advice | Completed | Long Rains | Impact Evaluation |
| The Impact of Heads-up SMS Messages on Call Pick-up and Listening Rates | PxD operates Ama Krushi, a free agriculture information service delivered over mobile phones, in collaboration with the State Government of Odisha Department of Agriculture, using a two-way Interactive Voice Response (IVR) platform with “outbound” push calls and an “inbound” hotline service. The outbound service is the primary way in which PxD provides farmers with timely, customized agricultural information. We wanted to increase farmers’ engagement with the outbound service, in terms of pick-up and listening rates. We explored whether sending a “heads-up” SMS message to farmers increases their likelihood of picking up and listening to calls. We find that these reminders, in particular a reminder 10 minutes before the call, increased the pick-up and listening rates. Non-smartphone users and users with above-median historical listening rates show larger gains in both pick-up and listening outcomes than smartphone users and less historically engaged users do. | 2021 | Odisha, India | Government of Odisha | Communication technology, Message timing and frequency | Completed | Kharif | A/B test |
| Framing of a Filter to Engage with the Seed Selector Tool | PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. The platform includes a Seed Selector Tool (SST) that allows farmers to learn about maize seed varieties that are best suited to their agro-ecological zones, by browsing a list of locally suitable, certified varieties. We implemented a new feature allowing SST users to directly browse the three top-yielding varieties in their agro-ecological zone. We conducted this experiment with over 160,000 users in the 2021 long rainy season (LR 2021) to test two different framings to label this new filter in the SST: “top-yielding seed varieties” and “expert recommended”. The objective of this experiment was to assess which framing leads to greater engagement with the SST, but we found no clear difference between the two framing options. The expert-recommended label slightly increased the likelihood of users completing the first step, but no significant differences were observed in the subsequent steps to complete their use of the tool. | 2021 | Kenya | J-PAL, Kenya Ministry of Agriculture | Communication technology, Message framing | Completed | Long Rains | A/B test |
| Gain- versus Loss-Framing for Opting-in to Cropping Series Messages | PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. In this experiment, we aim to test the relative effectiveness of gain- versus loss-framed messages in the cropping series (CS) opt-in invitation messages for long rainy season 2021 (LR 2021). The gain framing emphasized “improving harvests” while the loss framing emphasized “risking crop loss”. We find that, when compared to the neutral invitation message, the loss-framed message significantly decreased the likelihood of farmers opting-in to the CS, by nearly 7%. This suggests that farmers reacted strongly against a perceived risk of loss. There was no statistically significant difference in opt-in rates between the neutral and gain-framed messages. | 2021 | Kenya | Kenya Ministry of Agriculture | Message framing | Completed | Long Rains | A/B test |
| User Anniversary Messages to Re-engage Inactive Users | PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. To re-engage inactive users and acknowledge long-term users in a personal way, PxD sent an anniversary message showing appreciation of the user one year after the user’s registration. This A/B test compared two message variants—one asking about phone type and the other about land size—to assess which type of information request is more effective in prompting a user response. The phone-type information request elicited a stronger response than the land-size question. The anniversary messages achieved much higher response rates than previous information requests, which suggests that milestone-based, personalized messages can be an effective strategy for re-engaging dormant users and collecting valuable demographic information. We recommend further testing to explore the broader impact of such nudges. | 2020 | Kenya | Kenya Ministry of Agriculture | Communication technology, Message framing, Service design | Completed | Short Rains | A/B test |
| Agro360: Voice Messages in Bangla versus Local Dialects | mPower is a Bangladesh-based social enterprise that specializes in technology-based development solutions. As part of its agriculture program, Agro360, mPower sends text messages with crop management recommendations and weather alerts to farmers to improve the efficiency of their cultivation of their crops and to promote practices that mitigate the impact of climate change on their crops. For Rabi 2019, mPower partnered with PxD to understand the effect of sending voice messages in the local dialect on farmers’ engagement with the recommendations. Even though Bangla is the official language of Bangladesh, farmers in different parts of the country speak local dialects. According to the Cultural Survey of Bangladesh, there are 16 regional varieties of Bangla spoken in the target districts of the program. Even though farmers understand Bangla, they may feel more comfortable with messages in their local dialect. We tested farmers’ engagement with the service when the voice messages were delivered in their local dialect, compared to messages delivered in traditional Bangla as they had been during the previous seasons. The results show that farmers engage more with the voice message when it is sent in their local dialect, as reflected in the listening rate and the completion rate. | 2019 | Bangladesh | mPower | Agricultural management advice, Message narration | Completed | Rabi | A/B test |
| SMS Survey Design Tweaks | PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. This experiment tests how subtle changes in the framing or format of SMS survey questions and answers affect users’ response rates. We aimed to identify behavioral design elements that can improve the data quality and the user experience of SMS-based surveys deployed via the MoA-INFO platform. We find that small tweaks in message designs—from the opening-message framing to answer-option formats—make meaningful differences in survey completion rates and the time it takes for users to complete the survey. | 2020 | Kenya | Kenya Ministry of Agriculture | Message framing | Completed | Short Rains | A/B test |
| 2018 Rwanda Fertilizer Experiment | One Acre Fund (OAF) is an agricultural service provider that supports smallholder farmers in Africa in accessing agricultural inputs, training, and markets, to help the farmers increase their harvests and income. PxD and OAF began collaborating in 2016 on efforts to increase adoption of agricultural inputs and improve OAF operations in Kenya and Rwanda. Rwanda’s Ministry of Agriculture (MINAGRI), the Rwandan Agriculture Board (RAB), and OAF conducted a national radio campaign at the end of 2018 to address smallholder farmers’ misconceptions about chemical fertilizer. MINAGRI, RAB, and OAF observed that misconceptions impeded farmers’ adoption of fertilizers, which likely suppressed the yields and income of the farmers. OAF and PxD conducted a complementary phone-based intervention to send SMS messages to Farmer Promoters (FPs—model farmers elected by other farmers in their village to serve as volunteer extension workers). The SMS messages encouraged FPs to: (i) themselves adopt best practices associated with fertilizer use, and (ii) teach other farmers to do the same. We find that the intervention had a positive and statistically significant effect on the index of the six recommendations for fertilizer use and training we made to FPs in the treatment arms. SMS messages are a low-cost way of making recommendations, and this promising result shows they can effectively modify FP behavior. | 2018 | Rwanda | One Acre Fund (OAF) | Extension agents, Message framing | Completed | _Multiple seasons | Impact Evaluation |
| Changing the Frequency of Advisory Calls to Improve Farmer Engagement | PxD has operated the Krishi Tarang service in Gujarat since 2016 to provide free agriculture information via mobile phones using a two-way Interactive Voice Response (IVR) platform with “outbound” push calls and an “inbound” hotline service. We tested whether changing the frequency of advisory calls to farmers improves overall farmer engagement. Farmers were randomly assigned to receive standard weekly calls, or twice-weekly shorter calls, or to select their preferred frequency. The results suggest that both higher frequency and preference-based approaches increased listening rates, with stronger effects observed when farmers could choose their call frequency. | 2017 | Gujarat, India | J-PAL | Message timing and frequency | Completed | Kharif | A/B test |
| Agro360: Conversation versus Trivia | mPower is a Bangladesh-based social enterprise that specializes in technology-based development solutions. As part of its agriculture program, Agro360, mPower sends text messages with crop management recommendations and weather alerts to farmers to improve the efficiency of their cultivation of their crops and to promote practices that mitigate the impact of climate change on their crops. In Kharif 2019, mPower partnered with PxD to test advisory design tweaks to improve farmer engagement with the messages. This trial tested different narrative styles for the voice messages that were intended to capture farmers’ attention, by comparing the effects of messages beginning with short trivia facts and conversational-style advisory messages against the effects of standard messages. The results show that farmers who received conversational-style messages engaged more with the messages than farmers receiving the standard messages did, throughout the season. Notably, messages that started with a trivia fact increased the pick-up rates slightly but led to a considerable drop in the likelihood of listening to the full message. | 2019 | Bangladesh | mPower | Communication technology, Message narration | Completed | Kharif | A/B test |
| PRISE Pesticide Application Campaign SR 2020 | The Pest Risk Information Service (PRISE) is an early warning model, run by the Centre for Agriculture and Bioscience International (CABI), that predicts the optimal timing of pesticide application. PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. CABI partnered with PxD to send SMS messages to maize, bean, and tomato farmers registered on the MoA-INFO service, to help the farmers make better decisions about pesticide application when confronted with pests in their fields. Building on a 2019 pilot that provided maize farmers with optimal pesticide timing based on the PRISE model, CABI and PxD expanded the trial in 2020 to include three crop–pest combinations: maize and Fall armyworm (FAW), beans and bean fly, and tomatoes and tomato leafminer. Treatment farmers received targeted messages about optimal pesticide timing plus nudges to use specific MoA-INFO platform features (crop menus and the FAW monitoring tool). Control farmers received standard MoA-INFO cropping messages. We evaluate whether timely pesticide advice improves platform engagement and pest management practices, and we gathered farmer feedback on the PRISE model’s accuracy. The PRISE informational messages significantly increased menu access by maize, bean, and tomato farmers, while also increasing their platform opt-out rates. A vast majority of farmers (more than 85%) reported that the PRISE model gave accurate estimations. However, farmers indicated that their lack of resources to buy the required inputs was a notable limitation to them acting on the recommendations. | 2020 | Kenya | CABI | Pest management | Completed | Short Rains | A/B test |
| Kenya IVR Pilot | PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. PxD Kenya piloted automated phone calls delivering agricultural advice on banana pests and diseases to farmers using Interactive Voice Response (IVR) technology. The sample frame for this experiment was existing users of the MoA-INFO platform who had opted-in to receiving banana advice via SMS messages. The IVR service in this pilot was linked to a PxD-owned phone number, which ensured that all farmers subscribed to the service could call in and get advice for free. The messages were offered in the two languages (English and Swahili) available in the SMS system. The main aim of this pilot was to test whether the introduction of the IVR-based voice calls affected users’ SMS-platform engagement and knowledge gains. Another aim was to gain insight into the implementation of the new IVR feature and to monitor usage of this platform. While treatment farmers had a high rate of engagement with the IVR calls and gave positive feedback, the pilot did not detect a notable effect of the IVR intervention on MoA-INFO platform engagement or knowledge outcomes. | 2020 | Kenya | Kenya Ministry of Agriculture | Communication technology | Completed | Long Rains | Other |
| Kenya LR 2020 Cropping Series | PxD operates the MoA-INFO platform in collaboration with the Kenya Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. Farmers can access information whenever they like by sending the word “MENU” or “ORODHA” to access a complete list of topics. In addition, farmers can opt-in to weekly cropping series (CS) messages which offer advice on crop management practices—from land preparation to harvesting and storage—throughout the season. While all the content in the CS is available on the menu for farmers to access at any time, most engagement with the platform comes from farmers who opt-in to the weekly CS advisory. We previously found that the MoA-INFO platform could increase farmers’ knowledge about topics such as Fall Armyworm (FAW; for example, see “Kenya Fall Armyworm Trial 2019”). In this trial we evaluate the effects of the MoA-INFO service on farmers’ behavior, by randomizing farmers to receive invitations to specific CS topics. During the long rainy season 2020 (LR 2020), we selected seven CS topics that pertain to seed and fertilizer choice and post-harvest storage, and we randomized farmers to receive invitations to those seven CS topics. In September 2020, we collected information on the adoption of recommended practices and on crop yields, via a phone survey. Complementary information on platform engagement came from administrative data from the MoA-INFO platform. We find that higher adoption of recommended practices, as reported by farmers and measured using an aggregate index of practices, is correlated with higher yield. This finding suggests that recommended practices can improve agricultural outcomes. However, receiving SMS advice did not lead to statistically significant changes in the adoption of recommended practices or crop productivity (yield and harvest). These results suggest that our agronomic content and approach (i.e., advising on a set of good agricultural practices) has potential, and underscore the importance of enhancing the effectiveness of our information provision. | 2020 | Kenya | Kenya Ministry of Agriculture | Agricultural management advice, Communication technology | Completed | Long Rains | Impact Evaluation |
| The Effects of Fewer and Revised Recommendations on Service Engagement and Agricultural Practices | PxD operates Ama Krushi, a free agriculture information service delivered over mobile phones, in collaboration with the State Government of Odisha Department of Agriculture, using a two-way Interactive Voice Response (IVR) platform with “outbound” push calls and an “inbound” hotline service. This A/B test examines the impact of three adjustments to our standard Ama Krushi service on farmers’ engagement with the service and on the adoption of the inputs we recommended to paddy farmers during the 2020–21 Rabi season. Specifically: (1) We sent fewer messages on high impact practices; (2) we used a structured message development template to develop content; (3) for each message, we conducted qualitative interviews with farmers who were not in the sample but were eligible for inclusion in the sample, to gather feedback on the comprehensibility and actionability of the messages; and (4) we sent the advisory message a second time to help farmers remember the advice better. We find that providing farmers with fewer, re-framed messages over a season increases their pick-up rates by 4 percentage points (pp) and their listening times by around 20 seconds per paddy message. Although the analysis was not powered to detect adoption effects, our qualitative findings suggest various mechanisms by which fewer and simpler messages could drive the impact on engagement and practices. | 2020 | Odisha, India | Government of Odisha | Agricultural management advice, Communication technology, Message framing, Message narration, Message timing and frequency | Completed | Rabi | A/B test |
| Kenya Agricultural Lime Experience Trial 2017 | One Acre Fund (OAF) is an agricultural service provider that provides support for smallholder farmers in Africa in accessing agricultural inputs, training, and markets, to help the farmers increase their harvests and income. PxD and OAF began collaborating in 2016 on efforts to increase adoption of agricultural inputs and improve OAF operations in Kenya and Rwanda. Soil acidity is a major issue for many of OAF’s clients. Under a critical threshold (pH < 5.5), soil acidity negatively affects maize growth and can inhibit the efficacy of OAF products, such as fertilizer and hybrid seeds. Soil acidity can be resolved by the application of agricultural lime. Agricultural trials run by OAF indicate that spot applying (microdosing) lime increases yields significantly. These spot applications are the application of lime directly to the soil around the roots of maize plants. Despite the high return on and relatively low cost of purchasing lime, adoption of lime has been low. This trial implemented a series of interventions to test whether Field Officers (FOs) and OAF’s clients gaining experience with lime increases the likelihood of farmers purchasing lime the following season. We find that monetary incentives for FOs were highly effective in increasing lime adoption by farmers. However, receiving free lime decreased the likelihood of farmers adopting lime in the subsequent year, possibly because farmers were expecting lime to be free again. | 2016 | Kenya | One Acre Fund (OAF) | Agricultural management advice, Input recommendations, Service design | Completed | Long Rains | Impact Evaluation |
| Kenya Agricultural Lime Messaging Trial 2018 | One Acre Fund (OAF) is an agricultural service provider that supports smallholder farmers in Africa in accessing agricultural inputs, training, and markets, to help the farmers increase their harvests and income. PxD and OAF began collaborating in 2016 on efforts to increase adoption of agricultural inputs and improve OAF operations in Kenya and Rwanda. In 2017, PxD and OAF conducted a randomized trial “Kenya Agricultural Lime Messaging Trial 2017” that demonstrated SMS messages significantly increased the adoption of agricultural lime by smallholder farmers in western Kenya. Building on these findings, this follow-up trial aimed to validate the 2017 results and to (a) verify that SMS messages continue to increase lime uptake, (b) test whether referencing peer farmers who previously adopted lime could promote adoption through social learning, and (c) test the relative effectiveness of different SMS framings and message repetition on farmer adoption behavior. We used a multi-component randomized trial to test these hypotheses with farmers in previously studied districts and new districts. The sample was drawn from OAF’s core program, with randomization at the individual level, stratified by district. SMS messages were delivered during the input-adoption period. The results confirm that SMS messages increased lime adoption, and provide evidence supporting the effectiveness of digital outreach at scale. While differences between specific message framings and social learning nudges were not statistically significant, the results of the experiment suggest that message content and repetition may influence adoption behavior and offer valuable insights for refining future campaigns. | 2017 | Kenya | One Acre Fund (OAF) | Message framing, Message timing and frequency, Social learning | Completed | Short Rains | Impact Evaluation |
| Kenya Agricultural Lime Messaging Trial 2017 | One Acre Fund (OAF) is an agricultural service provider that provides support for smallholder farmers in Africa in accessing agricultural inputs, training, and markets, to help the farmers increase their harvests and income. PxD and OAF began collaborating in 2016 on efforts to increase adoption of agricultural inputs and improve OAF operations in Kenya and Rwanda. Prior to 2016, less than 3% of OAF clients in western Kenya purchased agricultural lime from OAF. To increase take-up, OAF designed a phone-based extension pilot that consisted of six rounds of text messages targeting clients who had signed up for the OAF package during the previous season in a selected district of western Kenya. The objective of the experiment was to evaluate the impact of SMS messages on lime adoption by farmers. The study tested whether more detailed, locally customized messages—containing site-specific information about soil acidity, lime dosage, total cost, and expected yield gains—would lead to higher adoption compared to simpler, generic messages. We tested this hypothesis by randomizing two different formats for SMS messages sent to farmers in a selected district of western Kenya, starting in 2016 to lead up to the period when farmers had to decide whether to request inputs from OAF for the long rainy season 2017. A control group of farmers received no messages. The SMS messages significantly increased lime adoption by over 3 percentage points (pp), which is a 30% increase relative to the 10% control mean. Customized messages produced slightly higher effects than general messages. Adoption patterns were consistent across administrative and phone survey data, although adoption measured via a phone survey gave higher point estimates than when adoption was measured via the administrative data on input purchases. | 2016 | Kenya | One Acre Fund (OAF) | Message framing, Social learning | Completed | Long Rains | Impact Evaluation |
| PRISE FAW Information Campaign SR 2019 | The Pest Risk Information Service (PRISE) is an early warning model, run by the Centre for Agriculture and Bioscience International (CABI), that predicts the optimal timing of pesticide application. PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. CABI partnered with PxD to send SMS messages with information about Fall Armyworm (FAW) to maize farmers about the time when pesticides were likely to be most effective for their area, based on the PRISE model. PxD randomly selected a sample of farmers from the MoA-INFO platform to receive a series of SMS messages with general advice on maize management practices and with FAW pest-management advice that corresponded to the forecasted planting dates for the farmer’s constituency. Control farmers had access to demand-driven FAW management advice, but did not receive push SMS messages. PxD conducted a follow-up phone survey to measure the effects on knowledge, practices, and the extent of FAW damage. The intervention messages significantly increased the number of times farmers accessed FAW content. It marginally improved farmers’ knowledge (statistically insignificant) and significantly increased farmers’ self-reported adoption of the natural solutions for managing FAW that were recommended by the SMS intervention. | 2019 | Kenya | CABI, Kenya Ministry of Agriculture | Communication technology, Pest management | Completed | Short Rains | A/B test |
| Differently Framed Blast Messages to Re-engage Sleeper Users | PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. As of April 2019, some 135,000 (37%) of the MoA-INFO platform users had sent only one message to the platform (either “FARM” or “SHAMBA”, to register on the platform) and had then never responded to any MoA-INFO messages. In the 30 days before the experiment, 58% of registered platform users had not sent any messages to the platform. We call these users, who complete the registration survey but never send any more messages to the platform, “sleepers”. PxD tested whether five messages framed in various ways re-engaged sleepers with the platform content. Compared to the four other messages, the message that directly showed the menu, to make it easy for the user to select content, resulted in a statistically significant higher response rate and prompted twice the number of users to access the content. | 2019 | Kenya | Kenya Ministry of Agriculture | Communication technology, Message framing | Completed | Long Rains | A/B test |
| Improving Invitation Messaging to Increase Registration Rates | PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. Blast messages from Safaricom (a large Kenyan phone company) are a critical method to attract new users to the MoA-Info platform. When PxD scaled to a nationwide campaign reaching over 10 million Kenyans, registration rates were at around 1%, which was significantly below the 3% for the best-performing messages in earlier pilots. This experiment tested two framing improvements—addressing farmer pest concerns, and appealing with social proof and urgency—to a message used previously. The experiment also tested two follow-up messages. Improved message framing significantly outperformed the previous nationwide campaign, resulting in up to 30% higher registration rates. The effects of the two improved versions were not significantly different from each other. The follow-ups were very effective and led to an extra 3.1% of the overall sample registering on the MoA-INFO platform. | 2018 | Kenya | Kenya Ministry of Agriculture | Communication technology, Message framing, Message timing and frequency, Service design | Completed | Short Rains | A/B test |
| Rewording and Restructuring Menu Options (ATA Experiment 109) | PxD is partnering with Ethiopia’s Agricultural Transformation Agency (ATA) to help improve the effectiveness of their voice-based mobile-phone advisory service, the 8028 hotline, by conducting continuous iterations and experiments, as well as by making suggestions for improvements to and customization of the service. The service has millions of registered farmers and represents the first in Africa to be maintained by a government entity at such a large scale. The 8028 hotline Interactive Voice Response (IVR) system uses phone key navigation. Menu options for advisory content were labeled using technical or scientific terminology (e.g., “Pre-planting”). Qualitative investigations, including focus group discussions and in-depth interviews, revealed that many farmers did not understand these labels. As a result, they often could not predict what kind of advice each menu would include. This likely limited their ability to access the information most relevant to them and may have reduced the overall usefulness of the service. While farmers showed a reasonable distribution of preferences across crop-specific menus in qualitative investigations, most callers defaulted to pressing option 1 in the main menu—possibly because they didn’t fully understand the meaning or purpose of the other options. Findings from “Rotate Menu Seasonally (ATA Experiment 108)” demonstrated that users’ choices were strongly influenced by menu position; here we examine whether unclear menu labeling also affects user behavior. To improve content access, we tested whether rewording and restructuring how menu options are presented would help users to make more informed choices to access relevant content. We find limited evidence that rewording menu options led to changes in user behavior; the majority of users still selected the first menu option. The experiment was conducted outside the seasonal crop menu rotation experiment, which means option 1 consistently referred to "pre-planting"—a possible reason for the persistent default behavior. | 2019 | Ethiopia | Ethiopian ATA | Communication technology | Completed | _N/A | A/B test |
| Advisory Opt-ins During Registration Compared with Opt-ins After Registration | PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. Farmers join the MoA-INFO platform through a registration survey initiated by sending the word “FARM” or “SHAMBA”. The registration survey collects data such as the farmer’s name and location to customize future messages, and introduces the farmer to other platform features and keywords like MENU and CHECK. The cropping series (CS) is a key feature of the MoA-INFO platform. In this trial we test whether more farmers opt-in to the CS if they are given the option during registration than if they are given the option after completion of their registration. The results show that including the opt-in question during registration significantly increased the maize CS opt-in rate from 26% to 76%, but reduced completion of the location section of the survey by 5.9 percentage points (pp). While it lowered opt-in rates for other crops, such as beans and Irish potatoes, the overall effect was net positive, leading to more farmers opting-in to at least one CS and receiving advice throughout the season. | 2019 | Kenya | Kenya Ministry of Agriculture | Communication technology, Service design | Completed | Long Rains | A/B test |
| Removing the Prompt to Save a Selected Crop to the User’s Profile (ATA Experiment 104) | PxD is partnering with Ethiopia’s Agricultural Transformation Agency (ATA) to help improve the effectiveness of their voice-based mobile-phone advisory service, the 8028 hotline, by conducting continuous iterations and experiments, as well as by making suggestions for improvements to and customization of the service. The service has millions of registered farmers and represents the first in Africa to be maintained by a government entity at such a large scale. The 8028 hotline Interactive Voice Response (IVR) system uses phone key navigation. When users call the 8028 hotline, they are prompted to select a crop to hear advisory content. When users select a crop they haven’t added to their profile before, the system prompts users to save the selected crop to their profile, before the system provides advisory content. This prompt was identified as a source of both confusion for users and drop-off from the system. We tested whether removing the prompt to save the selected crop to the user’s profile improved access to content. The intervention resulted in a modest positive effect: a one percentage point (pp) increase in the share of users accessing content, which would translate to approximately 5,000 additional messages accessed per month across all the 8028 hotline users. | 2017 | Ethiopia | Ethiopian ATA | Communication technology | Completed | _N/A | A/B test |
| Longer Pauses Between Language Options (ATA Experiment 103) | PxD is partnering with Ethiopia’s Agricultural Transformation Agency (ATA) to help improve the effectiveness of their voice-based mobile-phone advisory service, the 8028 hotline, by conducting continuous iterations and experiments, as well as by making suggestions for improvements to and customization of the service. The service has millions of registered farmers and represents the first in Africa to be maintained by a government entity at such a large scale. The 8028 hotline Interactive Voice Response (IVR) system uses phone key navigation. A PxD survey revealed that many farmers found the language options were presented too quickly, which made it difficult for the farmers to select their preferred language. Platform data analysis showed that approximately 15% of users failed to select a language, potentially due to the rapid pace of delivery of the options. We implemented an A/B test in which users in the treatment group experienced a longer pause than the control group experienced between the language options during registration. The intervention did not yield a statistically significant improvement in the rates of language selection or content access. | 2017 | Ethiopia | Ethiopian ATA | Communication technology | Completed | _N/A | A/B test |
| Postponing Registration (ATA Experiment 101) | PxD is partnering with Ethiopia’s Agricultural Transformation Agency (ATA) to help improve the effectiveness of their voice-based mobile-phone advisory service, the 8028 hotline, by conducting continuous iterations and experiments, as well as by making suggestions for improvements to and customization of the service. The service has millions of registered farmers and represents the first in Africa to be maintained by a government entity at such a large scale. The 8028 hotline Interactive Voice Response (IVR) system uses phone key navigation. First-time callers to the 8028 hotline are prompted to register by selecting their preferred language, and their region, zone, and woreda, before accessing any content. This registration process delays access to the core informational content of the IVR system and has been found to compromise the accuracy of the registration information that is collected. Specifically, while 85% of users correctly reported their region, only 65% accurately reported their zone, and just 15% provided correct woreda information. To address this issue, we implemented an experiment in which users in the treatment group, after selecting their language, bypassed the registration process and were immediately directed to the top menu to access content. Postponing registration significantly increased the share of users accessing content, from 52% to 63%. Based on these findings, the registration step was postponed to gather profile information in a later call. | 2017 | Ethiopia | Ethiopian ATA | Communication technology | Completed | _N/A | A/B test |
| Registration Survey Redesign | PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. In February 2019, the MoA-INFO platform was undergoing various expansions that required a redesign of the registration survey. First, the content on the platform was increased from a single-crop maize service to include two additional crops. Second, the customization of content was improved by targeting message content and timing with the farmer’s location information. Third, Safaricom (a large Kenyan phone company) planned to deliver the third nationwide blast campaign, which was expected to more than double the number of platform users. In the redesign process, we considered ways to optimize the new registration survey. The first consideration was whether to start the survey by asking the user’s first name. The potential benefit is that users who answer the question may feel trust in the platform when it subsequently refers to them by name. The potential downsides are that adding an extra message to the registration may decrease completion rates, and that the name question might make some users feel less inclined to complete the survey due to reservations about sharing personal information. The second consideration was whether to include questions asking if users farm potatoes and beans in addition to a question about maize. The potential benefit is that this information allows targeting of subsequent messages. The downside is that it adds two additional questions about potato and bean farming to the survey. After experimentally testing these variations, we find that asking users their first names increased the rates of both response to the first question and survey completion. However, asking whether users grow potatoes and beans decreased the registration survey completion rate. | 2019 | Kenya | Kenya Ministry of Agriculture | Communication technology, Service design | Completed | Long Rains | A/B test |
| Removal of Initial Jingles from Outbound Calls | PxD has operated the Krishi Tarang service in Gujarat since 2016 to provide free agriculture information via mobile phones using a two-way Interactive Voice Response (IVR) platform with “outbound” push calls and an “inbound” hotline service. Weekly push calls were being sent from the PxD system to all the active farmers. Every push call started with an 8-second branding jingle before the agricultural content. We observed, however, that many farmers dropped the call during the jingle before the agricultural content. In this experiment, we tested whether removing the jingle improved farmers’ access to agricultural content in outbound calls. There was a dramatic improvement in farmers’ engagement with the service. The calls without the jingle saw an increase in the listening rate by 17.8 percentage points (pp). | 2017 | Gujarat, India | J-PAL | Message framing, Message timing and frequency | Completed | Rabi | A/B test |
| Targeted Messages to Promote Flood-tolerant Seeds | PxD operates Ama Krushi, a free agriculture information service delivered over mobile phones, in collaboration with the State Government of Odisha Department of Agriculture using a two-way Interactive Voice Response (IVR) platform with “outbound” push calls and an “inbound” hotline service. There is strong prior evidence that flood-tolerant (FT) rice varieties—Swarna Sub-1 and CR 1009 Sub-1—improve productivity by reducing crop losses during floods. Despite their proven ability to increase rice yields in flood-affected years, adoption of these varieties remains low. PxD tested whether a short, targeted Ama Krushi advisory message about the benefits of FT seeds would increase their adoption. At the start of the 2020 Kharif season, roughly 10,000 Ama Krushi users in three lowland districts were randomly assigned to receive either the regular weekly seed advisory, which provided information on several locally appropriate seed varieties (control group) or the same weekly advisory with additional messages highlighting the benefits of the FT seeds for two consecutive weeks (treatment group). However, at implementation a substantial proportion of farmers with low and medium land types in the treatment group received only the additional messages focused on FT seeds. We find from the follow-up phone survey that sending the additional messages focused on FT seeds significantly increased the reported use of FT seeds; the effect was primarily driven by farmers with low and medium land types. This test builds on PxD’s existing body of evidence showing that simple, engaging messages can be effective in promoting behavioral change among farmers. | 2020 | India, Odisha, India | Government of Odisha | Agricultural management advice, Communication technology, Message framing, Weather information | Completed | Kharif | A/B test |
| Methods of Inviting Farmers to Refer Fellow Farmers | PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. This A/B test, conducted during the nationwide rollout of the MoA-INFO platform in July 2018, tested how referral invitations that were delivered via different channels—SMS only, a live phone call only, and both—affect referral behavior. The outcome measures were the likelihood of referring at least one farmer, the total number of referrals made, and the number of referred farmers who opted into the service. Half of the farmers selected to receive referral invitations were also randomly assigned to receive a message with information to dispel misconceptions about Fall Armyworm (FAW), in order to test whether these messages increased the likelihood of referring others. Farmers made referrals by texting fellow farmers’ phone numbers to a short code. Referred farmers received an opt-in prompt. We find that a live-call invitation to refer farmers significantly increased both the likelihood and number of referrals, as well as the number of successful opt-ins from the referred farmers. Combining the live calls with SMS invitations further increased referral activities, particularly the likelihood of referring someone, although the incremental effect on opt-ins is not statistically significant. Receiving the misconception-dispelling messages had no significant effect on referrals. | 2018 | Kenya | Kenya Ministry of Agriculture | Communication technology, Service design | Completed | Short Rains | A/B test |
| Information about Cumin: The Effect on Crop Choice and Protection | PxD has operated the Krishi Tarang service in Gujarat since 2016 to provide free agriculture information via mobile phones using a two-way Interactive Voice Response (IVR) platform with “outbound” push calls and an “inbound” hotline service. India produces about 80% of the world’s cumin, which is a dried seed used as a spice. Gujarat produces about 85% of the country’s total output of the crop. Cumin is a high-return crop, but is susceptible to pest attacks. In 2017, only 48% of the sixty thousand farmers who use PxD’s service in Gujarat intended to grow cumin. Farmers make crop choices based on many considerations, including the costs, risks, climate, and historical selling price. One key determinant of whether a farmer grows a particular crop is the information they have about that crop. We tested whether increasing the amount of information farmers received about cumin would increase their likelihood of cultivating cumin in the next Rabi planting season. Although we did not find a large increase in cumin cultivation, additional information about cumin led to a rise in the cumin share of total area under cultivation. Moreover, providing farmers with information about ways to prevent crop damage appears to have changed their behavior. | 2017 | Gujarat, India | J-PAL | Agricultural management advice | Completed | Rabi | A/B test |
| The Timing of UCAT SMS Reminders | PxD is partnering with Hanns R. Neumann Stiftung (HRNS) and TechnoServe (TNS) on the Uganda Coffee Agronomy Training (UCAT) program to provide a complementary digital service, which reinforces recommendations of Good Agricultural Practices (GAPs) via automated calls to subsets of coffee farmers. PxD is also offering a “standalone” customized digital advisory service to a subset of farmers in villages that were randomly selected to serve as a control group and did not receive the in-person UCAT program, as part of a large-scale impact evaluation. PxD sent SMS message reminders prior to Interactive Voice Response (IVR) push calls to these farmers who were receiving standalone digital agricultural advisory messages. To test the optimal timing of reminders, we randomly assigned farmers to receive SMS reminders one hour before the weekly push calls or 24 hours before the weekly push calls. We implemented the experiment for four weeks and measured farmers’ level of engagement with the platform, as call pick-up and completion rates. The 24-hour SMS reminder was more effective than the one-hour SMS reminder for increasing farmer engagement with weekly push calls. These effects, however, are driven largely by particular experiment weeks. | 2020 | Uganda | IFPRI, UCAT | Communication technology, Message timing and frequency | Completed | Long Rains | A/B test |
| The Effect of Reminder and Instructional Messages on Inbound Engagement | PxD operates Ama Krushi, a free agriculture information service delivered over mobile phones, in collaboration with the State Government of Odisha Department of Agriculture, using a two-way Interactive Voice Response (IVR) platform with “outbound” push calls and an “inbound” hotline service. This experiment tested the impact of sending a series of reminder and instructional messages in different styles on farmers’ likelihood of engaging with the Ama Krushi inbound service. Farmers were randomly assigned to one of five groups: a control group receiving no reminder, or one of four treatment groups receiving biweekly messages in one of four formats—a standard advisory, a conversational script between farmers, a farmer’s testimonial, or a social nudge highlighting service usage statistics. We also conducted qualitative feedback surveys to understand the farmers’ motivation and engagement with the inbound service. Reminder messages modestly increased the likelihood of farmers calling the inbound service, particularly farmers who had previously engaged with Ama Krushi. However, varying the message framing beyond the standard advisory did not enhance this effect, and no significant impact was found on the likelihood of farmers asking a valid question during the call. | 2020 | Odisha, India | Government of Odisha | Communication technology, Message narration, Service design | Completed | Kharif, Rabi | A/B test |
| Inbound Engagement: Remote Training | PxD operates Ama Krushi, a free agriculture information service delivered over mobile phones, in collaboration with the State Government of Odisha Department of Agriculture, using a two-way Interactive Voice Response (IVR) platform with “outbound” push calls and an “inbound” hotline service. This experiment built on earlier A/B tests aimed at increasing engagement with the inbound service by offering phone-based remote training to a subset of farmers who had previously received reminder and instructional messages. In this experiment we assessed whether combining remote training with reminders would improve farmers’ likelihood of calling into the hotline and asking valid agricultural questions, compared to farmers who received only reminders or no intervention. Adding remote training meaningfully increased farmers’ engagement with the service, in terms of both call-in rates and the likelihood of asking valid questions. The effect was particularly notable on farmers who were active outbound users and on smartphone users. These results indicate that remote training is a promising complement to reminder messages for enhancing farmer engagement with digital advisory services. | 2020 | India | Government of Odisha | Communication technology, Service design | Completed | Kharif, Rabi | A/B test |
| The Effect of Message Phrasing and Timing on the Use of the Fall Armyworm Monitoring Tool | PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. The Fall Armyworm (FAW) monitoring tool helps farmers assess FAW damage in their fields and provides tailored advice based on their observations. Farmers can access the tool by texting the word “CHECK” (“ANGALIA” in Swahili). As of July 2019, 22% of platform users accessed the FAW monitoring tool but less than 9% of those users completed using the tool. This A/B test tested whether variations in message phrasing and messaging time of day affected the uptake and completed use of the monitoring tool. Farmers were randomly assigned to receive either a regular invitation message that required them to go through multiple interactions to launch the monitoring tool, or an easy version that allowed farmers to launch the tool in one step. Message delivery time was also randomized across four time slots: 7AM, midday, 3PM, and 6PM. The easy message led to a 2 percentage point (pp) higher response rate and a significantly higher completion rate than the regular message did. Messages sent at 6PM yielded the highest tool access rate, while midday messages, despite a lower response, resulted in the highest completion rate. These findings inform best practices for driving engagement with interactive advisory tools via SMS. | 2019 | Kenya | Kenya Ministry of Agriculture | Message framing, Message timing and frequency | Completed | Long Rains | A/B test |
| The Effect of the Narrator’s Gender on the Rates of Engagement by UCAT Farmers | PxD is partnering with Hanns R. Neumann Stiftung (HRNS) and TechnoServe (TNS) on the Uganda Coffee Agronomy Training (UCAT) program to provide a complementary digital service by reinforcing recommendations of Good Agricultural Practices via automated calls to subsets of coffee farmers. We tested whether the gender of the narrator for Interactive Voice Response (IVR) agronomy messages affected farmer engagement. In one group the narrator's gender was switched to match that of the farmer’s, while in a second group the narrator's gender was switched to the opposite of the farmer’s. Overall, both male and female farmers increased engagement when switching from a male to female narrator, but not when switching from a female to male narrator. Our analysis also suggests—although only tentatively—that women presented a stronger response than men. | 2020 | Uganda | IFPRI, UCAT | Message narration | Completed | Short Rains | A/B test |
| Approaches to Asking Users for Their Location Information | PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. Kenya’s 2010 Constitution replaced the former provincial and district administrative structures with a devolved system of 47 counties. PxD therefore tested four different approaches to asking platform users for their location information. We tested whether the approach affected the response rate of users and the corroboration of responses with known administrative boundaries. Asking about counties yielded higher response rates and corroboration rates than asking about districts. Using filtered lists classified more data, albeit at a slightly lower corroboration rate, than unfiltered text responses did. Lastly, increasing the number of location questions reduced the overall data quality, which suggests that collecting more granular administrative data has diminishing returns. | 2018 | Kenya | Kenya Ministry of Agriculture | Message framing, Service design | Completed | Short Rains | A/B test |
| The Effect of a Language Preference Question on Quiz Scores | PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. To recruit farmers to the service, PxD worked with Safaricom (a large Kenyan phone company) to send SMS messages to owners of mobile phones in rural areas; the messages had a keyword in English or Swahili (“FARM” or “SHAMBA”) that farmers could use to register. Farmers who texted the keyword to the MoA-INFO shortcode were sent a registration survey. This experiment builds on a language prompt experiment “The Effect of a Language Prompt on Farmers’ Platform Engagement”, which tested whether offering users the option to switch languages affected registration, retention, and engagement with the MoA-INFO platform. That study found increased initial completion rates when users could choose their preferred language, but it did not find a sustained impact on engagement. This A/B test aimed to measure whether asking farmers about their preferred language affects their comprehension of the information in the registration survey content, as measured by their performance on a comprehension quiz. Farmers who had been sent a registration survey were randomly assigned to the treatment group, who received an SMS asking them which language they preferred, or the control group, who did not receive the language preference question. Approximately one thousand farmers were randomly selected to receive an SMS invitation to take a quiz to test their knowledge about Fall armyworm (FAW). This randomization was stratified by treatment status if the user had been offered the opportunity to switch languages in the registration survey, and language of registration message (English or Swahili), resulting in four groups of equal size ~250. Users who were given the option to switch languages scored slightly higher on the quiz when scoring only the questions for which they provided a response. The treatment had a slightly negative effect on quiz scores calculated by the number of correct responses out of all questions, whether or not farmers provided a response. These results are not statistically significant and the findings do not provide supporting evidence that the opportunity to switch languages has a positive effect on the FAW comprehension quiz scores. | 2018 | Kenya | Kenya Ministry of Agriculture | Communication technology, Message narration | Completed | Short Rains | A/B test |
| Framing of Location Requests | PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. Farmer’s location information can improve the targeting and relevance of advisory messages, but collecting accurate location information using SMS messages has been a challenge. PxD tested whether different framings of a location request would affect farmers’ likelihood to respond. Two randomly selected groups of farmers were sent messages asking their location. One message was framed in terms of improving recommendations and the other message was framed in terms of fighting Fall Armyworm (FAW). The response rate was measured for both groups and was found to be significantly lower with the FAW framing. | 2018 | Kenya | Kenya Ministry of Agriculture | Communication technology, Message framing, Service design | Completed | Short Rains | A/B test |
| Menu Access Rates for Different Menu Reminder Messages | PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. PxD tested whether reminder messages—either general or content-specific—increase menu access rates, and which reminder message generates the most interest. A randomly selected group of platform users was sent a reminder with general or content-specific messages to access the menu. Farmers who received the messages were between 2 and 6 percentage points (pp) more likely to access the menu, with the message about the origins of Fall Armyworm (FAW) increasing the likelihood the most. | 2018 | Kenya | Kenya Ministry of Agriculture | Communication technology, Message timing and frequency | Completed | Long Rains | A/B test |
| The Effect of a Language Prompt on Farmers’ Platform Engagement | PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. To recruit farmers to the service, PxD worked with Safaricom (a large Kenyan phone company) to send SMS messages to owners of mobile phones in rural areas, with a keyword in English and Swahili (“FARM” or “SHAMBA”) that they could use to register their phone number. Farmers who texted the keyword to the MoA-INFO platform were sent a registration survey. Safaricom sent messages in the language that users chose for the keyword to register their phone number. We tested whether giving users the option to switch languages affects the likelihoods of their registration on, retention in, and engagement with the system. When given the chance, 22% of users chose a different language from their keyword language. Having the option to switch the language increased the probability of completing the registration survey on the first day, but did not lead to a sustained increase in engagement. | 2018 | Kenya | Kenya Ministry of Agriculture | Communication technology, Message narration | Completed | Long Rains | A/B test |
| The Effect of an SMS Rating Survey on Platform Engagement and Practice Awareness | PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. Farmers are provided with four menu options: Pesticides, Managing Fall Armyworm (FAW), Detecting FAW, and FAW Origins and Lifecycle. Upon selecting a specific topic area, a farmer has a 25% chance of being asked to take a satisfaction survey via SMS. The survey asks the farmer to rate how useful the MoA-INFO platform is and if the farmer plans to follow the recommendations. Receiving the rating survey decreased the opt-out rate by 1.4 percentage points (pp) and decreased access to additional menu topics immediately following the survey by 6.1 pp. The pesticides menu had the highest average rating but the lowest likelihood of the recommendations being followed; the managing-FAW menu had the lowest average rating but the highest likelihood of the recommendations being followed. | 2018 | Kenya | Kenya Ministry of Agriculture | Communication technology | Completed | Short Rains | A/B test |
| Kenya Fall Armyworm Trial 2019 | One Acre Fund (OAF) is an agricultural service provider that provides support for smallholder farmers in Africa in accessing agricultural inputs, training, and markets, to help the farmers increase their harvests and income. PxD and OAF began collaborating in 2016 on efforts to increase adoption of agricultural inputs and improve OAF operations in Kenya and Rwanda. PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. In this project PxD and OAF provided information on Fall Armyworm (FAW) to OAF farmers via SMS. OAF invited its members to access information about FAW on PxD’s two-way MoA-INFO SMS platform. The OAF farmers who registered on the platform were randomly assigned to receive different volumes and types of messages. OAF conducted a follow-up phone survey after the farming season, which allowed us to quantify the effects of the service on farmer knowledge and practices. Treatment-group farmers who received push messages and reminders about the platform were 125% more likely to access the FAW menu compared to control-group farmers. Farmers’ knowledge about FAW was positively associated with the SMS service, although this effect was not statistically significant in most cases. Knowledge improvements were significant for some topics that control farmers understood at low levels. Similarly, the SMS intervention had (mostly insignificant) positive effects on farmers’ adoption of recommended practices. The probabilities of treatment farmers sharing the information with and recommending the platform to other group members were high. | 2019 | Kenya | One Acre Fund (OAF) | Communication technology, Pest management | Completed | Long Rains | Other |
| Rwanda Soil Health Trial 2020 | One Acre Fund (OAF) is an agricultural service provider that provides support for smallholder farmers in Africa in accessing agricultural inputs, training, and markets, to help the farmers increase their harvests and income. PxD and OAF began collaborating in 2016 on efforts to increase adoption of agricultural inputs and improve OAF operations in Kenya and Rwanda. This trial tested whether variation in SMS messages within farmer groups affected the adoption of recommended soil health inputs, including agricultural lime and fertilizer, by OAF farmers in Rwanda. Farmers from diverse-message groups received messages that were different from the messages received by their group members; farmers from same-message groups received the same messages as their group members did. We find that farmers from diverse-message groups were marginally less likely than farmers in same-message groups to order lime, although the difference was very small and only marginally significant after we accounted for multiple hypothesis testing. Diverse-message groups were no more likely than same-message groups to adopt fertilizer. We do not find differences in outcomes across the four message framings that we tested. These findings offer a counterpoint to earlier suggestive evidence from similar SMS trials in Rwanda (for example, the “2018 Rwanda Lime Trial”), which showed positive effects of message diversity within farmer groups. | 2019 | Rwanda | One Acre Fund (OAF) | Communication technology, Message framing | Completed | Season B | A/B test |
| Rwanda Soil Health Trial 2019 | One Acre Fund (OAF) is an agricultural service provider that provides support for smallholder farmers in Africa in accessing agricultural inputs, training, and markets, to help the farmers increase their harvests and income. PxD and OAF began collaborating in 2016 on efforts to increase adoption of agricultural inputs and improve OAF operations in Kenya and Rwanda. We evaluated the effects of an SMS campaign in Rwanda that encouraged farmers to adopt recommended soil health inputs, particularly agricultural lime. We first randomized whether individuals in a farmer group received identical messages, diverse messages, or no messages. In the groups that were assigned to receive messages, we randomized individual farmers to receive no message, two messages, or four messages in a six-day time window. We find that the SMS campaign increased lime adoption by 12% overall but did not significantly affect fertilizer use. Groups that received diverse messages saw higher lime adoption, but this difference is not statistically significant. We find that farmers who received four SMS messages were more likely to adopt recommended inputs than those who received only two messages. | 2018 | Rwanda | One Acre Fund (OAF) | Communication technology, Message framing, Message timing and frequency | Completed | Season A | Impact Evaluation |
| Kenya Repayment Trial 2019 | One Acre Fund (OAF) is an agricultural service provider that provides support for smallholder farmers in Africa in accessing agricultural inputs, training, and markets, to help the farmers increase their harvests and income. PxD and OAF began collaborating in 2016 on efforts to increase adoption of agricultural inputs and improve OAF operations in Kenya and Rwanda. OAF regularly tests different solutions to improve repayments. One such approach is outreach by call-center field officers (FOs) to struggling farmers and their group leaders (GLs), reminding them of the group-liability structure. This outreach encourages the farmers to make a payment to get back on a healthy repayment path. An important aspect of the calls is asking farmers if they would like to pledge to pay a certain amount by the end of the following week. On average, about half of the farmers make a pledge, and OAF shares this information with FOs, who can then follow up and enforce this informal commitment device. In the previous year OAF ran a “Kenya Repayment 2018 trial” to test the effect of FO calls to farmers; this trial found suggestive but statistically insignificant results that calls are effective at increasing repayment. In the 2019 long rainy season, we ran another trial in Kenya to test this intervention. In this second trial, we find that calls to farmers had no effect on overall repayment rates. | 2019 | Kenya | One Acre Fund (OAF) | Communication technology | Completed | Long Rains | Impact Evaluation |
| Kenya Down Payment Trial 2019 | One Acre Fund (OAF) is an agricultural service provider that provides support for smallholder farmers in Africa in accessing agricultural inputs, training, and markets, to help the farmers increase their harvests and income. PxD and OAF began collaborating in 2016 on efforts to increase adoption of agricultural inputs and improve OAF operations in Kenya and Rwanda. In this trial, OAF and PxD ran an experiment to test whether varying message content could improve an SMS program designed to nudge farmers to complete the OAF down payment by the deadline. We hypothesized that using diverse content and framings will increase the chances of reaching a farmer with a message that they find persuasive, and that sending different messages tailored to the times in the prepayment period will be particularly effective. To test this, we selected promising message styles from the previous year’s trial “Kenya Down Payment Trial 2018”. Farmers were divided into a control group, which received the same, randomly-assigned message three times, and a treatment group, which received all three messages in random order. We do not find a statistically significant difference in the likelihood of completing the down payment between farmers who were sent the same message and farmers who were sent different messages. In a context where farmers receive messages that are different from the messages of their fellow group members, there may be little or no additional benefit in sending varied messages to individual farmers. | 2018 | Kenya | One Acre Fund (OAF) | Communication technology, Message framing, Message timing and frequency | Completed | Long Rains | A/B test |
| Kenya Agricultural Lime Message Customization 2019 | One Acre Fund (OAF) is an agricultural service provider that supports smallholder farmers in Africa in accessing agricultural inputs, training, and markets, to help the farmers increase their harvests and income. PxD and OAF began collaborating in 2016 on efforts to increase adoption of agricultural inputs and improve OAF operations in Kenya and Rwanda. We implemented an experiment to test whether customizing SMS messages based on previous agricultural lime purchases could improve the uptake of lime. We used regressions to predict the likelihood of adopting lime based on the message each farmer received and a series of baseline characteristics from the previous trial “Kenya Agricultural Lime Messaging Trial 2018”. We randomized farmers to receive either the previous season’s overall best message identified in that trial, which emphasized yield gains, or the message predicted by the regression model to be most likely to encourage lime adoption. Contrary to the intervention’s intention, we find suggestive evidence that receiving customized messages based on prediction characteristics slightly decreased the uptake of agricultural lime and the quantity used. Our analysis of the model used for the predictions shows that message content did not improve the predictions of the model. Identifying more predictive baseline characteristics may help to improve model performance in future experiments on message customization. | 2018 | Kenya | One Acre Fund (OAF) | Communication technology | Completed | Short Rains | A/B test |
| Kenya Repayment Trial 2018 | One Acre Fund (OAF) is an agricultural service provider that provides support for smallholder farmers in Africa in accessing agricultural inputs, training, and markets, to help the farmers increase their harvests and income. PxD and OAF began collaborating in 2016 on efforts to increase adoption of agricultural inputs and improve OAF operations in Kenya and Rwanda. OAF’s credit program requires all group members to complete repayments to qualify for the loan in the following season. A simple model of group-liability lending suggests that informing members about the group’s progress toward repayment either spurs or reduces individual repayment. In a block randomized controlled trial with nearly 300,000 OAF members, we tested how SMS message reminders about the repayment status of the individual and the group affect the farmers’ loan repayment performance. We find that the effects of SMS message reminders about repayment status are heterogeneous. Reminding borrowers about their own repayment status increased the probability of on-time repayment; informing borrowers who had paid off their loans early about their group’s repayment status had, on average, a small but significant adverse effect on the repayment performance of their group members. These findings suggest that peer monitoring in groups may backfire in underperforming groups, by lowering the individual’s incentive to repay, and that individual messages are more effective in promoting timely repayment. | 2018 | Kenya | One Acre Fund (OAF) | Communication technology, Message framing | Completed | Long Rains | A/B test |
| Rwanda Agricultural Lime Trial 2018 | One Acre Fund (OAF) is an agricultural service provider that provides support for smallholder farmers in Africa in accessing agricultural inputs, training, and markets, to help the farmers increase their harvests and income. PxD and OAF began collaborating in 2016 on efforts to increase adoption of agricultural inputs and improve OAF operations in Kenya and Rwanda. In this experiment we examined the effects of an SMS campaign that promoted the adoption of agricultural lime (a relatively new input) by Rwandan farmers, in a group-based credit and extension program. The campaign sent messages to farmers organized in groups, and we experimentally varied the message diversity and intensity, as well as the message content, framing, and repetition. On average, receiving SMS messages increased the farmer’s likelihood of purchasing lime through the program by 20% over the adoption rate of 3.4% in the control group. The SMS campaign also had a small and marginally significant effect on the likelihood of lime adoption by farmers who do not own phones; this effect suggests the presence of spillover effects. Further analysis provides tentative evidence that sending diverse messages, instead of identical messages, to farmers within a group is more effective in increasing the likelihood of lime purchase. The overall treatment effect estimated for the full sample of phone owners and non-owners for groups that received diverse messages is twice as large as the estimate for those that received identical messages. An additional SMS message encouraging farmers to share information had no spillover effect. | 2017 | Rwanda | One Acre Fund (OAF) | Communication technology, Message framing | Completed | Season A | A/B test |
| Kenya Down Payment Trial 2018 | One Acre Fund (OAF) is an agricultural service provider that provides support for smallholder farmers in Africa in accessing agricultural inputs, training, and markets, to help the farmers increase their harvests and income. PxD and OAF began collaborating in 2016 on efforts to increase adoption of agricultural inputs and improve OAF operations in Kenya and Rwanda. This trial focused on identifying SMS nudges that meaningfully increase the rates of farmers qualifying for the OAF program to receive inputs on credit. We examined early qualification, which happens when a farmer completes the down payment of 500 KES by an early date to qualify for the loan. Early down payment offers two key advantages: It helps cash-constrained farmers secure program qualification before the Christmas period, which has increased spending needs, and it extends the loan repayment timeline once farmers qualify, thus potentially improving their repayment rates. We tested multiple SMS messaging strategies to encourage farmers to complete the down payment by an early date. Sending SMS reminders significantly increased down payment rates, compared to not sending messages. We find no meaningful differences in the effectiveness of different message framings or timing. | 2017 | Kenya | One Acre Fund (OAF) | Communication technology, Message framing | Completed | Long Rains | A/B test |
| Motivational Messages for Extension Workers in Rwanda Tree Program 2018 | One Acre Fund (OAF) is an agricultural service provider that supports smallholder farmers in Africa in accessing agricultural inputs, training, and markets, to help the farmers increase their harvests and income. PxD and OAF began collaborating in 2016 on efforts to increase adoption of agricultural inputs and improve OAF operations in Kenya and Rwanda. Governments in low-income countries rely on embedded community agents to provide frontline services in sectors critical for development such as education, health and agriculture. Understanding how best to motivate these agents, in settings where oversight might be difficult, is critical to improving development outcomes for the communities they serve. OAF, in collaboration with PxD, conducted a field experiment to evaluate the effect of phone-based motivational messages on the performance of agriculture extension agents in Rwanda. The extension agents were volunteer Farmer Promoters (FPs) at the village level, and paid socio-economic development officers (SEDOs) employed by the government at the cell level (administrative unit in Rwanda one level up from the village). Agents were tasked with mobilizing farmers for a nationwide tree-distribution agroforestry campaign that encouraged farmers to sign up to receive nursery trees and plant them on their farms. We implemented an SMS campaign to nudge agents to register farmers for the campaign and ensure that the farmers arrived on the tree-distribution day. The experiment tested the relative effects of messaging FPs, messaging SEDOs, or messaging both the SEDO and FP, for the village in question. We identified the effects of the SMS nudges on FP performance by the share of their target number of farmers who arrived on the tree-distribution day. We find that direct motivational messages to the FP village-level extension agents increased the farmer turnout for the campaign by 5 to 7 percentage points (pp) of the fraction of the target met over the control mean of 90%. Motivational messages sent to SEDO cell-level extension agents had no effect. Our findings demonstrate that non-financial motivational nudges aimed at frontline service-provider agents can improve program outcomes in developing country contexts. Our findings also highlight the potential of mobile phones to improve the delivery of agriculture extension services. | 2018 | Rwanda | One Acre Fund (OAF) | Communication technology, Extension agents, Message framing | Completed | Season A | Impact Evaluation |
| The Impact of Digital Extension Messages on the Prevention and Management of Fall Armyworm | Fall armyworm (FAW) is a pest that spread from the Americas to sub-Saharan Africa in 2016. It is a fast-reproducing species that causes substantial crop damage. PxD and the Centre for Agriculture and Bioscience International (CABI) implemented an SMS campaign in four provinces (Southern, Luapula, Central, and Eastern) of Zambia using a digital platform managed by the Zambian Ministry of Agriculture to provide smallholder farmers with timely advice on how to prevent and manage FAW infestations throughout the 2019–2020 season. We evaluated the impact of delivering digital extension messages on FAW prevention and management via the Zambia Integrated Agricultural Management Information System (ZIAMIS) platform to farmers registered for Zambia’s Farmer Input Subsidy Program (FISP). Intervention messages covered FAW monitoring, cultural pest control practices, preventive methods, and correct use of pesticides and fertilizers. We used follow-up questions to evaluate farmers’ actions based on whether they spotted FAW and used pesticides. Results show that farmers in the treatment group had significantly higher scores on both FAW knowledge and recommended practice adoption indices. The estimated treatment effects do not vary significantly across gender or SMS use frequency. | 2019 | Zambia | CABI, Smart Zambia Institute, Zambia Ministry of Agriculture | Communication technology, Pest management | Completed | Rainy Season | Impact Evaluation |
| Messaging Approaches to Promote Nutritious Vegetables | One Acre Fund (OAF) is an agricultural service provider that supports smallholder farmers in Africa in accessing agricultural inputs, training, and markets, to help the farmers increase their harvests and income. PxD and OAF began collaborating in 2016 on efforts to increase adoption of agricultural inputs and improve OAF operations in Kenya and Rwanda. Micronutrient deficiencies are widespread in rural Kenya. According to the Kenya National Micronutrient Survey conducted in 2011, 83.3% of preschool-age children in rural areas suffered from zinc deficiency, and 26.3% from anemia. Deficiencies in key micronutrients like iron and zinc can hinder growth and cognitive function and are major contributors to child and maternal mortality in the developing world, particularly in sub-Saharan Africa. In partnership with OAF, we evaluate one approach to addressing this issue: an SMS campaign informing OAF farmers about the agronomic and nutritional benefits of zinc-fortified beans and nutritious vegetables. OAF offers the opportunity for farmers to buy these seed varieties on credit. We find that SMS messages increased adoption of zinc-fortified beans and recommended vegetables by 1.8 and 1.0 percentage points (pp), respectively. These increases constitute increases over the control group of 89.3% (control mean 2%) and 3.4% (control mean 29.2%) in each recommended input. Based on the cost of SMS dissemination at OAF, we estimate that the intervention increased adoption at a cost of $0.57 and $0.97 per additional farmer adopting zinc-fortified beans and recommended vegetables, respectively. Furthermore, we find significant differences in adoption outcomes between different message framings: Content that emphasized agronomic properties and yield-potential was particularly effective at encouraging bean adoption, while messaging that focused on preventing anemia was particularly effective at encouraging vegetable adoption. We also tested if varying messages, both over time and across neighboring farmers, could facilitate social learning and increase adoption, but we find no evidence that message variation increased adoption. Lastly, we explore heterogeneity by gender, and find that, while adoption of zinc-fortified beans was significantly higher among women in the control group, men increased their adoption more than women did as a result of the treatment; we do not find significant gender heterogeneity in vegetable adoption. | 2019 | Kenya | One Acre Fund (OAF) | Communication technology, Gender, Message framing, Message timing and frequency, Social learning | Completed | Long Rains | Impact Evaluation |
| Intensive Training to Use the Krishi Katha Service | PxD is working with the West Bengal Accelerated Development of Minor Irrigation Project (WBADMI) to implement a mobile phone-based extension system, Krishi Katha, for smallholder farmers belonging to the Water User Associations (WUAs) formed under the ADMI project across West Bengal. The service uses a two-way Interactive Voice Response (IVR) platform with “outbound” push calls and an “inbound” hotline service. This experiment tested whether training farmers to use the Krishi Katha inbound service, and reminding them how to use it, improves the outcomes of service engagement, knowledge about pesticides used to tackle various pests and diseases, and adoption of recommended practices involving seed treatment and fertilizer application. Farmers were randomly assigned to three treatment arms at the WUA level so that some farmers had access to the inbound service only, some had access to both inbound and outbound services, and some received an additional package of intensive training and encouragement interventions. We find that the intensive training interventions significantly improved farmers’ use of the inbound service, but did not improve knowledge and adoption outcomes. | 2019 | West Bengal, India | ADMI | Agricultural management advice, Communication technology, Message narration, Message timing and frequency, Service design | Completed | Kharif | A/B test |
| Impact Evaluation of Ama Krushi | PxD operates Ama Krushi, a free agriculture information service delivered over mobile phones, in collaboration with the State Government of Odisha Department of Agriculture, using a two-way Interactive Voice Response (IVR) platform with “outbound” push calls and an “inbound” hotline service. Ama Krushi provides customized and real-time agricultural advice, including weather-based advisories, pest- and disease-management guidance, and best-practice advisories for soil health and nutrient management. Weekly advice is dynamically customized based on the common problems, such as pest outbreaks, that farmers report via the hotline. By June 2022, the service was reaching over 3.2 million farmers and covered 28 value chains, including several crops, animals, and fisheries. We evaluate the impact of the Ama Krushi digital agricultural advisory service at scale. We randomized the rollout of the service to 13,675 rice farmers in five districts, and measured the impact on agricultural outcomes using both survey and remote sensing data. Using survey data, we find that access to the digital service led to significant improvements in farmers’ knowledge and adoption of recommended practices, a modest increase in rice yield and harvest, and a large reduction in the likelihood of rice crop loss, on average. Further analyses suggest that the treatment impact is concentrated in areas hit by certain types of weather shocks; the treatment increased harvest by up to 9% and reduced severe crop loss by up to 21% in affected areas. We used vegetation indices (VIs) to construct an objective yield measure for all farmers in the study sample and confirm that our key survey results are robust against differential attrition, reporting biases, and survey sample selection. While the VI-predicted yield provides valuable validation of survey results, our analysis highlights the need for methodological improvements in the effective application of remote sensing data to measure program impacts on agricultural outcomes. | 2020 | India, Odisha, India | Government of Odisha, J-PAL | Agricultural management advice, Communication technology, Measurement methods, Weather information | Completed | Kharif | Impact Evaluation |
| Adding Menu Replay if No Option is Selected (ATA Experiment 102) | PxD is partnering with Ethiopia’s Agricultural Transformation Agency (ATA) to help improve the effectiveness of their voice-based mobile-phone advisory service, the 8028 hotline, by conducting continuous iterations and experiments, as well as by making suggestions for improvements to and customization of the service. The service has millions of registered farmers and represents the first in Africa to be maintained by a government entity at such a large scale. The 8028 hotline Interactive Voice Response (IVR) system uses phone key navigation. If a user is inactive for 10 seconds in any menu, then the system hangs up automatically. Users’ inactivity could be due to their difficulty in using the system (e.g., digital illiteracy) or because the options are read too quickly and the user has not yet decided which menu to access. We implemented an intervention to replay the menu if no option is selected in 10 seconds, for randomly selected farmers. We find that this feature significantly increased the probability of users accessing advisory content for the users affected by the treatment, meaning users assigned to the treatment group who did not make a menu selection within 10 seconds. | 2017 | Ethiopia | Ethiopian ATA | Communication technology | Completed | _N/A | A/B test |