The Effects of Fewer and Revised Recommendations on Service Engagement and Agricultural Practices
IND -20 -1496Last modified on December 19th, 2025 at 10:19 am
-
Abstract
PxD operates Ama Krushi, a free agriculture information service delivered over mobile phones, in collaboration with the State Government of Odisha Department of Agriculture, using a two-way Interactive Voice Response (IVR) platform with “outbound” push calls and an “inbound” hotline service.
This A/B test examines the impact of three adjustments to our standard Ama Krushi service on farmers’ engagement with the service and on the adoption of the inputs we recommended to paddy farmers during the 2020–21 Rabi season. Specifically: (1) We sent fewer messages on high impact practices; (2) we used a structured message development template to develop content; (3) for each message, we conducted qualitative interviews with farmers who were not in the sample but were eligible for inclusion in the sample, to gather feedback on the comprehensibility and actionability of the messages; and (4) we sent the advisory message a second time to help farmers remember the advice better.
We find that providing farmers with fewer, re-framed messages over a season increases their pick-up rates by 4 percentage points (pp) and their listening times by around 20 seconds per paddy message. Although the analysis was not powered to detect adoption effects, our qualitative findings suggest various mechanisms by which fewer and simpler messages could drive the impact on engagement and practices. -
Status
Completed
-
Start date
Q4 Nov 2020
-
End date
Q3 Jul 2021
-
Experiment Location
Odisha, India
-
Partner Organization
Government of Odisha
-
Agricultural season
Rabi
-
Experiment type
A/B test
-
Sample frame / target population
All Ama Krushi farmers who meet the following criteria: (1) Located in Balasore or Bargarh district; (2) “Active” farmers, i.e., have picked up at least one call in the six months before the start of the experiment; (3) Reported planting paddy in Rabi during profiling.
-
Sample size
35,573
-
Outcome type
Platform engagement, Farming practices, User experience
-
Mode of data collection
Partner administrative data, Phone survey
-
Research question(s)
Does providing fewer high priority advisory messages with structured framing increase service engagement and adoption of key agricultural practices?
-
Research theme
Agricultural management advice, Communication technology, Message framing, Message narration, Message timing and frequency
-
Research Design
Half of the 35,573 “active” Ama Krushi users in the Balasore or Bargarh district that reported planting Rabi paddy were randomly allocated to receive modified paddy advisory messages. The randomization was stratified by district, gender, and farmers’ Ama Krushi engagement during the last Kharif season (2020).
To measure the effect of modified messages on farmer engagement, we used the administrative data on engagement outcomes collected by the Ama Krushi system. We collected no baseline survey data; however, we collected survey data at the endline on farmers’ adoption of priority practices for a small share of our study sample to estimate the effect of the treatment on the adoption of inputs recommended in the paddy advisory. We attempted to interview 1,000 farmers about their agricultural practices, but only 556 completed the survey. There was no indication of differential attrition, but the small sample size reduces the experiment’s power to detect adoption effects.
Treatment effects were estimated using linear and logistic models for binary variables and linear models for continuous variables. We used robust standard errors and cluster standard errors where appropriate.
-
Results
We find that providing fewer and better-communicated messages increased farmers’ pick-up rates by 4 pp over a control group mean of 65%. The revised advisory also affected farmers’ listening behavior. Although listening rates were 5 pp lower in the treatment group than in the control group, listening times were around 10 seconds longer for all messages and about 20 seconds longer for paddy messages. This suggests that the average farmer took advantage of the additional information (i.e., the repetition of the detailed recommendation) in the revised messages.
Of the nine adoption outcomes that we analyzed, we found a statistically significant impact at the 10% level for just one (seed treatment), which is the rate that we would expect to occur by chance given the relatively small sample size for the endline survey.