The Effect of Reminder and Instructional Messages on Inbound Engagement
IND -20 -1417Last modified on December 19th, 2025 at 10:17 am
-
Abstract
PxD operates Ama Krushi, a free agriculture information service delivered over mobile phones, in collaboration with the State Government of Odisha Department of Agriculture, using a two-way Interactive Voice Response (IVR) platform with “outbound” push calls and an “inbound” hotline service.
This experiment tested the impact of sending a series of reminder and instructional messages in different styles on farmers’ likelihood of engaging with the Ama Krushi inbound service. Farmers were randomly assigned to one of five groups: a control group receiving no reminder, or one of four treatment groups receiving biweekly messages in one of four formats—a standard advisory, a conversational script between farmers, a farmer’s testimonial, or a social nudge highlighting service usage statistics. We also conducted qualitative feedback surveys to understand the farmers’ motivation and engagement with the inbound service.
Reminder messages modestly increased the likelihood of farmers calling the inbound service, particularly farmers who had previously engaged with Ama Krushi. However, varying the message framing beyond the standard advisory did not enhance this effect, and no significant impact was found on the likelihood of farmers asking a valid question during the call. -
Status
Completed
-
Start date
Q2 Jun 2020
-
End date
Q3 Aug 2020
-
Experiment Location
Odisha, India
-
Partner Organization
Government of Odisha
-
Agricultural season
Kharif, Rabi
-
Experiment type
A/B test
-
Sample frame / target population
Ama Krushi farmers
-
Sample size
15,914
-
Outcome type
Information access, Service engagement
-
Mode of data collection
PxD administrative data, Phone survey
-
Research question(s)
1. Are regular reminder and instructional messages effective in increasing usage of the IVR service?
2. Which message styles are most effective?
3. Do these messages have different effects on subgroups of users? -
Research theme
Communication technology, Message narration, Service design
-
Research Design
The sample included farmers who were: (1) “active” on the outbound service, meaning they had received and picked up at least one call in the previous six months; and (2) growing crops in the Kharif 2020 season. The different groups in this experiment were:
1. Control: Farmers received no reminder messages.
2. Standard: Farmers received reminders and instructional voice messages with information about the inbound service.
3. Conversational: Farmers received standard messages framed as a conversation between two farmers; the conversation reminded farmers about the inbound service and provided instructions on how to call in.
4. Farmer testimonial: Farmers received standard messages and a recorded testimonial from an Ama Krushi inbound-service user about their experience using the service.
5. Social nudge: Farmers received a short trivia-style announcement that provided statistics about how their peers use the inbound service, followed by standard messages.We sent treatment farmers one message framed in the style of the assigned treatment group every two weeks for a total of five messages over ten weeks. Control group farmers received no reminder messages.
-
Results
Reminder messages increased farmer engagement but varying the message framing from the standard style did not increase the effect. The standard reminder messages increased farmers’ propensity to call the hotline by ~1.8 percentage points (pp) over a control mean of 2.7 percent. This impact was greater on farmers who were already engaged with the Ama Krushi outbound service (2.4 pp more likely to call in) or had previous inbound engagement with Ama Krushi (1.6% of control group farmers who hadn’t called in, compared to 5.6% of treatment farmers who had previously called in). However, we don’t find any statistically significant impact of the messages on increasing a farmer’s likelihood of asking a valid question.