The Impact of Heads-up SMS Messages on Call Pick-up and Listening Rates
IND -21 -1651Last modified on December 19th, 2025 at 10:39 am
-
Abstract
PxD operates Ama Krushi, a free agriculture information service delivered over mobile phones, in collaboration with the State Government of Odisha Department of Agriculture, using a two-way Interactive Voice Response (IVR) platform with “outbound” push calls and an “inbound” hotline service. The outbound service is the primary way in which PxD provides farmers with timely, customized agricultural information. We wanted to increase farmers’ engagement with the outbound service, in terms of pick-up and listening rates. We explored whether sending a “heads-up” SMS message to farmers increases their likelihood of picking up and listening to calls. We find that these reminders, in particular a reminder 10 minutes before the call, increased the pick-up and listening rates. Non-smartphone users and users with above-median historical listening rates show larger gains in both pick-up and listening outcomes than smartphone users and less historically engaged users do.
-
Status
Completed
-
Start date
Q3 Jul 2021
-
Experiment Location
Odisha, India
-
Partner Organization
Government of Odisha
-
Agricultural season
Kharif
-
Experiment type
A/B test
-
Sample frame / target population
All active Ama Krushi users
-
Sample size
32,000
-
Outcome type
Service engagement
-
Mode of data collection
PxD administrative data
-
Research question(s)
1. Do heads-up SMS messages telling farmers they will be receiving a push call shortly increase the likelihood of the farmers picking up and listening to the weekly advisory calls?
2. Are heads-up SMS messages more effective for any particular kinds of farmers? -
Research theme
Communication technology, Message timing and frequency
-
Research Design
We selected approximately 32,000 farmers who were active Ama Krishi users as the sample frame. We then randomized farmers at the individual level into three groups:
1. Treatment 1: Received a heads-up SMS message 10 minutes before a push call.
2. Treatment 2: Received a heads-up SMS 1 hour before the push call.
3. Control group: Received no heads-up SMS message.We stratified the randomization by prior outbound engagement (above or below the median listening rate in the previous season), smartphone ownership, and gender.
We designed the intervention to test whether sending a heads-up SMS shortly before a scheduled advisory call increases the likelihood that farmers pick up and listen to the call. The experiment ran over four weeks to account for week-to-week variation and to estimate an average treatment effect.
We used administrative data to measure call pick-up and listening rates as primary outcomes of interest. Based on previous benchmarks, the experiment is powered to identify a Minimum Detectable Effect (MDE) of 1.9–2 percentage points (pp) in pick-up and listening rates, using an average baseline listening rate of 51.4% and a pick-up rate of 57.0%. Due to SMS delivery constraints, the sample size was adjusted to maintain statistical power.
-
Results
SMS message reminders, in particular a 10-minute reminder, increase the likelihood of farmers picking up and listening to advisory calls. We find a 5.9 pp increase in the pick-up rate (~17% over a control mean of 34%) and a 4 pp increase in the listening rate (~26% increase over a control mean of 15.4%) in the 10-minute group. The 1-hour group had a smaller but still significant positive impact.
We also identify significant heterogeneity in treatment effects: There were larger gains in both pick-up and listening outcomes for non-smartphone users and users with above-median historical listening rates than for smartphone users and less historically engaged users.