The Timing of UCAT SMS Reminders
UGA -20 -1427Last modified on December 19th, 2025 at 10:17 am
-
Abstract
PxD is partnering with Hanns R. Neumann Stiftung (HRNS) and TechnoServe (TNS) on the Uganda Coffee Agronomy Training (UCAT) program to provide a complementary digital service, which reinforces recommendations of Good Agricultural Practices (GAPs) via automated calls to subsets of coffee farmers. PxD is also offering a “standalone” customized digital advisory service to a subset of farmers in villages that were randomly selected to serve as a control group and did not receive the in-person UCAT program, as part of a large-scale impact evaluation.
PxD sent SMS message reminders prior to Interactive Voice Response (IVR) push calls to these farmers who were receiving standalone digital agricultural advisory messages. To test the optimal timing of reminders, we randomly assigned farmers to receive SMS reminders one hour before the weekly push calls or 24 hours before the weekly push calls.
We implemented the experiment for four weeks and measured farmers’ level of engagement with the platform, as call pick-up and completion rates. The 24-hour SMS reminder was more effective than the one-hour SMS reminder for increasing farmer engagement with weekly push calls. These effects, however, are driven largely by particular experiment weeks. -
Status
Completed
-
Start date
Q3 Aug 2020
-
End date
Q4 Oct 2020
-
Experiment Location
Uganda
-
Partner Organization
UCAT, IFPRI
-
Agricultural season
Long Rains
-
Experiment type
A/B test
-
Sample frame / target population
Farmers who do not attend in-person training but receive UCAT ICT treatment
-
Sample size
1,570
-
Outcome type
Service engagement
-
Mode of data collection
PxD administrative data, Partner administrative data
-
Research question(s)
Is sending a reminder one day before or one hour before scheduled push calls more effective?
-
Research theme
Communication technology, Message timing and frequency
-
Research Design
The study sample was drawn from the UCAT impact evaluation standalone treatment group. All farmers in this group, except for those who had never picked up any UCAT calls, were included in the sample. The resulting 1,570 coffee farmers were receiving standalone digital advisory treatment and were not receiving in-person training. The intervention was randomized at the individual level into two treatment groups, stratified by villages, the implementer location (HRNS or TNS), and the language the farmer speaks. The groups received reminders as follows:
1. T1 (n = 790): Received an SMS reminder one hour before the weekly push call.
2. T2 (n = 780): Received an SMS reminder 24 hours before the weekly push call.The content of the SMS reminder was identical for the T1 and T2 groups. The study ran for four weeks (from August 12, 2020 to September 15, 2020) with each farmer in the study receiving four reminders in total. Due to implementation issues, reminders were not sent during the week of September 2, so we extended the study for another week to complete the four weeks of implementation.
-
Results
The 24-hour SMS reminder was more effective than the one-hour SMS reminder for increasing farmer engagement with weekly push calls. Across four weeks of testing, farmers who received the SMS reminder 24 hours before the weekly IVR push call were 3.5 percentage points (pp) more likely to pick up the call than farmers who received the reminder one hour before (73.8% vs. 70.3%). This effect is statistically significant at the 5% level, but it was mostly driven by effects in the second week of implementation (week of August 17). Farmers in the 24-hour reminder group were also 3.7 pp more likely to listen to at least 90% of the call when they picked up (44.5% vs. 40.8%). Similarly, this pooled effect is statistically significant at the 5% level, but was driven by stronger effects in the first week of implementation.