Login to view full entry:
Automated versus Live Human Follow-up Calls
KEN -21 -1663Last modified on December 19th, 2025 at 10:41 am
-
Abstract
PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. To identify strategies for boosting farmers’ engagement with the service, we tested the effectiveness of different follow-up modalities (receiving an automated call or a live human call) for users who initiated the registration process.
Farmers who received blast invitations from Safaricom (a large Kenyan phone company) and started registration on the MoA-INFO platform in the long rainy season 2021 (LR 2021) were randomly assigned to a control group, or to two treatment groups that received either an automated follow-up call or a live human follow-up call.
Receiving an automated Interactive Voice Response (IVR) call is associated with a 7.6 percentage point (pp) increase in the probability of accessing the menu compared to control farmers. However, live human calls seem to have an impact on more measures of platform engagement than automated calls do. Therefore, having an initial human contact following registration may drive more engagement by farmers than having an automated call does. -
Status
Completed
-
Start date
Q1 Mar 2021
-
Experiment Location
Kenya
-
Partner Organization
Kenya Ministry of Agriculture
-
Agricultural season
Short Rains
-
Experiment type
A/B test
-
Sample frame / target population
Farmers who received Safaricom blast invitations and started registration in the MoA-Info platform in the LR 2021 season
-
Sample size
600
-
Outcome type
Platform engagement
-
Mode of data collection
Automated survey, Phone survey
-
Research question(s)
Do automated and live human follow-up calls to users who begin service registration increase engagement with the platform?
-
Research theme
Communication technology
-
Research Design
The sample consisted of 600 farmers who were part of the Safaricom blast and had initiated registration on MoA-INFO during the LR 2021 season. Farmers were randomly assigned to a control group or to two treatment groups that received either an automated or a live human follow-up call. Randomization was conducted at the individual farmer level and stratified by county.
Farmers were allocated into six groups, based on random assignment and whether they had opted-in to the LR 2021 cropping series (CS):
- Control group (n = 200)—No call:
• C1 (n = 100): Started registration but did not opt-in to the CS and were not assigned to receive CS or high-priority messages (in another A/B test).
• C2 (n = 100): Completed registration and opted-in to the CS.
- Treatment 1—Received an automated IVR call (n = 200)
• T1a (n = 100): Opted-in to the CS.
• T1b (n = 100): Did not opt-in to the CS.
- Treatment 2—Received a live human call from an enumerator (n = 200)
• T2a (n = 100): Opted-in to the CS.
• T2b (n = 100): Did not opt-in to the CS.
We used administrative platform data for menu browsing and content access to measure the farmers’ engagement with the MoA-INFO service after they had received the calls.
- Control group (n = 200)—No call:
-
Results
Receiving a live human call from an enumerator is positively associated with all engagement variables, and statistically significant, notably, for menu access.
Farmers who received an automated IVR call were 7.6 pp more likely to access the menu, compared to control farmers. Several coefficients are negative, although they are not significant. However, the sample size does not allow us to rule out that the IVR treatment may have a negative impact on engagement.
Giving a live human call to new MoA-INFO users appears to positively affect their future engagement with MoA-INFO services; this occurs mostly amongst users who have already opted-in to the CS. The impact of receiving an automated IVR message is more mixed, as this call seems to lead to lower engagement by treatment than by control farmers for many variables, although these results are not statistically significant. Therefore no definite conclusions can be drawn about the effectiveness of automated IVR calls for following up with new users.