Login to view full entry:
Blast Experiments: Names, Active Language, Proverbs, & Quiz Scores
KEN -19 -2641Last modified on October 29th, 2025 at 3:38 pm
-
Abstract
One Acre Fund (OAF) is an agricultural service provider that provides support for smallholder farmers in Africa to access agricultural inputs, training, and markets, to help the farmers increase their harvests and income. PAD and OAF began collaborating in 2016 on efforts to increase adoption of agricultural inputs and improve OAF operations in Kenya and Rwanda. PAD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS.
During the long rainy season 2019 (LR 2019), we sent regular push messages to OAF farmers encouraging them to access the Fall Armyworm (FAW) components of the MoA-INFO platform, namely the FAW menu of informational topics, the FAW monitoring tool, and the FAW misconception quiz. While sending these messages in the "OAF Kenya Fall Armyworm 2019" trial, we implemented a series of four A/B tests we called “Blast Experiments” to test the effects of message design tweaks on farmer access to the FAW components of the MoA-INFO platform.
1. Names: We tested the effect of including users’ first names in invitation messages on farmers’ access to the FAW components. The intervention increased access to both the FAW menu and the FAW misconception quiz by 3 percentage points (pp) over 19% in the control group, but did not have a statistically significant effect on access to or completion of the FAW monitoring tool.
2. Active language: We tested the relative effects of language, with active "now" framing compared to "at any time" framing, on farmer access to the FAW components. Using “send MENU… now” messages increased access to the menu by 3 pp over the 12% access rate of the “send MENU… at any time” messages.
3. Proverbs: We tested the effect of including one of various Swahili proverbs at the beginning or at the end of an invitation message on farmer access to the FAW components. Inserting a Swahili proverb at the start of an invitation message increased access to the FAW menu, but had no effect when the same proverb was placed at the end of the message.
4. Quiz scores: We tested the effect of including farmers’ prior best FAW-quiz score in invitation messages on their access to the FAW misconception quiz. The intervention increased access rates by 11 pp over 57% in the control group, which indicates that performance-based framing can meaningfully boost re-engagement. -
Status
Completed
-
Start date
Q2 Apr 2019
-
End date
Q3 Jul 2019
-
Experiment Location
Kenya
-
Partner Organization
Kenya Ministry of Agriculture , One Acre Fund (OAF)
-
Agricultural season
Long Rains
-
Experiment type
A/B test
-
Sample frame / target population
OAF members registered for MoA-INFO
-
Sample size
25,780
-
Outcome type
Information access, Service engagement
-
Mode of data collection
PxD administrative data
-
Research question(s)/hypotheses
Do push messages affect farmers’ access to FAW components of the MoA-INFO platform when the messages
1. include the farmer’s name,
2. use active “now” framing language compared to “at any time” framing, or
3. include Swahili proverbs?
4. For invitation messages, does the inclusion of a farmer’s prior best score on the FAW quiz affect access to the FAW misconception quiz on the MoA-INFO platform? -
Research theme
Communication technology
-
Research design notes
The push and invitation messages were randomized as the messages were sent, and the randomizations were not blocked or stratified in any way.
The main outcomes of interest are whether specific platform components (the FAW menu of informational topics, the FAW monitoring tool, and the FAW misconception quiz) were accessed or completed. Accessed means the users started the tool, either by replying to the invitation message or sending in a keyword (such as CHECK) to activate the tool. Completion means they finished using the tool. In the case of the FAW monitoring tool, users entered five values for their FAW infestation and the height of their maize to complete using the tool; the farmers then received a recommendation customized to the information they had shared.
For further information on the trial that these experiments were part of, see Kenya Fall Armyworm Trial 2019
-
Results
1. Names: Including users’ first names in message invitations increased access to the menu by 3 pp over a 19% control and to the FAW misconception information by 3 pp over a 19% control. However, it had no statistically significant effect on access to or completion of the FAW monitoring tool.
Our interpretation is that using a name can make a message more noticeable, and so is good for low commitment invitations like asking users to receive messages. But including the name makes no difference for higher commitment components such as the monitoring tool—where a user has to go out to their farm to use the tool.
2. Active language: Using the active language “send MENU… now” increased access to the menu by 3 pp over the 12% access rate of the “send MENU… at any time” message. The gap in access rates did not close over time, which suggests that the immediate effect of the “now” framing is more important than the potentially longer-term effect of “at any time” framing.
For the monitoring tool, “Send CHECK to 40130…” received a higher response rate than “Reply CHECK…” did. This might be because some low-end phones do not have a reply feature, so it is advisable to remind users of the MoA-INFO shortcode frequently.
3. Proverbs: Placing the proverb “One who studies goes on learning!” at the beginning of an invitation message increased access to the menu by 1.5 pp over the 11.5% of the control group. The same proverb at the end of the invitation had no statistical effect. No proverbs increased the rates of access to the FAW monitoring tool; we had placed proverbs only at the end of these messages, not at the beginning. For the misconception quiz, the proverb “One is never too old to learn!” increased access rates slightly more than the sentence “Test your knowledge of FAW!” did, although the difference was not significant.
4. Quiz scores: For users that had previously completed the Fall Armyworm misconceptions quiz, including their previous best quiz score (out of 5) in the invitation message increased their response rates by 11 pp over the control-group mean (57%).