Login to view full entry:
Improving Invitation Messaging to Increase Registration Rates
KEN -18 -1469Last modified on December 19th, 2025 at 10:19 am
-
Abstract
PxD operates the MoA-INFO platform in collaboration with Kenya’s Ministry of Agriculture to provide free agricultural recommendations to farmers via SMS messages. Blast messages from Safaricom (a large Kenyan phone company) are a critical method to attract new users to the MoA-Info platform.
When PxD scaled to a nationwide campaign reaching over 10 million Kenyans, registration rates were at around 1%, which was significantly below the 3% for the best-performing messages in earlier pilots. This experiment tested two framing improvements—addressing farmer pest concerns, and appealing with social proof and urgency—to a message used previously. The experiment also tested two follow-up messages.
Improved message framing significantly outperformed the previous nationwide campaign, resulting in up to 30% higher registration rates. The effects of the two improved versions were not significantly different from each other. The follow-ups were very effective and led to an extra 3.1% of the overall sample registering on the MoA-INFO platform. -
Status
Completed
-
Start date
Q4 Oct 2018
-
End date
Q4 Oct 2018
-
Experiment Location
Kenya
-
Partner Organization
Kenya Ministry of Agriculture
-
Agricultural season
Short Rains
-
Experiment type
A/B test
-
Sample frame / target population
Prospective MoA-INFO users (not yet registered)
-
Sample size
13,368
-
Outcome type
Service engagement
-
Mode of data collection
PxD administrative data
-
Research question(s)
Do improvements in and follow-ups to invitation messages increase the registration rates on the MoA-INFO platform?
-
Research theme
Communication technology, Message framing, Message timing and frequency, Service design
-
Research Design
Our sample comprised two groups of people whose phone numbers we had access to and who had not yet registered on the MoA-INFO platform:
- Default Group: ~8,350 people who had sent MoA-INFO a message that was not correctly formatted, meaning their registrations were unsuccessful.
- Referral Group: ~3,000 people who had been invited to join the service by a fellow farmer but had not yet registered.
In this experiment, these unregistered users were invited to join the platform. They were randomly selected to receive either a control message (one used in the previous nationwide campaign) or one of two improved treatment messages:
- Control: “The Ministry of Agriculture is launching a free SMS information service! Send the word FARM to 40130 to learn about Fall Armyworm.”
- Treatment 1: “Do you worry about pests attacking your crops? Send the word FARM to 40130 to learn about Fall Armyworm for free from the Ministry of Agriculture.”
- Treatment 2: “Send the word FARM to 40130 to learn about Fall Armyworm for free from the Ministry of Agriculture. Over 100,000 Kenyans already have. Don’t miss out!”
Follow-up messages were sent after 3 days, as follows:
- Follow-up 1: “Don’t forget to send the word FARM to 40130 to learn how to fight Fall Armyworm for free.”
- Follow-up 2: “Don’t forget to send the word FARM to 40130 to learn how to fight Fall Armyworm for free. Over 100,000 Kenyans already have. Don’t miss out!”
-
Results
In all specifications, namely the registration rate in week 1, the completed registration rate, and the total registration rate after two weeks, the treatment messages outperformed the control message. Improved messages resulted in up to 30% higher registration rates, with estimates of percentage points (pp) improvements between 1.1 and 1.9 compared to the control message.
The control mean for first-week registration was 5.13%, and both Treatment 1 and Treatment 2 increased this by approximately 1.65 pp (Treatment 1: +1.65pp, p < 0.01; Treatment 2: +1.67pp, p < 0.01). For completed registration in the first week, Treatment 1 increased registration by 1.14 pp (p < 0.05), and Treatment 2 by 1.45 pp (p < 0.01), compared to the control mean of 4.50%. Overall registration rates (control mean: 8.19%) were also higher in both treatment arms: Treatment 1 yielded a 1.93 pp gain (p < 0.01), and Treatment 2 a 1.42 pp gain (p < 0.05). Similarly, the completed registration rate overall (control mean: 7.59%) improved by 1.62 pp for Treatment 1 and 1.43 pp for Treatment 2 (both p < 0.05).
If such improvements had been applied to the October 2018 Safaricom blast messages, they could have yielded an estimated extra 22,000–38,000 registered members of the MoA-INFO platform.
The effects of the two improved versions were not significantly different from each other: Treatment 1 did better in the first week and Treatment 2 did slightly better in the second week. However, these results were not statistically significant. The follow-ups led to an extra 3.1% of the overall sample registering on the MoA-INFO platform, on top of a 6.23% registration rate in the three days after the first invitation message. Given the linear costs for messages, sending a second message thus lowers that average cost per registration.