Inbound engagement: remote training
IND -20 -1415Last modified on July 10th, 2025 at 8:36 pm
-
Experiment rationale and/or abstract
PxD operates a free agriculture information service delivered over mobile phones, in collaboration with a State Government Department of Agriculture in India using a two-way IVR platform with "outbound" push calls and "inbound" hotline service.
This experiment builds on previous rounds of A/B tests that seek to increase engagement with the inbound service. In this experiment, we extend phone-based remote training to a subset of farmers who received reminder and instructional messages about using the inbound hotline. We explore the impact of training on driving engagement, and in particular on asking a valid question, when compared with a pure control group or a group which only received reminders. -
Status
Completed
-
Start date
Q4 Oct 2020
-
End date
Q1 Jan 2021
-
Experiment Location
India
-
Partner Organization
_N/A
-
Agricultural season
Kharif, Rabi
-
Experiment type
A/B test
-
Sample frame / target population
Farmers on the service
-
Sample size
8,502
-
Outcome type
Information access, Service engagement
-
Method of Measurement
Platform administrative data
-
Research question(s)/hypotheses
Is a combined treatment of reminder messages and remote training effective in increasing the number of farmers that ask a valid question? What type of user is the training most effective for?
-
Research theme
Communication technology, Service design
-
Research design notes
There are three rounds of these inbound tests, the first two revolved around reminder messages. In Round 1 we sent reminders to roughly 4000 farmers. In Round 2 we sent a series of reminders to 15,914 farmers. 11,419 of these farmers were in the treatment group and 4,000 are in the control group and did not receive any messages.
In this third round, we build on Round 2 and randomly assign the treatment group farmers to one of three new treatment arms (including the training arm), stratifying along original treatment status (i.e. the type of reminder messages that farmers received), geography, and their engagement with the inbound service during the 10 weeks of the experiment (i.e. (i) if they didn’t call in at all (ii) if they called in but were unsuccessful (blank calls, calls under 60 seconds and invalid questions) (iii) if they called and were successful (feature accessed for over 60 seconds or valid question recorded).
The treatment arms are:
1. Control (T0): Farmers receive no treatment
2. Pest (T1): Farmers receive a series of reminder messages + two additional “use-case” messages that encourage farmers to record a question about pests
3. Training (T2): Farmers receive a series of reminder messages + remote training.
4. Reminder only (T3): Farmers only received a set of reminder and instructional messages in the first stageNote that we define a valid question as a farmer asking a query related to agriculture.
-
Results
We find that offering remote training to farmers in addition to reminders increases the likelihood of calling in by 1.3 percentage points. This is a 53 percent increase over the control group mean likelihood, which is 2.5 percent. We also find that 2.2 percent of this group ask a valid question compared with 0.1 percent in the control group. However the absolute numbers here are still low and hence we weren't able to assess the statistical significance.
In terms of heterogeneity, we also found that the training was more impactful for certain groups, including:
• Farmers who are active on the outbound service: Those who have an above median outbound listening rate in the 6 months before the original experiment call in at a higher rate (3% of farmers asked a question, as compared to 1.6% of the farmers who had a below median listening rate).
• Smartphone users: A larger proportion of smartphone users in the Training group called in as compared with non-smartphone users (3% versus 1.8%).