not-registered Login to view full entry:

Inbound engagement: remote training

IND -20 -1415

    Basic Information

  • Abstract
    PAD operates Ama Krushi, a free agriculture information service delivered over mobile phones, in collaboration with the State Government of Odisha Department of Agriculture using a two-way IVR platform with "outbound" push calls and "inbound" hotline service.

    This experiment built on earlier A/B tests aimed at increasing engagement with the inbound service by extending phone-based remote training to a subset of farmers who had previously received reminder and instructional messages. The objective was to assess whether combining remote training with reminders would improve farmers' likelihood of calling into the hotline and asking valid agricultural questions, compared to farmers who received only reminders or no intervention.

    The results suggest that adding remote training meaningfully increased farmers' engagement with the service, both in terms of call-in rates and the likelihood of asking valid questions. The effect was particularly notable among farmers who were active outbound users and among smartphone users, indicating that remote training is a promising complement to reminder messages for enhancing farmer engagement with digital advisory services.
  • Status
    Completed
  • Start date
    Q4 Oct 2020
  • End date
    Q1 Jan 2021
  • Experiment Location
    India
  • Partner Organization
    Government of Odisha
  • Agricultural season
    Kharif, Rabi
  • Research Design

  • Experiment type
    A/B test
  • Sample frame / target population
    Farmers on the service
  • Sample size
    8,502
  • Outcome type
    Information access, Service engagement
  • Mode of data collection
    PxD administrative data
  • Research question(s)/hypotheses
    Is a combined treatment of reminder messages and remote training effective in increasing the number of farmers that ask a valid question? What type of user is the training most effective for?
  • Research theme
    Communication technology, Service design
  • Research design notes

    There are three rounds of these inbound tests, the first two revolved around reminder messages. In Round 1 we sent reminders to roughly 4000 farmers. In Round 2 we sent a series of reminders to 15,914 farmers. 11,419 of these farmers were in the treatment group and 4,000 are in the control group and did not receive any messages.

    In this third round, we build on Round 2 and randomly assign the treatment group farmers to one of three new treatment arms (including the training arm), stratifying along original treatment status (i.e. the type of reminder messages that farmers received), geography, and their engagement with the inbound service during the 10 weeks of the experiment (i.e. (i) if they didn’t call in at all (ii) if they called in but were unsuccessful (blank calls, calls under 60 seconds and invalid questions) (iii) if they called and were successful (feature accessed for over 60 seconds or valid question recorded).

    The treatment arms are:
    1. Control (T0): Farmers receive no treatment
    2. Pest (T1): Farmers receive a series of reminder messages + two additional “use-case” messages that encourage farmers to record a question about pests
    3. Training (T2): Farmers receive a series of reminder messages + remote training.
    4. Reminder only (T3): Farmers only received a set of reminder and instructional messages in the first stage

    Note that we define a valid question as a farmer asking a query related to agriculture.

  • Results

  • Results
    We found that offering remote training combined with reminder messages increased the likelihood of farmers calling into the inbound service by 1.3 percentage points (a 53% increase over the control mean of 2.5%). Furthermore, 2.2% of farmers in the training group asked a valid agricultural question, compared to 0.1% in the control group. Although overall rates were low, these differences suggest a positive effect of training. The impact was notably stronger among farmers with higher prior outbound engagement (3% asked a question vs. 1.6% for low-engagement farmers) and among smartphone users (3% vs. 1.8% for non-smartphone users).