not-registered Login to view full entry:

UCAT Voice Gender A/B Test

UGA -20 -1411

    Basic Information

  • Experiment rationale and/or abstract
    The Uganda Coffee Agronomy Training (UCAT) program, implemented by Hanns R. Neumann Stiftung (HRNS) and TechnoServe (TNS), aims to increase coffee yields and improve the livelihoods of about 60,000 smallholder farmers by training them on Good Agricultural Practices (GAPs). PAD is partnering with HRNS and TNS to provide a complementary digital service, reinforcing recommendations via automated calls to a subset of farmers in the training program and a standalone digital advisory service on coffee farming to farmers in non-program areas. The training covers a broad set of GAPs for coffee husbandry in line with the respective regional/district coffee calendar.

    PAD tested whether the gender of the narrator for agronomy messages would affect farmer engagement. In one group the narrator's gender was switched to match that of the farmer’s, while in the other group the narrator's gender switched to not match that of farmer’s. These messages were sent to both UCAT standalone and reinforcement farmers who had already been receiving coffee agronomy messages. Overall, we find that farmers increased engagement when switching from a male to female narrator, but not when switching from a female to male narrator. Our analysis also suggests - although only tentative - that women presented a stronger response than men.
  • Status
  • Start date
    Q4 Oct 2020
  • End date
    Q4 Dec 2020
  • Experiment Location
  • Partner Organization
  • Agricultural season
    Short Rains
  • Research Design

  • Experiment type
    A/B test
  • Sample frame / target population
    Farmers receiving UCAT ICT treatment
  • Sample size
  • Outcome type
    System engagement
  • Method of Measurement
    Platform administrative data
  • Research question(s)/hypotheses
    Does the voicer gender of IVR push calls affect platform engagement (i.e. pick-up and completion rates)?
  • Research theme
    Message Narration, Female vs male, User engagement
  • Results

  • Results
    Switching the message content from being voiced by a man to being voiced by a woman increased the probability that female farmers picked up advisory calls by 2.6 percentage points over a control mean of 71%), while no effect was found among male farmers.

    We also found that switching from a male to a female narrator increased the listening rate - defined by the likelihood of listening to at least 90% of the message - by 3.1 percentage points (control mean: 69%) among male farmers and by 1.2 percentage points (control mean: 66%) among female farmers. The effect among male farmers faded over time while the effect for female farmers was sustained.

    The short-term nature of some effects may prevent such a change from generating long-run changes in farmers’ engagement. Moreover, only two narrators of each gender were involved in the recording, so we cannot rule out the possibility that we are capturing a “narrator” effect rather than a gender effect.