UCAT Voice Gender A/B Test
UGA -20 -1411Last modified on July 31st, 2025 at 2:01 am
-
Abstract
The Uganda Coffee Agronomy Training (UCAT) program, implemented by Hanns R. Neumann Stiftung (HRNS) and TechnoServe (TNS), aims to increase coffee yields and improve the livelihoods of about 60,000 smallholder farmers by training them on Good Agricultural Practices (GAPs). PAD is partnering with HRNS and TNS to provide a complementary digital service, reinforcing recommendations via automated calls to a subset of farmers in the training program and a standalone digital advisory service on coffee farming to farmers in non-program areas. The training covers a broad set of GAPs for coffee husbandry in line with the respective regional/district coffee calendar.
PAD tested whether the gender of the narrator for agronomy messages would affect farmer engagement. In one group the narrator's gender was switched to match that of the farmer’s, while in the other group the narrator's gender switched to not match that of farmer’s. These messages were sent to both UCAT standalone and reinforcement farmers who had already been receiving coffee agronomy messages. Overall, we find that both male and female farmers increased engagement when switching from a male to female narrator, but not when switching from a female to male narrator. Our analysis also suggests - although only tentative - that women presented a stronger response than men. -
Status
Completed
-
Start date
Q4 Oct 2020
-
End date
Q4 Dec 2020
-
Experiment Location
Uganda
-
Partner Organization
UCAT , IFPRI
-
Agricultural season
Short Rains
-
Experiment type
A/B test
-
Sample frame / target population
Farmers receiving UCAT ICT treatment
-
Sample size
4,002
-
Outcome type
Service engagement
-
Mode of data collection
PxD administrative data
-
Research question(s)/hypotheses
Does the voicer gender of IVR push calls affect platform engagement (i.e. pick-up and completion rates)?
-
Research theme
Message narration
-
Research design notes
To determine the effect of matching the narrator’s gender to that of the user on push call listening completion and pick-up rates, we randomly assign users into the following three groups:
- Control (n = 2,002): No change in the narrator’s voice gender (both male and female narrators had been used in recording content in four languages)
- Treatment 1 (n = 854): Switch IVR recording voice from not matching to matching the user’s gender
- Treatment 2 (n = 1,146): Switch IVR recording voice from matching to not matching the user’s gender
The sample includes 4,002 farmers in total, with 1,576 female and 2,426 male participants. Randomization was stratified by gender to ensure balanced representation across conditions. Users in the control group are those for whom the gender of the narrator’s voice remains the same before and after the introduction of the intervention. Users in each treatment group receive weekly IVR content tailored to different implementer (HRNS or TNS) categories. Treatment 1 (T1) includes users for whom the narrator’s voice was changed to match their gender. This includes female (male) farmers who previously received IVR content voiced by a male (female) narrator and, after the start of the experiment, began receiving content voiced by a female (male) narrator. Treatment 2 (T2) includes users for whom the narrator’s voice was changed to no longer match their gender. The matching or mismatching of narrator and user gender allows us to study the impact of perceived expertise and/or socio-cultural dynamics on the outcomes of interest.
The content is delivered in four different languages—English, Luganda, Runyankore, and Rutooro—using both male and female narrators. Farmers can call back and still access the advice for free if they miss the initial calls or wish to listen to previous messages. The duration of the experiment is eight weeks, after which the narrator’s voice gender is no longer based on the user’s gender.
- Control (n = 2,002): No change in the narrator’s voice gender (both male and female narrators had been used in recording content in four languages)
-
Results
Switching the message content from being voiced by a male to a female narrator increased the probability that female farmers picked up advisory calls by 2.6 percentage points, relative to a control mean of 71%. No significant effect was found among male farmers.
We also found that switching from a male to a female narrator increased the listening rate—defined as the likelihood of listening to at least 90% of the message—among male farmers by 3.1 percentage points (control mean: 69%) and by 1.2 percentage points (control mean: 66%) among female farmers. The effect among male farmers diminished over time, whereas the effect among female farmers was sustained.
Two caveats apply to using these insights in future service design work. First, the short-term nature of these effects may limit the potential for such a change to generate long-term improvements in farmer engagement. Second, because only two narrators of each gender were used in the recordings, we cannot rule out the possibility that these results reflect a “narrator effect” rather than a true gender effect.