Login to view full entry:
PaddyAI–Artificial Intelligence versus Human Engagement: A Comparison for IVR Advisories
INDIND -25 -3153Last modified on December 19th, 2025 at 10:42 am
-
Abstract
PxD operates the Coffee Krishi Taranga (CKT) platform in collaboration with the Coffee Board of India to provide a voice-based advisory service for coffee farmers through a two-way Interactive Voice Response (IVR) system. We have developed an AI-based chatbot feature, PaddyAI, using large language models (LLMs) to generate deeply customized and localized advisory on demand.
This A/B test is the first in a series of service improvement experiments as PxD integrates AI-enabled IVR advisories through PaddyAI. The objectives of this A/B test are to understand (1) whether AI-generated advisory content has similar user engagement compared to human-generated advisory content, and (2) whether an AI voice has similar user engagement compared to a human voice delivering the content. Findings from this A/B test will help improve the generation of advisory and the text-to-speech processes of PaddyAI. -
Status
Ongoing
-
Start date
Q4 Nov 2025
-
End date
Q4 Dec 2025
-
Experiment Location
India / Karnataka, India
-
Partner Organization
Coffee Board of India
-
Agricultural season
_N/A
-
Experiment type
A/B test
-
Sample frame / target population
Coffee farmers in Karnataka
-
Sample size
28,350
-
Outcome type
Platform engagement
-
Mode of data collection
PxD administrative data
-
Research question(s)
1. Does AI-generated advisory content have similar levels of farmer engagement compared to human-generated advisory content?
2. Does AI-voiced advisory have similar levels of farmer engagement compared to human-voiced advisory? -
Research theme
Agricultural management advice, Artificial intelligence (AI), Message narration, Service design
-
Research Design
The A/B test will run for seven weeks, during which coffee farmers will receive one advisory per week based on the coffee crop calendar.
Farmers will be randomly assigned with equal probability to one of three groups, stratified by district and gender. The groups are:
- T0: human-generated instructional advisories delivered with a human voice (status quo).
- T1: AI-generated instructional advisories delivered with an AI-generated voice.
- T2: AI-generated instructional advisories delivered with a human voice.
We will collect data for seven rounds (R1–R7). We will use PxD administrative platform data to assess user engagement, measured as:
- Listening rates: The proportion of the advisory call that the farmer listens to.
- Pick-up rates: The proportion of advisory calls that are picked up in Round 2 to Round 7 of calls.
- 80% listening rates: The proportion of the advisory calls when farmers listen to 80% of the content.