Research interviews conducted by artificial intelligence can help academics conduct studies on an unprecedented scale and produce more nuanced results, it has been claimed.
Two London School of Economics scholars have developed a chat bot powered by a large language model that, they say, can complete interviews with thousands of participants in a matter of hours.
Rather than having a standard set of multiple-choice and open text questions, as has typically been the case with online surveys, the chat bot takes a conversational approach, collecting interviewees’ responses and using them to generate new questions within a broad set of parameters.
Its creators, Friedrich Geiecke, an assistant professor of computational social science, and Xavier Jaravel, a professor of economics, say the tool employs best practices from academic literature—for example, encouraging participants to freely express their views and then posing follow-up questions to ensure clarity.
They also claim that the chat bot demonstrates “cognitive empathy,” using its follow-up questions to seek to understand an interviewee’s perspective as closely as they understand it themselves.
The tool is being made freely available for other researchers to download, adapt and use.
In trials, it demonstrated some impressive results. A team of sociology Ph.D. students from Harvard University and the London School of Economics that was asked to assess the quality of the interviews based on transcripts rated them as being broadly comparable to interviews conducted by human experts.
And when the almost 1,000 study participants were asked to evaluate their interaction with the chat bot, the majority said they had enjoyed it and preferred this mode of interview over open text fields. Only 15 percent of respondents said they would have preferred the interview to have been conducted by a person.
Respondents also tended to provide more detailed responses than they did with traditional open text boxes, with a 142 percent increase in the number of words written.
These advantages were particularly evident in relation to political questions. Here, the researchers found that participants preferred interacting with the chat bot because they viewed it as a “nonjudgmental entity,” allowing them “to feel more comfortable and to freely express their views.”
When combined with platforms that allow scholars to recruit large survey samples, such as Prolific, the tool could allow for research on a significant scale, at a cost of about 3 to 4 pounds ($3.79–$5.05) per participant.
Jaravel said they were “close” to incorporating voice interactions into the chat bot and were also looking at how AI could be used to analyze the responses collected, which would be helpful because the method can lead to researchers having “millions” of sentences to sift through.
While the tool might not be adopted immediately by the likes of anthropologists and sociologists, who tend to conduct very lengthy interviews with relatively small samples, he said the tool might serve to bridge the gap between qualitative and quantitative research for scholars such as economists and political scientists.
“I think this will probably transform the way economists and political scientists do surveys,” Jaravel told Times Higher Education. “As we add more features, we will get increasingly closer to qualitative interviewing, and that might be on the horizon, but that’s not the primary goal.
“The goal is that we can do something that’s really low cost, and not any more expensive than what political scientists and economists currently do, that’s going to have much more traction to extract information and keep the respondents engaged.”
