Header

UZH-Logo

Maintenance Infos

Zipf's Law in Human-Machine Dialog


Linders, Guido M; Louwerse, Max M (2020). Zipf's Law in Human-Machine Dialog. In: 20th ACM International Conference on Intelligent Virtual Agents, Virtual Event/ Scotland UK, 20 October 2020 - 22 October 2020. ACM Digital library, 1-8.

Abstract

Zipf’s law is a mathematically relatively simple formula stating that the frequency of a word is inversely correlated with its rank. Zipf’s law is well-known in computational linguistics and cognitivesciences alike. In the context of agent development, however, Zipf’s law has hardly ever been mentioned. This is surprising as principles regarding language likely benefit the development of conversational agents. This paper serves as a starting point to explore the role of Zipf’s law in agent development, showing that Zipf’s law also applies to dialog. Moreover, it can shed light on human-machine dialog. In addition to word frequency distributions that demonstrate Zipf’s law, we also included frequency distributions of words at specific positions in the sentence as well as turn lengths. Zipf’s law was found in the far majority of analyses we conducted. In addition, we investigated whether Zipf’s law can be used to detect differences between human and agent-generated speech through correlating the distributions and found that even though both the human and agent frequency distributions follow Zipf’s law, these distributions are not necessarily similar, shedding light on where agent dialog may distinguish itself from human dialog. The findings in this paper can thus serve as a way to monitor to what extent ubiquitous patterns in human-human dialog are found in humanmachine dialog.

Abstract

Zipf’s law is a mathematically relatively simple formula stating that the frequency of a word is inversely correlated with its rank. Zipf’s law is well-known in computational linguistics and cognitivesciences alike. In the context of agent development, however, Zipf’s law has hardly ever been mentioned. This is surprising as principles regarding language likely benefit the development of conversational agents. This paper serves as a starting point to explore the role of Zipf’s law in agent development, showing that Zipf’s law also applies to dialog. Moreover, it can shed light on human-machine dialog. In addition to word frequency distributions that demonstrate Zipf’s law, we also included frequency distributions of words at specific positions in the sentence as well as turn lengths. Zipf’s law was found in the far majority of analyses we conducted. In addition, we investigated whether Zipf’s law can be used to detect differences between human and agent-generated speech through correlating the distributions and found that even though both the human and agent frequency distributions follow Zipf’s law, these distributions are not necessarily similar, shedding light on where agent dialog may distinguish itself from human dialog. The findings in this paper can thus serve as a way to monitor to what extent ubiquitous patterns in human-human dialog are found in humanmachine dialog.

Statistics

Citations

Dimensions.ai Metrics
1 citation in Web of Science®
2 citations in Scopus®
Google Scholar™

Altmetrics

Downloads

32 downloads since deposited on 17 Aug 2023
32 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Conference or Workshop Item (Paper), not_refereed, original work
Communities & Collections:06 Faculty of Arts > Department of Comparative Language Science
Dewey Decimal Classification:000 Computer science, knowledge & systems
310 Statistics
410 Linguistics
Scopus Subject Areas:Physical Sciences > Artificial Intelligence
Physical Sciences > Human-Computer Interaction
Language:English
Event End Date:22 October 2020
Deposited On:17 Aug 2023 09:01
Last Modified:30 Sep 2023 02:09
Publisher:ACM Digital library
Series Name:Proceedings of the ACM International Conference on Intelligent Virtual Agents
ISBN:9781450375863
OA Status:Green
Publisher DOI:https://doi.org/10.1145/3383652.3423878
  • Content: Published Version
  • Language: English