Header

UZH-Logo

Maintenance Infos

MBIC – A media bias annotation dataset including annotator characteristics


Spinde, Timo; Rudnitckaia, Lada; Sinha, Kanishka; Hamborg, Felix; Gipp, Bela; Donnay, Karsten (2021). MBIC – A media bias annotation dataset including annotator characteristics. In: iConference, virtual, 17 March 2021. iSchools, online.

Abstract

Many people consider news articles to be a reliable source of information on current events. However, due to the range of factors influencing news agencies, such coverage may not always be impartial. Media bias, or slanted news coverage, can have a substantial impact on public perception of events, and, accordingly, can potentially alter the beliefs and views of the public. The main data gap in current research on media bias detection is a robust, representative, and diverse dataset containing annotations of biased words and sentences. In particular, existing datasets do not control for the individual background of annotators, which may affect their assessment and, thus, represents critical information for contextualizing their annotations. In this poster, we present a matrix-based methodology to crowdsource such data using a self-developed annotation platform. We also present MBIC (Media Bias Including Characteristics) - the first sample of 1,700 statements representing various media bias instances. The statements were reviewed by ten annotators each and contain labels for media bias identification both on the word and sentence level. MBIC is the first available dataset about media bias reporting detailed information on annotator characteristics and their individual background. The current dataset already significantly extends existing data in this domain providing unique and more reliable insights into the perception of bias. In future, we will further extend it both with respect to the number of articles and annotators per article.

Abstract

Many people consider news articles to be a reliable source of information on current events. However, due to the range of factors influencing news agencies, such coverage may not always be impartial. Media bias, or slanted news coverage, can have a substantial impact on public perception of events, and, accordingly, can potentially alter the beliefs and views of the public. The main data gap in current research on media bias detection is a robust, representative, and diverse dataset containing annotations of biased words and sentences. In particular, existing datasets do not control for the individual background of annotators, which may affect their assessment and, thus, represents critical information for contextualizing their annotations. In this poster, we present a matrix-based methodology to crowdsource such data using a self-developed annotation platform. We also present MBIC (Media Bias Including Characteristics) - the first sample of 1,700 statements representing various media bias instances. The statements were reviewed by ten annotators each and contain labels for media bias identification both on the word and sentence level. MBIC is the first available dataset about media bias reporting detailed information on annotator characteristics and their individual background. The current dataset already significantly extends existing data in this domain providing unique and more reliable insights into the perception of bias. In future, we will further extend it both with respect to the number of articles and annotators per article.

Statistics

Downloads

110 downloads since deposited on 07 Oct 2021
80 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Conference or Workshop Item (Paper), not_refereed, original work
Communities & Collections:06 Faculty of Arts > Institute of Political Science
08 Research Priority Programs > Digital Society Initiative
Dewey Decimal Classification:320 Political science
Uncontrolled Keywords:media bias, news slant, dataset, survey, crowdsourcing
Language:English
Event End Date:17 March 2021
Deposited On:07 Oct 2021 16:00
Last Modified:07 Mar 2022 15:40
Publisher:iSchools
OA Status:Green
Official URL:https://www.ideals.illinois.edu/handle/2142/109680
Related URLs:https://www.ideals.illinois.edu/handle/2142/109660 (Organisation)
  • Content: Published Version
  • Language: English