Annotation in action: Experimental perspectives on perceiving and labeling digital hate across four European countries

Autor(en)

Thomas Kirchmair, Kevin Koban, Jörg Matthes

Abstrakt

Digital hate annotation constitutes an important task for training automatic classification algorithms that are robust across country contexts yet also sensitive to their unique conditions. However, perceptual heterogeneity in how hateful content is evaluated introduces a great challenge for consistent rating and labeling, as interpretations of norm violations are shaped by individual and contextual factors. In this study, we conduct a 2 (uncivil vs. intolerant content) × 2 (ingroup vs. outgroup targeting) × 2 (annotator vs. user role) mixed online experiment in Austria, France, Hungary, and Sweden (N = 1,718), in which participants rate and label social media comments. Intolerance is consistently perceived as more severe and labeled more often as hateful compared to incivility across all countries. However, we find mixed support for differences between ingroup and outgroup targeting, only limited cross-country evidence that explicitly assigning participants an annotator role influences their severity ratings, and no evidence that it affects their labeling decisions compared to when they act as regular users. While the influence of sociodemographics on perception is highly dependent on the country context, we find migration attitudes to be a country-invariant predictor.

Organisation(en)

Institut für Publizistik- und Kommunikationswissenschaft

Journal

Human Behavior and Emerging Technologies

Band

2026

ISSN

2578-1863

DOI

https://doi.org/10.1155/hbe2/3332187

Publikationsdatum

03-2026

Peer-reviewed

Ja

ÖFOS 2012

508007 Kommunikationswissenschaft

ASJC Scopus Sachgebiete

Social Psychology, Allgemeine Sozialwissenschaften, Human-computer interaction

Link zum Portal

https://ucrisportal.univie.ac.at/de/publications/a37841fd-05b2-49a5-bb07-9e88c3bf6782