Profiling digital hate: A multidimensional measurement approach based on the Perspective API
- Author(s)
- Thomas Kirchmair, Kevin Koban, Jörg Matthes
- Abstract
The automatic classification of digital hate is a pressing challenge, yet many existing computational models remain opaque and insufficiently evaluated as measurement tools in social science contexts. This study examines the utility of Google’s Perspective API as a measurement instrument by modeling higher-order constructs of harmful discourse (i.e., incivility and intolerance) as outcomes of multiple lower-level behavioral indicators captured by distinct API scores rather than using a single aggregate score as a proxy for complex social behaviors while enabling the evaluation of state-of-the-art black-box classifiers beyond classification metrics. Drawing on 4,000 manually annotated English-language YouTube comments in the context of the Israel-Hamas war, we test whether multiple API scores predict incivility and intolerance using generalized linear and additive models, assess classification performance across non-hateful, uncivil, and intolerant content, and benchmark a recent deep learning model. Results show that Identity Attack is a strong predictor of intolerance, whereas Insult and Profanity are indicative of incivility. While classification performance is somewhat below state-of-the-art deep learning models, our approach offers important advantages: transparency, interpretability, accessibility for non-technical researchers, and potential cross-linguistic applicability. We argue that typology-driven, multi-indicator-based classification provides a practical and theoretically grounded complement to more aggregated black-box models, particularly in human-in-the-loop workflows that can help reduce annotator exposure through pre-filtering of content.
- Organisation(s)
- Department of Communication
- Journal
- Social Science Computer Review
- ISSN
- 0894-4393
- DOI
- https://doi.org/10.1177/08944393261469403
- Publication date
- 06-2026
- Peer reviewed
- Yes
- Austrian Fields of Science 2012
- 508007 Communication science
- Keywords
- ASJC Scopus subject areas
- General Social Sciences, Computer Science Applications, Library and Information Sciences, Law
- Portal url
- https://ucrisportal.univie.ac.at/en/publications/0bbbf366-b2bf-4704-a148-8bd7caa6a639