Topic Model Validation Methods and their Impact on Model Selection and Evaluation

Author(s)
Jana Bernhard, Martin Teuffenbach, Hajo G. Boomgaarden
Abstract

Topic Modeling is currently one of the most widely employed unsupervised text-as-data techniques in the field of communication science. While researchers increasingly recognize the importance of validating topic models and given the prevalence of discussions of inadequate validation practices in the literature, there is limited understanding of the consequences of employing different validation strategies when evaluating topic models. This study applies two different methods for topic modeling to the same text corpus. It uses four validation strategies to assess how the choice of validation method affects the final model selection and evaluation. Our findings indicate that different approaches and methods lead to different model choices and evaluations, which is problematic. This might lead to unwanted results in case the choice of model has a decisive impact on findings and, consequently, on theory development and practical implications.

Organisation(s)
Department of Communication, Research Group Data Mining and Machine Learning
Journal
Computational Communication Research
Volume
5
ISSN
2665-9085
DOI
https://doi.org/10.5117/CCR2023.1.13.BERN
Publication date
01-2023
Peer reviewed
Yes
Austrian Fields of Science 2012
602011 Computational linguistics
Keywords
ASJC Scopus subject areas
Computational Theory and Mathematics, Linguistics and Language
Portal url
https://ucrisportal.univie.ac.at/en/publications/topic-model-validation-methods-and-their-impact-on-model-selection-and-evaluation(d5618776-9547-4848-adb1-3e51ba006fbd).html