Measuring Organizational Constructs in Large-Scale Text: An interpretive framework combining topic modeling and BERT

Abstract

Measuring organizational constructs in large-scale unstructured text requires methods that preserve interpretive depth while enabling systematic, per-document classification. Structural Topic Modeling (STM) supports interpretive depth by surfacing patterns that inform construct operationalization, but its topic proportions are not designed as standalone per-document construct measures; transformer-based models such as BERT can scale measurement but require construct operationalization grounded in interpretive work. We address this challenge through a two-stage framework in which STM and BERT play structurally distinct roles. Coding schemes are developed through qualitative interpretation and STM analysis; BERT is then fine-tuned to apply these schemes across documents. We formalize two research approaches—construct-guided and pattern-discovery—that provide principled paths from different theoretical starting points toward a shared methodological endpoint: validated, auditable per-document construct measures. We illustrate the framework using more than 20,000 employee reviews. The framework supports iterative construct validation, using computational diagnostics to guide interpretive refinement and producing document-level measures suitable for theory-informed organizational analysis.