Industrial Log Analysis Revisited: A Task-Oriented Evaluation of Parsing and Anomaly Detection under Real-World Constraints

Context: Log analysis is a critical component for monitoring, diagnosis, and risk mitigation in software systems. However, most existing research evaluates log parsing and anomaly detection models on benchmark datasets derived from legacy or open-source systems, which fail to reflect the structural diversity, semantic density, and annotation constraints of real-world industrial logs. In industrial settings, detected anomalies can support early warning, root-cause localization, preventive maintenance, and operational risk control. Objective: This study aims to systematically examine how log parsing accuracy, anomaly detection performance, and annotation availability interact in industrial settings, and to assess whether improvements in parsing quality translate into meaningful downstream gains for anomaly detection. Method: In this work, we conduct a comprehensive empirical evaluation across four datasets, including one benchmark dataset and three large-scale industrial datasets collected from manufacturing, process control, and energy monitoring environments. Using a unified evaluation framework, we assess multiple state-of-the-art parsing and detection models under standardized conditions. We further analyze the impact of annotation scarcity, identify common industrial parsing errors, and perform controlled parsing corrections to isolate their downstream effects on anomaly detection. Results: Our results show that models achieving strong performance on benchmark datasets suffer substantial degradation on industrial logs, particularly under limited supervision. Semi-supervised approaches demonstrate greater robustness in low-label regimes, while continued annotation yields sustained performance improvements. Crucially, we find that improving parsing accuracy benefits anomaly detection only when corrections preserve anomaly-critical semantic cues, whereas many parsing refinements provide little practical gain. Conclusion: These findings highlight fundamental limitations of benchmark-centric evaluation and parser-centric optimization for industrial log analysis. We argue that log parsing should be treated as a task-aware semantic transformation, and that effective industrial monitoring systems must prioritize label efficiency, semantic fidelity, and downstream utility to achieve reliable performance in complex, real-world environments.