TY - GEN
T1 - Cross-modal Attention Network with Orthogonal Latent Memory for Rumor Detection
AU - Wu, Zekai
AU - Chen, Jiaxin
AU - Yang, Zhenguo
AU - Xie, Haoran
AU - Wang, Fu Lee
AU - Liu, Wenyin
N1 - Publisher Copyright:
© 2021, Springer Nature Switzerland AG.
PY - 2021
Y1 - 2021
N2 - In this paper, we design a cross-modal attention fusion network with orthogonal latent memory (CALM) to fuse multi-modal social media data for rumor detection. Given multimodal content features extracted from text and images, we devise a cross-modal attention fusion (CAF) mechanism to extract critical information underlying the modalities by intra-modality attention, and model the underlying relations among the modalities by inter-modality attention. In terms of the text, the natural sequential characteristics are critical to semantic understanding, while existing sequence models suffer from losing the information conveyed by the former words. To this end, we propose a Bi-GRU with orthogonal latent memory to extract the sequential features from the text, where the memory captures independent patterns. The fused content features and the sequential features can be used for rumor detection seamlessly. Extensive experiments conducted on two real-world datasets show the outperformance of the proposed CALM. (e.g., F1 -score is improved from 0.823 to 0.846 on Weibo dataset).
AB - In this paper, we design a cross-modal attention fusion network with orthogonal latent memory (CALM) to fuse multi-modal social media data for rumor detection. Given multimodal content features extracted from text and images, we devise a cross-modal attention fusion (CAF) mechanism to extract critical information underlying the modalities by intra-modality attention, and model the underlying relations among the modalities by inter-modality attention. In terms of the text, the natural sequential characteristics are critical to semantic understanding, while existing sequence models suffer from losing the information conveyed by the former words. To this end, we propose a Bi-GRU with orthogonal latent memory to extract the sequential features from the text, where the memory captures independent patterns. The fused content features and the sequential features can be used for rumor detection seamlessly. Extensive experiments conducted on two real-world datasets show the outperformance of the proposed CALM. (e.g., F1 -score is improved from 0.823 to 0.846 on Weibo dataset).
KW - Multi-modal
KW - Rumor detection
KW - Social media
UR - http://www.scopus.com/inward/record.url?scp=85121924889&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-90888-1_40
DO - 10.1007/978-3-030-90888-1_40
M3 - Conference contribution
AN - SCOPUS:85121924889
SN - 9783030908874
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 527
EP - 541
BT - Web Information Systems Engineering - WISE 2021 - 22nd International Conference on Web Information Systems Engineering, WISE 2021, Proceedings
A2 - Zhang, Wenjie
A2 - Zou, Lei
A2 - Maamar, Zakaria
A2 - Chen, Lu
T2 - 22nd International Conference on Web Information Systems Engineering, WISE 2021
Y2 - 26 October 2021 through 29 October 2021
ER -