MULTIMODAL FUSION NETWORK WITH LATENT TOPIC MEMORY FOR RUMOR DETECTION

Jiaxin Chen; Zekai Wu; Zhenguo Yang; Haoran Xie; Fu Lee Wang; Wenyin Liu

doi:10.1109/ICME51207.2021.9428404

MULTIMODAL FUSION NETWORK WITH LATENT TOPIC MEMORY FOR RUMOR DETECTION

Jiaxin Chen, Zekai Wu, Zhenguo Yang, Haoran Xie, Fu Lee Wang, Wenyin Liu

School of Science and Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

13 Citations (Scopus)

Abstract

In this paper, we propose a multimodal fusion network (termed as MFN) to integrate the text and image data from social media for rumor detection. Given the multimodal features, MFN exploits self-attentive fusion (SAF) mechanism to conduct feature-level fusion by assigning corresponding weights to the complementary modalities. In particular, the textual features are combined with the fused features in a skip-connection manner, as textual features tend to be more distinguishable compared with visual features. Furthermore, MFN introduces latent topic memory (LTM) to store the semantic information about rumor and non-rumor events, benefiting to the identification of the upcoming posts. Extensive experiments conducted on two public datasets show that the proposed MFN outperforms the state-of-the-art approaches.

Original language	English
Title of host publication	2021 IEEE International Conference on Multimedia and Expo, ICME 2021
ISBN (Electronic)	9781665438643
DOIs	https://doi.org/10.1109/ICME51207.2021.9428404
Publication status	Published - 2021
Event	2021 IEEE International Conference on Multimedia and Expo, ICME 2021 - Shenzhen, China Duration: 5 Jul 2021 → 9 Jul 2021

Publication series

Name	Proceedings - IEEE International Conference on Multimedia and Expo
ISSN (Print)	1945-7871
ISSN (Electronic)	1945-788X

Conference

Conference	2021 IEEE International Conference on Multimedia and Expo, ICME 2021
Country/Territory	China
City	Shenzhen
Period	5/07/21 → 9/07/21

Keywords

Multimodal fusion
Rumor detection
Self-attentive

Access to Document

10.1109/ICME51207.2021.9428404

Cite this

@inproceedings{e93384942e02469fbfd98d63f5b8dc6d,

title = "MULTIMODAL FUSION NETWORK WITH LATENT TOPIC MEMORY FOR RUMOR DETECTION",

abstract = "In this paper, we propose a multimodal fusion network (termed as MFN) to integrate the text and image data from social media for rumor detection. Given the multimodal features, MFN exploits self-attentive fusion (SAF) mechanism to conduct feature-level fusion by assigning corresponding weights to the complementary modalities. In particular, the textual features are combined with the fused features in a skip-connection manner, as textual features tend to be more distinguishable compared with visual features. Furthermore, MFN introduces latent topic memory (LTM) to store the semantic information about rumor and non-rumor events, benefiting to the identification of the upcoming posts. Extensive experiments conducted on two public datasets show that the proposed MFN outperforms the state-of-the-art approaches.",

keywords = "Multimodal fusion, Rumor detection, Self-attentive",

author = "Jiaxin Chen and Zekai Wu and Zhenguo Yang and Haoran Xie and Wang, {Fu Lee} and Wenyin Liu",

note = "Publisher Copyright: {\textcopyright} 2021 IEEE; 2021 IEEE International Conference on Multimedia and Expo, ICME 2021 ; Conference date: 05-07-2021 Through 09-07-2021",

year = "2021",

doi = "10.1109/ICME51207.2021.9428404",

language = "English",

series = "Proceedings - IEEE International Conference on Multimedia and Expo",

booktitle = "2021 IEEE International Conference on Multimedia and Expo, ICME 2021",

}

Chen, J, Wu, Z, Yang, Z, Xie, H, Wang, FL & Liu, W 2021, MULTIMODAL FUSION NETWORK WITH LATENT TOPIC MEMORY FOR RUMOR DETECTION. in 2021 IEEE International Conference on Multimedia and Expo, ICME 2021. Proceedings - IEEE International Conference on Multimedia and Expo, 2021 IEEE International Conference on Multimedia and Expo, ICME 2021, Shenzhen, China, 5/07/21. https://doi.org/10.1109/ICME51207.2021.9428404

MULTIMODAL FUSION NETWORK WITH LATENT TOPIC MEMORY FOR RUMOR DETECTION. / Chen, Jiaxin; Wu, Zekai; Yang, Zhenguo et al.
2021 IEEE International Conference on Multimedia and Expo, ICME 2021. 2021. (Proceedings - IEEE International Conference on Multimedia and Expo).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - MULTIMODAL FUSION NETWORK WITH LATENT TOPIC MEMORY FOR RUMOR DETECTION

AU - Chen, Jiaxin

AU - Wu, Zekai

AU - Yang, Zhenguo

AU - Xie, Haoran

AU - Wang, Fu Lee

AU - Liu, Wenyin

PY - 2021

Y1 - 2021

N2 - In this paper, we propose a multimodal fusion network (termed as MFN) to integrate the text and image data from social media for rumor detection. Given the multimodal features, MFN exploits self-attentive fusion (SAF) mechanism to conduct feature-level fusion by assigning corresponding weights to the complementary modalities. In particular, the textual features are combined with the fused features in a skip-connection manner, as textual features tend to be more distinguishable compared with visual features. Furthermore, MFN introduces latent topic memory (LTM) to store the semantic information about rumor and non-rumor events, benefiting to the identification of the upcoming posts. Extensive experiments conducted on two public datasets show that the proposed MFN outperforms the state-of-the-art approaches.

AB - In this paper, we propose a multimodal fusion network (termed as MFN) to integrate the text and image data from social media for rumor detection. Given the multimodal features, MFN exploits self-attentive fusion (SAF) mechanism to conduct feature-level fusion by assigning corresponding weights to the complementary modalities. In particular, the textual features are combined with the fused features in a skip-connection manner, as textual features tend to be more distinguishable compared with visual features. Furthermore, MFN introduces latent topic memory (LTM) to store the semantic information about rumor and non-rumor events, benefiting to the identification of the upcoming posts. Extensive experiments conducted on two public datasets show that the proposed MFN outperforms the state-of-the-art approaches.

KW - Multimodal fusion

KW - Rumor detection

KW - Self-attentive

UR - http://www.scopus.com/inward/record.url?scp=85121919671&partnerID=8YFLogxK

U2 - 10.1109/ICME51207.2021.9428404

DO - 10.1109/ICME51207.2021.9428404

M3 - Conference contribution

AN - SCOPUS:85121919671

T3 - Proceedings - IEEE International Conference on Multimedia and Expo

BT - 2021 IEEE International Conference on Multimedia and Expo, ICME 2021

T2 - 2021 IEEE International Conference on Multimedia and Expo, ICME 2021

Y2 - 5 July 2021 through 9 July 2021

ER -

MULTIMODAL FUSION NETWORK WITH LATENT TOPIC MEMORY FOR RUMOR DETECTION

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this