Aspect-attentioned Prompting for Multimodal Sentiment Analysis

Multimodal Aspect-based Sentiment Analysis aims at identifying specific aspects like celebrities, locations, and organizations, along with their corresponding sentiments. Typically, existing methods comprehend each aspect within images and their relationship without external knowledge. To this end, we propose an Aspect-Attentioned Prompting Framework (AAPF) with Vision-language Prompting (VP) and Attention Aggregation (AA) modules to leverage the fine-grained visual semantics corresponding to the conditional sentences. More specifically, the VP constructs aspect-oriented questions from the perspectives of objects, relations, and sentence-oriented questions, leveraging chain-of-thought prompting to guide the reasoning process. Given the questions, sentences, and images, a visual language model is exploited to perceive the aspect-sensitive intricate semantics, which are further filtered by CLIP scores and condensed into an aspect-relevant summarization. Furthermore, the AA module conducts complementary cross-attention on the summarizations and the conditional sentences to model their interactions by achieving aspect-attentioned features, which can be used to make sentiment predictions. Experiments conducted on two public datasets indicate that our approach achieves superior performance against the state-of-the-art methods.