Advanced search
1 file | 652.81 KB Add to list

LT3 at SemEval-2021 Task 6 : using multi-modal compact bilinear pooling to combine visual and textual understanding in memes

Pranaydeep Singh (UGent) and Els Lefever (UGent)
Author
Organization
Abstract
Internet memes have become ubiquitous in social media networks today. Due to their popularity, they are also a widely used mode of expression to spread disinformation online. As memes consist of a mixture of text and image, they require a multi-modal approach for automatic analysis. In this paper, we describe our contribution to the SemEval-2021 Detection of Persuasian Techniques in Texts and Images Task. We propose a Multi-Modal learning system, which incorporates “memebeddings”, viz. joint text and vision features by combining them with compact bilinear pooling, to automatically identify rhetorical and psychological disinformation techniques. The experimental results show that the proposed system constantly outperforms the competition’s baseline, and achieves the 2nd best Macro F1-score and 14th best Micro F1-score out of all participants.
Keywords
LT3

Downloads

  • 2021.semeval-1.145.pdf
    • full text (Published version)
    • |
    • open access
    • |
    • PDF
    • |
    • 652.81 KB

Citation

Please use this url to cite or link to this publication:

MLA
Singh, Pranaydeep, and Els Lefever. “LT3 at SemEval-2021 Task 6 : Using Multi-Modal Compact Bilinear Pooling to Combine Visual and Textual Understanding in Memes.” Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), edited by Alexis Palmer et al., Association for Computational Linguistics (ACL), 2021, pp. 1051–55, doi:10.18653/v1/2021.semeval-1.145.
APA
Singh, P., & Lefever, E. (2021). LT3 at SemEval-2021 Task 6 : using multi-modal compact bilinear pooling to combine visual and textual understanding in memes. In A. Palmer, N. Schneider, N. Schluter, G. Emerson, A. Herbelot, & X. Zhu (Eds.), Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021) (pp. 1051–1055). https://doi.org/10.18653/v1/2021.semeval-1.145
Chicago author-date
Singh, Pranaydeep, and Els Lefever. 2021. “LT3 at SemEval-2021 Task 6 : Using Multi-Modal Compact Bilinear Pooling to Combine Visual and Textual Understanding in Memes.” In Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), edited by Alexis Palmer, Nathan Schneider, Natalie Schluter, Guy Emerson, Aurelie Herbelot, and Xiaodan Zhu, 1051–55. Bangkok, Thailand: Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2021.semeval-1.145.
Chicago author-date (all authors)
Singh, Pranaydeep, and Els Lefever. 2021. “LT3 at SemEval-2021 Task 6 : Using Multi-Modal Compact Bilinear Pooling to Combine Visual and Textual Understanding in Memes.” In Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), ed by. Alexis Palmer, Nathan Schneider, Natalie Schluter, Guy Emerson, Aurelie Herbelot, and Xiaodan Zhu, 1051–1055. Bangkok, Thailand: Association for Computational Linguistics (ACL). doi:10.18653/v1/2021.semeval-1.145.
Vancouver
1.
Singh P, Lefever E. LT3 at SemEval-2021 Task 6 : using multi-modal compact bilinear pooling to combine visual and textual understanding in memes. In: Palmer A, Schneider N, Schluter N, Emerson G, Herbelot A, Zhu X, editors. Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021). Bangkok, Thailand: Association for Computational Linguistics (ACL); 2021. p. 1051–5.
IEEE
[1]
P. Singh and E. Lefever, “LT3 at SemEval-2021 Task 6 : using multi-modal compact bilinear pooling to combine visual and textual understanding in memes,” in Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), Bangkok, Thailand, 2021, pp. 1051–1055.
@inproceedings{8716832,
  abstract     = {{Internet memes have become ubiquitous in social media networks today. Due to their popularity, they are also a widely used mode of expression to spread disinformation online. As memes consist of a mixture of text and image, they require a multi-modal approach for automatic analysis. In this paper, we describe our contribution to the SemEval-2021 Detection of Persuasian Techniques in Texts and Images Task. We propose a Multi-Modal learning system, which incorporates “memebeddings”, viz. joint text and vision features by combining them with compact bilinear pooling, to automatically identify rhetorical and psychological disinformation techniques. The experimental results show that the proposed system constantly outperforms the competition’s baseline, and achieves the 2nd best Macro F1-score and 14th best Micro F1-score out of all participants.}},
  author       = {{Singh, Pranaydeep and Lefever, Els}},
  booktitle    = {{Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021)}},
  editor       = {{Palmer, Alexis and Schneider, Nathan and Schluter, Natalie and Emerson, Guy and Herbelot, Aurelie and Zhu, Xiaodan}},
  isbn         = {{9781954085701}},
  keywords     = {{LT3}},
  language     = {{eng}},
  location     = {{Bangkok, Thailand}},
  pages        = {{1051--1055}},
  publisher    = {{Association for Computational Linguistics (ACL)}},
  title        = {{LT3 at SemEval-2021 Task 6 : using multi-modal compact bilinear pooling to combine visual and textual understanding in memes}},
  url          = {{http://doi.org/10.18653/v1/2021.semeval-1.145}},
  year         = {{2021}},
}

Altmetric
View in Altmetric