Sentiment analysis in social media is a popular task attracting the interest of the research community, also in recent evaluation campaigns of natural language processing tasks in sev- eral languages. We report on our experience in the organization of SENTIPOLC (SENTIment POLarity Classification Task), a shared task on sentiment classification of Italian tweets, proposed for the first time in 2014 within the Evalita evaluation campaign. We present the datasets – which include an enriched annotation scheme for dealing with the impact of figurative language on polarity – the evaluation methodology, and discuss the approaches and results of participating systems. We also offer a reflection on the open challenges of state-of-the-art systems for sentiment analysis of microblogging in Italian, as they emerge from a qualitative analysis of misclassified tweets. Finally, we provide an evaluation of the resources we have created, and share the lessons learned by running this task for two consecutive editions.

Sentiment Polarity Classification at EVALITA: Lessons Learned and Open Challenges

Basile, Valerio;Patti, Viviana
2018-01-01

Abstract

Sentiment analysis in social media is a popular task attracting the interest of the research community, also in recent evaluation campaigns of natural language processing tasks in sev- eral languages. We report on our experience in the organization of SENTIPOLC (SENTIment POLarity Classification Task), a shared task on sentiment classification of Italian tweets, proposed for the first time in 2014 within the Evalita evaluation campaign. We present the datasets – which include an enriched annotation scheme for dealing with the impact of figurative language on polarity – the evaluation methodology, and discuss the approaches and results of participating systems. We also offer a reflection on the open challenges of state-of-the-art systems for sentiment analysis of microblogging in Italian, as they emerge from a qualitative analysis of misclassified tweets. Finally, we provide an evaluation of the resources we have created, and share the lessons learned by running this task for two consecutive editions.
2018
1
12
http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=5165369
Irony Detection; Sentiment Analysis; Social Media Analysis; Evaluation
Basile, Valerio; Novielli, Nicole; Croce, Danilo; Barbieri, Francesco; Nissim, Malvina; Patti, Viviana
File in questo prodotto:
File Dimensione Formato  
08550785.pdf

Accesso aperto

Descrizione: accepted maunscipt
Tipo di file: POSTPRINT (VERSIONE FINALE DELL’AUTORE)
Dimensione 1.54 MB
Formato Adobe PDF
1.54 MB Adobe PDF Visualizza/Apri
Sentiment_Polarity_Classification_at_EVALITA_Lessons_Learned_and_Open_Challenges.pdf

Accesso riservato

Tipo di file: PDF EDITORIALE
Dimensione 547.45 kB
Formato Adobe PDF
547.45 kB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2318/1684800
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 13
  • ???jsp.display-item.citation.isi??? 10
social impact