- Author
- Year
- 2017
- host editors
-
G.J.F. Jones
S. Lawless
J. Gonzalo
L. Kelly
L. Goeuriot
T. Mandl
L. Cappellato
N. Ferro - Title
- CLEF 2017 dynamic search evaluation lab overview
- Event
- 8th International Conference of the CLEF Association, CLEF 2017
- Book/source title
- Experimental IR Meets Multilinguality, Multimodality, and Interaction
- Book/source subtitle
- 8th International Conference of the CLEF Association, CLEF 2017, Dublin, Ireland, September 11–14, 2017 : proceedings
- Pages (from-to)
- 361-366
- Number of pages
- 6
- Publisher
- Cham: Springer
- ISBN
- 9783319658124
- ISBN (electronic)
- 9783319658131
- Series
- Lecture Notes in Computer Science, 0302-9743, 1611-3349, 10456
- Document type
- Conference contribution
- Faculty
- Faculty of Science (FNWI)
- Institute
- Informatics Institute (IVI)
- Abstract
-
In this paper we provide an overview of the first edition of the CLEF Dynamic Search Lab. The CLEF Dynamic Search lab ran in the form of a workshop with the goal of approaching one key question: how can we evaluate dynamic search algorithms? Unlike static search algorithms, which essentially consider user request’s independently, and which do not adapt the ranking w.r.t the user’s sequence of interactions, dynamic search algorithms try to infer from the user’s intentions from their interactions and then adapt the ranking accordingly. Personalized session search, contextual search, and dialog systems often adopt such algorithms. This lab provides an opportunity for researchers to discuss the challenges faced when trying to measure and evaluate the performance of dynamic search algorithms, given the context of available corpora, simulations methods, and current evaluation metrics. To seed the discussion, a pilot task was run with the goal of producing search agents that could simulate the process of a user, interacting with a search system over the course of a search session. Herein, we describe the overall objectives of the CLEF 2017 Dynamic Search Lab, the resources created for the pilot task and the evaluation methodology adopted.
- URL
- go to publisher's site
- Other links
- Link to publication in Scopus
- Language
- English
- Persistent Identifier
- https://hdl.handle.net/11245.1/084809d5-046b-4050-a88c-7f0822b01ba6
- Downloads
-
CLEF 2017 dynamic search evaluation lab overview(Final published version)
Disclaimer/Complaints regulations
If you believe that digital publication of certain material infringes any of your rights or (privacy) interests, please let the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the material inaccessible and/or remove it from the website. Please Ask the Library, or send a letter to: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands. You will be contacted as soon as possible.