Re: [EXT] How to algorithmicaly evaluate a ranking function ?

From: Harker, Karen <Karen.Harker_at_nyob> Date: Wed, 12 Jul 2023 21:31:46 +0000 To: CODE4LIB_at_LISTS.CLIR.ORG

 Not sure if it's relevant to your needs, but you could look at  https://fair-trec.github.io/ (TREC)

Karen R. Harker, MLS, MPH
Collection Assessment Librarian
UNT Libraries

-----Original Message-----
From: Code for Libraries <CODE4LIB_at_LISTS.CLIR.ORG> On Behalf Of Ohms, Jannis
Sent: Wednesday, July 12, 2023 11:41 AM
To: CODE4LIB_at_LISTS.CLIR.ORG
Subject: [EXT] [CODE4LIB] How to algorithmicaly evaluate a ranking function ?

[Einige Personen, die diese Nachricht erhalten haben, erhalten häufig keine E-Mails von j.ohms@tu-braunschweig.de. Weitere Informationen, warum dies wichtig ist, finden Sie unter https://aka.ms/LearnAboutSenderIdentification ]

Hi all,

I want to evaluate the ranking of my discovery system to tune the ranking function. are there datasets or benchmarks I can use?

i.e. a list of queries and the ranked results?

I want to evaluate different functions and weights  for this an automated repeatable approach that does not require  user tests  for every run would be nice

Thanks for your help

Jannis