Re: The next generation of discovery tools (new LJ article)

Maybe I was being too flip, and I do *often* find the first few items  
in a ranked set useful, but it isn't clear to me in many cases what  
the ranking has to do with user needs. The distance between the first  
item (or page) and the 10th item (or page) may be algorithmically  
predictable, but that is not to say that the 10th item is less useful  
to me than, say, item (or page) 17.

I have often been critical of presenting results in alphabetical  
order, since the alphabetical order of retrieved items is generally  
not meaningful in the context of the user's query. However, the  
alphabet is a *known order* those times when it IS relevant. Ranking  
by complex algorithms does not produce a known order. As a user, I  
can't manipulate the results to get the ranking that serves me best.

I don't think we should give up ranking -- the fact that it works some  
of the time is a reason to keep doing it. But I surely hope that we  
don't consider it the solution to the large retrieved set problem.

kc

Quoting "Beacom, Matthew" <matthew.beacom_at_YALE.EDU>:

> Karen,
>
> I don't see how the evidence David provided or Jonathan's analysis  
> would lead us to conclude that ranking is a crapshoot. The 1st ten  
> in any half-sensible ranking of a half-sensible search will not be  
> as likely to be relevant as the 10th ten (they will be more likely  
> to be relevant), which is what I think you meant by "a crapshoot."
>
> The rankings are crude approximations of relevancy, but they are  
> often pretty helpful. And a savvy searcher, who is after more than  
> the first likely thing that comes up, may be able to sort through  
> what rose up in the rankings to perform more suitable searches or  
> re-sort the results by another vector or reduce the results by some  
> facet or facets.
>
> Matthew
>
> -----Original Message-----
> From: Next generation catalogs for libraries  
> [mailto:NGC4LIB_at_LISTSERV.ND.EDU] On Behalf Of Karen Coyle
> Sent: Monday, March 28, 2011 4:20 PM
> To: NGC4LIB_at_LISTSERV.ND.EDU
> Subject: Re: [NGC4LIB] The next generation of discovery tools (new  
> LJ article)
>
> Thank you, David. This confirms Jonathan's analysis, that the set is
> compared to itself and therefore does not flatten out tail-like as I
> expected. That said, the most important part of what Jonathan said was
> that there is no particular correlation between Solr's determination
> of ranking and what the user experiences when looking at the results
> in a linear fashion.
>
> Can we just conclude that, with a few exceptions, ranking is a crapshoot?
>
> kc
>



-- 
Karen Coyle
kcoyle@kcoyle.net http://kcoyle.net
ph: 1-510-540-7596
m: 1-510-435-8234
skype: kcoylenet