Hello,
Many of you on this list are familiar with Vufind, developed at Villanova. Yale is currently beta testing its own implementation called Yufind (our main catalog interface is called Orbis). We have just completed a brief survey run in October of undergraduates who have used the Vufind beta implementation at Yale. Below is a brief synopsis and a link to the full survey. A System Usability Scale was included in the survey. The SUS is widely used in evaluating product usability. I would be very interested in hearing from other institutions who are implementing a other next generation type OPACs who would be interested in running the SUS. This could provide useful comparative data for libraries considering different OPAC products.
83 surveys were submitted by readers indicating they had used the Yufind search. Of those 83 responses, 79.3% found that results were what they had expected and 75.6% found what they needed. Respondents who tried facets tended to like them (85.0%), and those who didn't like facets commonly complained that results were oddly not related to their search or took too long to load. 57.8% of respondents preferred Yufind to Orbis, but when asked for the first place they would turn for a book search, slightly more respondents would turn to Orbis than to Yufind (40.2% to 35.4%). Some respondents indicated they were frustrated by the broad search in Yufind, which in some cases returned many highly irrelevant results, and by slow response times. Other respondents were unable to use Yufind to locate items they knew to be in the collection, such as the journal Nature.
Some changes could be made to Yufind to improve results, and a key area would be facets. In the short term, those facets with misleading metadata can be suppressed, including era and format. Additionally, facets can be made more relevant by narrowing the set of results to exclude those records that may only match one word in a search query. Finally, usability results indicated that facets should be presented in a list that the reader could manipulate with an alphabetical sort. This sort option should be implemented.
Survey results indicated that while Yufind is viewed as promising, it needs significant improvement. A search of the Yale collection should be more accurate so that very popular items such as Nature (as measured by holdings, circulation, or downloads) are easily located in the Yufind search. This type of customization to the Yale environment would help to improve Yufind's standing as a search interface for Yale readers. Another area where Yufind could distinguish itself would be by integrating seamlessly with other Yale services. Candidates would be easy export to Refworks/Endnote and ordering book delivery via Eli Express or interlibrary loan. The benefit of this emphasis in the Yufind implementation would be to provide a service that other search engines such as Google could not easily provide, thereby distinguishing the Yufind interface.
System Usability Scale: The System Usability Scale or SUS (developed by John Brooke) is an instrument composed of 10 statements (5 negative and 5 positive) that are scored on a 5-point Likert scale. 83 respondents answered the SUS section of this survey, returning a score of 66. In general, those systems which return a score over 70 are considered acceptable in terms of usability. Scores of less than 70 require further improvement and are considered "marginal.". A score of 66 places Yufind on the higher end of the marginal scale.
http://library.yale.edu/libepub/usability/studies/yufind_survey.doc
Sincerely ,
Katheen Bauer
Director of Usability and Assessment
Yale University Library
130 Wall Street
P.O. Box 208240
New Haven, CT 06520-8240
203-248-3359
kathleen.bauer_at_yale.edu
Received on Mon Dec 01 2008 - 16:33:26 EST