Letting User Search Behavior Lead Us: An Analysis of Search Queries & Relevancy in USC’s Web-Scale Discovery Tool
More and more academic libraries are providing users with a single search box from which to discover and access the majority of their print and online collections. The demand for this came from our users who are accustomed to and expect a Google-like experience when searching for information online. Yet, many librarians remain doubtful as to whether or not this one-size-fits-all single seach box can really deliver on its promises. Change can be uncomfortable, but public service librarians, faced with these prominently placed search boxes, must rethink how we teach information literacy sessions and provide reference service.
There has been a lot published about web-scale discovery tools since they were first introduced in 2008. Much of the literature pertains to how a particular library went about choosing and implementing a particular discovery tool. Several articles have recently begun to appear about how to teach information literacy skills using web-scale discovery tools. There have also been numerous articles written about usability studies of discovery tools. Most of these studies have looked at only a small handful of users, usually undergraduates, providing them with a set of tasks to complete and then interviewing them about their experiences. As of writing this proposal we have only found three studies that quantitatively analyze search queries and/or the search results of web-scale discovery tools. (Lowan, Sierra and Boyer (2011); Meadow and Meadow (2012); Asher, Duke and Wilson (2012)). There is clearly a need for more quantitative research to be done on how well discovery tools deliver on their promise to provide users with an easier to use interface and more streamlined and relevant results.
We will be conducting a research study that will follow our users’ lead by examining how they conduct research and experience results using the University of Southern California’s discovery tool, Quick Search (Summon). We will be looking at a semester’s worth of search queries and will be re-executing a random sample of those searches in order to assess the success and failure rate of retrieved results. Our research will generate evidence-based results regarding how users construct searches when faced with a single search box on a library’s homepage and whether or not USC’s iteration of Summon successfully delivers relevant content to our users. By gaining a better understanding about how our users approach a single search box we can better customize the Summon interface and display options and provide more informed and tailored instruction on how to successfully search our discovery tool. These results will also provide tangible evidence regarding whether or not web-scale discovery tools actually deliver on their promise to provide a Google like search experience, as our users expect, or if there is more work to be done to deliver on this promise.