Letting User Search Behavior Lead Us: An Analysis of Search Queries & Relevancy in USC’s Web-Scale Discovery Tool

Research into Practice
Beth
Namei
Reference & Instruction Librarian
University of Southern California
Christal Young
Research Project Coordinator
University of Southern California

Abstract

More and more academic libraries are providing users with a single search box from which to discover and access the majority of their print and online collections. The demand for this search option came from our users who are accustomed to a Google like search experience. Yet many librarians still remain dubious about whether or not this option can really deliver on its promise. Change can be difficult, but public service librarians have no choice; web-scale discovery tools demand that we rethink how we teach information literacy sessions and provide reference service. We will be conducting a study that will follow our users’ lead by examining how they conduct research and experience results using the University of Southern California’s single search box, Quick Search. We will be looking at a semester’s worth of Summon search queries and will be re-executing a random sample of those searches in order to assess their success and failure rate. Our research will look for patterns regarding how users construct searches and whether or not USC’s iteration of Summon actually delivers relevant content to our users. By gaining a better understanding about how users approach a single search box, librarians will be better equipped to teach students to effectively use our discovery tool. The results of my study will demonstrate whether or not discovery tools actually deliver on their promise to provide a Google like search experience, as our users expect, or if there is more work to be done to deliver on this promise.

Description

More and more academic libraries are providing users with a single search box from which to discover and access the majority of their print and online collections. The demand for this came from our users who are accustomed to and expect a Google-like  experience when searching for information online. Yet, many librarians remain doubtful as to whether or not this one-size-fits-all single seach box can really deliver on its promises. Change can be uncomfortable, but public service librarians, faced with these prominently placed search boxes, must rethink how we teach information literacy sessions and provide reference service. 

There has been a lot published about web-scale discovery tools since they were first introduced in 2008. Much of the literature pertains to how a particular library went about choosing and implementing a particular discovery tool. Several articles have recently begun to appear about how to teach information literacy skills using web-scale discovery tools. There have also been numerous articles written about usability studies of discovery tools. Most of these studies have looked at only a small handful of users, usually undergraduates, providing them with a set of tasks to complete and then interviewing them about their experiences. As of writing this proposal we have only found three studies that quantitatively analyze search queries and/or the search results of web-scale discovery tools. (Lowan, Sierra and Boyer (2011); Meadow and Meadow (2012); Asher, Duke and Wilson (2012)). There is clearly a need for more quantitative research to be done on how well discovery tools deliver on their promise to provide users with an easier to use interface and more streamlined and relevant results.

We will be conducting a research study that will follow our users’ lead by examining how they conduct research and experience results using the University of Southern California’s discovery tool, Quick Search (Summon). We will be looking at a semester’s worth of search queries and will be re-executing a random sample of those searches in order to assess the success and failure rate of retrieved results. Our research will generate evidence-based results regarding how users construct searches when faced with a single search box on a library’s homepage and whether or not USC’s iteration of Summon successfully delivers relevant content to our users. By gaining a better understanding about how our users approach a single search box we can better customize the Summon interface and display options and provide more informed and tailored instruction on how to successfully search our discovery tool. These results will also provide tangible evidence regarding whether or not web-scale discovery tools actually deliver on their promise to provide a Google like search experience, as our users expect, or if there is more work to be done to deliver on this promise. 

Presentation Materials

When

Date and time: 
Sat, 04/05/2014 - 3:45pm - 5:00pm

Where

San Martin [LL]