Conference Reports
Conference Reports Home

backReturn to OK-ACRL homepage

Embracing Assessment: New Measures for Academic Libraries

2001 Annual Fall Conference
Monday, October 22, 2001        9:00 am - 3:30 pm
Student Center
Oklahoma State University - Oklahoma City

The Oklahoma ACRL Chapter held its fourth annual fall conference on October 22, 2001 at the Oklahoma State University, Oklahoma City Campus. Three speakers shared both their theoretical and practical experiences in assessment in academic libraries and academia.

Peter Hernon | Annmarie Shirazi | Sheila Grant Johnson

Peter Hernon, Professor, Graduate School of Library and Information Science at Simmons College, Boston, MA

“Assessment of Library Services: An Examination of Service Quality AND Outcomes Assessment”

Hernon discussed the “Gap Theory of Service Quality” which includes: customers’ expectations and management’s perceptions of those expectations; and customers’ service expectation and the perceived quality of service. This theory looks at the library from the perspective of users, not librarians. These gaps are the basis of a customer-oriented definition of service quality that examines the discrepancy between customers’ expectations for excellent service and their perceptions of the actual service delivered. Hernon outlined and critiqued different service quality measurement tools, primarily SERVQUAL and LIBQUAL+, while also addressing problems inherent in surveys in general (i.e. poor response rates, overuse, not enough follow-up, do respondents reflect the population,...). Hernon cautioned whatever method is used, get feedback from your service population on the tool itself before you attempt to measure with it.

According to Hernon, outcomes assessment is generally aimed at information literacy skills. He pointed out the distinction between lower order (skills) and higher order (critical thinking and problem solving) objectives, and the need to focus on the latter. He also offered an example list of direct and indirect outcomes measures. Direct measures include pre/post tests, developmental portfolios, capstone courses, content analysis, think-aloud protocols, citation analysis, tests, videotape or audiotape evaluation, and exit interviews. Indirect measures include graduation rates, professional licensure, retention rates, focus groups, and surveys. These lists provide good starting places as we look for more ways to measure outcomes.

Dr. Annmarie Shirazi, Dean of Institutional Effectiveness, Oklahoma City Community College

“Caught Without a Leg to Stand On”

In the afternoon we heard from Annmarie Shirazi, OCCC’s Dean of Institutional Effectiveness. Using a fable about a little fuzzy guy and an axe-happy king, she addressed the concepts behind successful assessment implementations. After the more general overview, she spoke more directly on how program assessment works at OCCC. Shirazi divides objectives into three categories: process, satisfaction, and outcome objectives. She also mentioned a very interesting idea for collecting outcome data. She suggested that faculty members could add an essay question to an exam that could be given to the library rather than graded and then used in assessment. (For example, asking students to describe the process for finding material on a topic). A very important point in her talk was that doing the assessment isn’t enough; you have to use your results to make improvements to your services.

Sheila Grant Johnson, Assistant Dean of Libraries for Public Services and Collections, Oklahoma State University

"Assessing Library Services: The Ideal and the Real"

The final speaker was Sheila Grant Johnson, OSU’s Assistant Dean of Libraries for Public Services and Collections. She spoke about their participation in the ARL LIBQUAL+ project. LIBQUAL+ is a standardized service quality survey developed by Texas A & M University. Johnson mentioned some of the technical difficulties encountered, both in the initial set-up and during the actual administration of the survey. She then discussed the results of the survey. There was a very low return rate for the survey, so it’s difficult to come to any firm conclusions with the data in hand. Johnson was aware of the shortcomings of the survey, but is very interested in continuing with service quality assessment.


back Return to OK-ACRL homepage
Last Update: 23 May 2002
Comments to: