Laws, Sausages, and Statistics

I have just completed the annual task of compiling library statistics for ATS and ATLA a little later than usual, but still before the deadline. I should preface what follows by stating that I like statistics. I like being able to break complex processes down into neat rows of numbers and then lining up those rows of numbers on a spreadsheet. It gives me a secure feeling inside to see all that data marshalled into an eye-friendly format that can be surveyed historically for trends and changes. (Yes, I was one of the popular kids in high school.)

But there is that nagging feeling that an awful lot of ambiguity falls by the wayside when statistics are made. What about the book that comes with full PDF accessibility on a web site for registered purchasers? Did I purchase a book or access to a book? And if I purchased both, how much did each cost? If the publisher helpfully threw in a CD plastered inside the back cover, how do I record that much less circulate it? (Answer: Throw out the CD, and no one will ever know.) Then there are those misleadingly simple numbers about staff and staff salaries. I had one year where a part time professional librarian quit part way through the year. Do I then measure her contribution to staffing totals as 1/4 of a librarian? Or 5/12? The Devil is in the details, which means that there is something diabolical lurking in the white space around those neat rows of numbers.

What is especially troubling to my orderly soul is that we then construct empires out of aggregates of these numbers. "Peer comparison" is one of the benchmarks for how we're doing. It is always much easier to compare yourself to your peer when they are a compact, distant set of numbers, not a flesh and blood institution with crows feet and callused elbows and a host of other minor imperfections that jump out at you. I can say with great confidence that all the libraries in all the ATS schools reported total holdings of 49,280,000 volumes in the 2007-2008 ATS Factbook. (Don't believe me? You can double-check that fact -- I double-dog dare you.) The correlation between that number and the actual number of volumes in all of our libraries ... let's just say I hope that NASA doesn't depend on it to calculate the trajectory for the next Mars mission.

But, like many of my colleagues, I have turned a stony countenance toward those ambiguities, guesses, and WAGs and have now submitted my tidy spreadsheet of numbers to be laid quietly to rest in orderly rows with everyone else's numbers. In addition to being like laws and sausages, statistics are also like cemetaries. It's better not to dig up what lies beneath.

Library Assessment

This posting is thanks to the great session on library assessment at a recent CATLA Conference led by Robert Roethemeyer from Concordia Theological Seminary.

Over the past decade or so, theological schools have been adapting from an assessment process that tended to be quantitative to a process that is now more qualitative -- outcomes-based assessment that demonstrates how an institution is living up to both local institutional mission statements as well as national standards. As Mr. Roethemeyer pointed out in his presentation, most of the library statistics that we gather and report each year are focused only on quantitative measurements. Our door counts, circulation counts, and budget counts all tell what we did during the past year; they do not really tell how well we did our work.

One of the things that I learned at the CATLA conference was that in September 2006, the Association of Theological Schools published section 9 of their Handbook of Accreditation, Guidelines for Evaluating Library and Information Resources. These guidelines include a long list of questions to assist in qualitative library assessment. For example:

• Does the institution have a written plan with criteria for information resources, information technology, and policies for information management?


• How are software, hardware, and network resources evaluated and regularly upgraded in response to emerging technology?


• How does the library integrate print collections, access to electronic information, and other resources to foster information literacy? Are there adequate policies to guide this integration?


• How are librarians and information specialists directly involved in shaping the use of resources and in fostering the informational literacy of students and faculty? How are they involved in long-range curricular and institutional planning?

Questions that are easy to ask, but harder to answer. I left the CATLA conference reminded of two things that I have known but still do not always heed:

1) The library collection development policy remains a critical institutional document to assist with answering many of these questions. "Collection development policy", though, seems too limited in scope to address all the issues being raised today. I really need a "Library Service Plan" or an "Overall Informational Resources Master Plan for Getting Everybody the Information That They Need When They Need It". (Yes, I have applied for a trademark for that last one.)

2) Statistics have to be accompanied by narrative information. Library statistics alone will not demonstrate competence in providing library services. Part of my job as library director is to think about and record that narrative--no one else is tasked with the responsibility of focusing on these questions.

Like New Year's resolutions to lose weight or to refrain from frowning when singing praise choruses at church, I cannot promise that I will always heed these two items. But I thank Robert Roethemeyer and CATLA for keeping me mindful of them.