Tuesday, March 22, 2011

What We Mean When We Talk About Measurements and Metrics: CIL Day One

I’m on record as complaining that library schools and MLS/MLIS programs aren’t rigorous enough. It seems that anyone can get in, GRE scores aren’t required for most programs, and once you’re in, it’s hard to get less than a B (or 3.0) average. This has real world consequences.
First, much like other post-graduate programs that involve education and teaching, there’s a lack of respect when it comes to libraries and librarians that stem, in part, from this lack of rigor. We’re easy to pick on and an easy target.
A second consequence was on display at the first day of the Computers in Libraries conference, here in Washington, DC. Both sessions I attended were marred by a lack of measurement, or by a lack of anything resembling social science, in its many forms. Of the 62 ALA-accredited schools that grant MLS/MLIS degrees none require a research methodology course. Zero. None. (UPDATE on 7/6/11: take a bow, University of Washington! You require methodology coursework.) The end result is that we graduate without knowing how to know. Assuming that there is a knowable world out there, a positivist conceit on my part throughout this post, I’m dismayed by how much librarians don’t know about designing experiments and measuring their results.
Take social media as a hypothetical example. Prior to implementing a social media strategy, one should take the current state of affairs as a baseline. There are many ways to measure this, and, in fact, it should be measured in many ways because most things out there are multifaceted. So, how many unique hits does your website get per month? How about overall visitors? Are programs at the building well-attended? What qualifies as well-attended in the first place? Are patrons satisfied, based on conversations with them (something like a focus group) or surveys? This isn’t exciting, but it’s important. Next, what does success look like? We ask this question regarding Iraq, Afghanistan, and now Libya, and it should be asked for less serious matters as well. Once you implement a program, what do you expect to happen, and why?
Are your users on Facebook, Twitter, both, something else? Do they express a desire to “tag” the catalog via LibraryThing or something similar? How do they want to interact with your library? Implement your program based off what you've found, and compare data with the previous state of affairs. That's science.
Every single one of our daily interactions in a library is a data point. It’s a piece of information that tells us something. Leverage that. It doesn’t have to be a number. I know that math, and statistics, can be scary sometimes, and that ethnographic research can be nuanced and illuminating. There are many roads to Damascus.
So if your library wasn’t on Facebook and now it is, tell me what changed, and why. Did you get more visitors, both virtual and physical? Are the patrons more satisfied, as measured somehow? That’s science, and it’s time library schools put it back in “Library and Information Science.” It’s also time that conferences asked presenters for some rigor and analysis, instead of just telling stories (Update: I'm not anti-story; stories are emotional, I mean this in a good way, and help us connect to patrons, donors, and the outside world. They're great in annual reports, as are metrics. Stories with data are the best of both worlds). Although we’re not all academic librarians, being a librarian is an academic enterprise. Isn’t it time we acted like that?

UPDATE: 3/9/12 - a new post on a related topic, the rigor of MLIS programs.

3 comments:

  1. I had a social science research methods class when I took my MLS at Clarion University of Pennsylvania.

    ReplyDelete
  2. Required? I took one at UNC through WISE; I got my MLIS at Pitt.

    ReplyDelete
  3. Hi Jake,

    I too have a quibble about what would otherwise be a pretty damning statistic. I am getting my MLIS at the University of Washington, and our program does require a research methods class.

    However, I agree with the overall sentiment of this post. I'm really interested in assessment in libraries, and I'm actually getting a Masters in Public Administration in part to boost my ability to do things like measuring the activities and impact of libraries. Though my program does require research methods, I don't think it is a class that many of my fellow students are eager to take. I'm also not sure if it is really preparing students to do the kind of assessment you are talking about. This is in part because regular, integrated assessment still isn't common practice in libraries, and there aren't professional or industry standards for what to measure and what those measurements mean.

    I wish that I was being better prepared to do this kind of thing, while I'm still in school. Instead I think I'll be figuring a lot of it out on my own.

    ReplyDelete