“Peer reviewed” might not be code for awesome but hey! it’s not code for useless either

So I’ve spent a lot of time in the last year talking about how we need to understand what peer review really is. Most of that time it leads to posts about the limitations of the system. Today, not so much.

I keep going back and forth about whether to write this this morning because while I’ve been thinking about it since reading this post at Information Wants to be Free yesterday, it really isn’t just about that post.

And it really isn’t even about the one snippet of the post that got me worked up. Seriously, the snippet isn’t even about the main point of the post, and it isn’t expressing any sentiment I haven’t heard a million times before, starting in library school and again, and again and again since.

And it feels like piling on to just pull out one throwaway line and write a whole post about it, especially by someone who has been dealing with kidney stones, who I have never met in person, on whose blog I do not regularly comment, and who may have not even meant this just as it sounds. It’s like “nice to meet you, way to totally miss the point.” I did get the point of the post, and I realize this snippet isn’t it. But it’s a snippet from an academic library blog and it is expressing a sentiment that I have heard a million times, and I think it’s a problematic sentiment, especially from academic librarians. And my blog is also a blog and I need to have something to link to to respond to so here we go.

Here’s the snippet:

I don’t write for peer reviewed journals since I’m not tenure-track and I actually want my work to be read. So this doesn’t make me particularly annoyed. To me, it’s just another reminder that peer-reviewed journals are completely irrelevant to me. So many peer-reviewed journals publish absolutely useless studies that were primarily done for the sake of getting the authors tenure. But at least I felt they had some sort of quality standards.

Do you see what I mean? Maybe not. Here’s what I mean – how can we as academic librarians pretend to have any relevance at all when it comes to helping students find, use and create their own scholarship — to helping students be successful in college — when so many of us have absolutely no respect for what it is that scholars actually do?

Now, the first time I wrote that I wrote “for the scholarship in our own discipline” I get that she’s probably talking here about the library literature, not articles in Science or Nature or The William and Mary Quarterly or Physics Review Letters or The Journal of Modern Literature or The American Journal of Sociology, though there’s nothing there to really indicate that distinction. But it really doesn’t matter – I do think this goes deeper than saying the library literature blows.

I mean, there is an issue with the do as I say not as I do thing that must be going on when academic librarians who disdain what is in peer reviewed journals in library science tell students that they should care about the scholarly literature in their own disciplines. But most of the time, even when it is articulated as the library literature isn’t timely enough/ cutting edge enough/ rigorous enough to meet my needs – I don’t think that’s the whole picture. The perception that there is an academic/real world gulf is so ingrained in our culture that it’s okay to state it as kind of a truism. This kind of thing – look at the comments in this piece from the Librarian in Black last summer.

And that’s the deeper issue that I think is there. I think there’s a perception that academic studies not directly and deliberately intended to inform practice can have no relevance for practice. That knowledge for knowledge’s sake has no value or relevance to the real world and that in a field like ours that is dominated by practitioners that means the academic research going on is hopelessly, inherently useless to the vast majority of the field. The work being done in other fields might be valuable to those fields, but only because those fields are academic and not as about practitioners — it’s okay for them to be useless to practice, it’s okay for them to be academic and theoretical. Knowledge for knowledge’s sake is useful in those fields because those whole fields are somewhere other than the real world.

Which could be read as librarian self-deprecation or self-hatred – “we’re just not real academics – they’re good and we’re bad.” But I think this cuts deeper than saying the library literature could be better – I tried to parse this snippet this way, and I think the other people being quoted in the post are thinking of the issue that way – but I don’t think this statement can be read to mean the library literature needs to be better. There’s no way that the peer-review model can be the go-to place for practitioners who want cutting edge answers to current problems, who want what they get from blogs and other dynamic information sources – that’s not a matter of better or worse.

The truth of the matter is even if academic research in library science was as cutting-edge, current, and rigorous as any academic research could ever be – a lot of it would still not be intended to inform practice.

When I hear people talking about how useless or stultifying or hard to understand or badly written they find the peer-reviewed literature – there’s a pride there in being a practitioner instead of an academic. There’s a sense that we are doing the real work in a way the academics never can. There’s nothing wrong with being proud of practice, don’t get me wrong – I am absolutely not saying that. It’s the “instead of an academic” piece that I have issues with because theory/practice isn’t a zero-sum thing. There’s no need to do either/or. There is something wrong with cutting yourself off from something that can and does inform practice in ways that nothing else can simply because it wasn’t created specifically to do so.

And there’s especially something wrong with academic librarians cutting themselves off from that because a huge part of our job, from collection building to information literacy, is all about connecting students to scholarship. There’s no way to compartmentalize that to the library literature – there’s no way to say “well, I think the scholarly study in librarianship is useless but in sociology, or social work, it’s totally awesome.”

Because here’s another thing – when I hear people talking about how useless or stultifying or hard to understand or badly written they find the peer-reviewed literature, they sound just like year after year of students I’ve heard complaining about their classroom reading. Classroom reading not found by a keyword search in Library Literature, but carefully selected and assigned by experts in the field who are saying “this, this is an important thing you need to understand to understand what knowledge is in our discipline.” Yes, a lot of what is in the peer-reviewed literature, in all fields, is not well written. A lot of it is not well researched. A lot of it is published only because it needed to be for the author’s tenure hit. This isn’t just true for us – it’s true across the board. It might be more true for some fields and less true for others but it’s true on some level for all of them. And not recognizing that it is not ALL like that, that sometimes the language is hard because the concepts are hard, sometimes you have to read it three, four or five times not because it’s badly written but because it’s talking about really complex things that take three, four or five readings to understand means closing yourself off from a type of knowledge and a way of understanding that can absolutely inform practice — not understanding that will keep a student from being successful in college. More than that, I think not understanding that hurts the practitioner as well.

Most of our students are going to be practitioners, not academics. We can’t just assume that they will magically understand the value of knowledge for knowledge’s sake because they start taking 400 level classes. It takes a particular skill set to apply theory to practice – it takes practice to apply theory to practice. Our students don’t come to us knowing how to do those things. They need help understanding not just how to find scholarly sources – but how to read and use them. One of our writing faculty was telling me the other day that the students in her class, when they are required to find “speaker sources” – sources that take a stand on issues – almost never use academic sources even though they are required to cite them somewhere in the paper. They use the academic sources as background sources instead of speaker sources. See, the point is that they don’t have the skills or the knowledge yet to see the academic sources as speakers. They can easily identify a policy agenda, but they don’t know yet how to identify the scholarly argument, agenda or point of view. Just like we can easily identify a practical problem that needs solving, but we think that academic problems are pointless.

Our students will be better at what they do – whether that is working, voting, or heck, even dieting, if they have the skills to be informed by what the research says, what the science says – even though that research will almost never have been created simply to inform them. But I don’t know that they can learn that skill set or gain that understanding from librarians who don’t have it, or at least who don’t use or practice it, themselves.

The peer-reviewed literature is what it is. It can be a whole lot better – but that doesn’t mean more obviously and immediately practical. As someone who spends an awful lot of time going on and on and on and on about the problems with the library literature, I still have to say if you can’t find any research in that literature to inform your practice, you aren’t trying very hard.

See, here’s my last thing – sometimes the questions that scholars are interested in ARE different than the questions that arise out of daily practice. Sometimes the problems that they are passionate about solving are not the same problems that keep practitioners up at night. But the questions they ask and the answers they come up with are still valuable to practitioners if those practitioners are willing to accept the research for what it is instead of focusing entirely on what it is not.

There’s going to be a little feature in an OSU publication about undergraduate information literacy instruction at my library and I was looking at the most recent draft just before I went to read my feeds. The author came to watch one of my instruction sessions to get a feel for what that kind of teaching was like – and she told me that she saw my interactions with the students more as interactions between peers than traditional teacher/student. I thought about that and realized that what she was seeing was that, to me, the purpose of most undergraduate instruction — across the disciplines but especially in the library — is to bring these new college students into an existing community of scholars, and giving them the skills, concepts, data and sharing the knowledge that will let them find their own place within that community.

To do that, we don’t all need to be scholars ourselves in same way – we are practitioners and for most of us that is one of the wonderful things about librarianship. But we need to respect, value and celebrate those who are and what they do.

7 thoughts on ““Peer reviewed” might not be code for awesome but hey! it’s not code for useless either”

Thank you for summing up exactly what I was feeling and thinking as I read Meredith’s post, but in a much more articulate way. I admit to being biased because I am extended-term track and am in the midst of a research study I hope to publish in a peer-reviewed journal (outside of LIS lit, actually), but I do genuinely value scholarship in librarianship and across the disciplines. And I read it!

This mindset echoes one that drove me batty when I was in my MLIS program. Many of my peers saw no value in the theory we were learning and instead wanted a “practical” curriculum. This makes no sense to me at all. We need to be fluent in the theoretical frameworks of our profession or discipline so that we can apply our understanding and make critical decisions in whatever contexts we work. That’s critical thinking, right? It’s what we strive to teach our undergraduates to do. Application of knowledge and understanding in “real-world” contexts is the ultimate goal of critical thinking, and *my* real world is academia and teaching.

I was just in a discussion today at our center for teaching and learning about teaching critical thinking. One of my colleagues from the College of Ed who teaches math to pre-service teachers summed it up: we want to push beyond the “how” into the “why.” And I think that’s what the scholarship of librarianship gets at. Why teach, catalog, do reference, provide services, etc., if we don’t investigate why these activities are beneficial to our communities or how we might use research to improve the ways we practice?

That passage in Meredith’s post made me wrinkle my forehead a little, but instead of going home and thinking about why, I went home and thought about skiing and scotch so I’m glad you pulled it out to talk about it even though, yeah, it wasn’t the point of her post.

I’m wondering if the author who observed you meant your interaction with students was more like an interaction with peers because you were treating students like (potential anyway) scholars, or because she doesn’t see other professors talking with students about process and purpose and therefore doesn’t see it as “teacher-y” (teacherish? teacheresque?…professorial?)

Sadly, I think that many students are successful (if by success you mean graduate with a respectable GPA, which of course you may very well not) without ever understanding the purpose and worse maybe, the value of scholarly work. I think this is sad because it makes it hard for them to see the value in much of what they are asked to do as students and also because it feeds into the perception of higher ed as expensive (and not always effective) job training.

Being a part of helping students to see the value of scholarly work just seems so much more satisfying than so many of the other skills and even concepts on which we tend to focus.

I’m exhausted and feeling especially inarticulate, so let me first say: WORD. If nothing else I say makes sense, just refer to that.

I believe people unfairly attach negative aspects of the publishing industry to the work that is being published. Peer review is an important part of scholarship and I really dig it. I was always that student who cited RUSQ when others chose LJ. To some extent it’s different strokes for different folks, but that’s different than devaluing peer reviewed literature.

In my environment (community college) peer review is sometimes something to avoid and I have mixed feelings about that. It’s really more about accessibility of content for students in introductory classes… but I hate to think we’re “dumbing down” our goals and outcomes for our students. I also hate to frustrate them with content that is often just out of reach. Ergh!

You said: “to me, the purpose of most undergraduate instruction — across the disciplines but especially in the library — is to bring these new college students into an existing community of scholars, and giving them the skills, concepts, data and sharing the knowledge that will let them find their own place within that community.”

I have a lot of (probably false) negative perceptions of what it’s like to be a librarian for a ginormous university but this is definitely the part that I occasionally pine for. I occasionally have students who want to become members of a community of scholars.

Mostly I have students who want to get the info, write the paper, and then go to work/pick up the kids/mow the lawn, etc. Those students are bored and annoyed when I entreat them to consider their research papers as a contribution to a scholarly conversation.

It’s a good thing there are so many things that I love about community college librarianship–else it’d be tempting.

Good post. It is as stupid to resist useful information because it is published in a peer reviewed journal as it is to refuse to to learn from experts when you have the opportunity to do so face to face.

Apprenticeship is still critical in becoming a professional. For a student learning to use libraries, the opportunity to work in a small group with a librarian to answer questions should be priceless.

Note that the simple fact that someone is a professional librarian in a first rate academic library is probably a better certification of the information that person provides on her field of expertise than is publication in a peer reviewed journal.

Note too that editors are supposed to assure the quality of the things that they publish in their publications.

Publications from government agencies go through a process of review and validation that should warrant confidence from the reader, with the exception of politically motivated spin from our current administration. If scientific, those publications should be a trustworthy as publications in a journal, and of course they can convey information that could not be validated by scientific means.

A key problem with getting your information from peer reviewed journals is in terms of the balance of their publications. They don’t like to publish negative results, so a reader is likely to get too rosy a picture of the likelihood of hypotheses being validated. The most respected journals also emphasize radical departures from conventional wisdom, which surprisingly often prove to be wrong with further study.

Remember, the scientific community itself tends to depend increasingly on immediate access to findings from colleages provided via email and pre-print respositories. Of course, the scientists who would use the information in their research usually have the ability to judge its quality.

Still, the Internet changes everything. We need to think more of systems which allow readers to pool their opinions as to whether documents are trustworthy and useful, and indeed to assign weights to those opinions according to the expertise of the opinion givers.