Reality Check: ‘The applicant’s publication record is spotty’

To those of you who are also my Twitter friends, I apologize: I said I was finished with this issue and moving on, but it turns out it is still going around in my head and needs a bit more sorting out–not because I feel aggrieved (that, I’m basically over), but because I think it is symptomatic in ways that are worth further disucssion.

My immediate situation is this: I was invited to participate in a panel on ‘knowledge dissemination in Victorian Studies in Canada’ at the upcoming British Association of Victorian Studies conference. Specifically, I was asked to present a paper about my experience as a blogger and how it connects to broader issues about research, writing, and ‘knowledge dissemination.’ I was also invited to propose a workshop for graduate students on academic blogging (or, as I like to think of it, blogging as an academic). Naturally, I think this is all good. For one thing, it is encouraging to find that my blogging seems interesting and significant enough to other academics for them to want to hear more about it. Also, it seems like evidence that this activity of mine, which is not by any conventional definition “scholarship,” can nonetheless open a door for me into an event like this–at least to the organizers, that I have been blogging was a reason to include me in this gathering of scholars, not a strike against me. The panel was duly accepted by the conference organizers, and I spent some substantial time last month putting together the materials required to apply for funding to cover travel expenses and the conference registration (which in this case is quite a substantial sum, as it includes on-campus accommodation as well as a registration fee). Times are hard and budgets are tight, but participation in conferences is a recognized professional activity, the kind of thing we are supposed to do, to share our ideas and learn about other people’s. I do it rarely, because my experience is often disappointing, but BAVS is just the kind of conference that promises to be really worthwhile. Our attendance at international conferences also does a little to raise the profile of our home institutions–though in this particular context, I think it’s worth pointing out that my blog, where my name and institutional affiliation are clearly displayed, also does this. (I think it’s safe to say that many people who read my blog had never heard of Dalhousie University before they came here–though perhaps I underestimate the successes of our PR team in getting the word out. I’d actually be interested to know from readers if I’m right about this.)

Anyway, the long and the short of it is that the institution I sort of work for that I don’t regularly name here, the University of King’s College (please don’t ask me to explain the Dal-King’s relationship–nobody can do it–let’s leave it at saying that I’m a member of the “Joint Faculty”) promptly coughed up the money I asked them for. Since they are a small institution and in particularly challenging financial circumstances right now, their commitment to supporting faculty research and conference travel is particularly appreciated. Thanks, King’s! But they have a cap on these grants, and just getting to the UK is very expensive these days, even though it’s a modest 5-hour direct flight from here to Heathrow. Luckily, there’s a special fund available through Dalhousie for international conference travel, specifically intended to help with the likely shortfall between a standard-sized travel grant and the real cost of even a short trip. I applied for a modest additional sum from this fund, to make up the difference between the King’s grant and what I anticipated the whole trip would cost. This application was turned down.

This is perhaps an unnecessarily long preamble, especially since the real point of this post is not to complain about this outcome–or at least, not exactly. I understand that universities do not have enough money to pay for everything, and I’m actually readier than most of my colleagues to point to conference travel as something that ought to be more rare than it is, given the multitude of ways we can communicate with each other for free, or at least for no more than the already budgeted overhead costs for internet access. I myself last attended a conference in 2009. (That doesn’t mean that there is nothing valuable about bringing a community of scholars together, though, and providing the occasion and support for focused and also serendipitous face-to-face engagement.) Given the shortage of funds, I also see that applications have to be ranked and there will be some that aren’t ranked as high as others. I do think the rationale for these decisions should be explicit and transparent, so that we put in the best applications we can and can realistically assess our chances of success before going to the trouble of applying again. So on getting the bad news that I wasn’t getting any more money from my university towards this trip, I fired off a request for the reviewers’ assessments of the application, and here’s where I got my knickers in a bit of a twist, because the single negative comment in the two otherwise perfectly positive evaluations was, under Quality of Applicant,”The applicant’s publications record is spotty.” Given that neither reviewer objected to my budget or to anything about the conference or to my proposed participation in it (“the conference and the applicant’s contributions seem noteworthy,” remarked the same person), it doesn’t seem far-fetched to conclude that the “spotty” publication record accounts for the relatively low rating assigned the application by Reviewer #2 (of course they are anonymous, though my application wasn’t) and its mediocre overall final rating. Did that perception “cost” me the grant? I’m not sure, since I don’t know precisely where the cut-off point was for the fund/don’t fund decision, but something did, and that’s the only clue I have about what it might have been.

But here’s the thing. Yes, absolutely, my publication record over the last 5 years (which is what they asked about) is “spotty” if by that you mean that I haven’t published a peer-reviewed academic article since 2007. My anthology of Victorian writing on the novel did come out in 2009: I’m not sure where editorial projects rate on the whole academic prestige scale, but I can tell you it was a pretty big undertaking and included a fair amount of original scholarship, particularly of the “reading things from old periodicals and figuring out if they are somehow significant, based on research into secondary materials about Victorian theories of the novel” kind. During that 5-year period, though, I have also published 5 essays and 4 book reviews in Open Letters Monthly. Actually, those are all since October 2009–so in the past 18 months or so, I have been more active as a published writer than during any previous time in my life. Because the application asked us to highlight publications especially relevant to the proposed conference, I listed my essay on Ahdaf Soueif as well as my review of Brenda Maddox’s George Eliot in Love and my pieces on Felix Holt and Vanity Fair (remember, the panel is on ‘knowledge dissemination in Victorian Studies’). Though the blog is discussed in the description of my paper I submitted for the panel (included in the application file), I did not attempt to declare it a “publication” on the application form, even though it is in fact the writing most relevant to a paper about my experience as an academic blogging in Canada. As a blogger, there’s nothing “spotty” about my record at all: I have posted 2-3 pieces (sometimes more) every week on my blog since 2007, for a grand total of 600 posts (601, counting this one!). Some of them are incidental, some of them are substantial. Some reflect original (if sometimes incomplete) thinking about scholarly problems, many of them address critical and interpretive questions. I think I could make the case that on this occasion, I have every right to identify this material as in an important sense a relevant “publication”–but I didn’t. There’s no place on the form, after all, to justify doing so, and the package overall is pretty clear about the relevance of blogging because that’s what I was invited to the conference to talk about. Maybe I made a tactical error in trying to avoid directly confronting the whole “a blog is just meaningless self-publishing” thing, but you’d think a careful reviewer might have thought a little outside the box provided and seen that in this case, that “spotty” comment was kind of missing the point.

I did, however, present my Open Letters pieces as publications, though I did not put the little asterisks next to them to indicate “peer-reviewed.” I think this is where the “spotty” comment really comes from, and here’s where I think our reliance on (or our faith in) peer review does a disservice, not just to those of us doing other kinds of publishing, but to the principle that what matters is the quality of the work, not the system that grants it (or doesn’t grant it) an extrinsic stamp of approval. Nobody is going to actually read any publication I list on something like this, starred or not. The task of deciding whether I’m doing good work or not has been outsourced to the readers of academic journals. I’m sure I don’t have to tell an audience of bloggers that there are serious flaws with peer review (see here for lots more about it). There’s no good reason (except efficiency and habit–and I don’t underestimate the weight of these) to assume that the little asterisk means “job well done” while its absence means “not a real contribution to the academic enterprise.” Or, we shouldn’t assume that if we understand “the academic enterprise” a little more broadly than perhaps many people do. Consider the panel topic again: “knowledge dissemination.” That’s an ugly coinage, but basically doesn’t it mean getting the word out about what we know? Who says that the only important thing is getting the word out to other academics? (Indeed, who says that other academics get their information exclusively from academic sources?) In fact, though probably the humanities were not at the forefront of anybody’s mind when worrying about this, there’s been a national discussion in recent years about the importance of communicating scholarly research beyond the traditional frameworks (for example). My review of Brenda Maddox’s book has been viewed 561 times since it went live. As academic blogger Alex Reid has reported, statistics show that 93% of humanities articles go uncited. That doesn’t mean they aren’t read, but if they were serving the purpose they are supposed to, e.g. furthering academic understanding and debate, their citation rate would surely be close to their viewing rate. I can’t know what the results were of those 561 “hits,” but it seems fair to say I disseminated something there. My essay on Gone with the Wind has been viewed more than 9700 times since it went live, and though you would have to read it to know, it offers an extensive ‘expert’ reading that is enmeshed in my work on fiction and ethics, as well as on historiography: it just presents that expertise in an accessible, jargon-free (well, nearly!) way. The Ahdaf Soueif essay has been viewed a modest 282 times–but reading it over, I am convinced it makes an original contribution to our understanding of Soueif’s novels, and 282 is not bad if Alex Reid and his respondents are right that the average readership for a humanities article is somewhere between 2 and 7. Why would it be better if I had padded it around with footnotes and laid it to rest in the Journal of Middle Eastern Literatures? Of course, I realize it might have been rejected if I had sent it to the Journal of Middle Eastern Literatures. The whole thing would have been a much slower process, certainly: not just writing the kind of paper that can compete for space in such a publication, but waiting to see how I did. Heck, I waited 5 months for a reply to a preliminary inquiry about submitting it to another journal, one that claims to ‘welcome’ such inquiries and even to recommend them prior to our undergoing the full submission process. By writing it on my own terms, subjecting it not to academic peer review but to the scarily rigorous review of my co-editors at OLM, and then publishing it there, I seized an important moment and, yes, disseminated my knowledge. The only way in which I think it would be preferable to be in, say, JMEL, would be that an essay there would be found and cited by other scholars working on related topics in a way that probably the Open Letters piece won’t be–it won’t show up in the MLA bibliography, right? That said, any scholar with anything on the go these days will do a range of searches including a web search, so someone writing on Soueif is surely bound to find my essay if they are doing a half-decent job of research.

I can’t help but be aware, though, that to some extent I am rationalizing my own recent choices, the way I have prioritized my time. It has turned out that for me, it is not possible to do everything. One rationale often heard for academic blogging is that it can further a ‘proper’ research and publication agenda. This has happened for me to some extent, especially in the early days of my writing about Soueif and while I was contributing at The Valve, but that’s not really how I have been using my blog for some time. Instead of writing posts about Victorian literature, or Victorian studies, I have been writing about Vera Brittain and Elizabeth Bowen and Maj Sjowall and Per Wahloo–and Salley Vickers and Jennifer Egan and Morley Callaghan and Brian Moore. I have chosen to do this, as I have chosen to write essays and reviews for Open Letters on both Victorian and non-Victorian topics, in the full knowledge that I have only so much time for reading and writing and that these are not the kinds of reading and writing that will serve me best professionally. I know perfectly well that the kind of writing I’ve been doing doesn’t “count”; I feel guilty and inadequate and defensive about not having been equally productive at the kind that does, and I can hardly pretend to be surprised that there are consequences to this.I have thought often about how my decision to use the security of tenure to experiment with the parameters of my work as a critic will affect my chances of any further promotion: it’s ‘research and publications’ (that is, research that leads to academic peer-reviewed publications) that will count for that, no matter how enriched my teaching is as a result of my other intellectual explorations, or how good and even well-respected my non-academic essays might become.

A couple of my “tweeps” have kindly suggested that it’s not easy being “in the vanguard” or being an innovator in a system that is as rule-bound as academia (I’m extrapolating a bit from their 140-character replies to my venting!). I think that without making exaggerated claims for the value of the writing I’ve been doing, it’s safe to see the reviewer’s response to my profile as symptomatic of something like this. The categories and labels in use are no longer sufficient; the boxes we are given to fill in do not fit what all of us are doing; the patterns we are expected to follow need to be altered. A productive, respected (if I may?) blogger presenting a paper on blogging at a panel on knowledge dissemination should not need a string of unrelated peer-reviewed publications to prove herself.

And that really is my last word on this incident! (Well, except for any responses I might have to comments. And any follow-up tweets …. )

I disagree with you about the citation rate as a measure of the value of scholarship. I’m currently writing an article about a minor 20th C writer. There are about 50-60 academic articles, books, etc. on this figure total. The article looks at aspects of all this writer’s work, which ranges from philosophy to fiction to political pamphlets. I have read and taken notes on more-or-less all of these secondary materials. Even though I will probably only cite about 10-20% of them in my article, the ones that I do not cite were nonetheless valuable and contributed to my overall understanding of the writer. Furthermore, this has been the case in all the research I’ve ever done. Some journals encourage footnotes where you summarize a range of scholarly views very quickly, and those will have higher citation rates. If I were writing this article for a venue like that, I might very well cite more than half of the total scholarship.

The reason that relatively few journal articles get cited many times has been discussed by Robert Merton and others. Merton called it the “Matthew effect.” It’s usually just rote expectation rather than intrinsic merit, and there’s a strong cascading effect. My guess would be the that the widespread use of databases like JSTOR has actually changed the scholarly dynamics of citation.

I’m really glad you wrote out this longer version. There are a lot of important issues in there that deserve wider discussion (beyond “venting” about specific decisions). It seems that the only time they can be raised is when we have specific complaints. However, it is possible to move from the specific to identify general patterns that do need to be addressed.

First, something you didn’t mention but that is probably relevant is that this was likely an interdisciplinary panel looking at applications from across the university. In a discipline specific panel, the edited volume would likely have had more weight. Outside of the literary disciplines, the scholarly value of editions is poorly understood. This has been discussed with regard to large collaborative interdisciplinary projects at SSHRC for example. (see this paper from the Federation which I worked on http://fedcan.ca/content/en/339/humanities-project.html)

Second, I think the forms combined with our knowledge of the existing culture creates a situation in which many people have a lot of things they don’t know where to put. The SSHRC CV has “Other Research Contributions” which is where you could put those but even then, ppl are unclear what that means. Either we have to take some chances and just be bold, or we need some changes to the forms and/or guidelines for both applicants AND reviewers.

There is a debate about “relevance”, “wider impact”, and “knowledge mobilization” both here and in the UK. Currently it focuses on policy unfortunately, but I think there is a debate to be had about how we communicate humanities knowledge to wider audiences (perhaps not policy makers but the many others who might need/want it). Unfortunately many in the humanities have been taking a different approach and arguing that this whole debate is a travesty or something.

You are indeed in the vanguard and it is a rather uncomfortable place to be. I hope you can get to that conference so you can start to talk to others about how this works in the humanities and why Victorian Studies should not be dismissed out of hand as some trivial subject with no relevance beyond a narrow academic circle.

I agree with Jonathan re relationship between citation and readership/usefulness.

Also, citation indexes are a poor measure for humanities because they do not include works cited in books, only those cited in journals. Given the importance of book publishing in humanities scholarship this is very important to keep in mind.

In disciplines where almost all publishing is in academic journals (e.g. most sciences, economics), they are a pretty good measure of impact though they don’t take account of Jonathan’s point about works that influenced your thinking on the topic that was not used in a way that merited citation.

Jonathan, you’re right that relying on citation numbers is not an adequate way to measure the value of the scholarship–though I was talking less about its intrinsic value then about its impact, or the circulation of its ideas. The context here, after all, is ‘knowledge dissemination.’ Jo has a good point (and I know you’ve made it elsewhere, e.g. here) about the problems with citation indexes for humanities scholarship too. It’s obviously much harder to measure the readership of any piece of writing (who knows how many people who bought that massive bestseller The Girl with the Dragon Tatttoo read all or even most of it?), but the numbers given in the comments to Alex Reid’s post are certainly ones I have come across elsewhere (e.g. fewer than 7 readers per academic article). What does readership have to do with value? Good–and difficult–question, of course. I think I would dodge it, myself, and instead say, look, great, write for the other people in that circle of specialists on your “minor” novelist–that has one kind of value. But so does writing something for a different audience, especially if it ‘mobilizes’ that expertise. The two projects are complementary. Speaking of which, feel free to pitch a piece on your novelist for Open Letters!

Scholarship does circulate without being cited. Articles are read far more than they are cited (they should be, if someone is thorough in their scholarship, which is not hard to do with a relatively understudied writer).

I’m a big proponent of scholarly blogging, open access, and all the rest of it; but I’m suspicious of arguments for the value of those activities which appeal to the lack of usefulness of the the traditional scholarly article.

Hope this is not missing the point: we have every right to be offended about any element of such a decision that aggravates us. (With more teaching experience than most graduate students, I really concentrated on my publishing in my early years, only to hear from search committees that they were then concerned that I was too “research-y.” Argh. There is a piercing insult in every opinion, I must conclude.) Anyway… I am suprised that publication record plays a role in such a decision because, in part, of the difficulties associated with interdisciplinary adjudication — as one of your commentators has said. If you’re eligible and a group wants you, surely relative strength of publication history across disciplines is the least reliable “tie-breaker.” We do a bunch of things incorrectly here, but on this matter we determine whether one is really participating — rather than simply attending in some lesser, supporting capacity — and then hand out grants on a first-come, first-served basis for each of four segments of the annual calendar. We can — and do — agonize over size of the funding pool, size of the grant, etc, but I cannot imagine trying to decide on the basis of whether a book and two blog posts bests a collection, two conference papers, and a radio interview. How many angels can dance on the head of a pin, indeed…?

I’m so sorry to hear this, Rohan! How very frustrating. To me, it sounds like the same sort of situation as when learner drivers sitting their test are told they ‘didn’t check their mirrors enough’. It’s very hard to know how often a driver checks a mirror and so it generally means the driving examiner just didn’t want to put them through for nebulous reasons. Is it possible that your anonymous reviewer just didn’t like the idea of talking about blogging, was resistant to that area of publication, and turned you down out of prejudice? I mean, saying ‘spotty publication’ is a bit offensive, I think, the equivalent to an academic of saying their mother is ugly. Most publications. after all, come out of conference papers, so preventing you from going to a conference that might result in a peer-reviewed publication seems bizarre to me. Hence I keep falling back on this feeling that it was s knee-jerk reaction from a bit of a jerk.

Not missing the point at all, Craig. To be fair, I am inferring a lot from this one comment–but I can’t really see how else to explain the result, unless there is something else about my application or whatever that made it a lower priority than the others.

Thanks, litlove. You may be right that there are other factors, and as I said, I know there’s not enough money to do everything and in the end someone has to come up with a reason not to give it to somebody. I know I reacted this strongly precisely because this little comment said out loud something that I hear often from the nagging little voices in my own head, too. However, I will not be stopped! Not only do I have a new essay project that I think (hope!) will combine an idiosyncratic personal interest with a number of the research areas I’ve worked in, but I’ll be at the conference and making the most of it, on my own dime if that’s how it has to be. So there.

That is a shame that you got this negative response from one of the reviewers. I am not a professor though I write grants for a university, and I understand how the system works. For what is worth, I have been really enjoying your blog since I found it. I have recently become very interested in Victorian literature, and your blog offers some wonderful insights on the books I am reading and intend to read. You do it in a way that is accessible to me–a nonacademic who is a passionate reader and has a great deal of intellectual interest in Victorian studies. I have even marked off certain posts to return to when I begin reading books. (And by the way, I can’t wait to review again all your postings on the relationship between In the Eye of the Sun and Middlemarch! I loved Middlemarch when I read it years ago–it is on of my favorite Victorian novels so far though I haven’t read too many–and I started In the Eye of the Sun a while ago, but put it down. I am excited to pick it back up to read and then look at all your postings about the book.) So I appreciate what you do and believe you deserve funding for attending conferences.

Thanks very much for your comment, Ali, and your generous words about my posts. I hope you’ll pass on your response to In the Eye of the Sun: it’s a sort of odd book but it has really grown on me. Colleen at the blog Bookphilia has undertaken a Victorian reading project, so if you don’t follow her already you might want to pop over–and Wuthering Expectations, too, always has lots good to say. I think it is more than time I reread Middlemarch myself.

Rohan,
I have been stewing about this post since I read it yesterday. Anyone who spent even ten minutes at your blog with its carefully and beautifully crafted posts could see . . . well, it’s simply absurd to call your publication record spotty. What an injustice! Nevertheless, the bias against self-published work (which blog posts are, I suppose) remains. Even more important to go to the conference and make the case for the work you do.

I have so many thoughts about this, but for now I’ll focus on the way these rejection phrases–“publication record spotty”–stick in one’s head and eventually can become part of one’s personal comedy routine. I’ve had two such experiences.

I think what you’re doing here and in the longer OLM articles is wonderful and a great service to people who study and care about books. More traditional scholarship is important, but so is having a wider audience and reaching people outside the academic world. How very annoying that that one reader didn’t get the logic behind the conference and your qualifications for it!

Susan: Thank you so much for the link to the ‘Rejection Rhumba.’ I can imagine how hurtful that kind of rejection must have been, and I admire your spirit in responding to your pain with humor. I may well try to work something about the whole ‘spotty’ thing into my conference presentation, in the same defiant spirit…

Thanks, Dorothy. I agree that both kinds of work have value–and in this particular context, as you say, there’s a logic that should have made a difference to the assessment of my particular profile.