Slashdot videos: Now with more Slashdot!

View

Discuss

Share

We've improved Slashdot's video section; now you can view our video interviews, product close-ups and site visits with all the usual Slashdot options to comment, share, etc. No more walled garden! It's a work in progress -- we hope you'll check it out (Learn more about the recent updates).

A recent examination of current scientific publishing methods shows that they are problematic at best, treating the entire process like an economic system, with publishers as bidders at an auction, authors as sellers, and the community at large as consumers. "The authors then go on to discuss a variety of economic terms that they think apply to publishing, but the quality of the analogies varies quite a bit. It's easy to accept that the limited number of high-profile publishers act as an oligarchy and that they add value through branding. Some of the other links are significantly more tenuous. The authors argue that scientific research suffers from an uncertain valuation, but this would require that the consumers — the scientists — can't accurately judge what's significant. "

A recent examination of current scientific publishing methods show that it is problematic at best. Treating the entire process like an economic system, with publishers as bidders at an auction, authors as sellers, and the community at large as consumers.

Agreed. I think we need to switch this whole process to Vickrey Auctions [wikipedia.org]. Then you can explain to the authors of the papers that they will receive $75 for their paper instead of $100 because whoever bid $100 was gaming the system. Why is it suddenly so popular to turn everything possible into an auction system with 75 different flavors of said auction system?

I have this fantasy of writing a program which makes some big combinatorial breakthrough and from time to time to motivate myself I imagine what I might do with such a thing should I actually bumble into creating it.

I looked at scientific journals, and I honestly can't see much of an incentive to appear there. I mean sure, you might get published and that's got some merit, but it seems to me that these journals don't really make a lot of money for the scientists who write the articles. Does Science or Nature pay its writers? It can't be that much, even if they did. So what's the point?

From the scientist's perspective, if they have pure research, then, they can put it on a web site, such as the university web site or even their own, and just skip the b.s. Or, they can sell it. Either option is better to achieve the altruistic or commercial ends of the scientist than being in a magazine.

As a scientist who has published work in a few journals, I know that the process is arcane and fraught with peril. There are publishers who have axes to grind and it sometimes keeps good information out of the scientific discourse.
Of course, I can't offer a real solution because all peer-reviewed journals involve humans with all of our attendant weaknesses.

The authors then go on to discuss a variety of economic terms that they think apply to publishing

Which is probably the most problematic point.

While even in the current crisis "market models" are still hip, they don't give the answers to all questions. A scientific conclusion that starts with "if X were a market" must question, among all the other validations, the "if" part as well.

And while economy provides interesting theories that are helpful in many cases - just like evolution, it does not fit everywhere. So the very first thing that would've to be established is that the model fits.

In this case, I've not seen enough of that, so any conclusions drawn are meaningless until then.

Is this serious, or just push back from economists who are upset that a number of papers and editorials have recently appeared in high profile scientific journals questioning the description of economics as science? Allegories, for example, are not scientific.

I'm not convinced that this is true. The only people (besides the library) that receive print publications receive them as a part of their professional membership fees (or as part of a training grant). Most scientists, myself included, simply rely on the email TOC (which we receive much sooner than the hard copy) or go to the websites directly. I suspect that most journals 'make their cash' from institutional subscriptions, professional fees (in the case of Blood), and/or publishing fees.

I think there are a few websites geared towards solving problems with scientific publications (specifically in the life sciences). I think Labmeeting (labmeeting.com) is one of the earliest websites in this field and already has some good functionality for researchers.
As a researcher myself, I know that one of the biggest problems is getting scientists to use new tools. Hopefully, when the right tools comes along we will see some big changes in scientific research methods...

I'm a grad student in the natural sciences. Some other friends of mine and I started Labmeeting.com [labmeeting.com] because we are so eager to help change the way science gets published.

The current system of peer review is inefficient, arbitrary, and hidden from public view. We definitely need something new, but, as we said in our talk [blogspot.com] at BioBarCamp a while back, change needs to be gradual enough to preserve consensus.

That's why we're starting by just trying to make research tools that are useful to scientists in their everyday professional lives.

That this article got published in PLOS Medicine is a data point saying that the publication model for PLOS Medicine is flawed. That's about it.

The authors don't bother to back up any of their assertions. If there is a winner effect, for example, the most prestigious journals should have the highest rate of publication of junk results, whereas lower-ranked journals should be more accurate. So, is this true? Did the authors bother to look, or even to think about and discuss it?

Also, does "overpayment" correspond to "poor quality science" or to "only slightly more cool than the rejected paper, on second thought"?

Now, it is more true in the medical sciences that positive results are published that claim to show p0.05, but are one of a dozen similar studies 11 of which have not shown an effect (i.e. overall there was no significant finding). But this recognition has nothing to do with bidding per se; it's not that the journals are picking the high tail of a distribution of value so much as that they're seeking statistical significance without controlling for the number of times that the study was done.

And as the summary says (which is actually better than the research article itself, IMO), there are a number of other problems.

There are certainly ways that one might seek to improve scientific publishing. But this seems almost entirely off target and/or ill-supported to me.

If its a reputable journal, the evaluators assume that the articles have been thoroughly peer reviewed , and the quality can be taken as read.

The big assessment exercises (such as the 5-yearly RAE in the UK which determines the research ranking of universities) have to "assess" a metric shedload of papers - so they're not going to spend too much time on each one!

Of course, the reliability of this assumption is legendary [theregister.co.uk].

You make is seem so objective. As a scientist I can honestly say that publishing has become a racket. It used to be you sent a little postcard and received a copy of the article from the scientist who had published it. Now they want you to buy the damn thing on line or subscribe to that journal for hundreds if not thousands a year.

Peer review is often no more than an attempt to stifle other peoples work. At one time science was brought to the people..may be that's why we are such an scientifically illiterate nation! We still put all the articles in Ivory Tower Journals that few people in the mainstream read.

Finally the politics of publishing is worse than you think furthermore, many journals it's not about how good the work is but whether you can adapt it to their "publishing format."

Your postcard method will still work 100% of the time. Every scientist will be delighted to email you a pdf of his requested paper. However, the new "racket" system, though incredibly expensive, does give new options for much more efficient distribution. Yes, the price is a major problem, but it didn't supplant the old system.

The benefit to the author is that he can put the paper in his CV. The more big name journals you publish in, the more likely it is that you'll get grant funding and that all important tenure.

It's also more likely that someone will actually read your paper if it's in a big wide-circulation journal (e.g., Nature) instead of a hard-to-find low-circulation journal. This is particularly true for papers outside your own specialization where you won't necessarily have heard of them at a recent conference. The publication volume is just overwhelming — if you're going to stay current, you need someone else to filter most of the junk for you, and that's the service which selective journals like "Nature" (and review articles) ultimately provide.

The adage expresses an Essentialist [wikipedia.org] sentiment. Fire _reaches_ for the sky. Water _seeks_ the lowest level.

In this personification, "information WANTS to be free", there is a warning and deep truth (that any intelligence worker understands) - the tendancy of information to move, as to entropy, towards the greatest degree of freedom. In other words, human desires aside, whether you want it, or I want it, any restriction on the flow of information is imperfect and temporary at best.

Despite the summary, authors aren't literally like sellers at an auction. They don't receive any money for their papers. For journals with page charges they have to pay to publish!

Publishers are only "bidders" in the sense that they are competing to have the best papers published in their journals. But they don't compete monetarily — as I said, they don't pay authors. They compete through prestige: authors (often) want their papers to be published in the most famous, widely read, influential journal. Prestige is determined by selective choice in what the journals choose to publish, which scientific niche they occupy, etc.

You insensitive Clod, I'm a slashdot reader! I can all take things out of context and talk w/o RTFA!

However, yes, the reviewing mechanism is flawed. Not because of the money, but because of the "well-known" research groups in US universities. No wonder why several EU universities have decided to start their own conferences and journals.

Jefferson wanted us to farm our way to victory. Here's some primary source stuff on the subject for your edification/amusement.

What is "farming"? More to the point, what was farming in the 1770's? In the 1700's the industrial revolution had not yet begun. In those days, wealth was still centered to a large degree around land. Jefferson has no other context in which to suggest goals people should strive for.

But what was "farming"? Did he mean people should be wheat farmers, grain farmers, turnip farmers? Did the specific crop grown effect the desired outcome? No. What was important was the fact that, then as now, a farmer is a self employed individual who makes their own way in the world. It was self employment and independence, not turnip growing, that Jefferson was advocating.

Were he teleported into out modern society, I think Jefferson would promote professions that lend themselves to self employment.

You raise an interesting and valid point. Certainly Jefferson would be aghast at what "farming" has become now to say the last. And while the Industrial Revolution proper hadn't happened, certainly Britain and the rest of the world had experienced factories. Jefferson was appalled at a supposed "moral decline" that accompanied such things.

And certainly he would be more in favor of self employment, but there are many forms of self-employment that he would undoubtedly be dubious of to say the least. And I certainly wouldn't describe Jefferson as a fan of the notion that technology is key to national success; on this he could be quite backwards thinking.

What he seems to be in favor of is "national self-employment" with a blue-collar pastoral tinge. He (as I read it) wants to export raw materials earnestly earned by the honest sweat of one's brow. From the linked article: "for the general operations of manufacture, let our work-shops remain in Europe. It is better to carry
provisions and materials to workmen there, than bring them to the provisions and materials, and
with them their manners and principles."

Basically, what Jefferson wanted was an idyllic, pastoral America strengthened by honest individualized labor. To say that he would be lost in the face of the modern globalized technological economy is an understatement of monumental proportions.

But that's exactly my point. Of all the historical figures I could think about quoting for insight on the economy, Jefferson would most seriously be at the very bottom of the list.