120 computer-generated gobblygook research papers got published in scholarly journals

A researcher in France, Cyril Labbe, found a slight problem with some 120 papers that were published either by Springer or the Institute of Electrical and Electronic Engineers (IEEE).

The thing is, the “papers” were all computer-generated fakes.

The papers were made with software that creates random official-looking research papers on various topics in the computer sciences. The software, called SCIgen, produces nonsense results that are presented in correct format, and often have accompanying tables and graphs. References for the “paper” are also included.

This software was developed at the Massachusetts Institute of Technology (of course).

And not only did the papers make their ways into various research journals, but 30 of the research papers made their way into various conferences as well.

Springer and IEEE are now in the process of withdrawing those papers. Fraud in scientific papers is not new, but seems to be gradually increasing. This, however, is a huge number to have been released at the same time. We will have to read papers more skeptically in the future.

Just to show you how these “papers” can be created, I generated several “written” by some very famous authors. Here are the intros of each article, then at the bottom of the page I include an entire “research paper” that the computer generated, so you can see what the entire thing looks like. It’s a pretty amazing fake.

Fake paper 1

Fake Paper 2

Fake Paper 3

Fake Paper 4

Fake Paper 5

Fake Paper 6

Fake Paper 7

And here’s one of the fake research papers in its entirety – check out the footnotes, they’re particularly good.

Dr. Mark Thoma, MD
Mark Thoma, MD, is a physician who did his residency in internal medicine. Mark has a long history of social activism, and was an early technogeek, and science junkie, after evolving through his nerd phase. Favorite quote: “The most exciting phrase to hear in science... is not 'Eureka!' (I found it!) but 'That's funny.'” - Isaac Asimov

Let me add on more thing, if you please. I think far more important, and far more interesting, than the prestige level of the journals, is the motive behind their generation and submission. That might give a better sense of what the problems might be and how (and where) they could be addressed. And, if one was really interested in how embarrassing this is, that might be a better gage than journal reputation. At this point, no one seems to know what the motive or motives might be.

I don’t think it matters particularly much. They appeared in publications by Springer and IEEE, who are scientific publishers of some prestige. And, yes, there were papers that made it into Springer journals. Which ones? I don’t know, and I don’t think it’s terribly important. The linked article in Nature doesn’t say, either. That they were accepted into journals or conferences is embarrassing enough for the publishers and conference organizers. It indicates that there is something that publishers (in general, not just these two) need to address and perhaps a matter to be considered in the sociology of science and the structure of funding and tenure systems.

I was just trying to understand the scope of the embarrassment here. Academic publishing is a tiered system, with the journals at the top representing corresponding levels of rigor and prestige. The higher tier at which these articles appeared, the more embarrassing to those who published them.

Yes, Springer publishes journals but did these papers actually make it into those journals? Look at the headline of this post: “Gobbledygook papers published in scholarly journals.” The obvious question this headline raises is: Which journals? Having read the post, can you answer that question?

I Dont think anyone was trying to pass of fake papers as there own. Beyond the point of seeing if it could be done. I thought all the papers were generated and submitted to see if they would be accepted, AND THAT was the research. Not that joe schmo needed a paper to publish so he generated one with the software and submitted it on his behalf. Also, anyone that even reads a single paragraph from any of these papers should know its total non-sense. They are not even remotely legit sounding. Visually, they look great. Just looking at one you would have no problem believing it is a legitimate journal article. The second you read a sentance or two however, the illusion quickly fails. They are basically just random mix and matching between computer terminology and things from other papers it seems. I dont think you have to be a computer science expert to tell they are fake either. The sentences just dont even make any sense, in any context. You can tell it is giberish just by the sentence structure. If you actually know what the terms used mean, then it is even more ridiculous.

What Mark mentioned were two publishers. IEEE is a professional organization, but it is also a publisher of many significant scholarly journals in various fields that IEEE members work in. Springer also a well respected academic publisher who produces a number of scholarly journals. I’m not sure where you find the inaccuracy. Nor do I understand why you made the suggestion that the articles “were just distributed by publishing services” or were merely conference papers.

No, I’d expect them not to have the time to verify all the citations on 30+ term papers, and may be more inclined to accept information as presented, if it’s attributed properly, even if it may be slightly out of the realm of their expertise. But if a student submitted work and their primary reference was Gumby, I’m sure that would raise an eyebrow. But apparently not for the IEEE.

It’s not even about catching errors. Honest mistakes and miscalculations are all part of the peer-review process. The whole point is to have others assess your work, and correct it, or build off of it, to create a wider understanding. But when the second line of a paper is “Peter Rabbit, Vladmir Putin, Gumby, et al.” That alone should be more than enough for any editor to want to take a closer look at the validity of a submission. You don’t need any scientific expertise for that level of editorial review, you just have to be conscious and actually care about doing your job.

Point taken about peer review, but based on the information in the article I think the headline of this post is in error. The articles don’t seem to have appeared in “scholarly journals” but were just distributed by publishing services. The first indication was that you didn’t cite journal names but “IEEE” which is just a professional organization. It’s embarrassing that they were accepted at conferences and ended up in publishing databases but that’s different than, say, appearing on the pages of Nature.

You didn’t SAY it was evidence but you USED the information it as if it was evidence. Twenty years ago there was no such thing as Retraction Watch etc. etc. so it is hard to know if the phenomenon is increasing. It may be. I would bet that it is, but we don’t know. Similary, you jumped on my phrase “people in this business” as if it was something more than an obviously colloquial expression. Lighten up on that, please. If being an academic scientist was much of a business, maybe they would pay me more. As for ethics courses, in the US they are now required for any graduate student supported on an NIH or NSF grant. Other places? I have no idea. These course may help a little, but people determined to cheat will cheat. There are a lot of things messed up in the current research enterprise. This is one of many. A bigger scandal in my mind is the way universities blindly chase after meaningless “rankings”, and ever increasing growth of meaningless and flawed bibliometric data to assess people’s careers. Look, I don’t disagree with your main points, but I object to loose reasoning wherever it comes from. We don’t know if fraud is increasing because it is very difficult to know how much there was in the past.

I never said that it was evidence. However, the number of retractions/removals cited by just Retraction Watch is alarming. And most of the retractions that it publishes are not from vanity journals. Perhaps the “more people in the business” now is a key statement. I don’t think of medicine and science as being a business. They are professions and held to a higher standard. Some researchers have had 20+ papers retracted for improprieties with their “research.” I find that absolutely deplorable. Not only is it fraudulent to take research money and then manufacture data designed to get yet MORE research money. Not to mention enhancing one’s career at the expense of one’s fellows. Perhaps courses in ethics should be required for all science students and MD and DO students, as well.

I can hardly believe these bogus papers were published. After all, most teregrinographers agree that unstilted PRM becomes increasingly volatile at high levels of entuperation. Indeed, several researchers have verified the poly-ectopic symmetry of DHv12 when semi-arrayed with detectable allotreme slopes. To disregard this emerging zitherin conflict is worse than a semi-flanged haplo-pile in a multi-churn crolk machine :O

“seems like it” is not evidence. My sense is that there has always been fraud. Of course, it may be increasing, there are more people in the business now, especially in emerging countries, and a lot more crappy journals marketed as open access but really almost vanity publishing.

Checking whether or not the journals are peer reviewed or not would take a considerable amount of time. Additionally, whether or not a journal is peer reviewed doesn’t always work as a method to demonstrate the scientific integrity of a paper.

“Peer-reviewed” doesn’t necessarily mean that errors are caught and the paper isn’t published. It means that at least one person who has some degree of expertise in the field has read through the paper and detected no major flaws. However, many papers are so highly specialized, that a reviewer with a great degree of expertise on that particular topic, may be impossible to find. Or, if found, he may not be willing to review the paper. Also, it assumes that the reviewer is honest. Remember, Regnerus’ paper was peer reviewed, accepted for publication and published. In spite pf severe criticisms from other peers, it has never been retracted.

I think integrity has been sacrificed for profits. Individual copies of those papers sell for anywhere from about $25-$50+. A year’s journal subscription can be several hundred dollars/year/journal. I’m sure that making as much as possible is the goal of all of the publishers out there. Doubtless, they don’t have the staff to do a thorough job. Springer publishes 2200 journals. I wonder how many people they have on staff with expertise in relevant fields to even read through all of those papers for even gross errors, let alone catch errors in the tables or photographs?

One researcher submitted an obviously flawed paper to 300+ journals in 2013. About 150 journals accepted it for publication. That’s just one example. Plus the 120 above from the last few years. Additionally, “Retraction Watch” often posts 1-3 examples of papers being retracted each weekday. That adds up to a pretty respectable number. Perhaps academic fraud isn’t increasing. But it certainly does seem like it.

According to https://www.govtrack.us/congress/bills/ there are “There are 7,255 bills and resolutions currently before the United States Congress.” All are meant to make the rich rich, to pauperize workers, engage in wars of aggression and gut the Bill of Rights and entitlements.

Withdrawing those papers? Springer and IEEE should immediately dissolve their academic publishing in shame. It wouldn’t have taken more than three seconds to debunk each of those papers and reject them out of hand, even if you weren’t an expert on the topic. But apparently they have absolutely no editorial staff, no interest in actually verifying references, and just publish anything that is handed to them. I’d expect that from a high school teacher, not a respected academic/scientific organization. One paper sneaking through? Okay. Sure. Two or three? Then they have some serious staffing problems, and probably need to start firing people. More than that, and they have flushed away their academic credibility.

I think this post could be fleshed out a bit more. Who is Springer? What journals did the articles end up in? Were they published as peer-reviewed?

One of the best arguments against the corporatist right’s denial of climate change is that there have been almost no peer-reviewed studies showing that the climate is not affected by human activity. The peer-review and publishing process is essential to the integrity of scientific study, and anything that casts doubt on the process — or can be used as a cudgel to do so — is bad news.

Thanks again, John, for this brilliant effort in assisting us and the larger concerned community to
connect the dots. Shifted over here after rereading the latest excellent responses to AZ gov. veto.
I am of the opinion that we are seeing the bitter harvest of an educational system that has left
enormous numbers in “middle America” partially/poorly educated in all areas and not just in
“job skills”. I happened to ask the prof in a course I was taking a while back for those teaching
adult literacy what the average number of books read a year was for the average adult with
h.s. diploma. He looked at me with a very serious expression and said… zero. Regardless
of the actual number we have been building toward this moment for a long time. Coupled
with an unusually sluggish, apathetic public, what we are witnessing shouldn’t come as any
huge surprise. Our own homegrown oligarchs are now in a position to play on vague fears
and superstition like a strad that even good science seems powerless to refute/inform.
My newly minted tea party state passed voter ID among its first order of business. As in all
other states with similar “provisions”, reporters have asked why when there was absolutely no
evidence of voter fraud to justify such an intrusion. Their answer appears to be because we can?
Our smarmy governor was on cnn last week defending his position. No need to watch the link
really. Comments were interesting though and, of course, mirror those in these threads.

In the past, mostly these computer-generated things were accepted in computer science and engineering journals in China, and every scientist has received spam soliciting contributions to these more or less fake journals and conferences. I get several emails a day, literally. So the fact that those “journals” published goblygook is not exactly news. The fact that a journal published by Springer and IEEE got taken in is a little bit more surprising. In the molecular biology journals where I submit, the peer review process entails pretty careful scrutiny. As for the idea that fraud in science is generally increasing, there is not hard evidence for this one way or the other.