A blog on statistics, methods, philosophy of science, and open science. Understanding 20% of statistics will improve 80% of your inferences.

Friday, April 14, 2017

Five reasons blog posts are of higher scientific quality than journal articles

The Dutch toilet cleaner ‘WC-EEND’ (literally: 'Toilet Duck')
aired a famous commercial
in 1989 that had the slogan ‘We from WC-EEND advise… WC-EEND’. It is now a
common saying in The Netherlands whenever someone gives an opinion that is
clearly aligned with their self-interest. In this blog, I will examine the
hypothesis that blogs are, on average, of higher quality than journal articles. Below, I present 5
arguments in favor of this hypothesis. [EDIT: I'm an experimental psychologist. Mileage of what you'll read below may vary in other disciplines].

1. Blogs have Open Data, Code, and Materials

When you want to evaluate scientific
claims, you need access to the raw data, the code, and the materials. Most journals
do not (yet) require authors to make their data publicly available (whenever
possible). The worst case example when it comes to data sharing is the American
Psychological Association. In the ‘Ethical
Principles of Psychologists and Code of Conduct’ of this professional
organization that supported
torture, point 8.14 says that psychologists only have to share data when
asked to by ‘competent professionals’ for the goal to ‘verify claims’, and that these
researchers can charge money to compensate any costs that are made when they
have to respond to a request for data. Despite empirical proof that most scientists do not
share their data
when asked, the APA considers this ‘ethical conduct’. It is not. It’s an
insult to science. But it’s the standard that many relatively low quality scientific
journals, such as the Journal of Experimental Psychology: General, hide behind
to practice closed science.

2. Blogs have Open Peer Review

Scientific journal articles use peer review
as quality control. The quality of the peer review process is as high as the
quality of the peers that were involved in the review process. The peer review process
was as biased as the biases of the peers that were involved in the review
process. For most scientific journal articles, I can not see who reviewed a
paper, or check the quality, or the presence of bias, because the reviews are
not open. Some of the highest quality journals in science, such as PeerJ and Royal Society Open Science,
have Open Peer Review, and journals like Frontiers
at least specify the names of the reviewers of a publication. Most low quality
journals (e.g., Science, Nature) have 100% closed peer review, and we don’t
even know the name the handling editor of a publication. It is often impossible
to know whether articles were peer reviewed to begin with, and what the quality
of the peer review process was.

Some blogs have Open pre-publication Peer Review. If you read the latest DataColada blog post, you can see the two
reviews of the post by experts in the field (Tom
Stanley and Joe
Hilgard) and several other people who shared thoughts before the post went
online. On my blog, I sometimes ask people for feedback before I put a blog
post online (and these people are thanked in the blog if they provided feedback),
but I also have a comment section. This allows people to point out errors and
add comments, and you can see how much support or criticism a blog has
received. For example, in this blog on why omega
squared is a better effect size to use than eta-squared, you can see why
Casper Albers disagreed by following a link to a blog
post he wrote in response. Overall, the peer review process in blog posts
is much more transparent. If you see
no comments on a blog post, you have the same information about the quality of
the peer review process as you’d have for the average Science article. Sure, you
may have subjective priors about the quality of the review process at Science
(ranging from ‘you get in if your friend is an editor’ to ‘it’s very rigorous’)
but you don’t have any data. But if a
blog has comments, at least you can see what peers thought about a blog post,
giving you some data, and often very important insights and alternative
viewpoints.

3. Blogs have no Eminence Filter

Everyone can say anything they want on a
blog, as long as it does not violate laws regarding freedom of speech. It is an
egalitarian and democratic medium. This aligns with the norms in science. As
Merton (1942) writes: “The acceptance or
rejection of claims entering the lists of science is not to depend on the
personal or social attributes of their protagonist; his race, nationality,
religion, class, and personal qualities are as such irrelevant.” We see
even Merton was a child of his times – he of course meant that his *or her* race,
etcetera, is irrelevant.

Everyone can write a blog, but not everyone
is allowed to publish in a scientific journal. As one example, criticism
recently arose about a special section in Perspectives on Psychological Science
about ‘eminence’ in which the only contribution from a woman was
about gender and eminence. It was then pointed out that this special
section only included the perspectives on eminence by old American men, and
that there might be an issue with diversity in viewpoints in this outlet.

I was personally not very impressed by the published
articles in this special section, probably because the views on how to do
science as expressed by this generation of old American men does not align with
my views on science. I have nothing against old (or dead) American men in
general (Meehl be praised), but I was glad to hear some of the most important
voices in my scientific life submitted responses to this special issue. Regrettably,
all these responses were rejected. Editors can make those choices, but I am
worried about the presence of an Eminence Filter in science, especially one
that in this specific case filters out some of the voices that have been most important in shaping me as a scientist.
Blogs allows these voices to be heard, which I think is closer to the desired
scientific norms discussed by Merton.

4. Blogs have Better Error Correction

In a 2014 article, we published a
Table 1 of sample sizes required to design informative studies for different statistical
approaches. We stated these are sample sizes per condition, but for 2
columns, these are actually the total sample sizes you need. We corrected this
in an erratum. I know this erratum was published, and I would love to link to
it, but honest to Meehl, I can not find it. I just spend 15 minutes searching
for it in any way I can think of, but there is no link to it on the journal
website, and I can’t find it in Google scholar. I don’t see how anyone will
become aware of this error when they download our article.

When I make an error in a blog post, I can
go in and update it. I am pretty confident that I make approximately as many
errors in my published articles as I make in my blog posts, but the latter are
much easier to fix, and thus, I would consider my blogs more error-free, and of
higher quality. There are some reasons why you can not just update scientific
articles (we need a stable scientific record), and there might be arguments for
better and more transparent version control of blog posts, but for the
consumer, it’s just very convenient that mistakes can easily be fixed in blogs,
and that you will always read the best version.

5. Blogs are Open Access (and might be read
more).

It’s obvious that blogs are open access.
This is a desirable property of high quality science. It makes the content more
widely available, and I would not be surprised (but I have no data) that blog
posts are *on average* read more than scientific articles because they are more
accessible. Getting page views is not, per se, an indication of scientific quality.
A video on Pen Pineapple Apple
Pen gets close to 8 million views, and we don’t consider that high quality music
(I hope). But views are one way to measure how much impact blogs have on what
scientists think.

I only have data for page views from my own
blog. I’ve made a .csv
file with the page views of all my blog posts publicly available (so you
can check my claims below about page views of specific blog posts below, cf.
point 1 above). There is very little research on the impact of blogs on
science. They are not cited a lot (even
though you can formally cite them) but they can have clear impact, and it
would be interesting to study how big their impact is. I think it would be a
fun project to compare the impact of blogs with the impact of scientific
articles more formally. Should be a fun thesis project for someone studying
scientometrics.

Some blog posts that I wrote get more views
than the articles I comment on. One commentary blog post I wrote on a paper
which suggested there was ‘A surge of
p-values between 0.041 and 0.049 in recent decades’. The paper received 7147
view at the time of writing. My blog post received 11285 views so far. But it
is not universally true that my blogs get more pageviews than the articles I
comment on. A commentary
I wrote on a horribly flawed paper by Gilbert and colleagues
in Science, where they misunderstood how confidence intervals work, has
only received 12190 hits so far, but the article info
of their Science article tells me their article received three times as many
views for the abstract, 36334, and also more views for the full text (19124). On
the other hand, I do have blog posts that have gotten more views than this
specific Science article (e.g., this post on Welch’s
t-test which has 38127 hits so far). I guess the main point of these
anecdotes is not surprising, but nevertheless worthwhile to point out: Blog are
read, sometimes a lot.

Conclusion

I’ve tried to measure blogs and journal
articles on some dimensions that, I think, determine their scientific quality. It
is my opinion that blogs, on average, score better on some core scientific
values, such as open data and code, transparency of the peer review process, egalitarianism,
error correction, and open access. It is clear blogs impact the way we think
and how science works. For example, Sanjay Srivastava’s pottery
barn rule, proposed in a 2012 blog, will be implemented in the journal Royal
Society Open Science. This shows blogs can be an important source of
scientific communication. If the field agrees with me, we might want to more
seriously consider the curation of blogs, to make sure they won’t disappear in
the future, and maybe even facilitate assigning DOI’s to blogs, and the
citation of blog posts.

Before this turns into a ‘we who write
blogs recommend blogs’ post, I want to make clear that there is no intrinsic
reason why blogs should have higher scientific quality than journal articles. It’s
just that the authors of most blogs I read put some core scientific values into
practice to a greater extent than editorial boards at journals. I am not
recommending we stop publishing in journals, but I want to challenge the idea
that journal publications are the gold standard of scientific output. They fall
short on some important dimensions of scientific quality, where they are
outperformed by blog posts. Pointing this out might inspire some journals to improve
their current standards.

52 comments:

"If the field agrees with me, we might want to more seriously consider the curation of blogs, to make sure they won’t disappear in the future, and maybe even facilitate assigning DOI’s to blogs, and the citation of blog posts."

Yes!

Perhaps some sort of collaboration with preprint servers like Psyarxiv could be established. Of course at this point in time you could already just make a pdf-file of your blog and post it on Psyarxiv for instance, but perhaps something more fancy, and easier could be developed.

I guess/assume Psyarxiv already has some sort of mechanism in place that makes sure the preprints are preserved, and they already make it possible to cite them. The only thing not (yet) in place would be for google scholar to index them so possible citations would actually "count" more concerning things like h-index, etc.

Crossref (the people behind DOIs for scholarly articles) are working on this. Crossref Event Data is heading into Beta very soon. We're collecting mentions of scholarly articles on social media, blogs etc, via their URLs and DOIs into an open dataset that people can search and build on. More info at https://www.crossref.org/services/event-data and a lot more detail at https://www.eventdata.crossref.org/guide . I'm happy to discuss here or contact jwass@crossref.org

Point 4 is interesting -- I thought it was just me, but if you can't find your own correction, then I must wonder if the journals are having some problem server-side. See this meta-analysis, the correction to which, like yours, has gone missing: http://journals.sagepub.com/doi/full/10.1177/0146167213520459

""If the field agrees with me, we might want to more seriously consider the curation of blogs, to make sure they won’t disappear in the future, and maybe even facilitate assigning DOI’s to blogs, and the citation of blog posts."Yes!Perhaps some sort of collaboration with preprint servers like Psyarxiv could be established."

I feel like most people don't know about the Winnower, but yes, that's exactly what it allows you to do - getting DOIs for blog posts, which are tracked (at least) by Google Scholar, plus permanent archiving of a snapshot of the post (once you're confident it's truly "done") plus hosted peer review.

Hi Richard, thanks - I know about the Winnower (e.g., here's one of my posts on the Winnower https://thewinnower.com/papers/1403-p-curves-are-better-at-effect-size-estimation-than-trim-and-fill-and-michael-jordan-is-better-at-free-throws-than-i-am and there are several others). Yes, it's one way to get blogs a DOI. I just wanted to point out the general point (curating blogs) - The Winnower is indeed one way, but I'm not sure what the best way is, as of yet.

I would add that blogs are designed to fit human cognitive capacities. To reach a broad audience, they are written to be broadly comprehensible -- and as it turns out, that's a good thing for science (and philosophy), since it reduces the tendency to hide behind jargon, technical obscurities, and dubious shared subdisciplinary assumptions. The length of a typical substantive blog post (500-1500 words) is also, I think, a good size for human cognition: long enough to have some meat and detail, but short enough that the reader can keep the entire argument in view. These features make blog posts much easier to critique, enabling better evaluation by specialists and non-specialists alike.

You forgot the sixth argument: blogs are resourceful for the open education. I'd like to manage the "meta-blog" of your's, Andrew Gelman's, Uri Simonsohn's, and etc. This will be the open-accessed textbook of methodology and falsifiable scientific knowledge.

Well... blogs are also laden with useless comments, if not worse. Or they are taken hostage by few individuals discussing side issues at length, almost not related with the principal post.It take too much time to try to weed out these fields. Isn't it the purpose of peer reviewed journals to provide that service? The question remaining is : are reviewers doing their job? or do the filter convenient articles from disturbing ones?

What makes a "blog"? Is it the stylesheet? The interactivity? The social engagement? ..

Presenting scholarly work in a "blog" is irrelevant IMO. What matters is presenting in a way that's most suitable to the reader and improving the communication. It can be accompanied with interactive components to help the reader understand better. After all, what's the point if one can't communicate/educate or pass on their knowledge to the others?

I fully support the idea of publishing scholarly work at a "blog", but due to initiatives towards like openness, accessibility, decentralisation, interoperability. Some of which you touch on in your article (above).

There is an initiative called "Linked Research" https://linkedresearch.org/ ( https://twitter.com/linked_research ) which should interest you. It doesn't mandate precisely how one should publish but whether they can address some challenges like mentioned here http://csarven.ca/linked-research-scholarly-communication#user-stories . Part of that strives for improving human and machine-accessible scholarly knowledge (articles, peer-reviews..).

As for whether needing DOIs or not, they are a form of centralisation. DOIs do not ensure quality. Anything can be assigned (registered with) a DOI. For DOIs to be useful, they end up being used as HTTP DOIs (via dio.org). That is not better than "http://daniellakens.blogspot.ch/" where this article sits. The difference is that, you in a sense have more "control" over your domain (although in your particular case it is the owner of blogspot.ch) as opposed to the third-party doi.org. At the end of the day, you have more say over what happens under daniellakens.blogspot.ch vs doi.org (in which you practically have zero control). Moreover, to obtain a DOI, one would have to jump through hoops.. instead of simply just publishing work under a webspace that one controls (rents from). Similar arguments hold for ORCID. I've discussed some of this recently in this thread https://groups.google.com/a/force11.org/forum/#!topic/f11discussion/_jE0D4ns_RQ

Daniel, I've put together something in response to a climate conspiracy blog misusing your article. (Feel free to delete this comment as you did my previous, if you want to. I just wanted to let you know about it as a courtesy.)

Hi Sou, thanks for this blog - I agree with basically everything you say. My blog is indeed not a scientific argument - I have no data - and it is indeed, as you write, meant to make science journals consider improving some core scientific values. Some parts are illustrations of problems (me not being able to find my own correction is just problematic, even when n = 1) but my blog should not be used as an argument by climate skeptics to argue their blog is better than scientific articles. I am not an expert on climate change, but browsing the blog you link to, it did seem to be a good example of the blog posts I read and reference above (note that ALL blog posts I link to in this blog are written by scientists, for scientists).

P.S. Apologies for deleting your previous comment - I was deleting your double postings, but must have deleted both. If you still have it, feel free to add it again, although I think your blog is probably an excellent reply. I think you see the point of my blog, not as a scientific claim, but as an opinion piece that is aimed at making people think about the way we publish articles, and how we can improve. It's too bad a climate skeptic uses it as an argument, and it's good you try to combat that - but that is not my fight. I'm trying to increase awareness of ways to improve the quality of articles and journals in psychology, but good luck on your end.

No problem - and to correct my own comment: "it did seem to be a good example" - I actually meant, the climate change blog did NOT seem to be a good example. Many of the blog posts I discuss above could be published in scientific articles (and some end up as publications, both from my own blog, as from other blogs), exactly because they have higher quality standards.

"Despite empirical proof that most scientists do not share their data when asked, the APA considers this ‘ethical conduct’. It is not. It’s an insult to science."

Indeed it means that the paper is not science, but speculation, assertion or rhetoric. Science requires that results can be verified and that research is repeatable. If data, processing code and experimental protocols are not all shared then the results cannot be verified. It is possible that they can be repeated without data, but cost would usually prohibit as in most cases funding would not be forthcoming to repeat research that has already been published, because of a foolish assumption by funders that the original research was almost certainly competently carried out and analysed, so they do not expect new information or a published paper.

If even data is withheld then it is highly unlikely that the research can be challenged in any way, thus it is not science.

If I, with a mere B.A. in science, albeit from an excellent university, know this then why do the journal editors not? Why do the researchers, and more importantly their competitors who could gain by shaming them into releasing data, not know this?

20 years ago I was turned down for a junior job in scientific publishing. I am more confused than ever now as to why, given the incompetence of the majority of people now in senior positions, many of whom must have started their own careers around that time.

"The science the public does not see, neither in the media nor on blogs, is also important for science. We will need a way to disseminate science that also works for the other 99.9% of science. Blogs and "blog review" won't do for this part."

I do not find the majority of these arguments very convincing. That said, I am coming from a different field (medical physics), and I get the impression that these points may have more relevance in fields in which difficulty in replication is more of a problem.

First I would like to ask: Do these arguments apply to a distinct blogging platform that is used/curated by the scientific community? Or just any blogging platform (blogspot, wordpress, etc.)? If the former, what is the difference from existing platforms like pre-publication arxiv, or ResearchGate? If the latter, there are obvious problems of wide-spreading of popular but inaccurate information, lack of expertise in areas that require it, formation of positive feedback/amplification of misinformation, and overall "muddying of waters". I expect it is the latter, as this article itself is published on blogspot.

I will address each point separately:

1) Transparency in data/methods is of course desirable. Again this may be my field, but this is a non-issue. In my experience, all data/methods required for accurate replication or innovation of an experiment is included in the publication. In the few instances that I have requested code/data, it has been shared (admittedly I have not needed to do this often, so maybe I got lucky in those few instances). Looking at the links, evidence provided is only for psychology or medicine, so I believe this problem could exist in these fields and I agree needs to be rectified.

2) I think most view closed peer review as the lesser of two evils. Yes the issues Daniel brings up definitely exist, but you could apply those same arguments to open peer review. The peer review is still just as biased as the biases of the peers, that does not change if I know who the reviewers are. But now there is an added complication that it is possible for an author to accept illegitimate comments or reject valid comments because they know who they came from. So not only are the reviewers' biases involved, but the author's as well. This is a case where there is no perfect system, so it is a problem of minimizing negatives. Daniel mentions blogs have comments. But arxiv e-mail discussions and ResearchGate comment sections exist, so what would be the difference? Also, side question, by what metrics are Science and Nature "low quality journals"?

3) This point includes the biggest problem I have with this: "Everyone can say anything they want on a blog". This is a problem, as there is no system in place for fact-checking, and we have all seen the runaway train of misinformation in current media that could occur (scientists are human and are therefore not immune to this). I also disagree with the statement "it is an egalitarian and democratic medium"; Democratic does not mean egalitarian, and the two are often in conflict, especially on open source media such as blogging platforms where popular and easily digestible pieces are amplified in a positive feedback loop (accuracy is secondary), and boring pieces that contain more rigour and thoughtful analysis are suppressed.

4) Great point. For example, I am always frustrated when reading news and corrections are shoved in an impossible-to-find backwater after the mass majority have already read the main piece and considered it factual (especially if the error correction changes the entire conclusion of the writing!). Any existing corrections should be front and center on any article, which may be easier to do as journals shift to online platforms. However, the ability to update old articles creates new problems with dishonest authors modifying previous research to correspond to new data. Again, there is no realistic system to rigorously moderate this.

5) Again I agree with this, blogs are definitely more accessible, and I do agree that science should be more accessible. However I do see an advantage of having a distinction of easily-accessible "easy reading" format of blogs where I take in the information, but it is taken with a grain of salt, as (like Daniel says) "everyone can say anything they want on a blog". If I am looking for quality assurance, the absence of any system (however flawed) does not increase my expected level of accuracy. Proof of a flawed system does, however, decrease my expected level of accuracy.

Fundamentally, Daniel is an advocate of "open data and code, transparency of the peer review process, egalitarianism, error correction, and open access." With the exception of the stated misgivings of the second, I (and most) would agree with these. However, I see no reason why free-form blogging should be the solution instead of the existing platforms that are used and curated by the scientific community in a transparent manner, such as ResearchGate (https://www.researchgate.net/about).

Hi Cam, as you can see, I'm a psychologist, and if the situation is better in medical physics, good for you! If 1 is a non-issue in your field, even better, but as I explain, it's a big problem in my field. Ad 2 - open peer review is not just knowing *who* reviewed, but having access to the review. Again, arxiv and researchgate (really - researchgate? do people use that?) might have this solved, but they are more similar to blogs with respect to self-publishing than a journal, no? Your major (only?) issue seems to be with 3. If you think journals do a great job gatekeeping, that's great, and I think they often do a pretty good job. But not always, and I favor a review system where there is only an eye for quality (e.g., as Frontiers and PeerJ have) but not for novelty, eminence, etc. That way, you have the quality check, but not the eminence filter. See - I think we are pretty close to agreeing here, right?

Do you really not have access to the review? What is the point of the review process then? When I submit a paper I receive back an exhaustive list from each (anonymous) reviewer of comments, clarifications, and suggestions that must be addressed. If you do not have access to that then I fail to see the point, and I definitely understand the feeling of frustration with the process.

And yes, I am definitely an advocate of researchgate. It is presently a small community, but everything starts somewhere. I also believe it is a perfect compromise between our two positions, as there is open discussion about projects in progress and sharing of methods and ideas. I also like following projects that are of interest and get real-time updates on current research done and in-progress (I realize this sounds like a researchgate ad...I should state I receive no compensation for this post :P). But yes, it seems like there is less disagreement than it originally appeared.

Excellent and well thought out article. Thank you. I have been looking for an alternative to journals for decades. There has to be some form of peer review and open review through comments seems to be gaining integrity. The anti-troll community is becoming recognizable. The problem of publishing scientific research lies with accountability; without accountability there is no integrity. Journal editors across the board are not accountable to anyone, let alone being transparent. I very much like your idea of a blog coupled with an open archive. ViXra comes to mind. I am sure there are others. It would be remarkably easy for an accademic institution to give your suggestions a go.

One alternative to the monopolistic journals would be grassroots scientific publishing. Single scientists or groups should be able to start a journal relatively easily nowadays. As long as this does not have a reputation and no one sends in manuscripts, these journals can start open review of existing articles and Arxiv manuscripts.

Could be seen as a mixture between a blog and a journal. The Wordpress blogging software would be a good start to get this working.

Hi - I'm writing a thesis (ha, eventually) at Cranfield on epistemology that includes an academic section, and this blog covers some excellent points that have surfaced in bits of academia but don't seem to be resolved. To be a bit rude, it's not so much that blogs are good, but that journal articles are (often) so bad. Published open data (and code), open review, wide review, clear language, etc are all things that 'should' be part of everyday academic discourse but often (usually?) aren't.

This in turn leads to a community cognitive dissonance that non-academics (such as climage skeptics above) can poke at: that academics claim scientific certainty from expertise and the gold standard of peer review, but then academics (quite rightly) point out how peer review is really not very good. The result seems to be a bolstering of peer review, rather than attacking the problem at source. I had a senior lecturer say, when I gave a list of academic-reported problems with peer review, that I was undermining the whole way that science is done. Which is worrying; we should be looking to improve the quality of scientific output, not protect a tradition.

In any case the main core problem is that we don't have decent audit, and without audit we can't actually assure quality. Whether blog or journal article, there's really no way to tell if the source data was collected, made up down the pub, generated by code, or borrowed from some other dataset. Until we can fix this, we have a big hole in scientific discourse that is only poorly plugged by replication.

In a reply above you wrote "Hi Cam, as you can see, I'm a psychologist, "To my mind, this is the single most import sentence on this web page.It sets the context for this entire discussion.

As a scientific editor (which I sort of was once, though not in a journal), I would insist on having this in the first paragraph, and repeated a few times where appropriate throughout the text. E.g. At the start of the 3rd paragraph, something like "On psychology blogs, ..."

As you say, a blog can be read by a wider audience. So imagine a non-psychologist scientist reading your blog. - You describe how journals publish, archive, and vet. These vary by field. A lot. They even vary a lot between journals in a single field.- You describe how blogs publish their data, their overall approach, and how they serve as a basis for discussion. Again, this is all extremely depend on the field. Experimental psychology is undergoing a revolution, where the old approach of using statistics as an excuse for publishing is being challenged by the new-fangled view that statistics are a tool for distinguishing what we know from what we don't. So you have a huge generational gap with the old timers controlling journals and such, and the young crowd airing their views in more modern platforms, such as blogs. That situation is unique to psychology, and has little to do with the intrinsic quality of blogs vs. journal articles. Alas, the blog does not provide this context (BTW: fact-checking myself, I now found it in the sidebar. Maybe I should read those first :-)- You mention the APA and JEPG, whose roles are clear to a psychologist. But you do not indicate whether they are a dominant or negligible force in American psychology (or somewhere in between). - Nature and Science publish articles in many fields, with criteria that differ strongly between fields so that they achieve the mix of articles they desire. At first, it reads as though your describing their practices in general. But on a second read, I think you're only describing their practices for the field of psychology.

Hi Amnon, I've added an edit ([EDIT: I'm an experimental psychologist. Mileage of what you'll read below may vary in other disciplines].) Thanks for suggesting it. This post is quite popular, with over 12.000 hits in 2 days, many from outside my normal readership. Normally, I'd assume most readers know my blog, but this is attracting a lot of new readers, so adding this context indeed seems like a very good idea. Thanks.

We're seeing a huge amount of non-traditional scholarly activity (if you'll pardon the dry phrase) happening in blogs. Alternative metrics aka altmetrics (as distinct from altmetric.com) have been taking in to consideration the scholarly activity that happens around traditional publishing for a while. Crossref, the organisation who brought you DOIs for scholarly publications, thought that it would be a good idea to help collect this kind of data as a counterpoint to traditional publishing and citations. We're building Crossref Event Data, which is a free (libre, gratis) service for collecting mentions of articles on blogs and social media, so that it can be used by the community in all kinds of ways. Discoverability, recommendations, and yes, maybe more metrics. How you use it is up to you.

Your article raises a good point about blogs as primary methods of publishing rather than, for example, as a venue for the discussion of traditionally published articles. Establishing an open 'citation'-graph-like-dataset of blogs is a good first step toward that.

We're heading into Beta soon, and you can read more about it https://www.crossref.org/services/event-data . The User guide is a work in progress, but might answer any questions you have: https://www.eventdata.crossref.org/guide . You can also contact me at jwass@crossref.org if you have any questions.

Hi Joe, I think many journal articles do a good job, but I also think some high quality blogs outperform many lower quality journals - especially on core scientific values as I specify. It would be great if these contributions can get some more credit in the current academic system. They won't replace journal articles, but they can definitely complement it.

Is there scope to separate experimentation from analysis? Currently these are activities siloed together per-study.

What if (say) a pharmaceutical company could ONLY do research by publically doing a data study, or a series of them, which all release their raw data. Then, separately, analysis is performed on some or all of said data. Could that reduce the insidious practice of only publishing results that agree with what said company wanted to show in the first place?

The claim that blogs have open data and materials is only true if you get to select which blogs count and which do not. Blogs as a medium include both this web site and any other blog set up by the Church of Scientology, climate change deniers or other groups. The diffenece between genuine scientific blogs and opinion may seem stark, but it may not always be that way.

I also think that the claim that their is no eminance filter on the Internet is a bit misleading - PageRank and the ability to drive reader traffic to a website determine eminance. Its just a different set of eminance criteria. They are softer in that anything anyone publishes could be read, but the reality is that nobody reads anything that doesn't make it into their newsfeed or the first page of a google search result.

The error correction point is interesting, but this creates complications for citing blogs as supporting evidence shifts beneath your feet. I think this only works if you also adopt some way of recording version history.

Hi Daniel,thank you so much for this article! In German speaking humanities, there are only few scientific online-only journals - peer reviewed or not -, but lots of blogs have emerged, since in 2012 de.hypotheses.org (its French 'mother', hypotheses.org, is a little bit older) was launched as a platform for academic blogs (in humanities and social sciences) in German speaking countries. Some of them provide citation suggestions for their scientific articles (like us), and since https://www.rechercheisidore.fr/ harvests them, handle identifiers are assigned to the blogposts there - and I guess, it's only a question of months until persistens identifiers will be assigned to blog articles directly (we are negotiating with DARIAH-DE). Long-term preservation remains mostly a desideratum, yet, but the German National Library - who decided to assign ISSN to academic blogs in 2013 - does already archive some blogs - although only as complete images, not as single blog posts. Cf., for instance, http://d-nb.info/1052535798

In short: I agree with you, that the advantages of blogs should inspire some scientific journals to improve their standards, but academic blogs will become an even greater challenge for them in the future.

This (your) article was published at Principia-Scientific.org. (PSI) and I would be surprised if it creates a single comment; unless it is mine. But I've been wrong before.

I agree totally with what you wrote. Especially with the critical importance of data. You wrote: "I'm an experimental psychologist." And you focused more on the fact that as a scientist you were a psychologist than the fact that as a scientist you were an experimentalist. I expect that the community of psychologists have theoretical psychologists also.

The data of your science has to be analyzed statistically because your experiments cannot be rigidly controlled like many of the observations of the physical sciences can. Hence, you tend scholarly discount the significance of your anecdotal experiences. However, the physical scientist, Albert Einstein, is said to have stated: "The only source of knowledge is experience."

It is unfortunate that you are not really familiar with 'climate'. For I, an experimentalist, consider that none of the parties involved in this well-known controversy are willing to consider 'all' the data which is available to anyone with access to the internet. They would rather argue with words instead of calling attention to several decades of quantitative data (some recorded every minute twenty-four hours a day seven days a week) that would refute many an argument.

Yes, on your blog no one can prevent you from reporting your data and its analysis and what you conclude from it. And if you want respect you must make a scholarly effort that what you write warrants the respect of the peers you respect.

PSI is a good example for this discussion. These are people who deny the greenhouse effect exists. Consequently they are not able to get their nonsense published in scientific journals and started their own PSI journal/blog.

If you are not satisfied with the standards of the current psychology journals, it may be an idea to start your own with a group of colleagues. It should be easier to start a scientific journal. Open Access should help reduce the monopoly power of the publishers and I hope the above mentioned idea of grassroots scientific publishing can help in this regard.

Victor, thank you for your comments about PSI. What you did not say about PSI is it became a site where a broad spectrum of people, scientist or non-scientist can go and find late breaking news about all kinds of science. If you know of a better website (blogsite) that does this, please tell me.

I consider a fundamental problem of science is specialization so that one group of scientists have little interest in what another group of scientists is learning and not learning. But even a greater problem is that I find (no Daniel, I have not done a statistical study) most current scientists, if they read English, have not even read significant portions of Crew and de Salvio’s translation of Galileo’s classic book or significant portions of Motte’s translation of Newton’s classic book.

A scientist needs to be curious and I wonder if either of you know what language Crew and de Salvio translated to English. Or, which language Motte translated to English. I consider sometime current ‘scientists’ could profitably ponder is: Why did Galileo ask that his book be published in the common language of the common people of Italy while Newton later published in Latin, the intellectual language at the time both books were published.

The value of blogs is people with a great variety of experience can meet each other, as we three have, and begin to have a real dialogue instead of the one which Galileo had to invent.

As an illustration of a summation of what I would like you both to consider, go to: http://principia-scientific.org/ddt-case-study-scientific-fraud/

Pathetic that the author should need to comment on the generic use of "his" pronoun in the language of 1942. I can guarantee that if, by some incredibly unlikely statistical outcome, this brilliant blog post is read by anyone 75 years from now, those readers will note some perceived anachronism that dates the author to some less enlightened period -- as if anyone can avoid adopting and expressing themselves in the common language of their day, with whatever political/social/moral assumptions then shared.

Criticizing the manners past is gutless. The question we all have to ask every day if what are WE doing NOW that is accepted today that the future will justifiably condemn.

A very interesting angle. I immediately wonder if some of the up-sides you've listed from blogs might also be down-sides (eg, there is a trade-off involved on many of these points). Blogs are correctable, but are also editable, which could be a disaster for the wider scientific record. I accept that journal articles are editable in theory as well, but in practice, blogs come with an expectation that the content could be edited, journal articles come with the expectation they will remain in the original form. Additionally who's responsibility is it to preserve the archive of a blog? The blog platform? The author? The Wayback Machine? Journal articles are preserved by the publisher.

Beyond the importance of the wider scientific record, the loss of the eminence filter - while eliminating any kind of institutional bias and democratizing science - also might have negative consequences. If I have to identify the important work outside my field, I can use the eminence filter provided by journal reputations as a proxy. I have a starting point for what is considered the current state of the field. What is my starting point in a world of blogs? Whichever blog has been upvoted the most on Reddit? The first Google search results? What if in the comments, DrBob59 insists this is the best paper on the subject representing the latest in scientific discourse, and ProfJen17 denounces the work and all its supporters as pseudoscientific charlatans? The systemic curation achieved by the journal system may not be democratic, but in a sea of noise it is useful. Perhaps I should only dive into science outside of my field if I am prepared to fully understand it enough to make up my own mind. This is hardly helpful for democratization of science in a world with so many complicated and disparate disciplines...

I don't what your metric for scientific quality is. If it is a single figure of merit metric, you can rank order science blog posts and science journals. I assert that there will be a range, mostly overlapping, range of quality. I opine that the worst blog is worse than the worst journal and that the best journal is better than the best blog, but not by much in either case.