Discussion with Steven Pinker connecting cognitive psychology research to the difficulties of writing

Following up on my discussion of Steven Pinker’s writing advice, Pinker and I had an email exchange that cleared up some issues and raised some new ones.

In particular, Pinker made a connection between the difficulty of writing and some research findings in cognitive psychology. I think this connection is really cool—I’ve been thinking and writing about writing for awhile, now, but I’e never really seen the connection to psychology research. So I wanted to share this with you.

Pinker’s remarks came at the end of an email exchange. I’ll share the earlier messages to give the background, but by far the most interesting part is what Pinker said, so I’ll give that right away.

Here’s Pinker, discussing the difficulty of communicating complex ideas (in particular, in academic writing we typically aren’t just making the case for position B, we’re also arguing why previous position A, reasonable as it might sound, is not correct):

I address it in part in chapter 5 of The Sense of Style in discussing our comprehension of negation. The human mind cannot represent a proposition without a truth value – to think “X” is to think “X is true,” at least temporarily. Negation requires an extra mental step—which can easily fail when the person is overloaded or distracted. A number of systematic kinds of error and difficulty follow. My colleague Dan Gilbert has an insightful review of this literature in his 1991 article, “How Mental Systems Believe.”

On top of that people’s comprehension is often driven more by expectations than the literal content of the text (again, particularly when not paying close attention). When I see certain misinterpretations I often think of the puzzling neurospsychological syndrome called “deep dyslexia.” Surface dyslexia consists of misreadings based on alphabetic confusions – misreading “pear” as “bear,” for example. In deep dyslexia, the patient might misread “pear” as “apple.” The puzzle is that if the patient’s word-recognition system could parse the letters well enough to realize it referred to a fruit, it must have been because he matched the input with a stored template for “pear” – so why didn’t he successfully read it as “pear”? Presumably such patients’ semantic representations (the definitions in their mental dictionary) were so degraded that the word merely activated a coarse semantic ballpark, without enough precision to pinpoint the exact entry. From there sheer base-rate frequency determines the output. It’s a crude analogy to the way we non-brain-damaged people often parse a sentence coarsely enough to remind us of a semantic neighborhood and then we fill in the rest from base rates. Often a writer will have to anticipate this and explicitly disavow an expected confusion: “You probably think I mean this, but I really mean that.”

I’ve been trying to do this more often, for example in this 2011 article on the philosophy of Bayesian statistics, where I’m pretty explicit about what I’m disagreeing with (for example, the first section of this paper is entitled, “The Standard View of the Philosophy of Statistics, and Its Malign Influence on Statistical Practice”).

Still, I think the necessity of clearing-away-the-old creates an additional degree of difficulty in much of academic writing.

What I found exciting in Pinker’s note above was his connection of this vague idea, which I arrived at by introspection, to research in cognition.

Background

And here’s how we got there.

Our conversation started with Pinker reacting to my statement that it’s not so remarkable that academics don’t in general write well; after all, writing is hard.

Pinker wrote:

The paradox is that academics might be expected to do better than laypeople in their writing, since the communication of ideas is part of their job description. So if academics are no better than laypeople at writing, that raises a puzzle. And . . . academics are not just average writers – academic prose is often, notoriously, far worse than nonprofessional writing.

Pinker also pointed out a place where I’d misread what he’d written even though he’d taken “a great deal of care in crafting the sentence” (as he put it).

This was interesting and reflected experiences I’ve had, where I try so so hard to be clear, and people still come up with a misreading of my prose.

I reflected on this and replied to Pinker as follows:

My misreading of your article raises an interesting meta-issue about writing which might interest you, as I think it’s also relevant to the puzzle of why academic writing is often so bad, despite the fact that academics typically get a lot of practice at it.

What happens to me a lot in writing, whether it’s a scholarly article, a textbook, or a popular article, is that I am aware of a possible misunderstanding, and I carefully craft my sentences to avoid the possible pitfall—but people fall into it anyway! For example, when writing about significance testing, I am careful to avoid saying that the p-value is the probability the null hypothesis is true, so I use a very careful wording, but then people read my writing as if I’d said that wrong thing. I realize that in such cases it’s not enough to say it right and avoid the error; I really need to explicitly state that I’m not saying that other thing that people are expecting to hear. (Which may be what happened in the cases where I misread you; I read a sentence on a certain topic, was expecting to hear a certain thing (“professors have a foreign policy” or whatever), and then I heard it, misprocessing the incoming words. So in my more recent writing I’ve tried harder (not always with success) to not just avoid making an error but also to making clear to the reader where the error arises.

Anyway, back to academic writing. Perhaps academics are often in this situation: trying to explain an idea. while also explaining why a certain natural-seeming interpretation is not correct. Which means that, to be most effective, we have to both convey our idea and also convey the idea that we think is wrong. I think this is an inherently difficult task (and I suspect you’ll agree on this, as I know that a lot of your writing has this feature, that you explain the appeal of the wrong model in the context of explaining your own ideas). So . . . perhaps one contribution to the ugliness of academic writing is that academic writing, to be compelling, often has to do both these tasks, and that’s not easy.

And all this is, perhaps, especially true when academics are writing about topics of general interest (which is, of course, the time that outsiders are likely to encounter our work. A nonstatistician might well read my article on the statistical crisis in science; such an outsider is not so likely to read one of my articles that’s full of math): in our general-interest articles we’re often explaining how a certain common idea is actually wrong.

It was in response to this message that Pinker sent the note given above, with the connection to psychology research.

43 Comments

“The paradox is that academics might be expected to do better than laypeople in their writing”

They used to be better. Reading English speaking Scientists and Mathematicians from about 1800-1930 is an absolute pleasure. Even before that Newton was described as the one of the finest English prose writers that ever lived despite mostly writing in Latin. A high school graduate in 1900 typically wrote about as well as the average journalist today.

The paradox is why academics regressed and are such awful writers now. Incidentally, is it a coincidence that scientists from 1800-1930 achieved far more than from 1970-2015 whether measured per person, or per dollar, or per year?

There isn’t a shred of evidence to suggest science wide “diminishing returns” to research. “Diminishing returns” is however a mighty convenient excuse for any scientist whose career overlapped with 1970-2015.

I was not arguing that there are diminishing returns. My comment was just to express my amusement about by the phrases “diminishing fruit” and “low hanging returns.”

Now that we’re on the topic, thought: In statistics there are tons of new things to discover and lots of exciting research. I think the statistics research today is much more interesting than that of 1970. We’re starting from a much more advanced point.

Just after that came QM, relativity etc. & reinvigorated the field. Maybe someone will discover in 2020 something like that & you won’t be complaining any more about underachieving, stagnated scientists.

I’m not claiming science won’t make progress. I’m claiming it’s much less likely if they keep doing things they way they’ve been doing them recently. A small part of this is the current fad for poor writing standards.

Taken from an article on reviewing the collected works of Stokes talking about the origin of the famous Naiver-Stokes equations:

“[Article Writer] In recalling his early days at Cambridge, STOKES wrote ‘I thought I would try my hand at original research; and, following a suggestion made by Mr. Hopkins while reading for my degree, I took up the subject of Hydrodynamics, then at a rather low ebb in the general reading of the place, notwithstanding that George Green, who had done such admirable work in this and other departments, was resident in the University till he died.’ The basis upon which STOKES built his great theory of internal friction seems to have been a little reading and much daring and powerful pure thought:

[Stokes] In reflecting on the principles according to which the motion of a fluid ought to be calculated when account is taken of the tangential force, and consequently the pressure not supposed the same in all directions, I was led to construct the theory explained in the first section of this paper. … I afterwards found that Poisson had written a memoir on the same subject, and on referring to it I found that he had arrived at the same equations. The method which he employed was however so different from mine that I feel justified in laying the latter before this Society. (The same equations have also been obtained by Navier in the case of an incompressible fluid…, but his principles differ from mine still more than they do from Poisson’s) The leading principles of my theory will be found in the hypotheses of Art. I, and in ARt. 3.

[Article writer] Imagine the reception some dusty editor, nincompoop processional society, or book manufacturer’s clerk would give these lines today if they were submitted by a young college teacher, twenty-four years old and with no more than three short papers to his list of publications! An objective, impersonal, scientific style must be used, not to mention our system of references! The following revision is suggested:

[Fake Revised Stokes] In the present paper, hereinafter referred to as Ref. 1, a theory of unequal-in-all-directions internal fluid pressure is derived for compressible or incompressible viscous flows. No slip viscosity initial boundary conditions are preferred to slip stick slip. Basic operational definitions and laws are given in Sec 1 and Sec 3. It is seen that similar relationships have been derived by Poisson (Ref. 2) and Navier (Ref. 3). Pointing out the hypotheticalness of the intermolecular (interatomic) for laws in Ref. 2 and Ref. 3 it is felt by the present author that the physical basis of the theory of Ref. 1 is to hopefully be preferred by the good-physics-knowing theoretician.

[Article Writer] The catastrophe that has befallen the language of science in the past hundred years is only the outer dress of the catastrophe to method and thought and taste in natural philosophy.”

[me] The authors sarcasm fall’s flat today, but Stokes original lines would be written much worse than the joke revision suggests. Where’s the acronyms?

As is my wont when the subject of scientific writing comes up, I’ll mention the great Knut Schmidt-Nielsen. When he was a journal editor he would actually edit: he would take a sentence like “Desert-dwelling mammals, though well adapted to life in arid environments, nevertheless show signs of distress if deprived of water for long periods” and change it to “Even camels get thirsty.” Much academic writing would be improved by that sort of treatment.

“Both for method and for style, this paper would be rejected by the secret referees of any society’s journal today. The language of Stokes is plain English; today’s crab-dance of German noun piles tottering among Dutch passives and impersonals with quotation marks, dashes, and parentheses to string together street slang, Latin fustian, and ad-men’s gabble was still to be invented as the ‘scientific style’.”

Regarding “even camels get thirsty,” I think one point worth emphasizing is that such clear sentences do not always come naturally. Often I find it a struggle to express myself, and, once I do manage to get my ideas out on the page, they’re a mess. Then, after the paragraph has been written, I can go back and clean it up.

And the above applies for other units of writing, including “sentence,” “chapter,” “book,” and, perhaps, “career.”

So I think it’s important for people to realize that they can, and should, be their own editors.

How’s this? (Note the suppression of any pesky mention of your predecessors directly).

The equidirectional model (EDM) for fluid pressure can be generalized through assumption relaxation to a UDM (unequal in all directions model). While previous models have derived similar equations for compressible [1] and noncompressible [2] fluids UDM (if statistically significant, ie supported by a p value less than 0.05) more parsimoniously describes the observational framework.

But the interesting comparison is not with laypeople. When compared to other similar, educated people, say, practicing but non-academic engineers, doctors, lawyers etc. are academics at par on writing skills?

I think not. My anecdotal experience says that communication in other professional but non-academic sectors is far more clear & succinct.

In the olden days, college educated officers like General Grant were famous for their writing. More recent experience has been different.

The military found at one point that college educated officers were so much worse than their enlisted high-school educated counterparts, they started making officers take effective writing remedial courses.

My experience was that they were both on average illiterate. But the enlisted personal at least knew it and write as simple as possible as a result. The end product was usually passable. The Officers would write like they were 20th century french philosophers.

Here’s a typical example. This isn’t from an off-hand unedited publication. It was a funding announcement (proposal for bids) which are taken very seriously,usually written in a joint effort by Ph.D.’s and Senior military officers an highly edited:

Historically, conflict has centered on military action between nation states, but increasingly we are faced with insurgency and terrorism that operate across a broad spectrum involving political, economic, social, and informational activities. Unable to attain victory through military action, non-state actors attack asymmetrically. In this context, traditional defense, using only military actions, can result in doing more harm than good. Effective stabilization operations require knowledge of how insurgent networks effectively influence and are influenced by dynamic cultural and social factors as well as insight into how a wide variety of actions will result either in improvement or continued degradation. Networks are the systems of people, facilities, supplies and finances that terrorists use to produce, transport or employ IEDs. To effectively attack an IED network one must understand and penetrate the strategic, operational, and tactical environments where IED operations are conducted. This can be difficult because the adversary makes use of individuals who blend into a complex, often urbanized environment. In this environment the enemy is virtually undetectable and can remain hidden for extended periods of time unless we can force him to move, shoot, or communicate in reaction to us. To do this, our Concepts of Operations must have the coordinated ability to reduce the impact and influence of an adversarial network simultaneously at several small points or nodes or at least force them to maneuver into a presumed safe haven. Such unplanned migrations reduce adversary options and force them to interact among themselves, further exposing their networks.

I picked this example at random. You can find millions of others just like it or worse. This entire paragraph says nothing more or less than

“To defeat insurgents it helps to understand how they operate”.

That’s it. Nothing more. A person coming out of high school would have written just that and moved on.

Rahul, he lives about 100ft from a world class beach, bathed in perfect weather, which he mostly has to himself (and his kids), surfs every other day, and never ever comes into personal contact with academics, former academics, academic wannabees, statisticians, or anyone else who’s profession has the word ‘science’ in it.

Rahul: Your many posts here tend to be highly concise, clear and cogent, so you really don’t deserve this, but I could not resist.

I think you have answered your own question. Isn’t it in college that some people learn to write “cet[e]ris paribus”? What does it add to the rest of your statement? “Other things being equal, I find that hard to believe.” Why the qualification? What does it mean here? I can translate from Latin, but your last sentence says it all.

Well, I’m not ragging on Rahul’s English let alone his Latin, since both are better than mine (I failed or nearly failed just about every English or foreign language class I ever took). Maybe Rahul had an old school education and did improve through college, that’s why he has trouble imagining the monkey piles of poop thrown on the rest of us which resulted in retrogrades not improvements.

It’s a slow painful path to realizing that people 200 years ago were doing some key things substantially better (writing styles, research styles, non-frequentist statistics) and very predictably, were getting far better results for the amount of time/money/effort put into them.

I guess the thing that gets me, is that every academic I’ve ever met has strong, basically unchallengeable, opinions about the “scientific method”. These opinions include big things like philosophy and the foundations of statistics, all the way down to rules of thumb for Latex equation formatting.

They believe these things with such strong convictions that even funerals aren’t enough to eradicate any errors they might have. The next generation gets indoctrinated with the same blather.

And yet there’s almost no evidence for any of their opinions. Just the opposite in fact. Even a casual first hand look at the last 200-400 years provides ample evidence that there must serious flaws with their world-views. History may not teach what the problems are exactly, but it clearly shows major discrepancies exist.

It’s obvious that physics, economics, psychology, poli-sci and many others haven’t earned their keep in quite a long time. Certainly not in my lifetime. Yet every one of their practitioners is adamant that they know how science is done. They just know it. They will explain it to you at length and then go back to pissing taxpayers money down the drain writing the next god awful research crap in the hopes that the moron down the hall will think them “special” and will congratulate them for their latest publication.

Uh, sure. Wait one minute and I’ll drum up a p-value less than .05, n=100 coeds who happen to be enrolled in psych 101 this semester, article to ‘prove’ it using essentially ambiguous notions of ‘better’ and ‘worse’ writing.

After all this time at this blog Rahul have you learned nothing about the limits of studies? Blind prejudice may be wrong 50% of the time, but statistical studies are wrong 80%+ of the time.

Any study you could reasonably perform, if done right, will give you the answer “we don’t know”. That happens a lot.

“So justify it.”

I did. The military repeated found it necessary to give Officers specifically remedial training in effective writing and it wasn’t some mystery why.

” I decide based on some n=1,2,3 anecdotes “

Yes. Anecdotes or even rank biases while a poor way to do science are better than statistical studies.

How much statistics did Darwin use? None.

How much statistics were used to the create that carboyhdrate-protein-fat food pyramid which was taught to school kids for half a century and now appears to be completely, life destroyingly, wrong? Substantial.

But that was a long time ago. I’m not sure many politicians (let alone physicians, for example) put much effort into communication. And many writers in magazines and newspapers seem to do a very poor job.

One hypothesis: Academics are especially bad writers when they write journal articles. But not relatively bad writers compared to non-academics when they write books. Comments?

If so, maybe the format & motivation to write a journal article is what we ought to be scrutinizing.

My impression is that a fair bit of journal articles get written expressly to bag PhD’s, pad Resumes, impress hiring committees, cement tenure or convince funding agencies that their monies weren’t frittered away. In most of these uses, no one’s actually *reading* anything.

Effective, succinct communication is nowhere near the top of that list.

I’m told that people who watch lots of television dramas have a very mistaken overestimate of the prevalence of violent crime in the real world. Similarly, I suspect that people whose main exposure to science, and ethical issues in science, is via blogs and critiques get a very mistaken impression of what conducting science is like. It’s simply not the case that “a fair bit of journal articles get written expressly to bag PhD’s… [etc.]” or that (implicitly) the authors don’t care if their papers are read. Believe it or not, most people who work hard, for years, on experiments or analyses actually do want their work to have some sort of impact. This doesn’t necessarily mean that they’ll write the papers well, but that doesn’t mean that they’re cynics or slackers.

“But not relatively bad writers compared to non-academics when they write books.”

Not borne out by my experience. Who else writes books? Mostly journalists. (Leave aside autobiographers/memoirists who are famous for some other reason.) I believe journalists’ books tend toward higher readability/lower reliability (they can’t resist telling you the juicy but possibly apocryphal anecdote, or the so-crazy-it-might-just-be-right theory), while academic books tend in the opposite direction.

I find the prose in nonfiction books by academics typically less good than the prose in books by non-academics. The books can be good or bad for other reasons. Gladwell writes beautifully but his books make you dumber IMHO.

For me, as a scientist who puts a lot of care and effort into scientific writing, this thread is even more depressing than I thought it would be. Apparently it’s open season for gratuitous abuse of any scientific endeavour in the last few decades. No-one has even taken the trouble to find an example of bad current scientific writing in a place where it could and should obviously be better. Straw men and anecdotes do not add up to evidence.