Unsound vaccine thesis or how to review a PhD

How should PhD theses be reviewed to avoid a nonsense thesis? Especially if the work is between two disciplines.

Today a PhD thesis that might be considered an extended anti-vaccine essay was brought to my attention. The red flags start popping up from the very first sentences of the abstract. I got as far as the second sentence and honestly thought “Amateur hour?” The flaws are obvious — to anyone familiar with vaccine science.

But that looks to be the catch. This thesis is not from a Science department but from Humanities.

I’m a big fan of cross-disciplinary work.* A key element is that they really need to include solid expertise in both of the two disciplines spanned.

If you’re going to do work that builds on another specialist area, you really need to ensure that the underlying material you’re building on is sound. Was any attempt made to ensure the science in this thesis was sound? It doesn’t look so.

Fellow blogger and vaccine specialist, Helen Petousis Harris, has looked at the science of this thesis, as well as another fellow blogger, Alison Campbell. To be polite, it doesn’t come out well. I’d like to consider how this thesis has, or has not, been reviewed to raise the question of how reviews might avoid the antithesis of a thesis.

Helen calls it a ‘stealth’ thesis — a thesis done in a different division in the university under a ‘favourable’ supervisor, presumably to escape criticism.

When you read the abstract, it’s making statements about the science of vaccines not just about humanities aspects. A thesis is supposed to critically examine an issue. Critically doesn’t mean ‘be negative’, it means the student must examine in depth their hypothesis: they must critique it. This, in turn, means a thesis can realistically only be done in a department that has expertise in the topic. Humanities departments do not cover vaccine science.

So the first question is why was this being done in a humanities department? At the very least it would want to be jointly supervised with staff from a medical department familiar with vaccines.

Even then, there are (rare) opportunities for abuse, or more ordinarily, sloppiness. There are a tiny number of scientists with odd views about vaccines. (The thesis acknowledgements thank a few examples.)

Universities are given accreditation by their governments to confer particular types of degrees. With that universities are asked to hold to some (hopefully high) standards.

How might universities ensure that selection of Ph.D. advisors and reviewers avoid shoddy work being put out under their name, under their accreditation status? A few common steps are listed below.

Review of the thesis proposal

Thesis proposals are usually reviewed by the department. This should pick up things like that the thesis is cross-disciplinary in a way that need input from another department, that it looks unlikely to produce new understanding (a requirement of a PhD thesis), and so on.

It seems hard to believe this has happened (properly).

Probationary period reviews

Many modern Ph.D. theses are dependent on internal reports over the early period. For example, the University of Otago has a probationary of a year. If these fail, the student either cannot continue or is asked to bring their work up to a better standard.

A main tenant of a thesis is that the student critically examines evidence. This means they much consider all relevant aspects. For example, they cannot ignore vaccine science that does not appeal to them. On this note, an offer to help on the science was abandoned because the student would not take on board the science. This would demonstrates a failure of a aspect of being a Ph.D. student, one that I would like to think would fail a properly-done probationary review.

Ph.D. examination

Typically, these have two or more reviewers. In New Zealand three is typical: one internal examiner (from the department), on external examiner, and one international examiner.

How the examiners are chosen matters. The selection should provide sound critique of the work. The Ph.D. supervisor will, understandably, favour their student succeeding. But they must succeed on sound terms.

Some thoughts

There are several steps that should have brought in criticism that initiated action over concerns about this work. I think it’s fair to suggest University of Wollongong has a problem it needs to address in this department.

It makes me wonder if some of these steps want to include staff from outside of the department, perhaps at a faculty level with the idea the input that is more distant to the supervisor and his immediate colleagues is likely to be more partial and more able to speak out.

Perhaps independent adjudicators have their place in some steps?

Another thought was if the Ph.D. examiner’s names should be open. As a practical matter they will be known to the department. Aside from the committees, the examiners travel to the department for the examination.

These are just loose thoughts: I claim no special insight.

We should also remember that all human endeavours are prone human aspects of error, relations, unwillingness to speak up and so on. But these are things we ought to try mitigate against if we want quality.

Other fields

The broad issues I’ve touched on also apply to over work that (legitimately) span disciplines. Science communication for example wants to be build on sound science. Similarly, history and philosophy of science should not mangle the science.

They also apply to review of grants, and maybe job applications.

Footnotes

Image credit: Wellcome Library, London. (Wellcome Images http://wellcomeimages.org) The history of vaccination seen from an economic point of view: A pharmacy up for sale; an outmoded inoculist selling his premises; Jenner, to the left, pursues a skeleton with a lancet. Coloured etching, c. 1800. Copyrighted work available under Creative Commons Attribution only licence CC BY 4.0.

On an only tangentially-related note, I really wish the Taleban would stop attacking vaccine workers. Aside from the obvious upsetting tragedies, it’s one of the key things preventing polio from being eliminated world-wide. Only Afghanistan and Pakistan still have (endemic) polio.

* My own PhD is an example in a different way. Computational biology spans computer science and (molecular) biology. Today it is considered an established discipline in it’s own right, but at that time it was a fairly unusual mix.

5 Responses to “Unsound vaccine thesis or how to review a PhD”

So much more I could say. As Helen has pointed out, Orac has written well on this.
In the comments he writes,

“Certainly members of my thesis committee had to sign their names to my thesis on the front page. I believe that the published version of my thesis (which is now 22 years old) includes that page, but I don’t know for sure. I just looked at it and one of the signatures is illegible, but I know who it is because it’s my thesis advisor, whose signature was physician-like. In any case, it’s not a secret who made up my thesis committee.”

This seems a sensible idea to me.

There is also,

“I’m informed in the comments that Australian universities don’t do the traditional public thesis defense done in the US and Europe, but rather the thesis has to be read by two experts external to the University and the supervisor gets to make the call. Ugh.”

Anyone able to elaborate on this?

Some other snippets:

“Some departments (at least in physics; I don’t know about biomedical fields) have requirements that candidates demonstrate knowledge of the general field. That would help catch cases like this one before they start writing, but only if the requirement is enforced. Given that this Ph.D. was from a humanities department, I’m not sure such a requirement would have helped.”

“Well, in the US, at least in science departments, there is a preliminary examination that a student has to pass in order even to begin his thesis work. These days, frequently the test is to write a mock NIH R01 grant, which is then critiqued by faculty panels as though it were real.

Moreover, in the thesis defense, virtually anything is open to questioning, not just the candidate’s thesis work. In thesis defenses for biochemistry, for instance, on occasion candidates have been asked things like to draw out the Krebs citric acid cycle, with chirality and then show where a labeled carbon atom would end up based on where it was in the starting sugar. They can also be quizzed on recent research. True, in most thesis defenses, little of this happens. Usually the questions are pretty tightly related to the candidate’s thesis. However, the candidate knows going in that anything is fair game.”

“Another thought was if the Ph.D. examiner’s names should be open… Aside from the committees, the examiners travel to the department for the examination.”

It’s not generally the case for examiners for Australian PhD theses to visit the campus where the candidate is located. Usually (and I acknowledge that there may be other examination methods) the thesis is sent to the examiners by email. Those examiners will then provide a report from which the candidate makes revisions. As there is not usually a formal “defence” process (see disclaimer above) the amended thesis will be sent back to the examiners after which the thesis is “passed” or more revisions recommended (and so on). I’m not sure how many revisions are allowed as most of my friends who’ve been through this have only had minor or no revisions.

Aside from this clarification, thank you for your piece. This “thesis” is a travesty and a poor reflection on Australian higher education. As a hopeful PhD student in the future, it has at least informed me of where I will not be a candidate lest the stain of UoW attach itself to my field of interest.

Interesting to read how different countries do things. In most PhD examinations I know of candidates are directly interviewed by the examiners after the examiner has read their thesis. This is not the public defense, but effectively an “internal” process. Sometimes the external examiner will work on-line these days (e.g. via Skype), but it will still be “face-to-face”, so to speak.

Code for Life is the blog of Dr Grant Jacobs who has wide-ranging interests in science-related subjects, especially genetics, bioinformatics and science communication. To learn more about Code for Life (topics, copyright, comments, writing), see the introductory page Twitter: @BioinfoTools

Sciblogs Archive

Sciblogs is the biggest blog network of scientists in New Zealand, an online forum for discussion of everything from clinical health to climate change. Our Scibloggers are either practising scientists or have been writing on science-related issues for some time. They welcome your feedback!

Sciblogs was created by the Science Media Centre and is independently funded