The key is third party. Clinton is only having a lead of 1.7%, but the third party candidates (yes, there are more than two candidates in US presidential election – look at the picture below) have a vote share around 4% (http://fivethirtyeight.com/liv… ). If you want a popular vote system, you will likely to have elections in several rounds, like French presidential election. To explain this more clearly, let’s consider a more simple version: 2012 Olympics host city election (https://en.wikipedia.org/wiki/…):

Note in round 2 London was behind Madrid but once New York was removed London was top again. So it is essential that in a system where the outcome was decided by popular votes, several rounds are required to ensure the final selected one has genuine majority. Back to this US election, of course, it is hard to know how the third party voters would vote if the they can only choose between the two top candidates in an imaginary final round of voting.

Then there is another issue. A candidate can only choose their strategy based on the election rules. If the rule was popular vote, their strategy may be totally different. They may focus on their big stronghold states instead of smaller “swing states”. Clinton never visited Wisconsin during campaign, which costs her dearly. Trump know the election rules well (as well as tax rules… ) and understand where to spend his time, energy, and money, which ultimately lead to victory.

Like this:

Considering that the European Union (EU) make 13-65% of UK laws (depends on how you look at it), it is important to examine the degree of democracy of EU institutions for their legitimacy.

Some may correctly point out that, for instance, European Parliament whose members were elected, has significant power (though may not be as comparably powerful as a national parliament at national level). However, as we learnt from the history, there are no shortage of examples of democracy only on paper.

Obviously, the European Parliament is nothing like NPC. In fact, the problem is totally opposite: every party is a minority. For example, in 2014 European election in the UK, UKIP had taken 24 MEPs, making UKIP the largest among all UK political parties in this election. However, there were 751 MEPs in total in the European Parliament. This means a mere 3% representation, despite UK being the second largest economy in the EU, or around 17% of EU’s GDP; or having the 3rd larges population within the EU. With such a low representation, UKIP has no chance to effectively form/influence a governing body either on its own or through coalition (European Commissioners are appointed anyway). Now compare to the UK parliament which can form a government body by one or, less commonly two parties (coalition), when they have the majority of seats. While there were Scottish “minority” government, the ruling party still had large share of seats (e.g. the 3rd Scottish parliament). Everyone being a small minority means that there is no one who is accountable, and conversely no opposition to hold whoever to account. No wonder a shopkeeper had such a experience he shared in a TV debate: he asked every customer walking into his shop who their local MEP is, and not surprisingly no one had a clue.

As a consequence, this causes some sort of “fears” in England. Labour leader Ed Miliband had since been busy trying to comfort the voters that he would not do any deal with SNP, be it formal coalition nor “confidence and supply”. This had been a major question people asked at the televised leaders’ debates, especially the BBC Question Time special on 30 April 2015. The impact was probably so bad that, Nicola Sturgeon, the SNP leader has to clarify that she “love England”, and “I hope nobody in England is afraid of the SNP – there is absolutely no need to be” (http://www.independent.co.uk/news/uk/politics/generalelection/nicola-sturgeon-says-there-isnt-an-antienglish-bone-in-her-body-10228042.html).

Ironically, the rise of SNP did not give what Scottish people want. I guess most Scottish people want a left-wing party but yet a Conservative majority emerged, which could be explained by Labour’s loss both in Scotland and England.

Like this:

There are many ways to judge a statistical software, e.g. user interface, number of different statistical models it can estimate, speed, large data handling capabilities, data manipulation, complexity of scripts, graph qualities, and documentation.

This post talks about documentation. I have come crossed a number of statistical packages, Stata impressed me the most especially in terms of documentation. Its documentation is so good that you can literally use it as a text book to easily learn a new modelling routine, the history of statistics (for example how negative binomial models derives). It has many examples that can walk you through a statistical model which you are unfamiliar with, making it a pleasant learning experience.

The documentation of other packages, such as R, SAS, SPSS, I found are rather simple, with little explanation and examples, difficult to navigate through, and sometimes confusing. Recently I am exploring the weighted regression so I want to know what types of weights are available and how they differ from each other. I found this useful article online: What types of weights do SAS, Stata and SPSS support? There are four types of weights, while Stata supports all four types of weights and you would know which one to be executed by the software (actually you have to explicitly specify which weight type you want to use in your command), SPSS only supports frequency weights which is fine provided you do actually know this fact when you use “weight by” in your command (though it is not apparent from the syntax). Since SPSS only supports one type of weight, there is no confusion.

Now SAS, it is said that:

“You need to read the documentation for the proc that you are using to determine what kind of weight will be used with the weight statement. The weight statement used in one proc might assume frequency weights while another assumes probability weights. Some procs will handle the weights differently depending on the values of the weight variable. For example, if all of the values of the weight variable are integers (whole numbers), SAS will assume that you have a frequency weight. If you specify a different weight variable that has decimals, then the proc will assume that you have a probability weight. If you cannot tell from the documentation which type of weight will be used, you will either need to do some experimenting or contact SAS technical support.”

What? You have to check the type of weight case-by-case depending on the proc (SAS procedure)? This is fine but then even so there are probabilities that you would not know how a specific proc handles your weight variable. And sometimes you have to do experiments by your own or call the SAS technical support?

Like this:

I was asked several statistics related questions, which I think quite bizarre. The following is my take/reflections on these questions.

Unlike popular belief, an ordinary linear regression model (i.e. using ordinary least squares, or OLS method) does NOT require the assumption of normality (http://www.ats.ucla.edu/stat/stata/webbooks/reg/chapter2/statareg2.htm). Neither the dependent or independent variables are required to be normal. For residuals “normality is necessary only for hypothesis tests to be valid”.

With this in mind, you cannot ask what is the difference between an ordinary linear and logistic regression models on the basis that one requires normality the other does not, which is a false claim. The main difference between the two is one is for continuous dependent variables; and the other is for binary categorical dependent variables.

Now logistic regression. Sure it can be viewed as a special case of a generalised linear model (GLM) (so as ordinary linear model), however GLM is just one of many formulations. Logistic regression model can also be derived as a latent variable model (http://en.wikipedia.org/wiki/Logistic_regression#Formal_mathematical_specification). So if someone asks you “what do ordinary linear and logistic models have in common”, s/he needs to be specific of what s/he means (i.e. in terms of what?). One would never conclude/guess that “they are both GLM” if the person questioned thinks of a logistic model as a latent variable model.

Finally, how to assess whether a model is a good or bad model. Let’s be clear: the most important thing is model inference (statistics is all about making sense of data). Some are more interested in prediction. Fine. Then measures of goodness-of-fit should be used, such as R-squared, AIC, BIC, MAD etc. Some then suggested cross-validation. Let me say this: cross-validation is generally not a good idea in statistics and in many cases wrong. When you can “cross-validate”, it usually means that you have more data on hand. If you have more data, why not build your model based on the WHOLE sample rather than estimate a model using a crippled subsample? Even non-statisticians would understand that the more data, the better the model is. If you use a subsample, you would get a worse model, which would be less useful. As stated by Antonakis and Dietz (2011), “Nobel prizes have been earned in econometrics for methods to correct for truncated samples, among other contributions (e.g., Heckman, 1979; Tobin, 1958). In short, researchers must avoid or correct sample bias instead of creating it.” Cross-validation might be justifiable if your computer is so slow (for a given model) using the whole sample, so you may want to build a model based on a subsample and test the other part of the data. However, if we are talking about and committed to the “big data” analysis, this would not be an excuse and as such there is even less reason to conduct cross-validation.

It came to my mind that what you ask could actually reflect what level of your knowledge is. For example, if one only knows a logistic model is a GLM, s/he would just design a question as confused as above, leaving others wondering what the heck?!

Like this:

Having recently read this excellent book by Emanuel Derman, I would highly recommend it, especially those PhDs and of course, “financial engineers”.

The below are some of the highlights I liked and you may find them interesting. The best, of course, is to read the book itself if you have time. If you are just interested in his academic experience but not so into financial modelling, the first six chapters would be adequate.

Highlights:

We cannot simply look at the world around us and deduce Newton’s Laws or Maxwell’s equations. Data on its own does not speak… His method wasn’t based on observation or empiricism.

Part of the reason for the influx of physicists to other fields was the collapse of their traditional job market: academia.

Quants do not. Like academics trained in research, they prefer to do one thing from beginning to end, deeply and well.

“If you decide you don’t have to get A’s, you can learn an enormous amount in college.”

Finally, fiercely brightest among all the stars in the Columbia firmament was Tsung-Dao Lee, the embodiment and perhaps even the cause of all the good and bad qualities of the department.

In the end, character and chance counted at least as much as talent.

“More is Different!”

At age 16 or 17, I had wanted to be another Einstein; at 21, I would have been happy to be another Feynman; at 24, a future T D. Lee would have sufficed. By 1976, sharing an office with other postdoctoral researchers at Oxford, I realized that I had reached the point where I merely envied the postdoc in the office next door because he had been invited to give a seminar in France.

If you didn’t mind wasting the best years of your youth, graduate student life at Columbia was paradise.

Everything looks simple once you have been taught it.

It was disappointing to learn that even the Nobel Prize and almost eternal fame were not enough to overcome vanity and competition.

It took me seven years in graduate school to get my PhD, an astonishing ten percent of a lifetime.

Most of us grew to hate our stay in the physics department.

When trying to discover something new in any field, one has to spend many years thinking, making false starts, wandering down blind alleys and stumbling into ditches, only to emerge again and keep going. For this, a PhD is a good, if painful, training.

“This coercion had such a deterring effect [upon me] that, after I had passed the final examination, I found the consideration of any scientific problems distasteful to me for an entire year.”

I had imagined postdoctoral life as a sort of priesthood, the blissful apotheosis of a life dedicated to knowledge.

As we got older, we took solace in the stories of people who made great discoveries after the age of thirty.

Postdoc life was an atavism, a relic of a time long past…. Mitchell Feigenbaum, famous for his contributions to chaos theory, described it aptly: “These two-year positions made serious work almost impossible. After one year you had to start worrying about where you could go next”

“publish or perish,”

As an academic, you could work (or not work) wherever you liked. It was freedom, but now, with one year as a postdoc gone and the future looming, it sometimes felt like the freedom to fail.

Such small freedoms, together with the long vacations spent doing physics in interesting places, gave one the sensation of being nonmonetarily rich, and compensated for the low noncorporate salaries.

I was 33 years old and halfway through my third postdoc; where was this peregrination going to end? I concluded I either had to find a position as an assistant professor with a good chance of tenure, or else get out of physics.

If you aren’t Feynman, you’re no one.

My solitary personal life amplified the lonely activities of a theoretical physicist and an academic.

Until I experienced it, I wouldn’t have believed that an investment bank could be more collegial than a college.

“A truth that’s told with bad intent beats all the lies you can invent”

If they expelled me from the monastery, I didn’t intend to worship God in the world. I would rather quit religion forever.

Everything is interesting when you examine it closely enough to be able to reconcile its quality and its quantity; every field is fascinating when you have sufficient familiarity with its nuances and begin to try to bridge the gap between its form and its implementation.

“You can do what you want, but you cannot want what you want,”

In my previous life in physics, talent and skill were everything-you felt sorry for people who ceased creating in order to become administrators. But at the Labs, talent seemed to be a commodity, fodder for managers to buy and redistribute. Supervisors were actually forbidden from doing “technical work” on the grounds that competing with their employees in this way was demoralizing.

It was around this time, seeing Ed write and design code, that I realized how many physicists misperceived the nature of jobs and careers in the nonacademic world. Physicists tended to think they were so smart that, once they descended to a job in the “outside world,” their talent would allow them to work in a 9-to-5 mode and still outperform their colleagues. But, in any nonacademic job, there are people for whom that particular work is not a compromise but a passion and dedication, taken seriously. They, rather than the smart but coasting physicists, set the standards of excellence.

the willingness to do so is an essential part of graduate student and postdoc subculture… All of Stan’s hires came from a culture in which you did your own dirty work – you developed your own theory, did your own mathematics, and then wrote your own programs.

Most quants, then and now, came from abroad because immigrants often see the quickest path to success in hands-on work. It’s the next generation that prefers management and business school.

It was a shock to realize that people whose great experience and knowledge straddled both the quantitative and the trading worlds had, despite their sophistication, brought themselves into such a catastrophic state.

Everyone was so determined to not have his or her own time wasted that they collectively wasted everyone else’s.

Most profitable options strategies I have seen have had the same formula: Buy some simple, less attractive product wholesale, use financial engineering to transform it into something more appealing, and then sell it retail.

Calibration is absolutely critical.

So much of financial modeling is an exercise of the imagination.

I believe that you can summarize the essence of quantitative finance on one leg, too: “If you want to know the value of a security, use the price of another security that’s as similar to it as possible. All the rest is modeling. Go and build.

Financial economists grandiosely refer to this law as the law of one price, which states that securities with identical future payouts, no matter how the future turns out, should have identical current prices.

Many finance academics who should know better also seem to imagine it can be done, but they don’t live in the real world.

People learn from past mistakes and go on to make new ones.

In physics you’re playing against God, and He doesn’t change his laws very often. When you’ve checkmated Him, He’ll concede. In finance, you’re playing against God’s creatures, agents who value assets based on their ephemeral opinions. They don’t know when they’ve lost, so they keep trying.

Catastrophes strike when people allow theories to take on a life of their own and hubris evolves into idolatry.

With two major golfing wins the Northern Ireland star is hot property – but his new wealth won’t change him, according to friends.

and some text from an excellent book “My Life as a Quant” (by Emanuel Derman)

… when I arrived there, and very quickly realized that you got no respect in Area 90 unless you were a manager. In my previous life in physics, talent and skills were everything – you felt sorry for people who ceased creating in order to become administrators. But at the Labs [note: Bell Labs], talent seemed to be a commodity, fodder for managers to buy and redistribute. Supervisors were actually forbidden from doing “technical work” on the grounds that competing with their employees in this ways was demoralizing. Instead, managers became experts at intracorporate maneuvering.

So for the first piece of news, Rory repeatedly highlighted that he was not for money. Ok come on. Do I consider he is worth that? I am not sure. As my previous article (Another look at the PhD and scientist/academic as a career) discussed, compared to a Nobel-winning scientist, this does seems excessive; however compared to other professions such as the RBS CEO then this should be just ok. Maybe it’s just the reality of the academia.

But there is one thing that I do like about sports industry: talent and skills are everything you need. You played golf or tennis very well? Ok then good, continue to play, do use your talent and enhance your skills, and one day you will be rewarded well.

The same story can not be said in other places. Note the second texts I quoted above. Academia, like sports, used to be one of few places that you can make full use of your talent and skills. However, the glory days has been long in the past. As I discussed before (Another look at the PhD and scientist/academic as a career), the higher the ladder you climb, the less the front-line research you would be doing. You will eventually become a managers and/or sales person. You would spend more time on writing proposals, teaching, and managing students/postdocs. Now here is the interesting thing: during PhD and postdocs, you were trained to do research (mostly by yourself); but once you have some outputs (e.g. publications) and/or stayed long enough, you would be asked to be a ‘manager’ whose required skill set are quite irrelevant to what you had been doing before. Luckily they are not stupid (nor extraordinarily smart) so they can handle the change. But it seems a waste of talent. They do teaching as well, of course; however, with no formal/serious teaching qualifications, has been doing all research (again, most of time alone) prior becoming a ‘lecturer’, one has to wonder whether they can transfer to good teachers. Recently there has been some debate about young researcher going to be a high school teacher in China (http://blog.sciencenet.cn/home.php?mod=space&uid=660333&do=blog&id=634268). One thing the supervisor of this young man against him going to high school is that teachers there should have trainings in teaching, psychology in dealing with pupils etc. I understand that in order to teach in primary/high schools in the UK, you have to get a teaching qualification. However this is not the case in universities. You can do years of research and suddenly become a ‘lecturer’, with little or no teaching experience at all, let alone teaching qualifications. A visitor professor from the US once told me he has witnessed loads of ‘bad teachers’, but you know what, it doesn’t matter, as long as you get good publication record (ok, it’s not an excuse for falling asleep in university classrooms during lectures :)). It doesn’t take a PhD to realise that a person good at research is not necessarily good at teaching (think Sheldon Cooper!). (of course there are many other problems in teaching in academia. For one thing, how would you expect someone with no industry experience would help students find a job in industry?)

In any respect, a system fails if one cannot survive with or fully make use of their talent. If you do something really well in a given area, you are just a ‘technician’, doing some sort of ‘technical works’. Then you would want to be promoted (or they would promote you), becoming a ‘manager’ (or other types of staff) and pretty much abandon the ‘technical skills’ you learned or used before (of course you have to show that you can manage as well). Someone would say, ‘intracorporate maneuvering’ skill, or in a prettier word ‘management skill’, is also a sort of skill. True. But this skill does not have to be superior than other ‘technical’ skills. In fact, it is supposed to serve, rather than triumph over other skills that are usually essential in an organisation. Sadly, the reality is that people are just getting used to this trend, and they seem to think this is perfectly normal and just the way we should go on. This is when the nonsense becomes common sense.

Here science refers to more general meaning including social science. So when I say scientist it also includes those researchers doing engineering degree/subject. PhD refers to research doctoral degrees, not professional doctoral degrees such as doctor of medicine (MD) or Juris Doctor (JD).

PhD and how academia works

According to Wikipedia, ‘PhD’ was not invented until 1861 and only imported to the UK in 1917. Previously you did not need this thing to start an academic career, which made you wonder why one needs a PhD at all in academia: Archimedes didn’t have one; Isaac Newton didn’t have one; a current professor and also director and former head of the transport studies group (CTS) at Imperial College London doesn’t have one either. Andre Geim, the Nobel Prize winner in 2010, labelled the PhD work as going unnecessary depth and “so boring that I decided that I did not want to end up doing this for the rest of my life” (Renaissance scientist with fund of ideas).

This leads one to suspect whether PhD is really a necessary qualification or is it just a way for the professors (including ‘lecturers’) to use them as cheap labours. In the UK, PhD students typically receive £11k – £18k (depending where you are and which university you go) a year (tax free), which is far much lower than a full professor and even a lecturer. Some PhDs, mainly those from outside EU, are even self-funded. As this article points out, “universities have discovered that PhD students are cheap, highly motivated and disposable labour. With more PhD students they can do more research, and in some countries more teaching, with less money”; and clearly it is very often that “the interests of universities and tenured academics are misaligned with those of PhD students”.

This echoes what I have seen in universities. To see this clearly, we need to think about this question: who are the people actually doing research in universities now? The respected professors? Lecturers? No. The answer is PhDs and postdocs. With a few exceptions, permanent academics usually don’t do much research. One of my PhD supervisors once told me learn as much as you can while as postdocs as there would not time for me once as a lecturer. An academic is too busy with writing funding proposals. One source indicates that a faculty member usually spent 1/3 of the time to write proposals. Remember they also need another 1/3 of the time to teach (as told by my supervisor); and another portion of the time to manage the funded projects or network at various conferences – no wonder academics don’t have time to do research. This leaves PhDs and postdocs as the main labour (if not only) force doing research in universities. Yes they are the true researchers and scientists. The professors and lecturers? I would rather call them managers and/or sales person – they don’t do (much) research, but manage others (i.e. PhDs & postdocs) to do research for them. Now this is a huge difference.

So why do you want to do a PhD?

… in the first place? I think most people choose to do a PhD mainly for their love of science. For fame? Try to name three people who won Nobel Prizes in science in 2011. Money? Even as full professor what you earn would look rather pathetic compared to other professionals. In the UK a full professor could earn around £60-70k; in the US “the average pay of full professors in America was $109,000 in 2009” (Doctoral degrees The disposable academic). On the other hand, a top banker could easily earn £1 million basic salary PLUS another £1 million bonus (http://www.bbc.co.uk/news/uk-politics-16752358). In academia, the only way to achieve that level is to win a Nobel Prize every year. No, this is not a joke. Want some serious data? Let’s look at the salary comparison between PhDs and Non-PhDs (see below), you could easily realise that PhDs are worse off in Europe and US. Artificially PhDs still seem to earn more, but this is the comparison of 6-10 years after achieving “highest qualification”. Since PhD itself can take anywhere between 3-10 years (or even more, see the following paragraph), this means that those “non-PhDs” would have extra 3-10 years of professional experience compared to PhDs at the same life stage. Obviously non-PhDs are likely to earn more.

So I appreciate the passion for science. However, you are also a human being and you also have a life. “Science is a profession, not a religious vocation, and does not justify an oath of poverty or celibacy.” (Don’t Become a Scientist!). As one source put, “It is incomprehensible that you spend 10 years of your life educating yourself and then you are earning the same amount as a bus driver”. What even worse is the timing. In the UK you would spent 3 – 5 years after your first degree when you finish your PhD (some even spent 8 years for PhD alone). In the US, “half of all science and engineering PhD recipients graduating in 2007 had spent over seven years working on their degrees, and more than one-third of candidates never finish at all.” (The PhD factory). What this means to you is that after finishing your PhD you are mostly likely to be around 30 years old. At that age, you may have a partner that you want to marry to; you would need “a house in a good school district and all the other necessities of ordinary middle class life” (Don’t Become a Scientist!); you want your children to receive best education such as piano lessons with private tutors as well as sending them to expensive private schools. However you will soon discover that you would fall into that “postdoc trap” (discussed below) that paid poorly and without any job security. You need to prepare to move every couple of years to find a new position once your current contract ends. You may not want to buy a house because you don’t know where you are going in the following year. You have little hope to persuade your partner to move with you to another country (or state/city, well unless your partner is out of work anyway or willing to sacrifice). If you are a women, you may also have to delay having children (even though it’s a good age to have one) or without a proper family for years (Why women leave academia and why universities should be worried).

You, and your family, making all these sacrifices, hoping one day you could climb to the top of the pyramid of academia, with passion for science, will only find out that you eventually become a “manager” or a “sales person”. Is that really what you want (i.e. to be a scientist) in the first place? Sadly it is often the case that “having achieved the promised land, you find that it is not what you wanted after all.” (Don’t Become a Scientist!).

Yes again, I appreciate one’s love of science. If that is the case why not learn something that appeals to you by yourself. That is what you did for your PhD in most cases after all!

The postdoc trap

So assume you are ok with being a science “manager” or “sales person”, it still takes ridiculously long process to secure a permanent faculty position, making it far less attractive than normal ‘managers’ in a company. Of course I know that there are people who are very talented and/or lucky to get one soon after finishing PhD. The fate of majority of PhDs are however not so promising . They often need to do multiple postdoc posts for many years.

Take Andre Geim, the Nobel Prize winner mentioned above, for example: he did several postdocs and applied for the first permanent academic position only around the age of 35. A senior lecturer at University of Manchester stated “In my case, I had temporary contracts for seven years before obtaining my first permanent job as a lecturer, which is not unusual in my field.” One source said one needs “about six years” to obtain a tenure-track position (which just means one have a chance to get a ‘tenured’ position after another lengthy process). It is also reported that “In some areas five years as a postdoc is now a prerequisite for landing a secure full-time job.” From my personal experience, I have seen one doing postdoc for 5 years before finding a job in some second class university far from home in a quite remote place. Another postdoc in my office have been doing 5 years already without securing any tenured (or tenure-track) faculty job yet. I also personally knows someone who did postdoc for 10 years before he eventually gave up and went to be a high school teacher. You want some proper/reliable statistics? How about this: “In 1973, 55% of US doctorates in the biological sciences secured tenure-track positions within six years of completing their PhDs, and only 2% were in a postdoc or other untenured academic position. By 2006, only 15% were in tenured positions six years after graduating, with 18% untenured”. Also remember tenure-track doesn’t guarantee tenure (http://chronicle.com/article/Reactions-Is-Tenure-a-Matter/64321/). I heard MIT has a failure rate of 50%.

Remember spending more time as postdoc does not necessarily means you will get an academic job in the end: see for example “The NSF estimates that only 26 per cent of recent PhD recipients in the US will secure a tenure-track position. UK postdocs appear to have even more reason for pessimism: according to the Royal Society’s 2010 report The Scientific Century: Securing our Future Prosperity, 30 per cent of science PhD graduates go on to postdoctoral positions, but only about 12 per cent of those attain permanent research jobs.”

Obviously this issue is mainly caused by the over supply of PhD students (see The PhD factory). When in golden age during which the universities were expanding, it was much easier to find a faculty job. I routinely heard that, at least in my university, someone started a lecturer job before finishing his/her PhD (like 10-20 years ago). My PhD supervisor once told me that, there was only two PhDs at a research centre in Imperial College London back in 2001. Now there are around 50. Has the number of faculty positions increased by 25 times during this time period? And what are those PhDs going to do in the end?

As discussed above, when you finish your PhD, you are likely to be 30s; and then after all these series of postdocs you will be around 35-40 years old. Or that is 12-17 years later than your mates who left university after undergraduate education and now may be a manager in a Forbes 500 company living in luxury. If you happen to live in a US or US-style university also be prepared to spent another 6 or so hard-working years to get actually tenured. Good luck with that!

This is not surprising after all. Suppose an academic produces average 10 PhDs in his/her lifespan, unless the higher education expands significantly (bad news though: http://www.bbc.co.uk/news/education-18768857), there will be only one vacancy after he/she leaves. This means a failure rate of 90%. Simple maths. The PhDs are supposed to be smart and they should have known this.

To understand why such a miserable thing happened one needs to look at the history: “In the ’60s and early ’70s many profs were hired fresh from grad school. Postdoctoral ‘training’ was unnecessary. Were PhDs so much better then?” and “if postdoctoral ‘training’ was once unnecessary, why have it now? As the golden age ended, research jobs became scarce. So postdoctoral positions were created as a holding pattern for graduates. Circling like vultures, postdocs waited for older professors to die off. As fewer profs were replaced the stack grew. Postdoctoral ‘training’ became the norm.”. See? we are back to the situation described in the previous section. Postdoc – even the name was strange: is there a post master, post lecturer or post professor? It is clear that postdoc was created to feed the need of academic ‘managers’ who don’t have time to do research themselves. After PhD, instead of giving you a real job, the university/profs hires you as a postdoc as a cheap and disposable labour, leaving you on hopeless rolling fixed terms. One has to ask is it even legal?

So how about going to industry?

Ok, academia is difficult to squeeze into. We get it. Then how about going to industry? This sounds a brilliant idea. After graduation (at any degree), you may be able to enter a “graduate scheme” which in most cases means you’ll get a permanent job and if you perform well you will get promotion. On the contrary, in academia there is no such thing called “graduate scheme” after PhD and you are expected to do many years of postdoc (see the section above). Also industry seems to pay more than academia: the following figure compares the life time income between the two sectors:

Clearly staying out of academia seems good at least in terms of money. Yes there are some good things about academia, such as small chance to be sacked (only if you have tenure which could take ages) and flexible work (but on the other hand you don’t have benefits in the industry such as bonus, free Premier League tickets. Just saying…).

For PhDs, these all sound good, until you realise it is difficult to find a suitable job in industry. Officially, yes, majority of PhDs go to industry eventually (remember the 90% failure rate I mentioned above?). A survey for the Royal Society of Chemistry also suggests that “88% of the women don’t even want academic careers, nor do 79% of the men”. However finding a suitable job is not easy for PhD students. As this article noted: “The organisations that pay for research have realised that many PhDs find it tough to transfer their skills into the job market. Writing lab reports, giving academic presentations and conducting six-month literature reviews can be surprisingly unhelpful in a world where technical knowledge has to be assimilated quickly and presented simply to a wide audience.”

Indeed. About a year ago, our research project holds a seminar inviting people from academia and industry to contribute and exchanging ideas (i.e. “dissemination”). One person from the consultancy industry asked how long our project is. We said three years. The person said the academia had the luxury of time, as his projects were usually measured in months and projects in academia were measured in years. I have no idea what he would think if he knew that just literature review alone could take 6 months in academia! This is a classic example. Do employers really value the skills you learnt from PhD?

Another problem is that, “PhD courses are so specialised that university careers offices struggle to assist graduates looking for jobs, and supervisors tend to have little interest in students who are leaving academia.” I have to admit that as well: PhD is deep but very narrow. The Royal Statistical Society stated clearly that because of this very reason PhD can only count one year of experience when applying for the Chartered Statistician (normally it requires five years’ professional experience).

Worse still: “the skills learned in the course of a PhD can be readily acquired through much shorter courses. Thirty years ago, he says, Wall Street firms realised that some physicists could work out differential equations and recruited them to become ‘quants’, analysts and traders. Today several short courses offer the advanced maths useful for finance. ‘A PhD physicist with one course on differential equations is not competitive,’ says Dr Schwartz.” and ironically “a PhD may offer no financial benefit over a master’s degree. It can even reduce earnings”.

The fundamental problem is that PhDs/postdocs are designed for academia. This was deeply fitted into the DNA of these types of works. PhD/postdocs and industry do not fit by design. They are like two worlds, and they don’t really understand each other. My supervisor once said “academia views industry as knowing nothing about theory; and industry views academia as knowing nothing practical”. Go figure.

And this leads to another problem: “The longer you spend in science the harder you will find it to leave, and the less attractive you will be to prospective employers in other fields.” (Don’t Become a Scientist!). Or in other words “If someone is in the job market a long time … they haven’t met the standards elsewhere.” (The Postdoc Trap). Your experience as PhD and/or postdoc does not count “work experience” for industry. After spent 5-10 years in academic research you found you may still need to apply as a “graduate”. This is truly the lost ten years.

“Bad money drives out good”

So why universities and policy makers should care about all these issues? Sure, there may be an over supply of PhD students. But why not just let them compete for the few academic jobs and wait to harvest the best and smartest?

This kind of social Darwinism may be valid, only if there are no other choices. Competition doesn’t necessarily produce the best. Smart people could see this problem in a early stage and move to more promising sectors. I even suspect that the smartest are within banking and financial sectors in the current situation. Those who find difficult to get a job in industry may just cling to the academic system – they are not necessarily smarter nor loving the subject more.

Look, I am not wholly against choosing an academic career. Sure there are some perfect valid reasons, for example, it is much easier to find a faculty position in China than Europe or US because universities in China are expanding fast (i.e. more students each year – whether this is good or bad is another matter). Another example is foreign students looking for student visas. As Jonathan I. Katz stated in 1999: “The result is that the best young people, who should go into science, sensibly refuse to do so, and the graduate schools are filled with weak American students and with foreigners lured by the American student visa.” This seems to be true: “Dr Freeman estimates that in 1966 only 23% of science and engineering PhDs in America were awarded to students born outside the country. By 2006 that proportion had increased to 48%.” and “foreign students tend to tolerate poorer working conditions, and the supply of cheap, brilliant, foreign labour also keeps wages down.” (Doctoral degrees The disposable academic).

And also for some reason, it is much easier to secure a work visa in academia than in industry. In the UK, universities can easily justify why they need a person outside from EU and they usually have no problem supporting you for a work visa. In the US, it is said that an assistant professor (or even postdoc) can obtain an American green card within 2-3 years while those working in industry have to wait 5-6 years. Because of this, immigrants are likely just to stay in academia long enough (e.g. 5 years in UK) to secure a permanent residency before looking for other options. Of course, at that time, they may as well find that they stay in academia “too long” that they are not competitive in industry.