Wednesday, January 28, 2009

Graduate Student Folders Part II

We're getting to the point where we have to make our decisions on graduate student admissions. In the theory group at Harvard we got a sizable number of folders that we've cut down to about 25. We'll probably cutting those down to about 10 acceptances.

There's clearly an abundance of good people, who could become solid TCS researchers. And as we went over in my last post on this, it's difficult to choose based on an application who will become successful.

My guilt in having to choose from among these many talented people is assuaged mostly by my concern for the people who we do admit to graduate school, especially in TCS. Where will the jobs be when these people graduate? I've heard the arguments that CS is still expanding, well maybe not in America but elsewhere in the world, and that PhDs can find other work besides being a professor. I've seen that most of the other students I knew as a graduate student in Berkeley have ended up with good positions -- mostly professors -- albeit some after a number of years of postdocs. But still, at the end of the day, when I do the math, it just doesn't add up. I expect to be in my position for at least 30 years (barring the current curse of Harvard CS faculty to be moved into administration); how many professors (or scientists at research labs) can I produce over my lifetime?

The financial meltdown has only raised my concerns that in CS generally (and theory in particular) we've been living through a bubble, and we'll all be shocked when that bubble pops, though we shouldn't be.

I'm not so entirely pessimistic -- CS PhDs, for as far as I can see, should have no trouble getting pretty good jobs. But how many will get the jobs they think they're getting the PhD for? (Note the internal mental bias here -- professors remember mostly the students who did go on to get faculty/research jobs...) We'll see.

13 comments:

Anonymous
said...

if your students dont get jobs. i think I am stuck doing 30 years of postdoc.....Danm the day I chose to do a Ph.D. CS theory still seems fun and atleast a little applicable. Comm. theory is a disaster. Reminds of white mice running on wheels inside cages. Some paper, including some of mine, seem to be have done purely to kill time.But thankyou for as usual thinking beyond just math problems. Atleast someone knows what being and educator means. A heartfelt thankyou from the trenches of Ph.D glut.

I'm no expert on the job market, but this really doesn't seem that out of whack. First of all, as you mention, a lot of PhD's get jobs in industry. And I'm guessing that most (all?) of your students who want to will be able to get jobs at PhD-granting institutions. But at lower-tier universities, a lot of people get jobs at universities that don't grant PhD's. Most people who get a PhD will have no intellectual descendants, while a few institutions produce almost all of them. Higher tier universities feed lower tier universities, which feed lowest tier universities, which don't need to feed anything.

Of course, there may still be some oversupply (e.g. in the philosophy department at my low-ranked undergraduate university, all but one of the profs was Ivy-League, and the lone exception came from UCLA), but it's not as bad as your analysis seems to imply.

Let them make an informed decision. Point them to the CRA Taulbee survey when they walk thru your door wondering if a PhD is for them.

In general, I think money and employment opportunities are low priorities for the type of students that go into a PhD. They (we) are more like those "starving artists" one hears about, except without the starving part.

There is a PhD glut. In the sciences in general (to say nothing of the humanities) postdocs are in a Malthusian trap. We're lucky in TCS relative to, say, theoretical physics, since industry provides an escape valve that raises the equilibrium pay/benefits/quality of life for postdocs. The current environment is not at all healthy. These days, many strong PhDs from top 5 departments are struggling to get research positions.

One way to make the system sustainable (without crushing the hopes and dreams of most students) is to establish public research centers, where researchers *do not advise PhD students*. These researchers should outnumber PhD generating faculty enough so that all PhDs that are talented enough that they should be able to get research positions can get them (for whatever level of "talented enough" society judges appropriate).

In light of the above, I have a question for the academic CS establishment:Why aren't there CS national labs akin to Fermilab, Brookhaven, Los Alamos, etc?(Is it because we never built a devastating cyber-weapon of mass destruction?)

Actually, these other researchers don't need to be at a national research lab -- the key is that they don't generate new PhDs. In this regard, you could have a separate class of faculty at research universities that had the job of faculty at elite small liberal arts colleges. That is, teach well, do good research, but don't advise students.

I also wonder how much this affects tenured faculty, if at all. Are lousy career prospects in science driving top-talent into other fields? I'd like to see the data on that. If it's not affecting them much, then I predict the prospects for change are dim indeed.

One factor being overlooked is the pressure on junior faculty in CS departments to have Phd students. This pressure comes from the fact that most CS departments are in the School of Engineering where the equation more Phd students = more cheap labor = more grant money holds. Whereas theoretical CS being really a part of mathematics should operate under different rules. Math faculty feel little pressure to take on students -- and many in pure math are indeed extremely reluctant to do so, and thus a relative balance hold -- which is good for everyone involved.

To the poster who said that there is no pressure in Math to take on grad students:

There is no pressure on individual mathematicians as is not linked to grants and labs, but there is a lot of pressure on the Dept. as a whole since the Dept. needs the TAs, who do a lot of cheap teaching, particularly at large public universities.

Getting a Ph.D. to go work in industry is a waste of time. Better get a strong Master degree if you must do graduate work.

How's that for bad advice?

Getting a PhD to end up being a regular programmer surely is a waste of time, but there are plenty** of jobs out there which require a PhD.

You want to work in the core of a search engine, optimizing machine learning or ranking algorithms? A PhD would be great help. You want to work on the innards of an SQL engine? Again a PhD is almost a must. How about work on a biotechnology company doing advanced research in bioinformatics? You need a PhD too.

In fact, I'd say that any leading edge software company would benefit greatly from having a PhD for every 20-50 programmers and nowadays most of them do.

There is no pressure on individual mathematicians as is not linked to grants and labs, but there is a lot of pressure on the Dept. as a whole since the Dept. needs the TAs, who do a lot of cheap teaching, particularly at large public universities.

This is indeed true. However, my impression is that the drop-out rates from these programs are also substantial and thus the total number of Phds produced is rather controlled while maintaining a good supply of TAs. It isn't that bad for those who drop out -- since they experience what Math research is like and decided for whatever reason that it isn't for them.

I think somehow that the drop-out rates from CS graduate programs is nowhere that high and this might be a reason for the flux in the number of Phds produced. I am not arguing for having very high drop-out rates -- but somehow I find it difficult to believe that everyone who enters a Phd program is cut out for a career in research (whatever their initial inclination might be).

Anon wrote: Getting a Ph.D. to go work in industry is a waste of time. Better get a strong Master degree if you must do graduate work.

And AnotherAnon responded:

How's that for bad advice?

Getting a PhD to end up being a regular programmer surely is a waste of time...

This ThirdAnon would like to add:

Even if you're just going to be writing programs, a PhD is a great experience as long as you're not doing it for the sake of getting that degree. It opens you up to thinking in different ways, there's both less pressure and more pressure, you get to meet some of the smartest people at the cutting edge of science, live the grad-student life, etc, etc.

As long as you don't take the view that a PhD must lead to an academic career or a research/govt lab career, you'll be fine. Restricting yourself to those choices goes directly contrary to a basic fact about PhD's: a PhD is the beginning of a career, not the end of it. Also, just because you get a PhD, don't for a moment assume that learning stops. I got my PhD 15 yrs ago, have enjoyed careers in teaching, writing software, working at research labs... each one is different, comes with its own challenges and rewards.

Be open-minded!

As for the YetAnotherAnon who asked about leaders of the field getting together and trying to set directions for the field: I think those experiments have in general been bad ideas. I've seen such efforts by very eminent people take place at least twice in theoretical CS - FOCS'95 and STOC'00 - with disastrous results. In a brilliant essay, Razborov wrote:

Coming to another part of the question, "Where should Complexity Theory go?", there’s a lot of effort now to try to increase the impact of our field on key application areas. But in my opinion it is not quite clear whether Complexity Theory should go anywhere at all or it would be more useful staying where it is. The mission of this discipline is to provide a bridge for the traffic of ideas and concepts (with a handful of exceptions, not the results themselves!)between pure mathematics and Computer Science. Try to pull it to one side(say, for the purpose of making a junction instead), and firstly you will no longe rhave anything to cross the river upon, and secondly you may discover that the bridge’s remnants are less useful on the land than expected as the construction was designed for different purposes. However, in my opinion that part of semi-applied research in Complexity Theory which develops according to the internal logic of our field has to be strongly encouraged (continuing the analogy with the bridge, it is simply our duty to provide as convenient access to the traffic across it as we can manage). The work on efficient program verication mentioned above in quite a dierent context is one good example of this.

I agree with Jose's comment: people that apply for a PhD are probably motivated to do research, so financial/career considerations play a lesser role. Anyway, you can get pretty good positions outside academia.

I guess what will happen is: the more dedicated ones will do more postdoc years, and eventually get a permanent position. The rest will "drop out". It is true that we invariably always remember those that stayed in research... I guess it's the "confirmation bias".