I got a question from my family that’s a little more complicated than a quick blurb. Instead I put a more detailed response here.

I’m curious, how do you decide on which Hypothesis or Experiment to begin with in your work? Do you use any models to weigh impact, risk and learning?

First, working on a particular hypothesis is very different from working on a particular experiment. A hypothesis can be small scale, but the best ones address a lot of different things at once. Ideally, these are big picture sorts of things which make a number of testable predictions that you then check by experiment.

How you decide what to do next depends heavily on the kind of work you are doing. If you are in a field that is well developed and has established models, then you can work with a real theory instead of a hypothesis. If there is an established theory, often when you read the literature there are obvious next steps; the prevailing theory implies x, y and z; both x and z have been confirmed so it’s time to work on y.

Alternately, if you’re in a newish field or a field undergoing a paradigm shift, it’s common to just try established techniques that you know well and see what happens. Once you have some of that data you try to put together a model for how things might work and shift back to the other mode.

Impact is something that we try to weigh for grants and such, but often it is a waste of time to guess at impact. There’s an Asimov quote “The most exciting phrase to hear in science, the one that heralds new discoveries, is not ‘Eureka!’, but ‘That’s funny …’”. If you know too much about the impact of what you are doing in advance, it’s probably boring.

Risk isn’t typically a consideration unless there is some rare material or a live animal involved in the experiment. Generally the analogous consideration is resource allocation- time, materials, etc. vs. interest in the experiment. Typically we try to keep working on a few things at once so that if one set of experiments stops working there is something else to push forward on and minimize time lost on a project that needs troubleshooting. Mostly this isn’t rigorously modeled (again, if you know too much about how interesting your experiment is, it’s probably boring) but a rough intuition.

There is an interesting temporal aspect to how a lot of us weigh our own interest in a project vs. other people’s interest. Peer review usually leads us to weigh things that we are personally interested heavily early in a project. As a project progresses, colleagues’ interest is weighed more heavily in anticipation of reviewer objections. At the same time, projects nearing completion get more weight relative to other projects so that they can be published.