Innovation in breakthrough energy technologies is notoriously challenging, despite having potentially large rewards. Individual innovations are embedded in larger systems where change is very hard. These innovations often carry significant capital costs to demonstrate, commercialize, or reach economies of scale. Unlike the latest cell phone, consumers are often unwilling to pay more for a new energy innovation, especially when the rewards are in the future.
The Advanced Research Projects Agency - Energy (part of the U.S. Department of Energy and known as ARPA-E) presents one model for how public-private partnerships can overcome some of these challenges. Sometimes tools, like a renewable portfolio standards, are by themselves not enough to pull high-risk, high-reward breakthroughs into the market, and ARPA-E has provided another means of support to entrepreneurs.

With a careful, professional vetting process and clear focus on the specific technical barriers that currently stand between new technologies and mainstream use, ARPA-E has picked ‘potential winners’ to give them an opportunity to commercialize game-changing technology solutions. Overall, this care has paid off in terms of the very significant levels of private sector capital that has followed ARPA-E into these technology companies. The ARPA-E endorsement is proving valuable, in and of itself, because their selection process is credible.

Even so, not all investments will succeed — this is the inherently uncertain nature of innovation and partly why it is so hard for the private sector to invest. But even if not all will succeed, the potential for cheap, clean energy and to secure the position of an American company in the global sector will bring economic and other benefits that make the overall investment very worthwhile.

The key to success for ARPA-E and other agencies that place bets on specific technical solutions is to continue to be very careful in their vetting process. Innovators and entrepreneurs often overestimate their chance of success. They do best when their optimism is balanced by realistic managers who can help them focus, push them to abandon dead ends, and maintain a clear vision of the end goal (see Andrew Van de Ven, et al. The Innovation Journey).

By continuing to engage experts in technology, in markets and supply chains, and in finance, ARPA-E can avoid being swept away by innovators’ enthusiasm and maintain a hopeful but fundamentally cold-eyed perspective about how likely success really is for each investment. Both ARPA-E and the companies they invest in will do better as a result.

In the case of clean energy, there are clear reasons for the government to work with the private sector to develop breakthrough technologies and help them gain a foothold in the mainstream. Greater economic opportunities, increased energy security, and reduced health impacts all provide evidence that stepping into this role is economically efficient, but it can be politically difficult.

Especially with the current political dialogue, it is important that the public understands the necessary role that energy innovation plays in driving economic growth. In order to meet today’s energy challenges, we need to expand public support for government’s role in taking some of the risks, so that we can all reap the rewards.