2
Findings and Recommendations

I.NRC STUDY FINDINGS

The Department of Energy (DoE) SBIR program is making significantprogress in achieving the congressional goals for the program. The SBIR program is sound in concept and effective in practice at DoE. With the programmatic changes recommended here, the SBIR program should be even more effective in achieving its legislative goals.1

Overall, the program has made significant progress in achieving itscongressional objectives by:

Stimulating technological innovation; (see Finding G)

Using small business to meet federal research and development needs; (see Finding C and D)

Fostering and encourage participation by minority and disadvantaged persons in technological innovation; (see Finding E) and

These objectives are set out in the Small Business Innovation Development Act (PL 97-219). In reauthorizing the program in 1992, (PL 102-564) Congress expanded the purposes to “emphasize the program’s goal of increasing private sector commercialization developed through Federal research and development and to improve the Federal government’s dissemination of information concerning small business innovation, particularly with regard to woman-owned business concerns and by socially and economically disadvantaged small business concerns.”

Citation Manager

"
2 Findings and Recommendations ."
An Assessment of Small Business Innovation Research Program at the Department of Energy . Washington, DC: The National Academies Press,
2008 .

Please select a format:

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 25
2
Findings and Recommendations
I. NRC STuDY FINDINGS
The Department of Energy (DoE) SBIR program is making significant
progress in achieving the congressional goals for the program. The SBIR pro-
gram is sound in concept and effective in practice at DoE. With the programmatic
changes recommended here, the SBIR program should be even more effective in
achieving its legislative goals.1
A. Overall, the program has made significant progress in achieving its
congressional objectives by:
• Stimulating technological innovation; (see Finding G)
• Using small business to meet federal research and development needs;
(see Finding C and D)
• Fostering and encourage participation by minority and disadvantaged
persons in technological innovation; (see Finding E) and
• Increasing private sector commercialization of innovations derived
from federal research and development. (See Finding B.)
1These objectives are set out in the Small Business Innovation Development Act (PL 97-219). In
reauthorizing the program in 1992, (PL 102-564) Congress expanded the purposes to “emphasize the
program’s goal of increasing private sector commercialization developed through Federal research and
development and to improve the Federal government’s dissemination of information concerning small
business innovation, particularly with regard to woman-owned business concerns and by socially and
economically disadvantaged small business concerns.”

OCR for page 25
SBIR AT THE DEPARTMENT OF ENERGY
B. The DoE SBIR program is focused on commercialization and has
seen meaningful achievement. There are, nonetheless, opportunities
for improvement in commercialization.
1. A significant percentage of DoE SBIR projects are commercialized
to some degree.
i. Reaching the market. NRC Phase II Survey data indicate that 41 per-
cent of SBIR-funded projects reach the marketplace or have commer-
cialization underway.2 Using a different methodology, DoE’s internal
survey reported that 30.3 percent of firms reached the market.
ii. Revenue skew. The NRC Phase II Survey data also show that a
much smaller number (4 percent) of projects generate more than $5
million in revenues.3 The distribution of sales resulting from SBIR
awards from DoE is not qualitatiely different from the distribu-
tion of returns from private sector investments.4 These few projects
have, however, been extremely successful; in the case of Atlantia
Offshore, for example, a single Phase I and Phase II pair of awards
resulted directly in a product that generated over $500 million in
sales, in addition to other societal benefits. DoE’s survey of their
Phase II awardees indicated that 11 percent of companies reported
sales greater than $850,000.
iii. Licensing revenue. The NRC Phase II Survey indicates that licens-
ing revenues have not been a significant source of additional com-
mercial success at DoE.5
2See Figure 4-1. Twenty-four percent of NRC Phase II Survey respondents reported products/
services/or processes in use at the time of the survey, and 18 percent reported commercialization
underway (figures rounded). See NRC Phase II Survey, question 1.
3See NRC Phase II Survey, Question 4b.
4As with investments by angel investors or venture capitalists, SBIR awards result in highly con-
centrated sales, with a few awards accounting for a very large share of the overall sales generated by
the program. These are appropriate referent groups, though not an appropriate group for direct com-
parison, not least because SBIR awards often occur earlier in the technology development cycle than
where venture funds normally invest. Nonetheless, returns on venture funding tend to show the same
high skew that characterizes commercial returns on the SBIR awards. See John H. Cochrane, “The
Risk and Return of Venture Capital,” Journal of Financial Economics, 75(1):3-52, 2005. Drawing on
the VentureOne database, Cochrane plots a histogram of net venture capital returns on investments
that “shows an extraordinary skewness of returns. Most returns are modest, but there is a long right
tail of extraordinary good returns. Fifteen percent of the firms that go public or are acquired give a
return greater than 1,000 percent! It is also interesting how many modest returns there are. About 15
percent of returns are less than 0, and 35 percent are less than 100 percent. An IPO or acquisition is
not a guarantee of a huge return. In fact, the modal or “most probable” outcome is about a 25 percent
return.” See also Paul A. Gompers and Josh Lerner, “Risk and Reward in Private Equity Investments:
The Challenge of Performance Assessment,” Journal of Priate Equity, 1(Winter):5-12, 1977. Steven
D. Carden and Olive Darragh, “A Halo for Angel Investors,” The McKinsey Quarterly, 1, 2004, also
show a similar skew in the distribution of returns for venture capital portfolios.
5NRC Phase II Survey, Question 3.

OCR for page 25
7
FINDINGS AND RECOMMENDATIONS
iv. DoE awards are characterized by a relatively high degree of pri-
vate sales. Of the DoE SBIR award recipient firms that responded
to the NRC Phase II Survey and reported sales of some type, 76
percent sold to domestic private sector firms and 14 percent to
export markets.6
v. Limits on potential for broader commercialization. In cases
where the market is inherently limited, as with the case with sen-
sitive energy technologies, products developed with DoE SBIR
assistance often cannot become a large commercial successes.7
2. DoE has a substantial history of commercialization support and
has expanded its efforts in recent years.
i. DoE has sponsored a Commercialization Assistance Program (CAP)
for the past 17 years. That program has provided, on a voluntary
basis to Phase II awardees, individual assistance in developing busi-
ness plans and in preparation of presentations to potential invest-
ment sponsors. It is operated by a contractor, Dawnbreaker, Inc., a
private firm based in Rochester, NY.
ii. For the past 6 years, DoE has offered their Phase II awardees addi-
tional market identification and networking service which requires
much less time commitment from awardees than the very intensive
CAP.8 These programs have provided professional assistance in
business plan development and/or market evaluation to participating
SBIR companies.
iii. Systematic tracking is necessary to assess the impact of these efforts
on commercialization.9
3. Third-party investors can be encouraged by the validation effect
of SBIR funding.
i. Additional investments. Sixty-three percent of respondents indi-
cated that they had received or made additional investments in the
surveyed project, averaging just under $1 million per project.10
ii. No venture funding. For a number of reasons (at least until recently)
the energy sector in general and therefore DoE SBIR projects have
not been attractive to venture capitalists. No responding projects
indicated that they had received venture capital funding.11
6See Table 4-2.
7The case study of Diversified Technologies illustrates this phenomenon with regard to specialized
transformers. See Appendix D.
8DoE SBIR publications and Web site; interviews with DoE SBIR staff.
9DoE calls for its contractors, Dawnbreaker and Foresight, to track their clients’ performance for
at least two years.
10See NRC Phase II Survey, Questions 22 and 23.
11See Table 4-4.

OCR for page 25
SBIR AT THE DEPARTMENT OF ENERGY
TABLE 2-1 2005 DoE SBIR Initiatives in Support of Commercialization
Technology
Niche Virtual Deal
Assessment Simulator™
CAP Trailblazer (TNA) (VDS)
Contractor Dawnbreaker, Foresight Foresight Foresight
Inc. Science and Science and Science and
Technology, Technology, Technology,
Inc. Inc. Inc.
Start Date January 2005 January 2005 January 2005 January 2005
Completion Date December 2007 December 2007 December 2007 December 2007
Eligibility SBIR II only SBIR I only SBIR I or II SBIR I or II
SOURCE: Department of Energy SBIR program publications and Web site.
iii. Acquisition. In some cases, the technology developed had sufficient
commercial potential that investors bought the grantee company
outright.
iv. Multiple other funding sources. Many grantees have found addi-
tional funds from a wide range of sources, including angel fund-
ing. Sixty-three percent of NRC survey respondents attracted some
additional investment (excluding further SBIR awards).
4. Commercialization and selection. Commercialization potential is
now formally recognized as a selection criterion at DoE; it accounts for
one-sixth of the total score used in selecting applications for award. 12
C. The DoE SBIR program is operated in alignment with the depart-
ment’s mission. It has the potential to contribute to the missions of
the National Laboratories.
1. Effective mission alignment. All DoE awards appear to be selected
primarily on the basis of their potential to advance knowledge and
provide solutions in the field of energy. There is no evidence that DoE
awards are made in fields outside those linked to the department’s
mission.13
12Interviews with DoE SBIR staff.
13Possibly the most dramatic identified success from the program has been Atlantia’s SeaStar ®
technology, a cost-effective alternative for the development of otherwise inaccessible small oil fields
in deep water. The technology generated more than $500 million in revenues for Atlantia.

OCR for page 25
FINDINGS AND RECOMMENDATIONS
2. Agency technical managers are deeply involved in topic develop-
ment and selection.
i. The impact of an SBIR project on DoE’s mission to advance
economic and energy security is carefully considered during the
selection process. Awardees and DoE staff note that impact effects
are an important component in every application. In all the cases
examined, DoE SBIR funded projects related to the science and
technology of energy.
ii. Results from the NRC survey of Technical Topic Managers indi-
cate that of the DoE projects surveyed, 70 percent reported that the
project manager was involved in the generation of the topic that led
to SBIR award. Fifty-eight percent of project managers reported
involvement with the technology before Phase I began.
3. Limited participation of DoE National Laboratories in SBIR.
i. While the National Laboratories are an important source of techni-
cal reviewers for proposals (which is an important component of
the administration of the SBIR program at DoE), the Laborato-
ries themselves are not otherwise strongly involved in the SBIR
program.
ii. As a result, the potentially significant role of the National Labo-
ratories as partners with SBIR-award-recipient firms is not fully
realized.
D. The SBIR program at DoE has provided significant support for small
business, frequently acting as the impetus for project deployment and
the foundation of new firms.
1. SBIR awards from the Department of Energy fund the develop-
ment of technologies that, otherwise, might have developed more
slowly, if at all.
i. The project initiation decision. More than 80 percent of NRC
Phase II Survey respondents reported that they would likely or
certainly not have gone forward with their project in the absence of
SBIR funding.14
ii. New firm formation. Twenty-three percent of DoE respondents
indicated that their companies were founded entirely or partly
because of an SBIR award. Of the NRC Phase I Survey respon-
dents, 11 percent stated that their firms were founded or remained in
businesses due to SBIR Phase I funding they were awarded. Growth
within companies also occurred, with 34 percent of responding
14See Figure 7-1.

OCR for page 25
0 SBIR AT THE DEPARTMENT OF ENERGY
Phase I companies hiring one or more new employees as a direct
result of an award.15
iii. Critical source of early funding. Case study work suggests that
alternative very early stage venture funding for SBIR projects is
nearly nonexistent. Without exception, all of the case study com-
panies indicated that SBIR was vital to the development of their
technology. Most suggested that the technology would not have
been created if there had been no SBIR program. All credited SBIR
as having played a significant role in the company’s formulation or
development.
2. The DoE SBIR program supports the engagement of small busi-
ness in federal R&D.
i. Contracting mechanism. SBIR provides a contracting mechanism
that keeps small firms engaged in the innovation system. Many
small firms find other approaches to seeking federal or corporate
R&D support to be either too costly from a contracting standpoint
or require too great a loss of control to be viable. This appears to be
one reason why SBIR continues to be enthusiastically supported by
the small business community despite the fact that the award sizes
have stayed constant for over a decade.
ii. Application process as a filter. At the same time, the applica-
tion process—though simpler than that for other federal R&D pro-
grams—is nonetheless very demanding for a small firm, and the
success rate of Phase I applications is typically at 18-20 percent. 16
3. The SBIR program also provides additional benefits for participat-
ing small businesses.
i. Networks and partnering. SBIR facilitates technological innova-
tion through the creation of new partnerships and the strengthening
of networks of innovators. Among case study firms, networking was
identified as critical when attempting to find partners to assist with
commercialization.17
ii. Alternative path development. Companies often use SBIR to fund
alternative development strategies, exploring technological options
in parallel with other activities.
iii. Expand the company’s technology base. SBIR awards can help
build a company’s technology base, allowing the company to pursue
15Case studies of NanoSonic and Thunderhead Engineering, and Diversified Technologies, Inc.
16DoE SBIR program data prepared for the Subcommittee on Technology and Innovation, Com-
mittee on Science and Technology, U.S. House of Representatives, June 26, 2007. This success rate
is subject to some variance—as high as 25 percent and as low as 12.5 percent in some years.
17See Figure 7-2.

OCR for page 25
FINDINGS AND RECOMMENDATIONS
an expanded suite of research projects, bring in researchers with
new points of view and knowledge, and develop new markets.18
iv. Stimulate other forms of financing. SBIR awards can help a firm
attract other forms of financing, such as funding opportunities from
states (NexTech). Banks may also be more willing to provide busi-
ness loans to companies that have received a Phase II grant, recog-
nizing that a Phase II award is a good indicator of an innovation’s
potential marketability.19
v. Enhance credibility with federal agencies. SBIR awards typi-
cally lend credibility to a firm and enable it to earn other federal
grants and contracts, including those obtained through Broad Area
Announcements.20
vi. Enhance credibility with private-sector firms. SBIR awards can
provide a “stamp of approval,” allowing smaller companies to better
access large private sector companies, including government con-
tractors, and attract private investors. An SBIR award can instill
confidence that the smaller firm, and its product, have been vetted to
some degree, thus raising interest. Once one large company becomes
aware of an SBIR firm, its reputation grows and others are likely to
hear about the firm, its product, and its area of expertise.21
E. Support for minority- and woman-owned firms.
1. No long-term trend is apparent. Data from SBA indicates that
between 1992 and 2005, the Phase I share of woman- and minority-
owned businesses at DoE averaged just over 20 percent, with a decline
in the early years of this decade, followed by an increase, accelerat-
ing in 2005 (see Figure 2-1). Data for Phase II are similar, although
slightly lower. On average, woman- and minority-owned firms won
22.1 percent of Phase I awards from 2001-2005, and 19.1 percent of
Phase II awards.22
2. Lagging application success rates. For Phase I, applications from
woman-owned businesses have had a lower rate of success compared
to all other groups—by approximately 3-10 percentage points—in
18See case studies of NanoScience, PPL, Diversified Technologies, Inc., and Thunderhead Engineer-
ing in Appendix D. See also the case study of Pearson Knowledge Technologies in National Research
Council, An Assessment of the Small Business Innoation Research Program at the Department of
Defense, Washington, D.C.: The National Academies Press, 2007 Prepublication.
19See the Airak case study in Appendix D.
20See the IPIX and NexTech case studies in Appendix D.
21See the case studies of Atlantia, NanoScience, and NexTech in Appendix D.
22See Chapter 6.

OCR for page 25
SBIR AT THE DEPARTMENT OF ENERGY
30
Percent of All DoE Phase I Awards
25
20
15
10
5
0
1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005
Year
Woman-owned Minority-owned
FIGuRE 2-1 Shares of Phase I awards for woman- and minority-owned businesses,
1992-2005.
SOURCE: U.S. Small Business Administration, Tech-Net Database.
2-1
every year except one. For minority-owned companies, the success
rate is better than for woman-owned companies but still lags behind
the “other” category (neither woman-owned nor minority-owned). For
2002-2003, the success rate of minority-owned businesses was consid-
erably lower than that for woman-owned and all other businesses. 23
F. understanding firms winning multiple awards.
1. Wide distribution of funding.
i. SBA awards data indicates that between 1992 and 2005, 3,698
Phase I awards went to 1,459 different companies. The top 20 win-
ners accumulated 556 Phase I awards, or 8 percent of all awards
made during this period. No firm received 50 or more DoE Phase I
awards.
ii. Only five companies have been identified as receiving more than
15 Phase II awards between FY1992 and FY2003, and only two
received more than 20, with the maximum being 21.24
23Ibid.
24The top 20 percent of winning companies together received 11.1 percent of awards. This is sig-
nificantly lower than the Department of Defense.

OCR for page 25
FINDINGS AND RECOMMENDATIONS
2. Some products, involving complex, multidisciplinary technologies,
require multiple awards and a long gestation to develop.
i. Products developed within by SBIR-awardee firms often require
multiple SBIR awards and/or other sources of funding. Many firms
work in complex technology areas, where capabilities in comple-
mentary technologies are the basis for competitive success.
ii. Developing prototypes of such products, with associated business
plans and reliably estimated costs of production, can take a decade
or more to complete.
3. Some SBIR firms with multiple awards achieve commercial impacts
via spin-off firms.
i. Some DoE-SBIR firms with multiple awards specialize in solving
practical problems posed by government. The approach of Creare,
Inc. in Hanover, New Hampshire, for example, is to create and
transfer to spin-offs technologies that have particular commercial
potential. As a consequence, Creare alone has spawned 15 spin-off
firms employing over 1,500 people, with annual revenues reportedly
in excess of $250 million.25
G. Knowledge creation and dissemination effects.
1. The SBIR program at DoE supports knowledge transfer in several
ways.
i. Patents. SBIR companies at DoE have generated numerous patents
and publications, the traditional measures of knowledge transfer
activity.26 Forty-three percent of the projects responding to the NRC
survey reported at least one patent application. Of these, 37 percent
reported receiving a patent related to the SBIR project.
ii. Peer-reviewed publications. Forty-six percent of projects surveyed
by NRC resulted in at least one peer-reviewed article.27
iii. Knowledge transfer from universities. The NRC survey also sug-
gests that SBIR awards are supporting the transfer of knowledge,
firm creation, and partnerships between universities and the private
sector:28
25As a point of comparison, Xerox Technologies Ventures, the famed venture capital arm of Xerox
Corporation, generated 35 spin-off firms over a comparable time period.
26See NRC Phase II Survey, Question 18.
27Without detailed identifying data on these patents and publications, it is not feasible to apply
bibliometric and patent analysis techniques to assess the relative importance of these patents and
publications.
28See Table 4-14 in National Research Council, An Assessment of the Small Business Innoation
Research Program, Charles W. Wessner, ed., Washington, D.C.: The National Academies Press, 2008.

OCR for page 25
SBIR AT THE DEPARTMENT OF ENERGY
o In more than two-thirds of responding companies at all agencies,
at least one founder was previously an academic;
o About one-third of company founders were most recently
employed as academics before the creation of their company;
o At DoE, about 34 percent of projects had some alignment with a
university, through the use of university faculty as contractors on
the project, use of universities as subcontractors, or employment
of graduate students.
iv. Indirect paths. There is anecdotal evidence concerning benefi-
cial “indirect path” effects—that projects provide investigators and
research staff with knowledge that may later become relevant in a
different context—often in another project or even another com-
pany. While these effects are not directly measurable, discussion
during interviews and case studies suggest they exist.29
2. SBIR research quality.
i. The NRC survey of technical topic managers (TTM’s) indicated that
technical managers see research funded by SBIR as being largely
similar in quality to the research funded under other programs.
ii. Another survey question asked whether more high quality research
proposals were received than could be funded. Just under two-
thirds (62 percent) of DoE project managers reported more fund-
able projects than were funded.30 Success rates for SBIR Phase I
competitions are consistently below 20 percent. In most years, the
success rates have been about 15 percent.
H. The DoE SBIR program has not benefited from regular evaluation.
1. Prior to the congressional legislation authorizing this study, no
systematic, external program assessment had been undertaken
at DoE. As a result, program management is not sufficiently
evidence-based.
i. Partly as a result of insufficient resources, the program has insuf-
ficient data collection, limited reporting requirements, and limited
analytic functions. This limits the program’s capacity for self-
assessment and adjustment.
29For a discussion of the “indirect path” phenomenon with regard to the results of innovation
awards, see Rosalie Ruegg, “Taking a Step Back: An Early Results Overview of Fifty ATP Awards,”
in National Research Council, The Adanced Technology Program: Assessing Outcomes, Charles W.
Wessner, ed., Washington, D.C.: National Academy Press, 2001.
30Thirty-one percent reported about the right number of proposals, and eight percent reported fewer
fundable proposals than funds available.

OCR for page 25
FINDINGS AND RECOMMENDATIONS
ii. This lack of assessment means that program management does not
have adequate ongoing information about how their actions affect
outcomes such as commercialization, knowledge generation, and
networking.
2. Developing a culture of assessment. DoE has recently developed a
system for tracking outcomes internally. This represents a positive step
towards an assessment culture, but a range of issues still need to be
addressed and a more systematic approach to evaluation adopted.
I. Limited resources for program management.
1. The fraction of resources devoted to SBIR program management
at DoE is the lowest among agencies studied by the NRC.
i. Given the size of the program, the number of applicants each year,
and the requirements of the award process, the SBIR staff at DoE
is the minimum required to operate the program.
ii. DoE SBIR staff devote nearly all their time to managing the pro-
cesses for generating technical topics, and for receiving, evaluat-
ing, and selecting grant applications. This leaves little time for
other important tasks such as outreach, measuring Phase III activity,
encouraging Phase III activity (both within and outside the depart-
ment, including the national laboratories), documenting successes,
understanding failure, and developing and implementing program
improvements.
2. With most staff time devoted to compliance, there is limited oppor-
tunity for program enhancement.
i. Impact on assessment. Program managers do not have time or
sufficient resources to support interaction between SBIR firms and
technical staff; conduct internal and external assessment of the
performance of funded firms; and evaluate the commercialization
program and the overall effectiveness of awards.
o Limited post-award follow up. While DoE’s technical staff
have taken the lead in developing topics for the program solicita-
tion, they have had very little interaction with the companies that
receive SBIR awards.31
o Limited monitoring and assessment. Administrative staff or
technical staff report few opportunities to visit funded firms
or otherwise track their progress. Over the course of this NRC
31Interviews with DoE SBIR staff. Progress is being made in this regard. DoE technical staff has re-
cently demonstrated its direct involvement in a newly instituted Continuation Application Process.

OCR for page 25
SBIR AT THE DEPARTMENT OF ENERGY
study, the DoE SBIR staff has sought new approaches to encour-
age DoE’s technical staff to become familiar with the work of
SBIR funded companies.
o Limited evaluation. While the DoE SBIR staff maintain a
detailed database of commercialization outcomes reported by
funded firms, they do not as yet have the resources to validate
the reported data or employ them in formal program evalu-
ation. Evaluation of the effectiveness of commercialization
programs has also been lacking. Program staff do not have
adequate resources to assess the relative effectiveness of differ-
ent approaches undertaken by DoE to enhance commercializa-
tion outcomes.
ii. Impact on cycle time. DoE provides only one solicitation annually.
This contrasts in particular with DoD and NIH, which offer candi-
dates several opportunities to propose projects. However, multiple
solicitations require more resources, which are currently not avail-
able to the at DoE SBIR program.
iii. Impact on gaps between SBIR Phase I and Phase II funding.
The funding gap between Phase I and Phase II also affects compa-
nies. More than half the respondents to the NRC Phase II Survey
reported that they stopped work on their project during this period.
A small number of respondents (3 percent) reported ceasing opera-
tions entirely in this unfunded interval. DoE has not adopted any
of the measures to reduce the Phase I-Phase II gap implemented at
other agencies.32
iv. Impact on outreach. Citing time and resource constraints, DoE
SBIR staff decline most invitations to speak about the program. The
agency does participate in the SBIR National Conferences, spon-
sored by DoD and NSF. However, DoE has attempted to avoid state
and local conferences, largely because the limited resources and
available staff are focused on day-to-day program operations.33
II. NRC STuDY RECOMMENDATIONS
The recommendations in this section are designed to improve the operation
of the SBIR program at the Department of Energy. They complement the core
findings that the program is addressing its legislative goals—that significant
commercialization is occurring, that the awards are making valuable additions to
nation’s stock of scientific and technical knowledge, and that SBIR is developing
products that apply this knowledge to the Department of Energy’s missions.
32See NRC Phase II Survey, Question 28.
33Interviews with DoE SBIR program managers.

OCR for page 25
7
FINDINGS AND RECOMMENDATIONS
A. Develop linkages between DoE technical staff and SBIR awardees.
1. Provide contact information. Communication with applicants, par-
ticularly first-time applicants, is critical to the processes. DoE should
increase the opportunity for all potential applicants to interact with the
technical program managers, including the publication of their names
and contact information within the annual solicitation.34
2. Provide pre-release information on topics. Pre-proposal communica-
tion through a fair, transparent mechanism is likely to improve proposal
quality and, therefore, overall program effectiveness. Consider the
DoD Pre-Release information exchange model, whereby the relevant
technical officer for each topic is available via email and phone for
questions during a period before the official release of the solicita-
tion. At DoD this practice has had the effect of ensuring not only that
applicants have a better understanding of the work being solicited, but
also in generating higher quality and better focused proposals.
B. Improve commercialization support.
1. Develop a match-making function for SBIR awardees. Bring SBIR
participants together with potential corporate customers, perhaps in
trade show, technical challenge workshop, or technology demonstra-
tion/validation formats. These functions could include large corpora-
tions identified by the agency’s two SBIR commercialization assistance
vendors, including large energy technology corporations that serve as
DoE contractors.
2. A possible model. The Navy Opportunity Forum, initially based on a
DoE initiative, is well funded. Similar levels of funding and scale might
enhance DoE’s return on this type of activity. Another model is the
National Renewable Energy Laboratory’s Enterprise Growth Forum.
C. Explore reallocation of funding and topics between programs.
1. While SBIR funds are awarded by DoE using a competitive review of
the best submitted proposals, the success rates of Phase I awards vary
dramatically between the different programs. This disparity occurs
because DoE allocates SBIR awards back to individual programs based
upon a running average of the amount of SBIR funds contributed by
that program, rather than only on proposal quality.
2. DoE might consider whether current adjustments on an ad hoc basis
are sufficient, or whether a policy that allows individual DoE programs
to benefit from SBIR projects in excess of their contributed amounts
would prove more effective.
34DoE initiated an information release process in FY2006. The relevant technical officer is available
via email from the opening date to the closing date of the Funding Opportunity Announcement.

OCR for page 25
SBIR AT THE DEPARTMENT OF ENERGY
3. By allowing DoE programs to compete among themselves for a greater
share of total, SBIR can better address the agency’s mission and
encourage proposals of the highest quality.
D. Engage the National Laboratories: More outreach to the National
Laboratories is desirable. This could include:
1. Subcontracting. Encourage participation of National Laboratories
as subcontractors to the small business on SBIR projects, not least by
removing regulatory barriers to the use of National Laboratories as
subcontractors. More resources are likely to be required to carry out
these activities.
2. Improve tracking. Develop procedures to track the relationship
between National Laboratories and the SBIR program more formally,
including the documentation of Phase III successes.
E. Increase the participation and success rates of woman- and minority-
owned firms in the SBIR program.
1. Encourage participation. Develop targeted outreach to improve the
participation rates of woman- and minority-owned firms, and strategies
to improve their success rates. These outreach efforts and other strate-
gies should be based on an analysis of past proposals and feedback
from the affected groups.35
2. Improve data collection and analysis.
i. There appears to be room for further improvement in the participa-
tion of women and minorities in DoE SBIR program. DoE should
undertake efforts to assess why woman- and minority-owned compa-
nies have experienced relatively lower success rates and to examine
courses of action that may rectify this underrepresentation.
ii. The Committee also strongly encourages DoE to gather the data
that would track woman and minority firms as well as principal
investigators (PIs), and to ensure that SBIR is an effective road to
opportunity. The success rates of woman and minority Principle
Investigators and Co-Investigators can also provide a measure of
woman and minority participation in the SBIR program.
3. Encourage emerging talent. The number of women and, to a lesser
extent, minorities graduating with advanced scientific and engineer-
ing degrees has been increasing over the past decade. This means that
many of the woman and minority scientists and engineers with the
advanced degrees usually necessary to compete effectively in the SBIR
35This recommendation should not be interpreted as lowering the bar for the acceptance of proposals
from woman- and minority-owned companies, but rather as assisting them to become able to meet
published criteria for grants at rates similar to other companies on the basis of merit, and to ensure that
there are no negative evaluation factors in the review process that are biased against these groups.

OCR for page 25
FINDINGS AND RECOMMENDATIONS
program are relatively young and may not yet have arrived at the point
in their careers where they own their own companies. However, they
may well be ready to serve as principal investigators (PIs) and/or senior
co-investigators (Co-Is) on SBIR projects. Over time, this talent pool
could become a promising source of SBIR participants.36
F. Develop data for evaluation and conduct regular assessments.
1. Summary annual report. DoE should annually provide Congress
with an enhanced summary report on the SBIR program. This should
include descriptive statistics for applications, awards, and outcomes
along the dimensions identified in this report, including knowledge
creation, technology innovation, and impact on agency mission, as
well as commercialization. As part of this process, DoE should produce
regular reports from the commercialization database.
2. Company Commercialization Report. DoE may also wish to con-
sider whether the DoD model—and technology—for producing a
Company Commercialization Report, updated each time the company
applies for further awards, might be a useful way of generating better
data about commercial outcomes.
3. Regular assessments. The proposed annual report noted above could
become a focus for wider efforts to develop improved internal assess-
ment capabilities that can be used to enhance program operations. It
could also tie proposed improvements to data-driven analysis and, for
example, include an evaluation of the predictive power of selection
scoring with regard to commercialization and other outcomes. DoE
should also commission regular external arms-length evaluations to
assess the program progress and the impact of new initiative.
G. Provide additional management funding to develop and maintain a
results-oriented program with a focused evaluation culture.
1. Effective oversight relies on appropriate funding. A data-driven program
requires high quality data and systematic assessment. As noted above,
sufficient resources are not currently available for these functions.
2. Increased funding is needed to provide effective oversight, including
site visits, program review, systematic third-party assessments, and
other necessary management activities.
36Academics represent an important future pool of applicants, firm founders, principal investigators,
and consultants. Recent research shows that owing to the low number of women in senior research
positions in many leading academic science departments, few women have the chance to lead a spin-
out. “Underrepresentation of female academic staff in science research is the dominant (but not the
only) factor to explain low entrepreneurial rates amongst female scientists.” See Peter Rosa and Alison
Dawson, “Gender and the commercialization of university science: academic founders of spinout
companies,” Entrepreneurship & Regional Deelopment, 18(4):341-366, 2006.

OCR for page 25
0 SBIR AT THE DEPARTMENT OF ENERGY
3. To enhance program utilization, management, and evaluation, addi-
tional funds should be provided. There are three ways that this might
be achieved:
i. Additional funds might be allocated internally, within the existing
budgets of the services and agencies, as the Navy has done, with
commensurately positive results.
ii. Funds might be drawn from the existing set-aside for the program
to carry out these activities.
iii. The set-aside for the program, currently at 2.5 percent of external
research budgets, might be increased slightly, with the goal of pro-
viding additional resources to maximize the program’s return to the
nation.37
4. These recommended improvements should enable the DoE SBIR man-
agers to address the four mandated congressional objectives in a more
efficient and effective manner.
H. DoE should consider the creation of an independent advisory board
that draws together senior agency management, SBIR managers,
and other stakeholders as well as outside experts to review current
operations and achievements and recommend changes to the SBIR
program.
1. Augmenting the role of the current DoE SBIR Oversight Committee,
the purpose of such an advisory board is to provide a regular monitor-
ing and feedback mechanism that would address the need for upper
management attention, and encourage internal evaluation and regular
assessment of progress towards definable metrics.
2. The annual report of the DoE SBIR program, recommended above,
could be presented to the board. The board would review the report that
would include updates on program progress, management practices,
and make recommendations to senior agency officials.
37Each of these options has its advantages and disadvantages. For the most part, the departments,
institutes, and agencies responsible for the SBIR program have not proved willing or able to make
additional management funds available. Without direction from Congress, they are unlikely to do so.
With regard to drawing funds from the program for evaluation and management, current legislation
does not permit this and would have to be modified, therefore the Congress has clearly intended
program funds to be for awards only. The third option, involving a modest increase to the program,
would also require legislative action and would perhaps be more easily achievable in the event of an
overall increase in the program. In any case, the Committee envisages an increase of the “set aside” of
perhaps 0.03 percent to 0.05 percent on the order of $35 million to $40 million per year or, roughly,
double what the Navy currently makes available to manage and augment its program. In the latter case
(0.05 percent), this would bring the program “set aside” to 2.55 percent, providing modest resources
to assess and manage a program that is approaching an annual spend of some $2 billion. Whatever
modality adopted by the Congress, without additional resources the Committee’s call for improved
management, data collection, experimentation, and evaluation may prove moot.

OCR for page 25
FINDINGS AND RECOMMENDATIONS
3. The board could be assembled on the model of the Defense Science
Board (DSB) or perhaps the National Science Foundation’s Advisory
Board.38 In any case, it should include senior agency staff and the
Director’s Office on an ex officio basis, and bring together, inter alia,
representatives from industry (including award recipients), academics,
and other experts in program management.
38The intent here is to use the DSB or the NSF Board as a model, not something necessarily to be
copied exactly.

Bookmark this page

Important Notice

As of 2013, the National Science Education Standards have been replaced by the Next Generation Science Standards (NGSS), available as a print book, free PDF download, and online with our OpenBook platform.