Series Introduction

Networks dominate today's computing landscape and
commercial technical protection is lagging behind attack technology. As
a result, protection program success depends more on prudent management
decisions than on the selection of technical safeguards. Managing
Network Security takes a management view of protection and seeks to
reconcile the need for security with the limitations of technology.

That sounds like an 'academic' view

Last month, I disabused my audience of the notion
that the 'academic' view of security is one to be scoffed at. So
it's only fair that this month I lay my fair share of abuse on the
academics of the world. And don't worry - this month I will do so
without shame. But at the same time, I think it is really important to
understand the vital role of the academics in the present and future of
information protection.

You may reasonably ask why it is that I feel as if I
can talk from the government view, the industry view, and the academic
view. I seem to act as if I am from one or the other almost at will, as
if I was somehow all three. That's because I am. While I rarely take
on the view of an academic in this venue, in my spare time, I am indeed
on the faculty of the University of New Haven where I teach 6 graduate
courses and, at times, carry out funded research.

So I will don my academic hat for half of this
article to defend the vital role of academic institutions in information
protection - but before I do that...

Why Do We Fund Stupid Academic Projects?

OK - it turns out that unless you really know what
you are talking about in this field, it's just about impossible to tell
the difference between a stupid academic project proposed by someone who
doesn't know their field and a brilliant academic who is pushing the
world forward by leaps and bounds. And of course most of those who fund
academics couldn't tell a brilliant academic from a used car salesman.
That's why we have academics review each others proposals...

But wait a minute. Suppose our reviewers are not
the brilliant academics, but rather the used car salesmen? Once we start
to let the used car salesmen in, we will never get another legitimate
researcher. Big problem. In fact, there is a major conflict of
interest when you have reviewers selected from the pool of people you
fund, because mostly they want to be funded again, so they will taint
their evaluations - even if they do not intend to do so - by their
views. But experts are in competition for funds, especially in academia
where there is a shrinking pot of money and monies not going to one
group goes to another. In zero sum games with memory, you will find
that people who 'game the system' win and those who don't lose.

So my solution is simple. Have people from
unrelated fields review work so that they don't know good from bad when
it comes to the proposals. Then you will have a random chance of
funding the real experts as opposed to the used car salespeople - which
would be an improvement over what happens today. OK - perhaps this is
not ideal, just a minor improvement. But I do have some suggestions...

Don't fund people who haven't done their
homework. How can you tell? Pay some outside experts to do your
homework before you fund anyone in a field. If it costs you $40,000 to
find out about the state of the art in intrusion detection, it is a
great deal compared to funding $10M a year for several years to find out
what was already known many years before. That's a real example by the
way. If the people you are thinking of funding don't know at least as
much as the results of our national
technical baseline study on the subject from a few years back, DO
NOT FUND THEM. If they refuse to read such studies, don't invite their
proposals. If that's too hard for you, check against the 50 Ways to Defeat Your Intrusion Detection
System article and have them explain in detail to an expert how
their technique avoids these attacks - or at least why it is that it
will not do so and how much of an impact that has on the value of the
work.

Don't fund people who haven't invested some
time in the field. This can be detected by getting copies of some of
their published papers on the subject. Get them, read them, understand
them, compare them to the state of the art from the study you sponsored
on the subject (or the examples above), and determine whether they know
what they are talking about. If you need an expert to evaluate their
proposals, get one who you are NOT funding EXCEPT for the evaluation.
Pay the evaluators good money to spend the time needed to do a good and
fair job of it and have them evaluate the people, their previous work,
etc.

Do a bit of real science on your own. Test out
what they say and do against people who know how to bypass their
techniques. Not stupid crackers from the Internet - real experts.
Perhaps some serious red teaming groups who are good at identifying
problems and pointing them out. Preferable folks who aren't driven so
much by their egos as by getting the right answers.

All right - so I am a dreamer... so sue me...

Why don't academics understand us?

This is an easy one. Academics don't understand
your problems because you haven't told them about your problems. And
indeed, most of your real problems are probably pretty stupid and don't
require an academic breakthrough to solve. For example, academics are
terrible about understanding issues of the color of money. It sounds
stupid to them when you say that you can't buy a $5,000 product, but you
can spend 500 person hours at a loaded rate of $100 per hour evaluating
it. They would ask why you don't just buy it and try it and save the
$45,000 of wasted time. See how foolish they are?

...

That was a pause for dramatic effect... I am working
on a budget these days trying to figure out how to turn money that I
have but cannot spend into money that I can spend before I have to get
rid of the people who can do the work. It's simple enough - there is
money to do the work but the people who can do the work can't be paid by
the money allocated to do the work - instead we have to hire someone who
can't do the work and get rid of people who can do the work so that we
can get the work done. Of course this will cause us to be unable to get
the work done, so I am trying to turn the people who can do the work
into people who can get paid to do the work, but of course the people
who can do the work aren't qualified to do the work, while the people
qualified to do the work can't do the work.

WARNING - if this makes sense to you, you need a
vacation - as do I. The reason academics don't understand this sort of
thing is not because they are stupid - which is not to say that they are
all that smart - but rather because they are academics. They are people
who have trained themselves and oriented themselves toward solving the
deepest problems in their chosen fields using a set of mental and other
sorts of hard won tools and tricks to do so. So here's the solution:
(1) bring problems to academics that are suited to their ability to
solve them, and (2) if they don't understand you it is either because
you haven't explained yourself well enough or because it is not a
problem they are likely to be able to solve.

So what are they good for?

That's easy too... Really good academics are really
good at solving problems once and for all. That is, they are not in the
business of making band aides or building a better mousetrap. They are
in the business of figuring out new and better ways to limit bleeding
while not exposing wounds to septic threats and finding ways to limit
mouse traffic so that it doesn't do any harm or create any scares for
humans. If you ask an academic to build a better mouse trap, and if
they take the problem on that basis, they are either really desperate
for funding or they are not really academics.

So if you want to solve problems - really solve
them, and if you have the time and money required to do this task
properly - then academics are probably well suited to the task. If you
don't really want to solve the problem or don't have long term funding
or don't have enough funding to really solve the problem, then the
academics are not the right people to put to the task.

Here are some problems we might really want to solve
and solve well where we are foolishly sending our money to the wrong
people:

Finding a way to effectively deal with computer
network attacks.

Finding an effective way to manage risks
associated with computer crime on a national basis.

Developing a systematic, reliable, repeatable,
and scientifically valid way to do forensics examinations involving
digital systems and media which reproduces the sequence(s) of events
that led to the current situation in those systems/media.

Create a new methodology for analyzing systems
for vulnerabilities relative to threats and consequences and produce
a systematic method and set of tools for generating analytical results
comparing defensive measures.

Hopefully you get the idea.

So What are the Vital Roles of Academia?

Academia is vital for at last three things in
information protection: (1) Education, (2) Research, (3) Social issues.

Education: While you might think that
few would debate the role of academia in education, today, almost none
of the education in information protection is done by academia. This is
largely because academia has failed to take up the cause and because the
government, particularly the NSA, has historically created impediments
to academia in this area. The latter problem is now changing but the
lack of infrastructure in the form of competent educators in this arena
is creating impediments to quality undergraduate and graduate education.
This will continue to be a problem until someone starts educating the
existing crop of Ph.Ds in universities in this area. Such efforts have
been attempted, but they have fallen short due to lack of funding. The
introduction of $10M per year for a few academic institutions is a
pitiful attempt by the government to change this situation.

Research: Many think that research is
better done by businesses, but history has shown just how poor a choice
this is. The fact of history is that almost no fruitful research has
ever been done by businesses in the information protection arena, and
the little useful research that has been done has focussed on optimizing
specific mechanisms that are usually poor to begin with. The research
is aimed at making them less poor and generally is focussed on solutions
in the 6 month to 2 year time frame. Similarly, government has been
funding these time frames exclusively in recent years with very few
research grants running for more than a year. Historically, 5-10 year
research programs have been required for real progress in complex
subjects, and there is no doubt that information protection is highly
complex. The total US government funding to universities in this area
is probably in the range of $50M/year, which would not be so bad, except
that it goes almost entirely to politically chosen institutions which
are not competent to do research. Rather, they produce reports and fund
others to do little projects. The other chunks of money typically go to
trusted systems research, intrusion detection research, and cryptography
research. This is all fine and dandy, but this represents a total lack
of understanding about what is needed in research and what the future
might look like.

Social issues: Universities are really
good at looking at social issues, and university professors in some
social sciences departments do look at these issues, but the funding and
support in these issues is pitiful. Typically, the lack of cooperation
between competent computer scientists and competent social scientists
leads to computer science without adequate social science methodology
and social science without adequate technical expertise. The result is
that we are missing what is probably the most important aspect of
information protection research, we do few valid experiments in this
area, and we do essentially no research and development designed to
address these issues.

So it looks like the areas where university research
should and will ultimately play its most vital role are collapsing from
benign neglect. I should point out one other really important thing.
If you look at the history of information protection, you will find that
almost every breakthrough that produced substantial changes came from
middle-aged researchers in universities doing research funded for
periods of 5 years or more. If you look at the situation today, we are
practically guaranteed that these sorts of breakthroughs will not happen
for the next five years and that they won't be numerous for the next 15
years. And every year we wait, the situation gets grimmer - because the
total number of researchers in this area in universities is going down,
the total number of Ph.D.s available to do the work is too low to
sustain current levels of professors, and the best and the brightest
stars who created the breakthroughs we are still depending on today, are
nearing retirement.

Conclusions

Universities are poorly understood by industry and
government and have taken a lot of abuse lately. Their ineptitude in
the politics of funding has led to the movement of research dollars and
quality researchers out of universities and out of this field. The side
effect is that there are fewer and fewer quality researchers in
information protection and they are doing less and less research and
producing fewer and fewer new scholars in the field. Unless this
changes, we will soon see a near-total collapse of the capability in the
United states to do real research in this area. We are nearing this
collapse today.

Government and industry funding have been slow,
inadequate, and poorly targeted, has ignored the long term in favor of
the short term, and a direct result is the increasingly staggering
losses due to attacks on computer systems. We are losing scores of
billions of dollars a year because of inadequate protection, and yet the
total funding to stop these losses through research is less than one
tenth of one percent of the losses.

The situation could not be clearer. Unless and
until we start spending more money more wisely for long term research at
the Ph.D. involving academic institutions with quality programs, we
will continue to sink into increasingly horrendous losses. If we don't
reverse the trend soon, we may even start to find that the efficiencies
brought about by improvements in information technologies, which we
spend billions for each year, are more than offset by the losses
associated with the poor quality of protection associated with those
technologies.

About The Author:

Fred Cohen is researching information protection as a
Principal Member of Technical Staff at Sandia National Laboratories,
helping clients meet their information protection needs as the Managing
Director of Fred Cohen and Associates, and educating cyber defenders
over-the-Internet as a practitioner in residence in the University of
New Haven's Forensic Sciences Program. He can be reached by sending
email to fred at all.net or visiting http://all.net/