The best place to start is with what "The State of Information Security 2003" survey doesn't include. It doesn't include some stark bit of data that will make you slap your forehead and exclaim, "Oh, that's the problem!" It doesn't include figures that suggest a secret formula for setting a security budget. Nowhere in its hundreds of pages of raw numbers will you find

The Answer, because The Answer is a fiction, even if the problem is not. Information security is a difficult, nuanced and immature craft. Silver bullets are for people who aren't serious about solving the problem.

What this survey does include, in its depth (more than 7,500 respondents) and intricacy (44 questions cross-tabulated by company size, security budget, geographical region and dozens of other categories) is a comprehensive profile of the imperfect and evolving world of information security.

According to the survey findings, it seems you're all just now coming to terms with information security as a problem. You understand that fixing the problem won't be easythat it will take a complex combination of infrastructure, education, proactive risk analysis and regulation. But at the same time, you seem to be hoping against hope that an easier way out will present itself. You know you need to do more, but the survey shows that you're not yet doing it. It's the classic economic principle known as the Problem of the Commons: Information security is a problem, but it's not my problem.

And one can hardly blame you for taking such a stance. Information security, right now, is a confused and paradoxical business. For example:

You've increased spending significantly, and you're told this is a good thing, and yet it has had zero effect in mitigating security breaches.

You're constantly warned about "digital Pearl Harbors," and yet the vast majority of incidents you report are relatively small, don't last long and don't cost much.

You're told that aligning security and business strategies is a top priority, and yet those who have fared best in avoiding breaches, downtime and security-related damages are the least likely to be aligned with the business.

But in another sense, you seem to be contributing to the confusion.

Respondents who suffered the most damages from security incidents were two times more likely than the average respondent to plan on decreasing security spending next year.

Those with the most damages were nearly half as likely to list staff training as one of their top three priorities.

A quarter of you neither measured nor reviewed the effectiveness of your information security policies and procedures in the past year.

In short, the survey shows that as much as the nascent information security discipline has grown since its baptismon Sept. 18, 2001 (one week after the terrorist attacks and the day the Nimda worm hit)it hasn't much improved with age.

Can we suss out any prevailing trend at all? If there's one there, it's hard to tell. In this particular survey, trends drift aimlessly. Positive correlations are rare. What you do about information security and what actually happens seem only vaguely allied.

Except for one case, where a connection was clear. In this survey, confidence in security correlates to better security, irrefutably. In other words, those who feel like they're doing better, are doing better.

What follows are the five cuts we made of "The State of Information Security 2003," including the aforementioned confidence correlation. Each provides insight into some aspect of this confused and complex discipline. In one, there's even a calculationan innovative method for benchmarking security spending called the per capita expenditure.

Forget silver bullets. Hard data, and lots of it, is what you need to start improving information security. And here it is.Fuzzy LogicIt is frustratingly difficult to find any relationship at all between good security and spending. And sometimes there's even a negative relationship.

Companies with $500,000 or more in damages were more than twice as likely to plan to cut security spending as companies that suffered no monetary loss in damages.What the Numbers MeanSince companies' size, and therefore their budgets, varied so widely across the survey's more than 7,500 respondents, the relative measure of security spending as a percentage of the overall IT budget provides a better comparative measure than the total spent on security. The mere single percentage point between the highest spenders and lowest spenders (when cross-tabulated with breach data) shows that those suffering fewer security incidents don't necessarily spend more to stay secureor, to flip it over, those who are hit the hardest by breaches aren't spending any less than those untouched.

So you can't accuse the companies suffering breaches of not spending enough. But perhaps they're not spending well. The hardest question for IT security officers to answer clearly isn't How much should we spend? but rather How should we spend?The answer: Probably by devoting less to technology.Security expert Bruce Schneier thinks the wanton deployment of technology hasn't helped because it hasn't been matched by a similar deployment of the soft stufftraining, education and awareness (see "The Evolution of a Cryptographer" in the September issue).

"Computer security folks are always trying to solve problems with technology, which explains why so many computer solutions fail so miserably," he says. "Most of the time, the security problems are inherently people problems, and technologies don't help much."

Take photo IDs, for instance. Schneier says that technologists want to add this or that to make IDs harder to forge, but what about the people who bribe the issuing officials to get real IDs in fake names? (At least two of the9/11 terrorists did that.) The technology that makes an ID harder to forge doesn't solve that problem.

In addition to the willy-nilly deployment of technology, some companies are also not using the technology to its full potential.

Consider that seven out of 10 survey respondents used intrusion detection systems, eight of 10 used firewalls, and nine of 10 used antivirus software. But only 50 percent of events were detected through those technologies or through security service providers managing those technologies for a company. The other half were detected the hard wayby customers, colleagues or news outlets alerting the company of a breach, or worse yet, by the damages the event caused.

Companies have deployed so much technology, and it has generated so much data in the form of log files, that they have given up trying to interpret the data. The haystack has gotten too big to look for needles in it, says Andrew Toner, partner in PricewaterhouseCoopers' security practice. "When [organizations] give up, that's when breaches are going to happen."

One interpretation for the disturbing trend of budget cuts by companies that were hit hardest by hacks is that they just gave up. Another possible explanation is that these companies are hard hit by something elsethe economyand they are cutting budgets across the board regardless of security breaches.

But it's just as likely that they've decided that the money they had spent was money down the drain. Why? Information security, for whatever reason, hasn't yet adopted risk management as a philosophy. It's still treated binarily: Either you're safe, or you're not. Either the money you spent worked, or it didn't. And that must change.

"People think in terms of threats, not in terms of risk," says T. Sean McCreary, a risk management specialist at The Motorists Insurance Group who previously served as a security manager and safety manager at two prisons. "Risk management allows you to assemble threats into some order or importance so the available funds can be used most effectively to prevent and prepare for the identified risks."

"Because it's harder," McCreary says. "It takes more time and effort, and, of course, more knowledge than they have."To-Dos1. Target spending on the soft stuffawareness, education, risk management traininginstead of throwing more technology at the problem.

2. Take better advantage of the technology you do have by interpreting the data it generates, not just letting it block attacks.

The Confidence Correlation

Those who are very confident in their security have stronger security infrastructures in place, and they spend more on security as a percentage of their IT budgets.What the Numbers MeanStructure and dedicated resources breed confidence. And confidence, experts say, breeds better security. In a sea of data that fails to reveal relationships between security and best practices, the confidence factor is a welcome sight. We can even go so far as to herald the one-quarter of respondents who called themselves "very confident" in their organizations' security as security leaders. That group tends to create far more structure around security within the organizationin other words, making it a discipline and not something that happens as part of the IT group. They hire more security executives and give those executives more control over policy, spending and staffs.

Another key point: The more confident a company is in its security, the less likely the security is controlled by the IT department. Many believe that IT's oversight of information security has been a limiting factor in improving itthat, if the CSO reports through the CIO, it's like having the fox guard the henhouse. If the CIO, for example, controls both the CRM implementation, which he's been told to get done in one year for $2 million, and is also in charge of information security, which will add time and money to that project, to which master does he answer?

At the very least, IT leaders should be self-policing and conducting independent audits of their security practices. But the numbers in that regard don't suggest companies are. About 75 percent of companies don't perform third-party assessments of privacy standards, and 60 percent don't audit security standards. No one indicated that systems were tested for security/policy compliance.

Extracting information security from the IT department overnight may not be wise either, but a good way to start the process of separating the two would be to conduct third-party audits and verification that security isn't getting subverted.

Bill Spernow, former director of IT for the Georgia Student Finance Commission, says the first thing he did when he got his job was to fight for, and win, independence from the IT department. "It's the biggest battle I had there," he says. "If I see a CISO reporting to some IT component, I see a position that's not working, guaranteed. The conflict of interest is just too much to overcome. Having the CISO report to IT, it's a deathblow."To-Dos1. Create structure around information security by hiring a CSO or creating an executive security committee.

2. Consider extracting the information security function from the IT department.

Little Bangs Everywhere

Major security breaches are the exception, not the rule. Most security incidents lasted less than a day and cost less than $100,000. And most companies had 10 or fewer such events in the past year.

What the Numbers Mean

Terrorists can shut down the Internet or the power grid. A hacker can take down your whole company. Both plausible headlinesor lines from consultants trying to sell their servicesfrom the past year. But survey data shows that you're not dealing with the Great Chicago Fire. You're dealing with lots of little brush fires.

The question then becomes: Are the little hacks common because you haven't done a good job of protecting your enterprise? Are the big-bang incidents rare because you have? Or are you simply lucky enough to have avoided the big problems but not lucky enough to ward off the smaller incidents?

In any case, you're exposed to the smaller incidents. And Howard Schmidt, vice president and CISO of eBay (and former special adviser to the White House for cyberspace security), thinks the prevalence of little bangs everywhere does not suggest you've done a good job steeling yourself against major attacks. Instead, he sees a severe lack of discipline everywhere.

"If anything, the more you take care of the little stuff, the less likely someone will be able to pull off a big attack," says Schmidt. "I see it all the time. Companies are always pushing, 'Let's just open this one little port.' Then next thing you know they want another port, and another. And that leads to all these vulnerabilities, which turn into little brush fires. No one draws the line and says no. Instead of creating a culture of security, we're often creating a culture of getting around security."

The way technology is designedbased on open architecturesonly fosters that kind of shortcut culture.