As I hope most of you know, Carolyn Vella, my significantly better half, and I have a new book out: Competitive Intelligence Rescue: Getting It Right. We were very fortunate to get some pre-release reviews, which said, in part

“In this book you get to listen in on real conversations and solve real issues.”

“Long experienced in competitive intelligence, Vella and McGonagle provide insightful lessons for those who need intelligence to compete, profit, and succeed.”

“This is an essential addition to every librarian’s shelf.”

“Anyone involved in CI, or trying to rescue their CI program, will find Vella’s and McGonagle’s book informative, insightful, practical, and executable.”

There are several undercurrents emerging about competitive intelligence. I see them as including the feeling that competitive intelligence is (A) easy and/or (B) not really needed because it is obvious/lacking insights. Now to be fair, comments like this are usually coming from people who either (A) don’t actually do competitive intelligence or (B) don’t use competitive intelligence, but they are still out there.

Why such negativism?

One reason may be the Internet. Now, I know we blame everything from childhood obesity to dangerous driving on the Internet, but there may be some connection here. The Internet, expanding as it does every day, particularly by adding governmental records and personal data on rapidly increasing basis, has made it easier to develop vast amounts of raw data in a relatively short time. This has several important consequences.

One consequence is that the time we have to do our tasks, including CI, has not increased, but when the amount of data we receive has increased, we are spending, therefore, less time on analysis. That is a real problem when research indicates that, given the opportunity, more time should be spent on analysis and less on data gathering to produce actionable CI.

A second consequence is that, since people doing CI, either full-time or part-time, are gathering lots of data, they feel that it is necessary to do something with all that data. The result: the daily/weekly briefing/newsletter/report. In other words, an internal daily newspaper. As we can see the fate of daily newspapers in the Internet age has not been particularly promising, why should we think that the fate would be any better for CI activities that rely on these fast, and consequently analysis light/free, vehicles? It is always been my professional position that a full-time CI unit should use such vehicles sparingly; frankly, use them only as a way to drag the readers into using the CI team for more advanced, more valuable research.

The third consequence is that past CI efforts, in the minds of at least some end-users, have created the illusion that they’ve already collected “everything there is to know”. In other words by consuming the harvest of low hanging fruit, our CI consumers filled up on junk food. (Yes I know it’s a mixed metaphor.)

Whether you are consuming CI, providing your own CI, or just learning more about CI, be sensitive to this: the availability of too much raw data is impeding the provision of even minimally important amounts of analysis. In addition, quick may have already replaced good. (Remember those quaint signs that say “You can have any 2 of these 3: quick, good, or cheap.”)

When I was talking to a friend about this, someone who had been a teacher who was reflecting on the research he saw from college students, he made an observation which I will credit to him (without using his name at his request) – if research and the related analysis was likened to the preparation of a meal, then Internet-based research produces the equivalent of a TV dinner. To the naked eye, it looks like a meal, but for the consumer, it lacks the taste, nutrition, and visual appeal of a well-sourced, and well-prepared fine meal.

By now, it should be clear that having access to the Internet is not the answer to anything. It is akin to saying that having access to the Pennsylvania Turnpike means you can drive safely anywhere. The issue is where are you trying to get and how? Or, for the Internet, what data are you seeking to locate to then analyze?

A recent article on the current state of information overload in Time rightly observed “[F]or the most part, answers are good to know. You just have to ask the right questions.[1]”

Now, I am not saying that doing some basic Internet research at the beginning of your own competitive intelligence project is not a good idea. Actually, it is a very good place to start. But it is only that, a starting point, not a destination. Use it to gain a general idea of your target(s), the competitive environment, and what people/organizations/resources with access/experience/knowledge could provide further data. But always apply your analysis and analytical tools to what you find, including a determination of how reliable the source for data is, as well as deciding how likely it is that the data you found there is correct (caution: these are 2 different issues).

But, in spite of all of our experience in CI, we still hear “Why do we need someone doing CI? Just have him/her look it up on the Internet!” Of, course, that is based on the erroneous inference that the Internet is a vast, indexed, and juried reference work, easy to use and highly reliable, rather than a vast trackless wilderness filled with information, misinformation, dated information, disinformation, and outright nonsense.

As this same article rightly put it, “[I]nformation is not knowledge or wisdom, and data can mislead.” Be careful out there.

Continuing on this occasional topic, yesterday I was reading a profile of the company that was touting its early warning system. The company described how it’s set up a team to monitor and deal with “megatrends”, trends at least 5 years out that could have a material impact on the business. To summarize it, a senior officer was selected to identify these overarching trends and their potential impact on the business. Following the definition of the trends, a team analyzed the impact of the trends on the company’s strategic planning and how it would affect its efforts in the future. The entire company’s planning team was then taken through a series of workshops on the process to integrate the megatrends into their planning, to identify potential successes and the routes to get there.

So?

There are at least three things wrong with this, maybe more:

First is the process of identifying the so-called “megatrends”. Without more on the process, it seems very introverted and therefore likely to miss the trends that really matter, because they are almost by definition the trends that are not being noticed by the company at present.

Second, this process is relatively static, in that it appears that the identification of the “megatrends” is a one time or, optimistically, an annual effort. However, megatrends do not show up and grow on schedule. For a case example of that, witness the relatively fast creative destruction of the American newspaper and magazine industry by the Internet.

Third the whole process seems to be premised on the fact that if the company can identify megatrends then it can identify new opportunities for the future. It totally ignores the fact that some megatrends may well provide only threats not opportunities.

The first stages of the early process are among the most important, and require not a top-down team, but an integrated effort using employees from across the company as well as consultation with outside experts. This example is one that will not go well.

Earlier, I said that the Internet did not create competitive intelligence. Rather, I suspect competitive intelligence has been able to grow due to the Internet, although in some ways, the existence of the Internet may be poisonous to competitive intelligence.

At its early stages, when CI specialists were digging into the backgrounds of publicly traded corporations, they became expert at the relatively arcane subject of US Securities and Exchange Commission filings. They also learned where such filings were housed as well is how to order them quickly. That meant that the turnaround time for analysis based on these filings could be a matter of weeks.

With the advent of what were then called online databases, people in competitive intelligence had faster access to these documents; the major delay was waiting for them to be put online. Initially, most of these documents were not quickly obtainable in that form, as the original text transmissions were at the order of 300 baud. To understand that speed, imagine you are reading the annual report of a corporation in the form of a crawl under your favorite news program. It was painfully slow.

As the Internet made access to historical, particular public documents easier, CI specialists hit their stride. Having spent time learning how to exploit the information, the increase in the speed of obtaining that information enabled them to produce refined analyses in a very short time. However, too often end users just accept the contents of the SEC filings without waiting for insightful analysis.

Unfortunately, the myth of the Internet has not helped the continuing development of CI. Too many people believe that “everything is on the Internet”, which is patent nonsense. The most important thing that CI can provide is a window on the future, not just a look back at the (recent) past. In the words of an old Microsoft commercial, they want to know “where do you want to go tomorrow?” Looking backwards does not answer that question.

As the Internet provides more real-time linkage to real people, its impact on CI is beginning to change, in directions as yet unknown. The willingness of people to talk about themselves and their jobs on social and business networking sites can be a terrific assist to CI specialists, particularly those seeking interview targets. However, as businesses and other enterprises realize the importance of social media, they will probably begin to exercise control over what their employees, and later their suppliers and vendors, say and where they say it.

I had an interesting discussion today with a colleague at a private firm. We were discussing some research issues and reflected on the still common (and dangerous) belief by many in business that “everything we need is on the Internet, so you just have to Google it”.

There are so many things wrong with that statement that I almost don’t know where to begin.

First, with all deference to Google, and I have the greatest respect for the company and its products, a Google search cannot find “everything” on the Internet. Many pages have content, the so-called hidden web that cannot be indexed by search engines. That includes everything from magazines whose archives are password restricted to subscribers to the records of local governments, where an individual has to sign in to access its records.

As for “on the Internet”, the continuing explosive growth of the Internet has actually made some kinds of research more difficult, not easier. It used to be that one could feel relatively comfortable doing a fairly simple, broad search under a target’s name on one of the many search engines and come up with what was available on the target. Now that is just impossible. Before you even begin your research, you have to narrow it down, or otherwise you will be faced with the daunting prospect of going through all 546,317 “hits”. Frankly, a half-million hits is often worse than none at all. Searching the Internet is more like fishing: you first have to make at least an initial determination of where you going to start based on what you want to find – you cannot just go out and start fishing.

As for “everything” actually being on the Internet, nothing could be further from the truth. For those of us in competitive intelligence, the most important things missing on the Internet are what has not yet happened, what people are thinking (but have not said), what they are planning, and where they’re going tomorrow. The best the Internet can provide is access to hints of these things and possibly leads to be exploited by interviews or even elicitation.

So don’t let others get fooled (and don’t fool yourself) about the Internet. It is incredibly valuable. But like the latest pharmaceutical or surgical procedure, it merely represents an advance, but not a solution to all intelligence problems.

Sorry to be so cynical, but in competitive intelligence, you learn that being honest is the first step towards being a skilled intelligence analyst and a good intelligence consumer.