Last week, Bloomberg Businessweek published an interesting piece on US immigration policy[1]. In it were the details of a dispute between ICE (US Immigration and Customs Enforcement) and the Transactional Records Access Clearinghouse (TRAC), a “small research group at Syracuse University”.

That dispute involves ICE records dealing with deportations and detainers. After providing this data to TRAC under the FOIA (US Freedom Of Information Act), ICE stopped last year.

Its reason? The “records TRAC has asked for don’t exist in the form requested and [ICE] says producing responses would require searching its database, a process [ICE] claims amounts to creating new records”. And ICE says that under FOIA, it is not required to create new records. So more data for TRAC.

In response, a lawyer for TRAC asserts “That just can’t be right, because that’s basically true of all [US] government records right now.” Depending on how the litigation comes out, the FOIA may become increasingly useless to researchers, including those of us in CI.

Maybe this is just one agency out of sync, a one-off situation? No, rather it is just the gradual continuation of a long-term trend that access to government records under Federal and State law is being narrowed for any number of reasons, some good, some not so good. For example, in 2007, Carolyn Vella and I wrote “As the war on terrorism continues, we can expect that access to more and more data currently held by the government will be impacted. Most likely, it will be subject to the typical back and forth of politics.” [2]

“Plus ça change, plus c’est la même chose.” Or,”The more that changes, the more it’s the same thing.“

The AP has reported on yet more efforts to restrict the scope of state open record/FOIA laws and limiting the public’s access to them. [1]

As you can see from the text and from many of my previous blogs on this (just search “FOIA”), the pressures come from several directions, as they have in the past, including:

From police, prosecutors and their supporters, usually aimed at limiting or barring access to body-cam recordings of police-involved activities, or of access to related information, such as emergency/911 call recordings and records.

From regulated businesses, seeking to keep their information, always called “sensitive”, “confidential”, or “proprietary” (without any supporting evidence), from competitors and from the public (AKA their own customers). I guess that they finally figured out that sound CI research procedures call for checking these records. Ah, the price of success!

From governmental units themselves, seeking to remove from the public view more and more of their records, reports, and even proceedings. This is done by changing rules, just flatly declining to provide requested records, and by hindering access through fees and lawsuits. Oddly, the same entities often pat themselves on the back for their transparency. Go figure.

So, keep on using state and federal laws to access records in your CI efforts, but do not expect the same success as you have had in the past. Also, from now on, when reading about open records, and the like, keep in mind the new terms you should use:

Time magazine recently published an interesting piece titled “The real costs of ‘forced transparency’”[1]. The focus of that was on the impact of WikiLeaks’ “disclosures” of US intelligence agencies’ ability to access data in private and government hands, the impact on national governments, and their possible reactions and responses.

I would add to that good analysis two more potential impacts:

These same revelations, on the ease of generating “forced transparency”, may feed the slowly growing trend of the US Government to resist providing online access (and offline access as well) to Freedom Of Information Act (FOIA) documents and data. The rationale offered, valid or not, would probably be along the lines that such access can only assist hacking efforts by opening ‘back doors’.

These same revelations will, I suspect, also cause businesses providing many of the filings that those of us in competitive intelligence are interested in to (a) resist making certain filings citing a fear that their confidential data and documents can no longer be protected, and (b) press for changes to the FOIA (and other laws and regulations) to reduce such sensitive filings.

Of course, if the federal government moves in that direction, I expect that the states will follow – not necessarily quickly but inevitably.

Last week, I posted my take on the future of FOIA (Freedom of Information Act) requests on the US government. What I neglected to do was to discuss the possible impact of relatively recent changes made in the US law by the FOIA Improvement Act of 2016 offered as improving transparency and access. From here, it gets a little technical.

Two of the relevant 2016 changes, as summarized by the US Department of Justice, are as follows:

“Agencies ‘shall withhold information’ under the FOIA ‘only if the agency reasonably foresees that disclosure would harm an interest protected by an exemption’ or ‘disclosure is prohibited by law.’

“Agencies shall ‘consider whether partial disclosure of information is possible whenever the agency determines that a full disclosure of a requested record is not possible.’”[1]

Now, to be fair, these changes would appear to undercut my negative view of the future use of the US FOIA in CI. However, they do not.

As for #1, this does not change the current underlying interpretation of the FOIA that its Exemption 4 of the FOIA still covers

“two distinct categories of information in federal agency records, (1) trade secrets, and (2) information that is (a) commercial or financial, and (b) obtained from a person, and (c) privileged or confidential.”[2]

In other words, anything falling into either category of Exemption 4 cannot be released.

The current interpretations of the scope of Exemption 4 are very broad. With a few exceptions, the federal courts have held that “trade secrets” here have a meaning broader than the usual meaning. That is, it covers “virtually any information that provides a competitive advantage” [3]. That means more is kept from release than is covered by what most of us understand a trade secret to be.

As #2’s “privileged or confidential”, the current standard is not merely whether the “information would customarily [not] be disclosed to the public by the person from whom it was obtained” [4], but rather

“commercial or financial matter is ‘confidential’ for purposes of the exemption if disclosure of the information is likely to have either of the following effects: (1) to impair the Government’s ability to obtain necessary information in the future; or (2) to cause substantial harm to the competitive position of the person from whom the information was obtained” [5].

In other words, what the FOIA protects from disclosure in the context of CI is, in practice, even broader than the plain language of Exemption 4. So, adding language which directs an agency to “foresee” this or “consider” that will not change the overly protective standards now in place.

I have written a lot about accessing government records because that process has, historically, been a very good way to access some competitively useful data on businesses. But that access at the federal level has gradually tightened with each new President.

As we all know, following 9/11, access to government records, particularly federal ones, changed under that relatively new Presidential administration. A lot of data about a lot of facilities, ranging from nuclear energy facilities to dairy farms, was taken off line and otherwise made inaccessible. The stated reason: foiling potential terrorist assaults on “soft” civilian targets by denying them information on the facilities.

Another change occurred starting in 2009, again with a new Presidential administration. At the federal level, some aggregated data became more accessible, but access to other data became more difficult. Let me give just two examples:

First, we saw a pattern of federal officials using non-governmental emails, evidently to foil civilian access to their working emails via FOIA (Freedom Of Information Act) requests.

Second, the response time to FOIA requests at many federal agencies drifted from being measured in weeks to being measured in months – or longer. The reasons included an avalanche of broad, politically-oriented requests, plus apparent diminutions in both resources allocated to FOIA requests as well as in willingness to release any data to the public.

So, what may happen how? Well, when the new President’s complete cabinet is approved, we will have a very pro-business administration. While that is good for business, it may not be as good for those seeking business information from federal filings and records. I can foresee a greater willingness to grant business requests to withhold business data from FOIA demands on the basis that the data is “competitively sensitive”.

On the other hand, the new administration seems to exhibit a different attitude towards the federal bureaucracy. I suspect that could translate into a willingness to be more approving of FOIA requests dealing with lower level and regional agency decision-making, which could involve also releasing involve business data.

The result? Probably faster responses to non-political FOIA requests, but a greater reluctance to release business data. But, in any case, with a new Presidential administration will come some change in FOIA and how it works (or does not work).

I read an interesting article dealing with the Freedom of Information (FOI) law in the United Kingdom[1]. Interestingly, this law only came into force 10 years ago[2] but, unsurprisingly, took nine years to get that point. It deals with accessing information held by “public authorities”, the equivalent of our state, local, and federal government.

Listen to a familiar observation:

“[T]he Freedom of Information law has been met with resistance. Local governments in particular complain that they lack the resources to process a growing volume of requests.”

Just what are those costs? The article does acknowledge it is “impossible to calculate precisely” the cost of compliance with the law, but reports that one estimate, at the local authority level, indicates that the cost to these local authorities is about $46 million per year. Rough estimates at the national level were at over $20 million per year (neither of the surveys appear to have netted out the fees paid by those requesting FOI research and copies). But, just looking at the national level, the author discloses that this represented only 0.0019% of the entire UK budget, or less than taxpayer payments for the annual travels of Prince Andrew.

In the US, we also hear about the costs to governments to reply to these requests, some of which certainly are a silly as this one received in the UK – “How many residents in Sutton own an ostrich?”. But there is more to government opposition, here and in the UK, than costs and silliness.

In the UK, the existence of FOI has had results that, to say the least, make elected and appointed officials uncomfortable:

In one case a local council that was spending thousands of dollars every year sending a delegation to Japan for a flower festival stopped doing it when it realized that a FOI disclose meant they “couldn’t justify doing” this.

One FOI request led to a parliamentary expense scandal in 2009. This scandal resulted in the imprisonment of five members of Parliament and two members of the House of Lords.

A more recent FOI request disclosed that one national minister was spending over $100,000 year on tea and biscuits.

Any wonder why governments, including my own here in the United States, are not keen on expanding these laws or, frankly, even seeing that they are properly adhered to?

Most of the article focuses on the United States, where the Affordable Care Act has produced yet another surprise. Under its provisions, starting 2014, pharmaceutical and device firms will have to disclose payments and other “transfers of value” to physicians. They also must report research fees and doctors’ investment interests.

The article points out that this does not limit these firms’ interactions with doctors, and that certain practices, such as free drug samples, are not covered. It also points out the new law does not regulate transactions with others in the healthcare system, such as hospital administrators, pharmacists and nurses.

The piece does note, at the end, that this law “will also provide [pharmaceutical firms] with exhaustive data on how much they and their competitors spend to market drugs to that doctor.” This coupled, with data already available on each physician’s prescriptions, would presumably allow pharmaceutical firms to determine which marketing results in which prescription sales.

In addition, while the article did not mention this, it enables these same drug firms to track what their competitors are doing and determine their own competitive counter marketing strategies as well.

I don’t use this to point out that this is a particularly effective tool, particularly since it will not even be in place until 2014.

However, it does also provide an object lesson for those of us interested in Competitive Intelligence. That lesson: look closely at any legislative or regulatory initiative that uses the magic words “Sunshine”, “Transparency”, “Open Records”, and the like. With some careful study and practice, they can often be used to develop new and occasionally powerful sources of CI. Of course, as we progress on one front, we often relapse on another. In that respect, I suggest reading one of the depressing recent articles about the continued restrictions imposed by the federal government on document production under the Freedom Of Information Act. In some ways, we might well call that the freedom from information act.

But, as the aphorism goes

“When one door closes, another opens; but we often look so long and so regretfully upon the closed door that we do not see the one which has opened for us.” Alexander Graham Bell.