Sunday, July 27, 2014

Top-Down vs. Bottom-Up

This is a post I’ve wanted to write for a
while. I have been talking about the
“top-down” and “bottom-up” approaches to BES Cyber System identification since last
fall, but – as with everything else I write – my views have evolved since
that time. So this post is hopefully my
definitive take on this topic, and I can get on to others I want to write[i].

Before I start, I want to point out that
there is another use of the words “top-down” and “bottom-up” in regards to CIP
version 5. This is exemplified by Dr.
Joe Baugh of WECC, who has used (in the CIP-002 presentation found at this
link, slide 20) the words to mean how one approaches CIP-002-5 R1 in the first
place: either by evaluating an inventory of BES assets (substations, etc)
against the criteria in Attachment 1 (“top-down”), or by evaluating an inventory
of BES cyber assets against those same criteria (“bottom-up”).

Joe says either approach is acceptable, but
entities will find the latter to be much more burdensome, since they will have
to first evaluate all of their cyber
assets as BES Cyber Assets. This means they’ll
have to have a complete inventory of High, Medium and Low cyber assets (since
at this point they don’t know how they’re classified). As a result, just about all entities are
using what he calls the “top-down” approach.

This is not how I’m using the terms. I have been using them – since last October –
in a much narrower sense: they are two different approaches to identifying BES
Cyber Systems at a Medium or High-impact asset/Facility. In the bottom-up approach as I define it, the
entity starts by applying the definition of BES Cyber Asset to each of its
cyber assets. The next step is to
combine those cyber assets into BES Cyber Systems, with no hard-and-fast rule
about how to do so – other than the rule that every BCA needs to be part of at
least one BCS.

What I call the top-down approach is to start
with the BES Reliability Operating Services (BROS), discussed in the Guidance
and Technical Basis section of CIP-002-5.
The entity needs to determine which BROS apply to the asset being evaluated
(e.g. in a generating station, it is likely that the “Controlling Frequency”
BROS will apply, while the “Controlling Voltage” BROS may not). Then it needs to identify the systems that
support one or more BROS at the asset (e.g. the DCS in the generating
station). These will then be the BES
Cyber Systems (they need to be further broken down into their component BES
Cyber Assets for completeness’ sake, but the unit of compliance is BCS, not
BCA).

When I first realized that both of these
approaches were viable, I thought it was an either/or question: which one was
required by CIP-002-5 R1? And the answer
to that was clear: the bottom-up approach is actually built into the
requirement, whereas the top-down approach only comes from reading the Guidance,
which is of course not part of the standard for compliance purposes.

When I say the bottom-up approach is built
in, I mean it is implicitly built in.
One of the many endearing features of CIP-002-5 R1 is that it never
explicitly orders the entity to identify
their BES Cyber Systems in the first place; it starts right out by telling you
to classify your BCS[ii]. But obviously you can’t classify what you
haven’t identified, so first you have to figure out how to identify BCS. And since the BCS definition refers to BCAs,
you logically have to start with BCAs – then group them into BCS. So the bottom-up approach is the only one
that you can derive from simply reading the requirement.

However, the BROS are discussed at length in
the Guidance section, and I know that many auditors consider the top-down
approach to be the approach for BCS
identification. I actually know of only
two auditor presentations that have squarely addressed the BCS identification
issue. Kevin Perry of SPP, in his
webinar on CIP v5 last February, only talked about the top-down approach. On the other hand, Joe Baugh of WECC, in his
presentation linked above (starting on slide 40), only discusses what I call
the bottom-up approach, and never even mentions the BROS.

Does this mean that SPP entities should use
top-down, WECC entities should use bottom-up, and all other entities are simply
SOL[iii]? At the moment, I’d say yes, but as I’ve said
many times, I’m hoping there will be a comprehensive re-interpretation of
CIP-002-5, almost certainly by NERC. I
would think that re-interpretation would address this issue, along with the
many other issues
I’ve been writing about for more than a year.
And if this doesn’t happen? Well,
we can all take comfort in the fact that McDonald’s is still hiring.

And what do I believe is the right
approach? I have been saying for a while
(e.g. in this
article in the June issue of Power
magazine) that it is best to use both approaches, one as a check on the
other. Practically, I’ve been saying the
entity needs to start with the top-down approach, which of course yields a list
of BES Cyber Systems. However, the
entity needs to then run the bottom-up approach, going at least as far as the
step of identifying BES Cyber Assets.
Then the entity needs to confirm that each BCA is contained in one or
more BCS.

I was fairly happy with that idea until I had
lunch recently with the CIP manager on the generation side of a large IOU. He pointed out to me that, in large
generating plants (those over 1500MW and subject to criterion 2.1), this will
place a big burden on the entity. It
does this because the bottom-up approach requires a complete inventory of cyber
assets, and large plants can literally have thousands of cyber assets –
“programmable electronic devices”.

You may say at this point (especially if
you’re on the Transmission side of the house), “Well that’s too bad, but
CIP-002-5 R1 clearly requires the entity to consider every cyber asset at a
Medium plant against the definition of BES Cyber Asset; this can only be done
if there is a complete inventory.”
However, this misses one important point (and I can say that I missed it
until my friend reminded me): not all BES Cyber Assets at a criterion 2.1 plant
will be Medium BCAs. Those that don’t affect
1500MWwill be Lows[iv], and
CIP-002-5 says in two places that an inventory of Low cyber assets isn’t
required[v].

Does this mean that, at least in a criterion
2.1 plant, the bottom-up approach really isn’t feasible? In general, I’d say yes. The entity first needs to do the top-down approach
to produce the list of BCS. It then
needs to determine which if any of these BCS affect[vi]
1500MW. Finally, the entity needs to
identify the component BES Cyber Assets of each BCS - as well as those cyber
assets that are networked with one or more BCAs, since they will be Protected
Cyber Assets. Any cyber assets that
aren’t identified as BCA or PCA by these steps will be Low impact and don’t
need to be inventoried.

However, I can think of an example where the
top-down approach clearly isn’t enough in a 1500MW+ plant. At SPP’s BES Cyber System identification exercise
I attended in Dallas in February, they of course advocated the top-down
approach. But they also pointed out the
case of an environmental system in a large plant that is designed to trip the
plant if there is an environmental excursion of more than ten minutes. Since environmental protection isn’t one of
the BROS, this system wouldn’t be identified by the top-down approach; yet it
clearly would be identified in the
bottom-up approach, since it has an effect in under 15 minutes. SPP clearly expects such systems to be
included in the entity’s list of Medium impact BES Cyber Systems.

Therefore, my modified rule for the criterion
2.1 generating stations is: a) Use the top-down approach to get your list of
BCS; and b) Augment that list with any other systems that may not fulfill a
BROS but clearly can have a fifteen-minute impact. In any case, you shouldn’t have to inventory
all of your cyber assets at the plant, as long as you can show that cyber
assets not inventoried are all on separate networks from those that contain
Medium BCS.

So at least in a criterion 2.1 generating
station, the top-down approach (with the slight modification just described) is
the only feasible one. Does that mean
it’s the only feasible approach across the board – for substations, control
centers, etc?

I say the answer to this is no, for two
reasons. The first reason – the
weightier from a “legal” point of view – is that IMHO criterion 2.1 is the only
one that can lead to both Medium and Low BCA/BCS at a single asset/Facility.[vii] If there can be only one classification of
BCS, then every cyber asset at a Medium asset/Facility will need to be
considered as a BCA, meaning it will need to be inventoried. The second reason is that control centers and
substations (and smaller generating stations) have much more manageable numbers
of cyber assets; at them, I don’t believe it’s a great burden to do a complete
inventory.

So here is my final “ruling” on the question
of the top-down vs bottom-up approaches to BES Cyber System identification:

At criterion 2.1
plants, use the modified top-down approach I outlined above.

At all other High
or Medium impact assets/Facilities, combine the two approaches so one
checks the other.

Of course, as with everything else having to
do with CIP v5, your friendly local Regional Entity auditor will have to decide
this question for you (and hopefully he/she will get some guidance from NERC,
as I think I’ve already said ten times in this post).

The views and opinions expressed here are my
own and don’t necessarily represent the views or opinions of Honeywell.

[i]
What I’m hoping will come soon is a new post in which I “rewrite” CIP-002-5,
the way I think it should have been written in the first place. Of course, this is for heuristic value only,
since there is no longer any chance that CIP-002-5 can actually be changed
(unlike the last
time I rewrote CIP-002-5 - in my comments to FERC last June - when I really
did hope that FERC would direct NERC to rewrite the standard. Of course, that didn’t happen). But since I still think that somebody –
almost certainly NERC – needs to take an extraordinary action and come out with
a comprehensive re-interpretation of CIP-002-5, by rewriting the standard now
I’ll at least provide my view on what that re-interpretation should look like.

[ii]
You may find this statement confusing, since CIP-002-5 R1.1 and R1.2 clearly
order the entity to “Identify” BCS, not classify them. But since this is only done in the context of
classifying BCS (i.e. R1.1 is about “identifying” High BCS and R1.2 is about
“identifying” Medium BCS), this is a misuse of the term ‘identify’. There should first be a separate requirement
saying you should identify your BCS (using the top-down or bottom-up
approaches) at High and Medium assets/Facilities, followed by a requirement
something like the current R1 (but much more clearly written), in which you determine
which of those BCS you’ve identified are in fact either High or Medium ones
(I’m skating over a whole bunch of other issues here. I’m hoping I can address them all when I
“rewrite” CIP-002-5 in a future post).

[iii]
I believe this is a NERC Glossary term, but if not I’m sure it’s in Webster’s.

[iv]
This is my interpretation. Criterion 2.1
doesn’t explicitly say that any cyber asset that doesn’t affect 1500MW will be
a Low. This is one of the many areas in
CIP-002-5 R1 and Attachment 1 where some sort of ruling needs to be provided by
NERC.

[v]
It actually says a list of Low BES Cyber Systems isn’t required, but you
couldn’t have that list without having a list of Low BCAs.

[vi]
I’m simplifying by saying “affect”; see criterion 2.1 for the full story.

[vii]
I’m glossing over a whole bunch of considerations here, of course. I’ve addressed them in previous posts, but
when I do my rewrite of CIP-002-5 R1 I hope to address them all in a logical,
consistent fashion.