I would add:
If an Information Element is required more than once in Template,
the different occurrences of this Information Element SHOULD follow
the logical order of their treatments by the Metering Process.
For example, if a selected packet goes through two hash functions,

and if the two hash values are sent within a single template, the
first occurrence of the hash value should belong to the first hash
function in the Metering Process. For example, when exporting the
two source IP addresses of an IPv4 in IPv4 packets, the first
sourceIPv4Address Information Element occurrence should be the IPv4
address of the outer header, while the second occurrence should be
the inner header one.

In section 9 "The Collecting Process's Side", at the very end, I would
add:

The Collector MUST support the use of Templates containing multiple occurrences of
the similar Information Elements.

Isn't that quite exactly what I proposed in my mail of 2005/08/04? It
was a long mail I admit, so maybe nobody read it. ;-) That mail is
attached, if somebody now is interested to read it.

I would say that the problem your proposal is trying to solve and the problem
that this proposal is trying to solve are subtely different.
You are trying to solve the issue of building a flow record with element
captured from different observation points. The problem with using the
same I.E.s for the same field but seen in different places is that you
can't tell which entry in the record applies to which field. Even adding
an order dependence doesn't solve this issue because there is no
requirement to put all versions of the element in the record. This is
detailed in your proposal but your solution is quite a complex addition
to the protocol.
There is a seperate problem when it comes to having the same element
be applicable more than once at a single observation point. Actually
there are two similar problems:
1. The same element applies more than once but you want to be able to
selectively report only some of them. i.e. MPLS labels. In this
case we have to use seperate I.E.s
2. The same element applies more than once but you need to report them
all.
This is the issue Benoit's proposal is an attempt to solve. For example
a classification routines may classify a packet as classes 1, 10 and 15
similtaneously because these classes have independent properties. In
this example order may not seem important, but if you want to match the
classes with some statistics about each class then making it order
dependent allows the order of each to be alligned.
Andrew

But doesn't make this all the post-/pre- I.E. discussion obsolete?
Wouldn't it be enough just to export two I.E. of the same type? Of
course, with the post-I.E. you have more semantics. But wouldn't it be
better to give this semantics by other means?

But there are certain amiguities. For instance in your example (IP in
IP), if you also export the packet size (for single packet reports) or
in octet counters in general, is it corresponding to the outer or the
inner packet? That's why I also proposed a kind of separator I.E. with
length 0, which separates different groups of I.E. in the template. And
in each group, every I.E. MUST appear only one time. That way you always
know which I.E. belong together.

That way it's clear, that the counter corresponds to the first packet
state size, which in the IP in IP scenario is the outer packet size.

Jürgen stated to my proposal, that you try to avoid more complex
concepts if possible. But as it seems, it's getting more complex anyway.

Cheers,
Sven
------------------------------------------------------------------------
Subject:
[ipfix-info] Proposal for IFPIX observations at middleboxes
From:
Sven Anderson <Sven.Anderson@CS.Uni-Goettingen.DE>
Date:
Thu, 04 Aug 2005 19:05:42 +0200
To:
ipfix-info@net.doit.wisc.edu
To:
ipfix-info@net.doit.wisc.edu
Hi all,
I will try to explain again my idea for reporting flows from middleboxes
as clear and short(?) as possible:
When IP packets travel through a middlebox, their properties may change.
Examples would be TTL, IP adresses (NAT), DSCP or even fragmentation or
replication (multicasting). That means, that if an Observation Domain
includes several Observation Points, and the same packet passes more than
one of these Observation Points, some properties can be ambiguous in this
Observation Domain.
You could solve this by saying, that every propery of a flow-report
has to derive from the same observation point. But this would be
restrictive. Sometimes it is desirable, that you can classify a single
Flow by properties, that derive from _different_ Observation Points. For
example a flow-definition that includes the source IP address before and
after a NAT, or the DSCP at three different Observation Points (before
ingress linecard, after ingress linecard, after egress linecard).
To realise this, you need a way to use certain properties more than one
time in a flow-template, to correspond to the different states at the
different Observation Points. One way to do this, is the introduction of
additional post- I.E.s, which then correspond to the first and the last
Observation Point in the Observation Domain. But this is a restriction to
two special Observation Points, and the second example from above is not
solvable with this approach.
A more general solution would be to allow the use of the same I.E. more
than one time in the same template. The order of the multiple I.E.s
corresponds to the different observations as the packet travels through
the middlebox. The problem here is, that you cannot tell then, to which of
the different observations the _singular_ I.E.s belong to.
To solve this, you need a possibility to build goups of I.E.s in the
Template: The I.E. values in one "Observation Group" always derive from
the same Observation Point (for each packet!). The Observation Groups are
ordered accordingly to the sequence of Observation Points the packet is
passing. Of course there are also fields which don't depend on the
Observation Point and can be reported in any Observation Group, like
ingressInterface (not egress!), as it is always the same, no matter where
in the middlebox the packet is observed. (You may say, that these fields
_have_ to be reported in the first group then, if this is an advantage.)
Important to note is, that packets of a Flow defined by (for example) two
Observation Groups don't necessarily pass the same sequence of two
Observation Points. They just share the same Flow poperties at the their
first and second Observation Points respectively. To make sure, that all
packets passed the same sequence of Observation Points you have to include
an (yet to be defined) I.E. "observationPointID" as a Flow Key in each
Observation Group.
BTW.: It's possible that a Flow has the same observationPointID in
different Observation Groups. For example if your Flow contains two
Observation Groups, corresponding to Observation Points at the ingress and
egress interface, you could have Flows where ingress and egress interface
and therefore the observationPointID is the same (e.g. after some NAT).
The next question is: what is a good way to define these groups? Here are
just two ideas:
- my first idea, which I descibed in an former mail, was to define
"Combined Templates", which are a ordered list of other Templates. Each
Template in the Combined Template represents an Observation Group. The
reports for Combined Templates are just normal reports with all the Fields
of all the listed Templates concatenated. The disadvantage is, that a
change of IPFIX-PROTO is necessary.
- another idea is to define a special separator I.E. with length 0, like
"newObservationGroup". This pseudo-field separates the Fields in the
Template into different Observation Groups. In the reports they don't
appear, but the collector has the template and knows where to split. Here

no change in IPFIX-PROTO is necessary. (Jeroen Massar had a very similar
idea)

I think this concept is a superset of the proposals of Jürgen and Benoit.
Instead of using post-I.E.s Benoit could use Templates like this (using
the second grouping-mechanism):
<sourceIPv4Address>
[...different Fields...]
<octetDeltaCount>
<DSCP>
*<newObservationGroup>*
<DSCP>
The DSCP field in the second ObservationGroup is the same as his postDSCP
field.
Jürgen would do it maybe like this:
<destinationIPv4Address>
*<newObservationGroup>*
<sourceIPv4Address>
<destinationIPv4Address>
[...different Fields...]
<octetDeltaCount>
instead of using an preDestinationIPv4Address Field.
Why do I like this concept:
- it includes all the possibilities of the other solutions.
- you can have as many Observation Groups as you want. (I have an
VPN-Relay here, where I will need 3 Observation Groups, which is not
possible with the pre-/post- solutions.)
- it is always clear, which fields derive from the same Observation Points
(same packet state). This is especially true for the counter Fields! You
can have even the same counter in different Observation Groups, if you
expect the packet size to be changed.
- you don't need any post-/pre- variants of the I.E.s You just need the
newObservationGroup I.E.. The observationPointID is a good idea anyway, I
think.
- you don't need complicated semantics, you just report an ordered
sequence of packet states. (And that's all you know, in fact.)
I'm not really sure, but I think, that the in- and out- counters also get
obsolete then. Wouldn't it be the same as having a counter in the first
and last Observation Group?
A side note: I think a packet-altering instance - like a NAT process -
which is reporting information to the metering process, should always be
seen as _two_ observation points, one before and one after the change.
Anyway, this is my "work in progress" idea. I hope I could present this
quite complex object in an understandable manner. Now tell me, why it is a
bad idea. :-)
Cheers,
Sven

--
to unsubscribe send a message to psamp-request@ops.ietf.org with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/psamp/>