Sven,
[sorry for the delay in replying to you]
Even if I agree that we will certainly have to solve the issue of
exporting structured records in clean way in a future IPFIX version,
even if I agree that your proposal is powerful, I was aiming for a
quick editorial change in [IPFIX-PROTO]. Let's not forget that the
IPFIX drafts are right now under IESG review.
Regards, Benoit.
Benoit Claise wrote:
I would add:
If an Information Element is required more than once in Template,
the different occurrences of this Information Element SHOULD follow
the logical order of their treatments by the Metering Process.
For example, if a selected packet goes through two hash functions,
and if the two hash values are sent within a single template, the
first occurrence of the hash value should belong to the first hash
function in the Metering Process. For example, when exporting the
two source IP addresses of an IPv4 in IPv4 packets, the first
sourceIPv4Address Information Element occurrence should be the IPv4
address of the outer header, while the second occurrence should be
the inner header one.
In section 9 "The Collecting Process's Side", at the very end, I would
add:
The Collector MUST support the use of Templates containing
multiple occurrences of
the similar Information Elements.
Isn't that quite exactly what I proposed in my mail of 2005/08/04? It
was a long mail I admit, so maybe nobody read it. ;-) That mail is
attached, if somebody now is interested to read it.
But doesn't make this all the post-/pre- I.E. discussion obsolete?
Wouldn't it be enough just to export two I.E. of the same type? Of
course, with the post-I.E. you have more semantics. But wouldn't it be
better to give this semantics by other means?
But there are certain amiguities. For instance in your example (IP in
IP), if you also export the packet size (for single packet reports) or
in octet counters in general, is it corresponding to the outer or the
inner packet? That's why I also proposed a kind of separator I.E. with
length 0, which separates different groups of I.E. in the template.
And in each group, every I.E. MUST appear only one time. That way you
always know which I.E. belong together.
Template example:
<sourceIPv4Address>
<octetDeltaCount>
*<newObservationGroup>* (pseudo I.E. with length 0)
<sourceIPv4Address>
That way it's clear, that the counter corresponds to the first packet
state size, which in the IP in IP scenario is the outer packet size.
Jürgen stated to my proposal, that you try to avoid more complex
concepts if possible. But as it seems, it's getting more complex
anyway.
Cheers,
Sven
Hi all,
I will try to explain again my idea for reporting flows from
middleboxes
as clear and short(?) as possible:
When IP packets travel through a middlebox, their properties may
change.
Examples would be TTL, IP adresses (NAT), DSCP or even fragmentation or
replication (multicasting). That means, that if an Observation Domain
includes several Observation Points, and the same packet passes more
than
one of these Observation Points, some properties can be ambiguous in
this
Observation Domain.
You could solve this by saying, that every propery of a flow-report
has to derive from the same observation point. But this would be
restrictive. Sometimes it is desirable, that you can classify a single
Flow by properties, that derive from _different_ Observation Points.
For
example a flow-definition that includes the source IP address before
and
after a NAT, or the DSCP at three different Observation Points (before
ingress linecard, after ingress linecard, after egress linecard).
To realise this, you need a way to use certain properties more than one
time in a flow-template, to correspond to the different states at the
different Observation Points. One way to do this, is the introduction
of
additional post- I.E.s, which then correspond to the first and the last
Observation Point in the Observation Domain. But this is a restriction
to
two special Observation Points, and the second example from above is
not
solvable with this approach.
A more general solution would be to allow the use of the same I.E. more
than one time in the same template. The order of the multiple I.E.s
corresponds to the different observations as the packet travels through
the middlebox. The problem here is, that you cannot tell then, to which
of
the different observations the _singular_ I.E.s belong to.
To solve this, you need a possibility to build goups of I.E.s in the
Template: The I.E. values in one "Observation Group" always derive from
the same Observation Point (for each packet!). The Observation Groups
are
ordered accordingly to the sequence of Observation Points the packet is
passing. Of course there are also fields which don't depend on the
Observation Point and can be reported in any Observation Group, like
ingressInterface (not egress!), as it is always the same, no matter
where
in the middlebox the packet is observed. (You may say, that these
fields
_have_ to be reported in the first group then, if this is an
advantage.)
Important to note is, that packets of a Flow defined by (for example)
two
Observation Groups don't necessarily pass the same sequence of two
Observation Points. They just share the same Flow poperties at the
their
first and second Observation Points respectively. To make sure, that
all
packets passed the same sequence of Observation Points you have to
include
an (yet to be defined) I.E. "observationPointID" as a Flow Key in each
Observation Group.
BTW.: It's possible that a Flow has the same observationPointID in
different Observation Groups. For example if your Flow contains two
Observation Groups, corresponding to Observation Points at the ingress
and
egress interface, you could have Flows where ingress and egress
interface
and therefore the observationPointID is the same (e.g. after some NAT).
The next question is: what is a good way to define these groups? Here
are
just two ideas:
- my first idea, which I descibed in an former mail, was to define
"Combined Templates", which are a ordered list of other Templates. Each
Template in the Combined Template represents an Observation Group. The
reports for Combined Templates are just normal reports with all the
Fields
of all the listed Templates concatenated. The disadvantage is, that a
change of IPFIX-PROTO is necessary.
- another idea is to define a special separator I.E. with length 0,
like
"newObservationGroup". This pseudo-field separates the Fields in the
Template into different Observation Groups. In the reports they don't
appear, but the collector has the template and knows where to split.
Here
no change in IPFIX-PROTO is necessary. (Jeroen Massar had a very
similar idea)
I think this concept is a superset of the proposals of Jürgen and
Benoit.
Instead of using post-I.E.s Benoit could use Templates like this (using
the second grouping-mechanism):
<sourceIPv4Address>
[...different Fields...]
<octetDeltaCount>
<DSCP>
*<newObservationGroup>*
<DSCP>
The DSCP field in the second ObservationGroup is the same as his
postDSCP
field.
Jürgen would do it maybe like this:
<destinationIPv4Address>
*<newObservationGroup>*
<sourceIPv4Address>
<destinationIPv4Address>
[...different Fields...]
<octetDeltaCount>
instead of using an preDestinationIPv4Address Field.
Why do I like this concept:
- it includes all the possibilities of the other solutions.
- you can have as many Observation Groups as you want. (I have an
VPN-Relay here, where I will need 3 Observation Groups, which is not
possible with the pre-/post- solutions.)
- it is always clear, which fields derive from the same Observation
Points
(same packet state). This is especially true for the counter Fields!
You
can have even the same counter in different Observation Groups, if you
expect the packet size to be changed.
- you don't need any post-/pre- variants of the I.E.s You just need the
newObservationGroup I.E.. The observationPointID is a good idea anyway,
I
think.
- you don't need complicated semantics, you just report an ordered
sequence of packet states. (And that's all you know, in fact.)
I'm not really sure, but I think, that the in- and out- counters also
get
obsolete then. Wouldn't it be the same as having a counter in the first
and last Observation Group?
A side note: I think a packet-altering instance - like a NAT process -
which is reporting information to the metering process, should always
be
seen as _two_ observation points, one before and one after the change.
Anyway, this is my "work in progress" idea. I hope I could present this
quite complex object in an understandable manner. Now tell me, why it
is a
bad idea. :-)
Cheers,
Sven
|