Re: passive packet measurement

[Matt Grossglauser <mgross@research.att.com>]

Hi Carter,

Thanks for your comments! Before responding to specific
items, let me outline some commonalities and key 
differences between your draft and the psamp draft.

My impression is that the goals we have in mind with
psamp are somewhat broader than what you had in mind,
in that we view psamp as enabling both "macroscopic"
(such as obtaining a breakdown of relative contributions
of tcp/udp port numbers over your entire traffic) and 
"microscopic" measurement tasks (such as capturing icmp
packets destined to a certain address). This underlines
the importance of sampling: most macroscopic measurement
tasks do not require packet headers of all the relevant
packets, but a comparatively small, random subset is
representative for the entire traffic. We want to exploit
this to reduce measurement overhead, hence the central
role of sampling in our proposal.

We seem to agree on the following design goals:
- must be able to select which packet fields are extracted,
  to minimize overhead
- simplify measurement device by minimizing local 
  processing; ship measurement records as quickly as
  possible

Below are some thoughts on the questions you have raised.

Matt



> I don't believe that you made the case for packet
> sampling.  While it maybe useful, packet selection
> techniques that are purely filter based are also
> perfectly adequate for controlling the load that a
> packet monitor may encounter.  A facility that
> measures and reports only on ICMP packets is more
> than legitimate, its actually useful, and sampling
> isn't appropriate in this case.

As described above, I think filtering and sampling fulfill
different roles. Filtering really helps to "drill down" 
into the traffic to zero in on a specific problem. Sampling
is important whenever we would like to compute aggregate
statistics over a large number of packets.
 
> Your sampling types and descriptors are a bit primitive.
> You could easily beef that up by having some real
> statistical sampling methods described.

I'm not sure I understand your comment. What statistical
sampling methods do you have in mind? If you are referring
to post-processing methods (e.g., computing estimators,
their confidence intervals, etc.), then we do not think 
this should be done in the device, but rather in the
backend.
 
> A framework for packet measurement should handle
> full wire-line measurement as well as packet selection
> techniques.  I don't have a problem with having a
> sampling technique of "all packets".
> 
> It seems that your describing an implementation, not a
> framework.  No sense in calling it a framework unless you
> plan on really building a framework.  If you're building
> a framework, then your section 3. Measurement Functionality
> shouldn't read like an implementation, but rather a design.
> So, I should be able to do filtering any way I want, it
> shouldn't have to be tied into the Hashing function.  

We do intend it as a framework, and have tried to avoid
implementation details. The sampling based on hash functions
serves a very specific purpose: to be able to make identical
sampling decisions for packets at different nodes, and then
match up these samples to reconstruct what happened to these
packets. Examples include IP traceback, where we would like
to figure out at which ingress nodes attack packets have
entered the measurement domain, and trajectory sampling,
a method to obtain the full paths of the sampled packets,
which is useful for a range of troubleshooting and traffic
engineering tasks.

> Actually
> if I'm really thinking about hardware and performance,
> I'm going to want to Filter before I Hash.  Possibly
> there is an advantage to doing some Filtering in conjunction
> with the Hashing function, but it can't be the only way
> to do filtering.  I believe that a framework should identify
> the issues, and suggest a possible implementation, as an
> example.

Absolutely: in which order you filter and hash does not matter;
the same set of packets results. This is therefore a choice
left up to the implementer.
 
> You shouldn't allow state variables to be supplied to the
> record after any period of delay, as there is no assurance that
> the variables will be applicable.  If the value isn't the
> actual value that the network element used to dispatch/process/
> whatever the packet, then you risk jeopardizing the credibility
> of the entire mechanism.

Good point.

to unsubscribe send a message to psamp-request@ops.ietf.org with
the word 'unsubscribe' in a single line as the message text body.