Re: draft on sampling techniques

[Cristian Estan <cestan@cs.ucsd.edu>]

Tanja Zseby wrote:

> ...
>
> > 1) For content based sampling, you say that we can base the decision 
> > on field values or hash functions. Do we really want all devices in 
> > the network to support functionality as "sample all packets coming 
> > from IP address X"? Is this useful? Is this dangerous? Is this prone 
> > to misconfigurations that would result in too many packets being 
> > selected?
> > We *must* specify what the hash functions are that we base the 
> > sampling decision on and what fields they take as input and also in 
> > section 4.2 how the network operator can "seed" them so that the 
> > system cannot be manipulated/evaded by "the bad guys". This is 
> > because devices from different vendors have to be consistent. I would 
> > incline towards not making the fields the hash is computed on 
> > configurable (i.e. hard code in the standard what (invariant) fields 
> > we hash on). Other opinions? 
>
> I agree that supporting all options might be too much for achieving 
> consistent standard schemes that are supported by all vendors. Maybe 
> we can check what field combinations are useful and define a few (2 or 
> 3 ?)  different hash-based schemes. An example would be maybe schemes 
> with or without parts of the payload or maybe a "lightweight" version 
> with less fields (it first needs to be clear, when taking less fields 
> would really save resources.) 
I think having just one would be even better :-) In the trajectory 
sampling paper Nick and Matt say that the hash should be computed on the 
entire IPv4 packet except 6 bytes of the IP header which can change hop 
by hop: the first two which contain the type of service byte and from 
the 9th to the 12th which contain the TTL and the header checksum. I say 
we either use their definition or ignore bytes after the xth where x has 
to be at least 32 so that we also hash on the TCP ports, sequence number 
and ack number. We would need the input of some people with hardware 
experience to see if we need this cap and if so, what its value should be.

My knee-jerk reaction to how to do this in IPv6 is to exclude the first 
two bytes and bytes 7 and 8 or just 8 and that we should go at least up 
to byte 52. Is there anyone familiar with IPv6 on the list who would 
like to take a closer look at this?

IMO we should *not* include any layer 2 fields into the input to the 
hash function, because the purpose of this scheme is to track packets 
across the whole network. On the other hand it might be useful to 
include some of those fields with the actual sampled packet, but that is 
a different issue.

> > ...
> > 3) I believe this document should also contain the safeguards we need 
> > to put in to avoid floods of sampled packets due to misconfiguration 
> > or attacks on the measurement infrastructure. The simplest thing that 
> > comes to my mind is defining a leaky bucket regulator that would 
> > limit the rate and the burst size for the samples being sent to the 
> > collection station. 
>
>
> Maybe we can add this proposal to the security considerations ? Could 
> you maybe write a few sentences that can be included in the next 
> version of the draft ?
Below. Feel free to modify it wherever you think it's necessary.

The amount of measurement result data produced by devices implementing 
packet sampling is a security concern. Large volumes of data can result 
because of attacks on the measurement infrastructure, misconfigurations, 
bugs in the implementation of the sampling function or the randomness of 
the sampling function. The traffic measurement modules performing the 
sampling of packets should protect against such floods of sampled 
packets. This protection is enforced through safe defaults for the 
sampling parameters and explicit safeguards.

If the sole consumer of sampled packets is a metering process local to 
the device that implements the sampling function, the manufacturer may 
choose not to implement safeguards and not to use safe defaults. If 
there is at least one consumer of traffic reports containing the sampled 
packets outside the device implementing the sampling function and the 
sampling function ensures that the number of sampled packets is no 
larger than a fixed fraction of the packets, the sampling function must 
implement safe defaults but may not implement explicit safeguards. Such 
sampling functions include systematic sampling and n-out-of-N sampling.

The default values of the sampling parameters should be safe. They 
should ensure that the number of sampled packets is no more than 0.01% 
of the packets carried by the link observed or that they do not add up 
to more than 0.01% of the link capacity. These constraints should hold 
over any 30 second time interval. A configuration of the sampling 
function that samples no packets at all is safe.

Explicit safeguards have to limit the average rate at which samples are 
generated (in sampled packets/second) and the maximum burst size (in 
sampled packets). The average rate and the maximum burst size are 
parameters of the safeguard. The default values for these parameters 
must be such that they ensure limits on the number of samples at least 
as strong as the safe defaults above. The implementation can allow the 
operator to change these parameters. The implementation may refuse 
parameters for the safeguard it considers too aggressive. The 
implementation must ensure that the sampling parameters are such that 
under normal operation, the number of sampled packets is within the 
limits of the safeguard. When the safeguard mechanism removes sampled 
packets because they go beyond the allowed limits, it should indicate 
this. The counter for sampled packets we include with each of them 
should be incremented for the sampled packets discarded by the 
safeguard. To implement the safeguard one could use a leaky bucket 
descriptor as in [Cruz87]. Another acceptable implementation of the 
safeguard which does not require timers is given below.

The device implementing the sampling function could use a safeguard 
counter initialized to the maximum burst size. Whenever a packet is 
sampled by the sampling function, if the safeguard counter is above 
zero, it is decremented and the sampled packet is processed further. 
Otherwise, the counter for sampled packets is incremented but the 
sampled packet is not processed. Once in every N packets, the safeguard 
counter is incremented unless it is at the maximum burst size, in which 
case it is not. The parameter N limits the average rate of the sampled 
packets to a fraction of 1/N of the transmitted packets. If the sampling 
function is probabilistic sampling with a fixed sampling probability, it 
should be strictly smaller than 1/N. Leaky buckets operate in a similar 
fashion, except they use timers instead of packets to increment the 
safeguard counter and therefore limit the average rate of the sampled 
packets to a fraction of the link capacity.

[Cruz87] Rene L. Cruz, A calculus for Network Delay and a Note on 
Topologies of Interconnection Networks, Ph.D. Thesis, University of 
Illinois, issued as Report UiLU-ENG-87-2246, July 1987

Cheers,

Cristian


--
to unsubscribe send a message to psamp-request@ops.ietf.org with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/psamp/>