[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: draft on sampling techniques



Tanja Zseby wrote:

...


1) For content based sampling, you say that we can base the decision on field values or hash functions. Do we really want all devices in the network to support functionality as "sample all packets coming from IP address X"? Is this useful? Is this dangerous? Is this prone to misconfigurations that would result in too many packets being selected?
We *must* specify what the hash functions are that we base the sampling decision on and what fields they take as input and also in section 4.2 how the network operator can "seed" them so that the system cannot be manipulated/evaded by "the bad guys". This is because devices from different vendors have to be consistent. I would incline towards not making the fields the hash is computed on configurable (i.e. hard code in the standard what (invariant) fields we hash on). Other opinions?

I agree that supporting all options might be too much for achieving consistent standard schemes that are supported by all vendors. Maybe we can check what field combinations are useful and define a few (2 or 3 ?) different hash-based schemes. An example would be maybe schemes with or without parts of the payload or maybe a "lightweight" version with less fields (it first needs to be clear, when taking less fields would really save resources.)
I think having just one would be even better :-) In the trajectory sampling paper Nick and Matt say that the hash should be computed on the entire IPv4 packet except 6 bytes of the IP header which can change hop by hop: the first two which contain the type of service byte and from the 9th to the 12th which contain the TTL and the header checksum. I say we either use their definition or ignore bytes after the xth where x has to be at least 32 so that we also hash on the TCP ports, sequence number and ack number. We would need the input of some people with hardware experience to see if we need this cap and if so, what its value should be.

My knee-jerk reaction to how to do this in IPv6 is to exclude the first two bytes and bytes 7 and 8 or just 8 and that we should go at least up to byte 52. Is there anyone familiar with IPv6 on the list who would like to take a closer look at this?

IMO we should *not* include any layer 2 fields into the input to the hash function, because the purpose of this scheme is to track packets across the whole network. On the other hand it might be useful to include some of those fields with the actual sampled packet, but that is a different issue.

...


3) I believe this document should also contain the safeguards we need to put in to avoid floods of sampled packets due to misconfiguration or attacks on the measurement infrastructure. The simplest thing that comes to my mind is defining a leaky bucket regulator that would limit the rate and the burst size for the samples being sent to the collection station.


Maybe we can add this proposal to the security considerations ? Could you maybe write a few sentences that can be included in the next version of the draft ?
Below. Feel free to modify it wherever you think it's necessary.

The amount of measurement result data produced by devices implementing packet sampling is a security concern. Large volumes of data can result because of attacks on the measurement infrastructure, misconfigurations, bugs in the implementation of the sampling function or the randomness of the sampling function. The traffic measurement modules performing the sampling of packets should protect against such floods of sampled packets. This protection is enforced through safe defaults for the sampling parameters and explicit safeguards.

If the sole consumer of sampled packets is a metering process local to the device that implements the sampling function, the manufacturer may choose not to implement safeguards and not to use safe defaults. If there is at least one consumer of traffic reports containing the sampled packets outside the device implementing the sampling function and the sampling function ensures that the number of sampled packets is no larger than a fixed fraction of the packets, the sampling function must implement safe defaults but may not implement explicit safeguards. Such sampling functions include systematic sampling and n-out-of-N sampling.

The default values of the sampling parameters should be safe. They should ensure that the number of sampled packets is no more than 0.01% of the packets carried by the link observed or that they do not add up to more than 0.01% of the link capacity. These constraints should hold over any 30 second time interval. A configuration of the sampling function that samples no packets at all is safe.

Explicit safeguards have to limit the average rate at which samples are generated (in sampled packets/second) and the maximum burst size (in sampled packets). The average rate and the maximum burst size are parameters of the safeguard. The default values for these parameters must be such that they ensure limits on the number of samples at least as strong as the safe defaults above. The implementation can allow the operator to change these parameters. The implementation may refuse parameters for the safeguard it considers too aggressive. The implementation must ensure that the sampling parameters are such that under normal operation, the number of sampled packets is within the limits of the safeguard. When the safeguard mechanism removes sampled packets because they go beyond the allowed limits, it should indicate this. The counter for sampled packets we include with each of them should be incremented for the sampled packets discarded by the safeguard. To implement the safeguard one could use a leaky bucket descriptor as in [Cruz87]. Another acceptable implementation of the safeguard which does not require timers is given below.

The device implementing the sampling function could use a safeguard counter initialized to the maximum burst size. Whenever a packet is sampled by the sampling function, if the safeguard counter is above zero, it is decremented and the sampled packet is processed further. Otherwise, the counter for sampled packets is incremented but the sampled packet is not processed. Once in every N packets, the safeguard counter is incremented unless it is at the maximum burst size, in which case it is not. The parameter N limits the average rate of the sampled packets to a fraction of 1/N of the transmitted packets. If the sampling function is probabilistic sampling with a fixed sampling probability, it should be strictly smaller than 1/N. Leaky buckets operate in a similar fashion, except they use timers instead of packets to increment the safeguard counter and therefore limit the average rate of the sampled packets to a fraction of the link capacity.

[Cruz87] Rene L. Cruz, A calculus for Network Delay and a Note on Topologies of Interconnection Networks, Ph.D. Thesis, University of Illinois, issued as Report UiLU-ENG-87-2246, July 1987

Cheers,

Cristian


--
to unsubscribe send a message to psamp-request@ops.ietf.org with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/psamp/>