Hi Tanja,I agree that supporting all options might be too much for achieving consistent standard schemes that are supported by all vendors. Maybe we can check what field combinations are useful and define a few (2 or 3 ?) different hash-based schemes. An example would be maybe schemes with or without parts of the payload or maybe a "lightweight" version with less fields (it first needs to be clear, when taking less fields would really save resources.)
A couple of observations about the draft:
1) For content based sampling, you say that we can base the decision on field values or hash functions. Do we really want all devices in the network to support functionality as "sample all packets coming from IP address X"? Is this useful? Is this dangerous? Is this prone to misconfigurations that would result in too many packets being selected?
We *must* specify what the hash functions are that we base the sampling decision on and what fields they take as input and also in section 4.2 how the network operator can "seed" them so that the system cannot be manipulated/evaded by "the bad guys". This is because devices from different vendors have to be consistent. I would incline towards not making the fields the hash is computed on configurable (i.e. hard code in the standard what (invariant) fields we hash on). Other opinions?
It can indeed get complex and only in some cases the gain justifies the stratification effort. So it probably would be o.k. to make stratified sampling just an optional feature. I am not sure about the classification policy (whether packets can be counted for multiple classes) We also discussed this in IPFIX but I think we did not yet decide how classifiers should behave. Maybe we can look how this is solved in other groups ?
2) Stratified sampling looks like it can get complex (although I never designed a switch or router), but it's useful. I propose we make it possible that devices don't support it at all or offer only limited support (e.g. limiting the number of classes and the type of classification they support). Also we need to clearly define what kind of classification rules we want. One option is to use the filtering rules for classification: the filter not only says if we want the packet or not but it also says which category it is in which decides what sampling parameters apply to it. Can the classes to which various parameters apply be overlapping? If so, does the packet go to all or just one of them (e.g. decrementing the counters that keep track of how far the next sampled packet from that class is)? If just one which one?
3) I believe this document should also contain the safeguards we need to put in to avoid floods of sampled packets due to misconfiguration or attacks on the measurement infrastructure. The simplest thing that comes to my mind is defining a leaky bucket regulator that would limit the rate and the burst size for the samples being sent to the collection station.
For me it would be o.k. to have it in one document, but following the last discussions on the list I assumed that others think that filtering and sampling should be in a separate documents ?
4) Definition of filters might go here too.
--Cheers, Cristian Tanja Zseby wrote:Dear psamp people,
I started a draft on sampling techniques for packet selection (document is attached). The document tries to define some terminology and describes various sampling methods and their parameters.
If you have any comments or if you like to contribute some text please let me know.
I know that there were once some volunteers for writing psamp documents. Are there people already working on other documents than the framework draft ?
Kind regards
Tanja