RE: comments to draft-psamp-framework-05.txt

[<duffield@research.att.com>]

Derek,

> -----Original Message-----
> From: Derek Chiou [mailto:dchiou@avici.com]
> Sent: Monday, March 01, 2004 2:21 AM
> To: psamp@ops.ietf.org
> Subject: comments to draft-psamp-framework-05.txt
> 
> Hi,
> 
> I have some (very late, I know and am sorry) comments to
> framework-05.txt. Some of these comments have probably been covered
and
> a decision made at previous meetings; if so, please ignore.
> 

Thanks. And apologies for the further delay in my reply. 

> Thanks.
> 
> Derek
> 
> >
> >      * Reporting Process: creates a report stream on packets
selected by
> >        a selection process, in preparation for export. The input to
the
> >        reporting process comprises that information available to the
> >        selection process per selected packet, specifically:
> >
> >           (i) the selected packet's content;
> >
> >           (ii) information derived from the selected packet's
treatment
> >           at the observation point;
> >
> >           (iii) any selection state maintained by the inputting
> >           selection process, reflecting any modifications to the
> >           selection state made during selection of the packet.
> 
> Should the selection state presented to the reporting process be pre-
> modification instead of post-modification?  It's fairly
straightforward to
> go from pre-modification to post-modification, but it may be much
harder
> to go from post-modification to pre-modification.  Pre-modification
> selection state, however, may be difficult to implement (have to keep
> around copies of all selection state until it is decided whether the
> packet will be selected or not) and thus should not be a requirement.
> 

Post-modification: incorporating the modifications arising from
selecting the packet. 


> >
> >      * Secure Export:
> >
> >        (i) confidentiality: the option to encrypt exported data MUST
be
> >        provided.
> 
> I know these requirements have already been discussed, so I'll just
ask a
> question and make a comment.  What sort of encryption are we going to
> require?  This requirement could severely limit the sampling
frequency.
> 

A good question; specifics will have to be described in later PSAMP
documents. There is a corresponding confidentiality requirement in
IPFIX. Since PSAMP will use IPFIX export, we'll need to make sure that
whatever encryption mechanisms are required there can operate on PSAMP
packet reports at produced at sampling rates that are useful in
practice.

> >
> >        (ii) integrity: alterations in transit to exported data MUST
be
> >        detectable at the collector
> 
> What sort of integrity-detecting mechanisms are required?  Again, this
> requirement could severely limit the sampling-frequency.
> 

Again, we need to assess proposed IPFIX solutions to same problem.

> >
> >   4.2 PSAMP Packet Selection Operations
> 
> >      A spectrum of packet selection operations is described in
detail in
> >      [ZMRD03]. Here we only briefly summarize the meanings for
> >      completeness.
> >
> >      A PSAMP selection process MUST support at least one of the
> >      following selectors.
> 
> To me, it seems that the mask/match filtering is the selector that
doesn't
> belong.  Some argument can be made that the others can roughly
approximate
> each other, but not the mask/match.  Thus, requiring both a sampling
> selector and a mask/match or just a sampling selector seems more
> reasonable.
>

Not sure I understand your intent in the last sentence. Do you mean that
to provide some kind of sampling operation would be mandatory for PSAMP,
with mask/match optional?

 
> >
> >   4.3 Input Sequence Numbers for Primitive Selection Processes.
> >
> >      Each instance of a primitive selection process MUST maintain a
> >      count of packets presented at its input. The counter value is
to be
> >      included as a sequence number for selected packets. The
sequence
> >      numbers are considered as part of the packet's selection state.
> 
> What if we are using a compound selector?  Do we tack on the sequence
> numbers for each of the selectors, or just the first one or just the
last
> one?  There could be several.  From an implementation point of view,
only
> passing the last one is the simplest.
> 

I envisaged sequence numbers primarily as being useful for sampling,
where they could be used to determine the attained sampling rate. This
is useful for estimating actual traffic volumes from the samples. One
could also envisage sequences numbers being kept when filtering: it
would enable determination of the proportion of traffic that matches the
filter. At present we allow compound selectors to compose sampling and
filtering in either order. How about keeping an input sequence number
for each selector in the compound?


> 
> >
> >      Use of input sequence numbers enables applications to determine
the
> >      attained frequency at which packets are selected, and hence
> >      correctly normalize network usage estimates regardless of loss
of
> >      information, regardless of whether this loss occurs because of
> >      discard of packet reports in the measurement or reporting
process
> >      (e.g. due to resource contention in the host of these
processes),
> >      or loss of export packets in transmission or collection. See
> >      [RFC3176] for further details.
> >
> >      As an example, consider a set of n consecutive packet reports
r1,
> >      r2, ..., rn, selected by a sampling operation and received at a
> >
> >   Duffield (Ed.)            Expires April 2004                [Page
13]
> >
> >
> >
> 
>    Internet Draft      Packet Selection and Reporting      December
2003
> >
> >
> >      collector. Let s1, s2, ..., sn be the input sequence numbers
reported
> >      by the packets. The attained selection frequency, taking into
> >      account both packet sampling at the observation point and
selection
> >      arising from loss in transmission, is R = (n-1)/(sn-s1). (Note
R
> >      would be 1 if all packet were selected and there were no
> >      transmission loss).
> >
> >      The attained selection frequency can be used to estimate the
number
> >      bytes present in a portion of the observed packet stream. Let
b1,
> >      b2, ..., bn be the bytes reported in each of the packets that
reached
> >      the collector, and set B = b1+b2+,...,+bn. Then the total bytes
> >      present in packets in the observed packet stream whose input
> >      sequence numbers lie between s1 and sn is estimated by B/R,
i.e,
> >      scaling up the measured bytes through division by the attained
> >      selection frequency.
> >
> >
> >   4.4 Composite Selectors
> >
> >      The ability to compose selectors in a selection process SHOULD
be
> >      provided. The following combinations appear to be most useful
for
> >      applications:
> >
> >      * filtering followed by sampling
> >
> >      * sampling followed by filtering
> >
> >      Composite selectors are useful for drill down applications. The
> >      first component of a composite selector can be used to reduce
the
> >      load on the second component. In this setting, the advantage to
be
> >      gained from a given ordering can depend on the composition of
the
> >      packet stream.
> >
> >   4.5 Constraints on the Sampling Frequency
> >
> >      Sampling at full line rate, i.e. with probability 1, is not
> >      excluded in principle, although resource constraints may not
> >      support it in practice.
> >
> >   4.6 Criteria for Choice of Selection Operations
> >
> >      In current practice, sampling has been performed using
particular
> >      algorithms, including:
> >
> >      * pseudorandom independent sampling with probability 1/N;
> >
> >      * systematic sampling of every Nth packet.
> >
> >      The question arises as to whether both of these should be
> >      standardized as distinct selection operations, or whether they
can
> >      be regarded as different implementations of a single selection
> >      operation.
> >
> >   Duffield (Ed.)            Expires April 2004                [Page
14]
> >
> >
> >
> 
>    Internet Draft      Packet Selection and Reporting      December
2003
> >
> >
> >
> >      To determine the answer to this question, we need to consider
> >
> >      (a) measured or assumed statistical properties of the packet
> >      stream, e.g., one or more of the following:
> >
> >           (i) contents of different packets are statistically
> >           independent
> >
> >           (ii) correlations between contents of different packets
decay
> >           at a specified rate
> >
> >           (iii) contents of certain fields within the same packet
are
> >           significantly variable and exhibit small cross correlation
> >
> >      (b) the desired reference sampling model, e.g., one of:
> >
> >           (i) sample packets with long term probability 1/N
> >
> >           (ii) sample packets independent with probability 1/N
> >
> >      (c) the set of possible alternatives and implementations, e.g.,
one
> >      of:
> >           (i) pseudorandom independent sampling with probability 1/N
> >
> >           (ii) systematic sampling with period N
> >
> >           (iii) hash-based sampling with target probability 1/N
> >
> >      (d) the tolerance for error in the applications that use the
> >      collected packet reports.
> >
> >      We can say that a given alternative from (c) reproduces a
reference
> >      model (b) for the applications if the results obtained using
them
> >      are sufficiently accurate in (d) for traffic satisfying an
assumed
> >      statistical properties in (a). Clearly, application to evaluate
> >      methods in (c) requires developing agreement on the relevant
> >      properties in (a), (b) and (d).
> >
> >      Example: systematic sampling with period N will not count the
> >      occurrence of closely space packets (less than N counts apart)
from
> >      the same flow. Thus for applications that are concerned with
the
> >      joint statistics of multiple packets within flows, systematic
> >      sampling may not reproduce the results obtained with random
> >      sampling sufficiently accurately.
> >
> >   5. Reporting Process
> >
> >   5.1 Mandatory Contents of Packet Reports (MUST)
> >
> >      The reporting process MUST include the following in each packet
> >      report:
> >
> >   Duffield (Ed.)            Expires April 2004                [Page
15]
> >
> >
> >
> 
>    Internet Draft      Packet Selection and Reporting      December
2003
> >
> >
> >
> >           (i) the input sequence number(s) of any sampling operation
> >             that acted on the packet in the instance of a
measurement
> >             process of which the reporting process is a component.
> 
> Should there be an additional sequence number on the packet reports
> themselves?  Without a sequence number on the packet reports, it's
> possible for the reports to be misinterpreted.  For example, if a
> mask/match selector is either configured incorrectly or the hit rate
on it
> is under what is suspected, it's hard to tell that from report packets
> being dropped just from the primitive selector sequence number.  Hash-
> based selection can have the same problem.  Tacking on an extra
counter
> here isn't much work from an implementation standpoint, compared to
> everything else we are requiring.
>

In order to detect export loss, sequence numbers could be attached to
the packet reports, or to the export packets (which might contain more
than one packet report). Can IPFIX folks tell us what sequence numbers
are used for IPFIX export?

 
> >
> >      The reporting process MUST be able to include the following in
each
> >      packet report, as a configurable option:
> >
> >           (ii) a basic report on the packet, i.e., some number of
> >           contiguous bytes from the start of the packet, including
the
> >           packet header (which includes link layer, network layer
and
> >           other encapsulation headers) and some subsequent bytes of
the
> >           packet payload.
> >
> >      Some devices hosting reporting processes may not have the
resource
> >      capacity or functionality to provide more detailed packet
reports
> >      that those in (i) and (ii) above. Using this minimum required
> >      reporting functionality, the reporting process places the
burden of
> >      interpretation on the collector, or on applications that it
> >      supplies.
> >
> >      On the other hand, some devices may have the capability to
provide
> >      extended packet reports (see Section 5.2 below). These devices
may
> >      exercise the option not to provide basic reports.
> >
> 
> 
> >
> >   7.7 Collector-based Rate Reconfiguration
> >
> >      Since collector-based rate reconfiguration is a new proposal,
this
> >      draft will discuss it in some detail.
> >
> >      The collector can detect congestion loss along the path from
the
> >      exporting device to the collector by observing packet loss,
> >      manifest as gaps in the sequence numbers, or the absence of
packets
> 
> Just to reiterate, gaps in the selector sequence numbers may or may
not
> indicate loss between the exporting device and the collector.  We are
> assuming we can determine the fraction of packets that will be
selected,
> based on the selector.  Our assumptions may be incorrect.
> 

This is an interesting point. In order to determine the attained
sampling rate of the selection process-export_and_transmission
combination, one only needs the selector sequence numbers on reports
that reach the collector. You can still estimate the actual traffic
presented to the selector by dividing the traffic rate seen at the
collector by the attained sampling rate.

But if you want to break down the attained sampling rate into components
i.e. the selection rate attained at the observation point, and the
transmission rate when exporting, then yes, you need to attach an
additional sequence number, either to the reports, or to the export
packets. Or both, if you want to determine if reports are lost before
transmission, e.g., if they timeout in a buffer while waiting to
transmit.



> >      for a period of time. The server can run an appropriate
congestion-
> >      control algorithm to compute a new export rate limit, then
> >      reconfigure the export process with the new rate.  This is an
> >      attractive alternative to requiring the export process to
receive
> >      acknowledgment packets.  Implementing the congestion control
> >      algorithm in the collector has the added advantages of
flexibility
> >      in adapting the sending rate and the ability to incorporate new
> >      congestion-control algorithms as they become available.
> >
> 
> 
> 
> 
> --
> to unsubscribe send a message to psamp-request@ops.ietf.org with
> the word 'unsubscribe' in a single line as the message text body.
> archive: <http://ops.ietf.org/lists/psamp/>


Nick


--
to unsubscribe send a message to psamp-request@ops.ietf.org with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/psamp/>