[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
RE: comments on draft-ietf-psamp-framework-03.txt
Benoit,
Thanks for your extensive comments. I'm sorry for the delay in replying. Responses are inline below.
Nick
> -----Original Message-----
> From: Duffield,Nicholas G (Nick)
> Sent: Wednesday, October 15, 2003 12:30 PM
> To: Duffield,Nicholas G (Nick)
> Subject: RE: comments on draft-ietf-psamp-framework-03.txt
>
>
>
> -----Original Message-----
> From: Benoit Claise [mailto:bclaise@cisco.com]
> Sent: Wednesday, September 17, 2003 9:07 PM
> To: psamp@ops.ietf.org
> Subject: comments on draft-ietf-psamp-framework-03.txt
>
> Dear all,
>
> I guess this is the time of the year when we start working for the IETF
> again... ;)
> Here is a list of comments on the framework version 3 draft.
> As always, feel free to start a new thread on a specific topic discussed
> below, with a new email subject.
>
> 1.
> The title "A Framework for Passive Packet Measurement"
> I don't think this is adequate: there is not even the term "sampling"
> which reflects our charter and sampling is part of the abstract section.
> Furthermore, I think the term "passive" is implicit. Anyway we don't find
> this term in the charter and the abstract section, just a few instances
> in the draft.
> What about "a framework for packet sampling and reporting"?
>
Yes, this is tricky. The feeling was that "sampling" might be interpreted as only statistical sampling, to the exclusion of other packet selection methods. On the other hand "measurement" does not convey that typically only a subset of packets would be selected. Any comments from the group on this?
> 2.
> The "Elements, Terminology, and Architecture" section, the definitions
> should be somehow ordered or logically bound
> The rfc-editor complained about that in one of my draft.
> Just one example: the measurement process definitions defines terms that
> will come later in the paragraph.
>
That's deliberate: the exposition of the architecture is top down.
> 3.
> The "measurement process" could be renamed the "metering process" to be
> more inline with IPFIX.
> This would give the great advantage that the "IPFIX reference Model" (*)
> could be simply applied to PSAMP.
> (*) see section 4 of http://www.ietf.org/internet-drafts/draft-ietf-
> ipfix-arch-01.txt
I've resisted calling the psamp version a metering process in order to confusion when ipfix and psamp coexist. Psamp measurement and ipfix metering are analogous in that they both produce objects ready for export (packet reports for psamp, flow records for ipfix). But if, say, psamp measurement process is used to select packets from which ipfix will form flow records, then having a common name might cause confusion.
> And the term "measurement" is used a lot in the draft, with different
> meanings: once it's report packet, an other time it's the selection
> process
> * Transparency: allow transparent interpretation of measurements as
> communicated by PSAMP reporting, without any need to obtain
> additional information concerning the observed packet stream.
>
> * Robustness to Information Loss: allow robust interpretation of
> measurements with respect to reports missing due to data loss,
> e.g. in transport, or within the measurement, reporting or
> exporting processes. Inclusion in reporting of information
> that enables the accuracy of measurements to be determined.
> I think the term measurement should be avoided, if possible!
>
Agreed. "Measurement" will be avoided when a more specific term will do the job.
> 4.
> * Packet Stream: a sequence of packets, each of which was observed
> at the observation point. Note that when packets are sampled
> from a stream, the selected packets usually do not have common
> properties by which they can be distinguished from packets that
> have not been selected. Therefore we define here the term
> stream instead of flow, which is defined as set of packets with
> common properties [QuZC02].
>
> * Observed Packet Stream: the packet stream comprising all packets
> observed at the observation point.
>
> I have some difficulties to understand the difference between those 2
> definitions.
That's a reasonable response. The point was to distinguish between the (full) packet stream seen at the observation point, and (usually) partial packet streams e.g. the output of a selector, en route to another. Will clarify.
>
> 5.
> I have another problem (that could be related to the point 4) with the
> terminology.
> The Observation Point seems to relative to only the Observed Packet
> Stream.
> So there is something wrong with:
> * Packet Stream: a sequence of packets, each of which was observed
> at the observation point. Note that when packets are sampled
> from a stream, the selected packets usually do not have common
> properties by which they can be distinguished from packets that
> have not been selected. Therefore we define here the term
> stream instead of flow, which is defined as set of packets with
> common properties [QuZC02].
>
> If my assumption is correct, there is another problem.
> * Selection Process: a selection process selects a substream of
> packets from the observed packet stream. A selection process
> entails the composition of one or more selectors in succession,
> acting on each packet in the observed packet stream. When
> selectors are composed, the output stream packet issuing from
> one selector forms the input packet stream for the succeeding
> selector.
> The "observed packet stream" should be replaced by the "packet stream"
> Or "a selection process selects a substream of Packet Stream"
>
> And...
> * Selector (or selection operation): a configurable packet
> selection operation that acts on single packets. It takes as
> its input, the content of a single packet from a packet stream,
> information derived from the packet's treatment at the
> observation point, and selection state that may be maintained
> at the observation point. If the packet is selected, this same
> information may be considered as the output. Selectors may
> change the selection state.
>
> What if we have several selectors in a raw? Do we have several observation
> points?
> I guess no according to the observation point definition
>
> Anyway, I think a small diagram would be usefull!
> Something like:
>
> Observation Point primitive selector 1 primitive selector
> 2
> (selection process 1) (selection
> process 2)
> Observed Packet Stream -> -> Packet Stream -> Packet
> Stream
>
> To clarify:
> - one or more observation points
> - observerd packet stream versus packet stream
>
I agree this needs work. Thanks for the suggestion of a diagram.
> 6.
> I think that capital letters should be used throughout the document for
> definitions from the terminology section.
>
OK
> 7.
> * Selector (or selection operation): a configurable packet
> selection operation that acts on single packets. It takes as
> its input, the content of a single packet from a packet stream,
> information derived from the packet's treatment at the
> observation point, and selection state that may be maintained
> at the observation point. If the packet is selected, this same
> information may be considered as the output. Selectors may
> change the selection state.
> Isn't a circular definition?
>
Yes. Will fix
> 8.
> * Reporting Process: the creation of a report stream of information
> on packets selected by a selection processes, in preparation
> for export. The input to a reporting process comprises that
> information available to a selection process, for the selected
> packets. The report stream contains two distinguished types of
> information: packet reports, and report interpretation.
> Just my personal view, but I start to be confused with all those streams:
> packets, observed packets, substream.
> Why not just put report in this case?
> There are some instances of this report stream throughout the draft.
Stream is used to refer to a collection of objects (reports, packets, ...).
The reason for this was that formally some of the operations may not act on single objects alone (e.g. if you have selection state that is influence by history).
>
> 9.
> * Measurement packets: one or packet reports, and perhaps report
> interpretation, are bundled by the export process into a
> measurement packet for export to a collector.
> I think that "measurement packets" is a confusing term! What is measured
> in the packets? We just have a sample of a packet plus some report
> interpretation.
> Why not "export packets"? This would even be more consistent with IPFIX
>
Agree.
> 10.
> * Collector: a collector receives a report stream exported by one
> or more measurement processes. In some cases, the entity that
> hosts the measurement and/or export process may also serve as
> the collector.
> measurement processes -> export processes
>
Agree
> 11.
> * Flexibility: to support selection of packets using different
> network protocols or encapsulation layers (e.g. IPv4, IPv6,
> MPLS, etc), and under packet encryption.
> I think that some clarifications are needed because I guess that you don't
> want to decrypt all packets just for PSAMP ;)
>
> 12.
> * Parallel Measurements: multiple independent measurement
> processes at the same entity.
> What's an entity?
> A PSAMP device?
> An Observation Domain like in http://www.ietf.org/internet-drafts/draft-
> ietf-ipfix-protocol-00.txt
Thanks for the pointer. Reusing appropriate ipfix terminology would be good.
> 13.
> * Timeliness: reports on selected packets MUST be made available
> to the collector quickly enough to support near real time
> applications. Specifically, any report on a packet MUST be
> dispatched within 1 second of the time of receipt of the packet
> by the measurement process.
>
> Where does the 1 second requirement come from?
> Should just say: with a configurable time ...
Proposed new material for later Section (much lifted from WG discussion)
Low measurement latency allows the traffic monitoring system to be
more responsive to real-time network events, quickly identifying
sources of congestion. Timeliness is generally a good thing for the
switches/routers performing the sampling since it minimizes the
amount of memory needed to buffer samples.
Keeping the packet dispatching delay to under 1 second has other
benefits besides limiting buffer requirements. For many
applications a 1 second time resolution is sufficient. Applications
in this category would include: identifying sources associated with
congestion; tracing denial of service attacks through the network
and constructing traffic matrices.
The 1 second rule in these situations eliminates the need for
timestamping by clocks synchronized clocks in measurement devices,
or for the measurement devices and collector to maintain
bi-directional communication in order to track clock offsets. The
collector can simply process packet reports in the order that they
are received---using its own clock as a "global" time
base---avoiding the complexity of buffering and reordering samples.
See [DuGeGr02] for an example.
> And potentially say: minimum value of zero, which means immediate export
>
Agree.
> 14.
> * Secure Export:
>
> - confidentiality: the option to encrypt exported data MUST be
> provided.
>
> - integrity: alterations in transit to exported data MUST be
> detectable at the collector
>
> - authenticity: authenticity of exported data MUST be
> verifiable by the collector in order to detect forged data.
>
> The motivation here is the same as for security in IPFIX
> export; see Sections 6.3 and 10 of [QZCZ03].
> IPFIX specifies a SHOULD for confidentiality, not a MUST
OK, will clarify. My recollection was that a MUST (provide option of confidentiality) was agreed at the last WG meeting.
>
> 15.
> * Content-independent Sampling: a sampling operation that does not
> use packet content (or quantities derived from it) as the basis
> for selection is called a content-independent sampling
> operation. Examples include systematic sampling, and uniform
> pseudorandom sampling driven by a pseudorandom number whose
> generation is independent of packet content. Note that in
> independent sampling it is not necessary to access the packet
> content in order to make the selection decision.
> independent sampling -> content-independent sampling
>
> 16.
> * Hash-based selection: a filter specified by a hash domain, a hash
> function, and hash range and a hash selection range.
>
> * Selection range: a subset of the hash range. The packet is
> selected if the action of the hash function on the hash domain
> for the packet yields a result in the hash selection range.
> Selection range -> Hash Selection Range
>
> 17.
> * Attained Sampling Frequency: Given a subset of packets in a stream
> input to a sampling operation, the attained sampling frequency is
> the ratio of the sample size to the pool size.
> I don't understand the underline part of the sentence.
> I was thinking the "the number of packets in the Observed packet stream"
> Do you want to take into account the case of filtering and sampling?
>
The language needs modification. But my feeling was that knowing the attained sampling rate was interesting only for sampling operations, since sampled packets are intended to be representative of the observed stream (so you might want to divide by the sampling rate to estimate the amount of original traffic). For (match/mask) filtering this seems less interesting, since the selected traffic is by design not representative. (But you would want to know the attained sampling rate for hash-based selection, even though this is formally a filter).
> 18.
> Section 4.2 Packet Selection Operations for a PSAMP
> -> for a PSAMP device?
>
> 19.
> Minor comment.
> When a reference is a RFC, I would clearly put as an RFC.
> So instead of [PAMM98], I would put [RFC2330]
> And instead of [PPM01], I would put [RFC3176]
> This is an IETF draft, after all ;)
>
OK.
> 20.
> * Hash domain: a subset of the packet content and the packet
> treatment, viewed as an N-bit string for some positive integer
> N.
>
> * Hash range: a set of M-bit strings for some positive integer M.
> Should we specify M<N?
>
> 21.
> I see a couple of "w.r.t."
> Is it accepted in a draft?
>
Will spell out.
> 22.
> 4.2 Packet Selection Operations for a PSAMP
>
> A PSAMP selection process MUST support at least one of the
> following selectors.
>
> * Systematic Time Based:
>
> * Systematic Count Based:
>
> * Uniform Probabilistic:
>
> * Non-uniform Probabilistic:
>
> * Probabilistic n-out-of-N:
>
> * Match/Mask Filtering:
>
> Match/Mask clearly explains this is filtering. I would add sampling or
> filtering in front of the other as well!
> For clarity purposes
OK.
>
> 23.
> * Uniform Probabilistic: <New Line> packets are selected independently
> with
> fixed sampling probability p.
> 24.
> * Match/Mask Filtering:
>
> ...
>
> Match/mask operations SHOULD be available for different
> protocol portions of the packet:
> packet -> packet header?
>
> 25.
> Match/mask operations SHOULD be available for different
> protocol portions of the packet:
>
> o the IP header (excluding options in IPv4, stacked headers in
> IPv6)
>
> o transport header
>
> o encapsulation headers (including MPLS label stack, ATOM)
>
> When an entity offers Match/Mask filtering in the selection
> process and, in its usual capacity other than in performing
> PSAMP functions, identifies or processes information from one
> or more of the above protocols, then the information SHOULD be
> made available for filtering. For example, when an entity
> routes based on destination IP address, that field should be
> made available. Conversely, an entity that does not route is
> not expected to be able to locate an IP address within a
> packet, or make it available for filtering, although it MAY do
> so.
> So basically, you want to be able to add some extra IPFIX fields in the
> report interpretation, fields that relate to the packet report.
> Right?
>
As an example, yes, if they are available. Will give this example.
> 26.
> o Trajectory Sampling: all routers use the same hash selector;
> the hash domain includes only portions of the packet that do
> not change from hope to hop (e.f. TTL is excluded).
> hope -> hop
>
> 27.
> * Router State Filtering:
> This class of filters selects a packet on based on the following
> conditions, combined with the AND, OR or NOT operators:
> on -> of
>
> 28.
> * Router State Filtering:
> This class of filters selects a packet on based on the following
> conditions, combined with the AND, OR or NOT operators:
>
> o Ingress interface at which packet arrives equals a specified
> value
> o Egress interface to which packet is routed to equals a
> specified value
> o Origin AS equals a specified value or lies within a given
> range.
> o Destination AS equals a specified value or lies within a given
> range
> o Packet violated acl on the router
> o Failed rpf
> o Failed rsvp
> o No route found for the packet
>
> acl, rpf, rsvp -> acronyms should not be used.
> And the references should be inserted
OK
> Don't we want to say that these are just examples, because this is the
> chapter starting by:
> "A PSAMP selection process MUST support at least one of the following
> selectors."
> My point is how to know if we are compliant? Do we really need all of
> these specific examples to be compliant to the Router State Filtering?
>
I'm open to comments from the group on what needs to be included here.
> 29.
> 4.6 Criteria for Choice of Selection Operations
>
> In current practice, sampling has been performed using particular
> algorithms, including:
>
> - pseudorandom independent sampling with probability 1/N;
> - systematic sampling of every Nth packet.
>
> The question arises as to whether both of these should be
> standardized as distinct selection operations, or whether they can
> be regarded as different implementations of a single selection
> operation.
>
> To determine the answer to this question, we need to consider
> ...
>
> I would not sure about the goal of this chapter?
> Is the goal to say that we could have the same results (the same accuracy)
> by having different selection operation?
> If yes, how is it important from a PSAMP device point of view?
> Don't we deal here with the interpration of the data, that should be done
> on the collector?
> Please shed some light!
This section raises the issue that we need some rational way to decide which selection operations are needed, and what variations should just be regarded as implementation details.
>
> 30.
> . The charter speaks of "Target for inclusion the packet's IP
> header, some subsequent bytes of the packet, and encapsulating
> headers if present." I think this is important notion that should be
> included in the draft.
> Section 5.1 includes:
> The reporting process MUST be able to include the following
> in each packet report, as a configurable option:
>
> (ii) some number of contiguous bytes from the start of the
> packet.
>
> I would change it to:
> (ii) some number of contiguous bytes from the start of the
> packet. Target for inclusion the packet's IP header,
> some subsequent bytes of the packet, and encapsulating
> headers if present."
>
OK.
> 31.
> 5.2 Recommended Contents for Packet Reports (SHOULD)
>
> The reporting process SHOULD provide for the inclusion in packet
> reports of the following information, inclusion any or all being
> configurable as a option.
>
> ...
>
> (v) selection state associated with the packet, including:
>
> - timestamps
>
> TimestampS? Plural? Which ones?
> This should be the timestamp at which the packet is observed at the
> observation point
OK.
>
> 32.
> When an entity that hosts a reporting process and, in its usual
> capacity other than performing PSAMP functions, identifies or
> process one or more of the above fields, then the contents of each
> such field(s) SHOULD be made available for optional reporting. For
> example, when a device routes based on destination IP address, that
> field should be made available. Conversely, an entity that does
> not route is not expected to be able to locate an IP address within
> a packet, or make it available for reporting, although it MAY do
> so.
>
> This is a very broad statement, which means: any header field from any
> protocol known to a router.
> Sure you want to say that?
> This goes well beyond to the few limited fields that the IPFIX information
> model contains!
>
We should be prepared to go beyond the ipfix information model, since ipfix if ip flow centric; there other traffic out there (e.g. AToM). If we need to limit to some protocols of interest, that would be fine, but we need the list to be extensible as new protocols come along
> 33.
> 5.3 Report Interpretation
>
> Information for use in report interpretation MUST include (i)
> configuration parameters of the selectors of the packets reported
> on; (ii) format of the packet reports (iii) configuration
> parameters and state information of the network element; (iv)
> indication of the inherent accuracy of the reported quantities,
> e.g., of timestamps; (v) identifiers for observation point,
> measurement process, and export process.
>
> state information of the network element ->
> state information of the network element (if relevant to the selection
> process)
> Because not all state information are interesting.
Not sure what you mean by "relevant". You might want to report state (e.g. routing state) even though it is not used for selection.
>
> And why do we need the identifiers for the measurement and export
> processes if we have the selector info?
Here's a case where its useful to have process ids: you want to reconfig selectors because information is getting lost (from congestion in observation point, or at transmission).
>
> 34.
> Section 6 Parallel Measurement Processes
>
> ...
>
> Each of the parallel measurement processes SHOULD be
> independent. However, resource constraints may prevent complete
> reporting on a packet selected by multiple selection processes. In
> this case, reporting for the packet MUST be complete for at least
> one measurement process; other measurement processes need only
> report that they selected the packet. The priority amongst
> measurement processes to report packets MUST be configurable.
>
> I don't understand the underline sentence!
Will make more explicit. Idea is that the selectors must at least count the selected packets even if there is no time to complete the report. This way, at least the collector can get some info on the report stream.
> BTW, do you mean packet report?
Will make uniform.
>
> 35.
> 7.5 Configurable Export Rate Limit
>
> ...
>
> At times, the reporting process may generate new reports or report
> interpretation faster than the allowed export rate.
>
> reports -> packet reports
> Surely other instances, like in 7.7.1
>
>
> 36.
> 7.6 Congestion-aware Unreliable Transport .................. 17
> 7.7 Collector-based Rate Reconfiguration ................... 17
> 7.7.1 Changing the Export Rate and Other Rates ........... 17
> 7.7.2 Notions of Fairness ................................ 18
> 7.7.3 Behavior Under Overload and Failure ................ 19
> Should be changed to
> 7.6 Congestion-aware Unreliable Transport .................. 17
> 7.6.1 Collector-based Rate Reconfiguration ................... 17
> 7.6.1.1 Changing the Export Rate and Other Rates ........... 17
> 7.6.1.2 Notions of Fairness ................................ 18
> 7.6.1.3 Behavior Under Overload and Failure ................ 19
> As this "Collector-based Rate Reconfiguration" is only proposed as a
> solution for the congestion-aware unreliable transport.
OK.
> 37.
> 7.7.2 Notions of Fairness
>
> In some cases, it may be reasonable to allow the collector to have
> flexibility in deciding how aggressively to respond to congestion.
> For example, the PSAMP device and the collector may have a very
> small round-trip time (RTT) relative to other traffic. Conventional
> TCP-friendly congestion control would allocate a very large share
> of the bandwidth to this traffic. Instead, the collector could
> apply an algorithm that reacts more aggressively to congestion to
> give a larger share of the bandwidth to other traffic (with larger
> RTTs).
> Add the RTT acronym after round-trip time, as RTT is used at the end of
> the paragraph.
>
OK.
> 38.
> I guess the section "7.7 Collector-based Rate Reconfiguration" will
> dissappear at some point in time, as PSAMP selected IPFIX for export...
Yes, if ipfix will do the job.
>
> 39.
> 8 Configuration and Management
>
> ...
>
> PSAMP requires a uniform mechanism with which to access and
> configure the MIB. SNMP access MUST be provided by the entity
> hosting the MIB.
>
> I don't understand the underline sentence. I only know of SNMP...
First sentence is the requirement. Second sentence is a solution.
>
> 40.
> 9.1.3 Hashing
>
> Hashing functions vary greatly in complexity. Execution of a small
> number of sufficient simple hash functions is implementable at line
> rate. Concerning the input to the hash function, hop-invariant IP
> header fields (IP address, identification) and TCP/UDP header
> fields (port numbers, TCP sequence number) drawn from the first 40
> bytes of the packet have been found to possess a considerable
> variability; see [DuGr01].
>
> identification, what does it mean?
IP Id field
>
> 41.
> 10.2 Passive Performance Measurement
>
> ...
>
> In this application, Trajectory Sampling is enabled at all network
> ingress and egress interfaces.
>
> Why upper cases?
>
> 42.
>
> The capabilities are positioned as suppliers of packet samples to
> higher level consumers, including both remote collectors and
> applications, and on board measurement-based applications. Indeed,
> development of the standards within the framework described here
> should be open to influence by the requirements of standards in
> related IETF WGs, for example, IP Performance Metrics (IPPM)
> [PAMM98] and Internet Traffic Engineering (TEWG) [LCTV02].
>
> "To be influenced by"?
>
> 43.
> The capabilities are positioned as suppliers of packet samples to
> higher level consumers, including both remote collectors and
> applications, and on board measurement-based applications. Indeed,
> development of the standards within the framework described here
> should be open to influence by the requirements of standards in
> related IETF WGs, for example, IP Performance Metrics (IPPM)
> [PAMM98] and Internet Traffic Engineering (TEWG) [LCTV02].
> Conversely, we expect that aspects of this framework not
> specifically concerned with the central issue of packet selection
> and report formation may be able to leverage work in other
> WGs.
>
> WGs -> Working Group
>
OK.
> 44.
> The PSAMP device term must be defined.
> For reference only, in http://www.ietf.org/internet-drafts/draft-ietf-
> ipfix-arch-01.txt
> * IPFIX Device:
> A device hosting at least an observation point, a metering
> process and an exporting process. Typically, corresponding
> observation point(s), metering process(es) and exporting
> process(es) are co-located at this device, for example at a
> router.
Thanks for pointer.
>
>
> I might have a long list of comments, anyway the draft version improved a
> lot compared to the previous version.
>
Thanks for all the comments!
> Regards, Benoit.
--
to unsubscribe send a message to psamp-request@ops.ietf.org with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/psamp/>