Here is a list of comments on the framework version 5 draft.
As always, feel free to start a new thread on a specific topic discussed below, with a new email subject.
I divided the issues in MAJOR and NON-MAJOR (I didn't want to say MINOR as some of them are not minor)
Note that as it was decided that this draft would be a standard track document, this has been considered in the review. And this is the reason why some new issues appeared!
Note also that some of the comments were already discussed in https://ops.ietf.org/lists/psamp/psamp.2003/msg00038.html. If this is the case, a reference to this posting will be provided.
We should have a set of normative versus informative references.
In normative, we will have [PSAMP-PROTO], [PSAMP-INFO], [PSAMP-MIB], [PSAMP-TECHNIQUE] I guess
Does it mean that we should wait for the 3 drafts above to be completed before submitting this one as RFC?
2. Terminology section
As a standard track document, it must be the same as the PSAMP protocol draft (or almost), which in turn will be the same as the IPFIX protocol draft (for which we just agreed upon)
Does it mean that we should wait for the [PSAMP-PROTO] to be completed before submitting this one as RFC?
BTW, I fully agree with Tom Petch's comment that the abstract is confusing.
See post https://psg.com/lists/psamp/psamp.2004/msg00023.html
What is the goal of the document? A framework + requirement + architecture?
Two minor comments
- "describes" must be changed by "specifies".
- PSAMP acronym must be explained.
4. Draft table of content for the requirements.
3. Requirements................................................7 3.1 Selection Process Requirements..............................7 3.2 Reporting Process Requirements..............................8 3.3 Export Process Requirements.................................8 3.4 Configuration Requirements..................................9But there are requirements all over the place in the draft.
5. Reporting Process..........................................15 5.1 Mandatory Contents of Packet Reports (MUST)................15 5.2 Extended Packet Reports (MAY)..............................16or
7. Export Process.............................................19My confusion comes from the section 3 which seems to be a complete list of requirements but actually it's not...
Just another example of the confusion: Section 4.2 "PSAMP Packet Selection Operations" is full of requirements.
So should we have a new subsection 3 with potential further references to this section?
Some more comments on section 3:
- Does section 3 define only the generic requirements?
- Some of the requirement in section 3 don't have a MAY, SHOULD, MUST or lower case may!
- I don't understand some requirement. For example:
* Transparency: allow transparent interpretation of the report stream, without any need to obtain additional information concerning the observed packet stream. * Robustness to Information Loss: allow robust interpretation of the report stream with respect to packet reports missing due to data loss, e.g. in transport, or within the selection, reporting or exporting processes. Inclusion in reporting of information that enables the accuracy of measurements to be determined.What does it mean: interpretation of the report stream?
It means everything and nothing? How to determine what is required?
Maybe we want to distinguish in these subsections what we would like to achieve and
what the requirements are!
5. Terminology section.
The order (as described by Tom Petch in https://psg.com/lists/psamp/psamp.2004/msg00023.html) is not adequate and intuitive.
For example, some terms are used but defined later: reporting process, selectors, etc...
As noted https://ops.ietf.org/lists/psamp/psamp.2003/msg00038.html point 2 and point 5, some diagrams would be useful.
Observation Point primitive selector 1 primitive selector 2 (selection process 1) (selection process 2) Observed Packet Stream -> -> Packet Stream -> Packet Stream To clarify: - one or more observation points - observed packet stream versus packet stream Another diagram is needed with the selector, composite selection process, primitive section process, composite selector.
6. Upper cases for the terms defined in the terminology section
As agreed in point 6 of https://ops.ietf.org/lists/psamp/psamp.2003/msg00038.html
Specifically with the forward definition in the terminology section.
7. A new architecture section?
Should a new architecture section be developped in this draft around the text below?
Anyway, this paragraph doesn't belong to the terminology section
Various possibilities for the high level architecture of these elements are as follows. MP = Measurement Process, EP = Export Process +---------------------+ +------------------+ |Observation Point(s) | | Collector(1) | |MP(s)--->EP----------+---------------->| | |MP(s)--->EP----------+-------+-------->| | +---------------------+ | +------------------+ | +---------------------+ | +------------------+ |Observation Point(s) | +-------->| Collector(2) | |MP(s)--->EP----------+---------------->| | +---------------------+ +------------------+ +---------------------+ |Observation Point(s) | |MP(s)--->EP---+ | | | | |Collector(3)<-+ | +---------------------+8. Section 4.1 "packet selection terminology"
This is exactly (most of the time) the same as the http://www.ietf.org/internet-drafts/draft-ietf-psamp-sample-tech-04.txt
What's the point to duplicate the info in this draft? We should just have a reference to the other draft.
On the top of that, not all the terms are used in this draft.
Anyway some comments (so maybe for Tanja?):
- Approximative Selection: selection operations in any of the above categories ??
- size of the population -> population size
Attained Selection Frequency: the actual frequency with which packets are selected by a selection process. The attained sampling frequency is calculated as ratio of the size of a sample size to the size of the population from which it was selected.- the paragraph is almost the same as [PSAMP-TECHNIQUE] but not quite -> very confusing
9. Packet Report and Packet Interpretation.
Those 2 terms were nice to describe the PSAMP requirements.
But, as we know that we will use IPFIX and as we know we don't have those two terms in IPFIX, do we need them in this draft?
I explain! In section 5.4, we see
Information for use in report interpretation MUST include (i) configuration parameters of the selectors of the packets reported on. (ii) format of the packet report; (iii) indication of the inherent accuracy of the reported quantities, e.g., of the packet timestamp. (iv) identifiers for observation point, measurement process, and export process.(i) will use the concept of selector ID
(ii), (iii) will be sent as information element in a template
(iv) observation point, export process will be sent as information element in a template
So it's not like we would send a separate packet with all the report interpretation information. So I'm thinking the concepts of report interpretation doesn't make sense from a [PSAMP-PROTO] point of view and is even confusing. Why have it here?
On the same topic, the following is not true, as described in [PSAMP-PROTO], as I just shown:
It is not envisaged that all report interpretation be included in every packet report. Many of the quantities listed above are expected to be relatively static; they could be communicated periodically, and upon change.10. Section 7.2
- "Note that IPFIX being still developped; this is given only as a possible example" Is this appropriate for a standard track doc.? Should we wait for IPFIX to be published before this draft?11. Transport protocol
In section 7.3, 7.5, 7.6, I think we should clearly express the choice from IPFIX.
And clearly give the options.
Section 7.7 must dissapear!
- this section must be changed according to the new abstract
- "describes" must be changed by "specifies".
- this section speaks of "capabilities of network elements" while the abstract speaks of "deployment in router interfaces"
Is it network element wide or router interface wide. We must be consistent!
- "(iii) describe a protocol by which information on sampled packets is reported to applications;"
not needed as IPFIX is selected
- "(iv) describe a protocol by which packet selection and reporting are configured."
We speak of a MIB here. But I guess we don't want to describe SNMP...
I would just express: specify some requirements for a MIB
- The following paragraph is not adequate anymore as IPFIX and its transport protocol are selected.
Conversely, we expect that aspects of this framework not specifically concerned with the central issue of packet selection and report formation may be able to leverage work in other Working Groups. Potential examples are the format and export of reports on selected packets, which may leverage the information model and export protocols of IP Flow Information Export (IPFIX) [QZCZ03], and work in congestion aware unreliable transport in the Datagram Congestion Control Protocol (DCCP) [FHK02], and related work in The Stream Control Transmission Protocol (SCTP) [SCTP] and the SCTP Partial Reliability Extension [SCTP-PR].2. Selection State definition.
(ii) a timestamp of observation of the packet at the observation points;observation point must be singular
3. Packet Report definition.
It's not obvious at all from the definition that you mean what is the section 5.1. Some examples or clarifications needed.
4. New section "PSAMP and IPFIX interfaction" for the text below.
The PSAMP measurement process can be viewed as analogous to the IPFIX metering process. The PSAMP measurement process takes an observed packet stream as its input, and produces packet reports as its output. The IPFIX metering process produces flow records as its output. The distinct name “measurement process” has been retained in order to avoid potential confusion in settings where IPFIX and PSAMP coexist, and in order to avoid the implicit requirement that the PSAMP version satisfy the requirements of an IPFIX metering process (at least while these are under development). The relation between PSAMP and IPFIX is further discussed in [QC03].(at least while these are under development)? Not adequate for standard track!
Draft [QC03] is not valid anymore
5. A space to be deleted
A specific reporting processes meeting these requirements, and th e requirement for ubiquity, is described in Section 5.6. References.
Instead of having unreadable references like [QZCZ03], the following references would make more sense, and are used in the IPFIX and PSAMP protocol drafts: IPFIX-REQ, IPFIX-PROTO, IPFIX-INFO, PSAMP-MIB, PSAMP-FRAME, PSAMP-INFO, PSAMP-TECHNIQUE
7. Following the previous point, we would need a "PSAMP Documents Overview".
In IPFIX serie of drafts, we have the following. To be adapted for PSAMP
IPFIX Documents Overview The IPFIX protocol provides network administrators with access to IP flow information. The architecture for the export of measured IP flow information out of an IPFIX exporting process to a collecting processing is defined in [IPFIX-ARCH], per the requirements defined in [IPFIX-REQ]. [IPFIX-PROTO] specifies how IPFIX flow record data, options record data and control information is carried via a congestion-aware transport protocol from IPFIX exporting process to IPFIX collecting process. IPFIX has a formal description of IPFIX information elements (fields), their name, type and additional semantic information, as specified in [IPFIX-INFO]. Finally [IPFIX- AS] describes what type of applications can use the IPFIX protocol and how they can use the information provided. It furthermore shows how the IPFIX framework relates to other architectures and frameworks.8. section 4.2 "PSAMP Packet Selection Operations.
No upper cases in Spacing and Interval Length.
9. [ZMRD03] not in the reference section
10. The following paragraphs should be in [PSAMP-TECHNIQUE] only, IMHO
When the hash function is sufficiently good, hash-based selection can be used to approximate uniform random sampling over the hash domain. The target sampling frequency is then the ratio of the size of the selection range to the hash range. Applications of hash-based selection include: (i) Trajectory Sampling: all routers use the same hash selector; the hash domain includes only portions of the packet that do not change from hop to hop. (For example, in an IP packet, TTL is excluded.) Hence packets are consistently selected in the sense that they are selected at all routers on their path or none. Reports packets also include a second hash (the label hash) that distinguishes different packets. Reports of a given packet reaching the collector from different routers can be used to reconstruct the path taken by the packet. Trajectory sampling is proposed in [DuGr01]; further description is found in [ZMRD03]; some applications are described in Section 10. (ii) Consistent Flow Sampling: the hash domain is a flow key. For a given flow, either all or none of its packets are sampled. This is accomplished without the need to maintain flow state. Some applications need to calculate packet hashes for purpose other than selection (e.g. the label hash in trajectory sampling). This can be achieved by placing a calculated hash in the selection state, and setting the selection range to be the whole of the hash range.11. As noted in https://ops.ietf.org/lists/psamp/psamp.2003/msg00038.html point 29, I don't think that the section 4.6 "Criteria for choice of selection operations" make sense in this draft. It could make sense in [PSAMP-TECHNIQUE] but not here. And even in [PSAMP-TECHNIQUE], I would put it as an appendix.
12. Section 5.1
- The input sequence number should be part of the packet report or the packet interpretation?
It doesn't match the definition of the packet report! See also point 9 of the MAJOR issues
- "These devices may exercise the option not to provide basic reports. "This sentence should be removed because covered the MAY in the first sentence of section 5.2
- Need references for IPv4, IPv6, MPLS, AToM, BGP AS + acronyms - "The accuracy of any timestamp reported MUST be supplied in the report interpretation and made available in the MIB." But we are in the "extended packet reports" section13. Section 5.3
"If an IPFIX metering process [IPFIX-PROTO] ..."
Otherwise what does "if IPFIX is supported" mean?
14. This is a general comment on the draft. I see a lot of justifications on why we should do this or why we should do that?
Is this really appropriate? Maybe we just want to specify what PSAMP is instead of justifying everything.
Just one example:
The requirements for robustness and transparency are motivations for including report interpretation in the report stream. Inclusion makes the report stream self-defining. The PSAMP framework excludes reliance on an alternative model in which interpretation is recovered out of band. This latter approach is not robust with respect to undocumented changes in selector configuration, and may give rise to future architectural problems for network management systems to coherently manage both configuration and data collection. 15. Section 5.4 "To conserve network bandwidth and resources at the collector, the export packets MAY ..." 16. Section 7.2 - "The report stream MAY ..." 17. section 7.3 ")" alone 18. section 7.4 "described in section 5.5" 19. Section 8 "collector-based rate configuration" to be removed 20. Section 10. (i) PSAMP aims for ubiquitous deployment of packet measurement, including devices that are not expected to support IPFIX. This offers broader reach for existing applications. In practice, it will never happen! We must support IPFIX because we must export some information elements. 21. Section 10. MIB-II needs a reference. 22. the reference section is not up-to-date D03 and QC03 don't exist anymore 24. The PSAMP device term must be defined, as discussed in https://ops.ietf.org/lists/psamp/psamp.2003/msg00038.html For reference only, in http://www.ietf.org/internet-drafts/draft-ietf-ipfix-arch-01.txt * IPFIX Device: A device hosting at least an observation point, a metering process and an exporting process. Typically, corresponding observation point(s), metering process(es) and exporting process(es) are co-located at this device, for example at a router. That's it for now ;) Thanks Nick for editing the draft. See you in Seoul Regards, Benoit.