|
Here is a list of comments on the framework version 5 draft. As always, feel free to start a new thread on a specific topic discussed below, with a new email subject. I divided the issues in MAJOR and NON-MAJOR (I didn't want to say MINOR as some of them are not minor) Note that as it was decided that this draft would be a standard track document, this has been considered in the review. And this is the reason why some new issues appeared! Note also that some of the comments were already discussed in https://ops.ietf.org/lists/psamp/psamp.2003/msg00038.html. If this is the case, a reference to this posting will be provided. MAJOR --------- 1. References We should have a set of normative versus informative references. In normative, we will have [PSAMP-PROTO], [PSAMP-INFO], [PSAMP-MIB], [PSAMP-TECHNIQUE] I guess Does it mean that we should wait for the 3 drafts above to be completed before submitting this one as RFC? 2. Terminology section As a standard track document, it must be the same as the PSAMP protocol draft (or almost), which in turn will be the same as the IPFIX protocol draft (for which we just agreed upon) Does it mean that we should wait for the [PSAMP-PROTO] to be completed before submitting this one as RFC? 3. Abstract BTW, I fully agree with Tom Petch's comment that the abstract is confusing. See post https://psg.com/lists/psamp/psamp.2004/msg00023.html What is the goal of the document? A framework + requirement + architecture? Two minor comments - "describes" must be changed by "specifies". - PSAMP acronym must be explained. 4. Draft table of content for the requirements. 3. Requirements................................................7
3.1 Selection Process Requirements..............................7
3.2 Reporting Process Requirements..............................8
3.3 Export Process Requirements.................................8
3.4 Configuration Requirements..................................9
But there are requirements all over the place in the draft.For example. 5. Reporting Process..........................................15
5.1 Mandatory Contents of Packet Reports (MUST)................15
5.2 Extended Packet Reports (MAY)..............................16
or7. Export Process.............................................19My confusion comes from the section 3 which seems to be a complete list of requirements but actually it's not... Just another example of the confusion: Section 4.2 "PSAMP Packet Selection Operations" is full of requirements. So should we have a new subsection 3 with potential further references to this section? Some more comments on section 3: - Does section 3 define only the generic requirements? - Some of the requirement in section 3 don't have a MAY, SHOULD, MUST or lower case may! - I don't understand some requirement. For example: * Transparency: allow transparent interpretation of the report
stream, without any need to obtain additional information
concerning the observed packet stream.
* Robustness to Information Loss: allow robust interpretation of
the report stream with respect to packet reports missing due to
data loss, e.g. in transport, or within the selection, reporting
or exporting processes. Inclusion in reporting of information
that enables the accuracy of measurements to be determined.
What does it mean: interpretation of the report stream?It means everything and nothing? How to determine what is required? Maybe we want to distinguish in these subsections what we would like to achieve and what the requirements are! 5. Terminology section. The order (as described by Tom Petch in https://psg.com/lists/psamp/psamp.2004/msg00023.html) is not adequate and intuitive. For example, some terms are used but defined later: reporting process, selectors, etc... As noted https://ops.ietf.org/lists/psamp/psamp.2003/msg00038.html point 2 and point 5, some diagrams would be useful. Observation Point primitive selector 1 primitive selector 2
(selection process 1) (selection process 2)
Observed Packet Stream -> -> Packet Stream -> Packet Stream
To clarify:
- one or more observation points
- observed packet stream versus packet stream
Another diagram is needed with the selector, composite selection process, primitive section process, composite selector.
6. Upper cases for the terms defined in the terminology section As agreed in point 6 of https://ops.ietf.org/lists/psamp/psamp.2003/msg00038.html Specifically with the forward definition in the terminology section. 7. A new architecture section? Should a new architecture section be developped in this draft around the text below? Anyway, this paragraph doesn't belong to the terminology section Various possibilities for the high level architecture of these
elements are as follows.
MP = Measurement Process, EP = Export Process
+---------------------+ +------------------+
|Observation Point(s) | | Collector(1) |
|MP(s)--->EP----------+---------------->| |
|MP(s)--->EP----------+-------+-------->| |
+---------------------+ | +------------------+
|
+---------------------+ | +------------------+
|Observation Point(s) | +-------->| Collector(2) |
|MP(s)--->EP----------+---------------->| |
+---------------------+ +------------------+
+---------------------+
|Observation Point(s) |
|MP(s)--->EP---+ |
| | |
|Collector(3)<-+ |
+---------------------+
8. Section 4.1 "packet selection terminology"This is exactly (most of the time) the same as the http://www.ietf.org/internet-drafts/draft-ietf-psamp-sample-tech-04.txt What's the point to duplicate the info in this draft? We should just have a reference to the other draft. On the top of that, not all the terms are used in this draft. Anyway some comments (so maybe for Tanja?): - Approximative Selection: selection operations in any of the above categories ?? - size of the population -> population size Attained Selection Frequency: the actual frequency with which
packets are selected by a selection process. The attained
sampling frequency is calculated as ratio of the size of a sample
size to the size of the population from which it was selected.
- the paragraph is almost the same as [PSAMP-TECHNIQUE] but not
quite -> very confusing9. Packet Report and Packet Interpretation. Those 2 terms were nice to describe the PSAMP requirements. But, as we know that we will use IPFIX and as we know we don't have those two terms in IPFIX, do we need them in this draft? I explain! In section 5.4, we see
Information for use in report interpretation MUST include
(i) configuration parameters of the selectors of the packets
reported on.
(ii) format of the packet report;
(iii) indication of the inherent accuracy of the reported
quantities, e.g., of the packet timestamp.
(iv) identifiers for observation point, measurement process,
and export process.
(i) will use the concept of selector ID(ii), (iii) will be sent as information element in a template (iv) observation point, export process will be sent as information element in a template So it's not like we would send a separate packet with all the report interpretation information. So I'm thinking the concepts of report interpretation doesn't make sense from a [PSAMP-PROTO] point of view and is even confusing. Why have it here? On the same topic, the following is not true, as described in [PSAMP-PROTO], as I just shown: It is not envisaged that all report interpretation be included in
every packet report. Many of the quantities listed above are
expected to be relatively static; they could be communicated
periodically, and upon change.
10. Section 7.2- "Note that IPFIX being still developped; this is given only as a possible example" Is this appropriate for a standard track doc.? Should we wait for IPFIX to be published before this draft?11. Transport protocol In section 7.3, 7.5, 7.6, I think we should clearly express the choice from IPFIX. http://ipfix.doit.wisc.edu/archive/2377.html And clearly give the options. Section 7.7 must dissapear! NON-MAJOR ISSUES ------------------------- 1. Motivation - this section must be changed according to the new abstract - "describes" must be changed by "specifies". - this section speaks of "capabilities of network elements" while the abstract speaks of "deployment in router interfaces" Is it network element wide or router interface wide. We must be consistent! - "(iii) describe a protocol by which information on sampled packets is reported to applications;" not needed as IPFIX is selected - "(iv) describe a protocol by which packet selection and reporting are configured." We speak of a MIB here. But I guess we don't want to describe SNMP... I would just express: specify some requirements for a MIB - The following paragraph is not adequate anymore as IPFIX and its transport protocol are selected. Conversely, we expect that aspects of this framework not
specifically concerned with the central issue of packet selection
and report formation may be able to leverage work in other Working
Groups. Potential examples are the format and export of reports on
selected packets, which may leverage the information model and
export protocols of IP Flow Information Export (IPFIX) [QZCZ03],
and work in congestion aware unreliable transport in the Datagram
Congestion Control Protocol (DCCP) [FHK02], and related work in The
Stream Control Transmission Protocol (SCTP) [SCTP] and the SCTP
Partial Reliability Extension [SCTP-PR].
2. Selection State definition. (ii) a timestamp of observation of the packet at the
observation points;
observation point must be singular3. Packet Report definition. It's not obvious at all from the definition that you mean what is the section 5.1. Some examples or clarifications needed. 4. New section "PSAMP and IPFIX interfaction" for the text below. The PSAMP measurement process can be viewed as analogous to the
IPFIX metering process. The PSAMP measurement process takes an
observed packet stream as its input, and produces packet reports as
its output. The IPFIX metering process produces flow records as its
output. The distinct name “measurement process” has been retained
in order to avoid potential confusion in settings where IPFIX and
PSAMP coexist, and in order to avoid the implicit requirement that
the PSAMP version satisfy the requirements of an IPFIX metering
process (at least while these are under development). The relation
between PSAMP and IPFIX is further discussed in [QC03].
(at least while these are under development)? Not adequate for standard
track!Draft [QC03] is not valid anymore 5. A space to be deleted A specific reporting processes meeting these requirements, and th e
requirement for ubiquity, is described in Section 5.
6. References.Instead of having unreadable references like [QZCZ03], the following references would make more sense, and are used in the IPFIX and PSAMP protocol drafts: IPFIX-REQ, IPFIX-PROTO, IPFIX-INFO, PSAMP-MIB, PSAMP-FRAME, PSAMP-INFO, PSAMP-TECHNIQUE 7. Following the previous point, we would need a "PSAMP Documents Overview". In IPFIX serie of drafts, we have the following. To be adapted for PSAMP IPFIX Documents Overview
The IPFIX protocol provides network administrators with access to IP
flow information. The architecture for the export of measured IP
flow information out of an IPFIX exporting process to a collecting
processing is defined in [IPFIX-ARCH], per the requirements defined
in [IPFIX-REQ]. [IPFIX-PROTO] specifies how IPFIX flow record data,
options record data and control information is carried via a
congestion-aware transport protocol from IPFIX exporting process to
IPFIX collecting process. IPFIX has a formal description of IPFIX
information elements (fields), their name, type and additional
semantic information, as specified in [IPFIX-INFO]. Finally [IPFIX-
AS] describes what type of applications can use the IPFIX protocol
and how they can use the information provided. It furthermore shows
how the IPFIX framework relates to other architectures and
frameworks.
8. section 4.2 "PSAMP Packet Selection Operations.No upper cases in Spacing and Interval Length. 9. [ZMRD03] not in the reference section 10. The following paragraphs should be in [PSAMP-TECHNIQUE] only, IMHO When the hash function is sufficiently good, hash-based selection
can be used to approximate uniform random sampling over the hash
domain. The target sampling frequency is then the ratio of the
size of the selection range to the hash range.
Applications of hash-based selection include:
(i) Trajectory Sampling: all routers use the same hash
selector; the hash domain includes only portions of the packet
that do not change from hop to hop. (For example, in an IP
packet, TTL is excluded.) Hence packets are consistently
selected in the sense that they are selected at all routers on
their path or none. Reports packets also include a second hash
(the label hash) that distinguishes different packets. Reports
of a given packet reaching the collector from different
routers can be used to reconstruct the path taken by the
packet. Trajectory sampling is proposed in [DuGr01]; further
description is found in [ZMRD03]; some applications are
described in Section 10.
(ii) Consistent Flow Sampling: the hash domain is a flow key.
For a given flow, either all or none of its packets are
sampled. This is accomplished without the need to maintain
flow state.
Some applications need to calculate packet hashes for purpose
other than selection (e.g. the label hash in trajectory
sampling). This can be achieved by placing a calculated hash
in the selection state, and setting the selection range to be
the whole of the hash range.
11. As noted in
https://ops.ietf.org/lists/psamp/psamp.2003/msg00038.html point 29, I
don't think that the section 4.6 "Criteria for choice of selection
operations" make sense in this draft. It could make sense in
[PSAMP-TECHNIQUE] but not here. And even in [PSAMP-TECHNIQUE], I would
put it as an appendix.12. Section 5.1 - The input sequence number should be part of the packet report or the packet interpretation? It doesn't match the definition of the packet report! See also point 9 of the MAJOR issues - "These devices may exercise the option not to provide basic reports. "This sentence should be removed because covered the MAY in the first sentence of section 5.2 - Need references for IPv4, IPv6, MPLS, AToM, BGP AS + acronyms - "The accuracy of any timestamp reported MUST be supplied in the report interpretation and made available in the MIB." But we are in the "extended packet reports" section13. Section 5.3 "If an IPFIX metering process [IPFIX-PROTO] ..." Otherwise what does "if IPFIX is supported" mean? 14. This is a general comment on the draft. I see a lot of justifications on why we should do this or why we should do that? Is this really appropriate? Maybe we just want to specify what PSAMP is instead of justifying everything. Just one example: The requirements for robustness and transparency are motivations
for including report interpretation in the report stream. Inclusion
makes the report stream self-defining. The PSAMP framework
excludes reliance on an alternative model in which interpretation
is recovered out of band. This latter approach is not robust with
respect to undocumented changes in selector configuration, and may
give rise to future architectural problems for network management
systems to coherently manage both configuration and data
collection.
15. Section 5.4
"To conserve network bandwidth and resources at the collector, the export packets MAY ..."
16. Section 7.2
- "The report stream MAY ..."
17. section 7.3
")" alone
18. section 7.4
"described in section 5.5"
19. Section 8
"collector-based rate configuration" to be removed
20. Section 10.
(i) PSAMP aims for ubiquitous deployment of packet
measurement, including devices that are not expected to
support IPFIX. This offers broader reach for existing
applications.
In practice, it will never happen!
We must support IPFIX because we must export some information elements.
21. Section 10.
MIB-II needs a reference.
22. the reference section is not up-to-date
D03 and QC03 don't exist anymore
24. The PSAMP device term must be defined, as discussed in https://ops.ietf.org/lists/psamp/psamp.2003/msg00038.html
For reference only, in http://www.ietf.org/internet-drafts/draft-ietf-ipfix-arch-01.txt * IPFIX Device:
A device hosting at least an observation point, a metering
process and an exporting process. Typically, corresponding
observation point(s), metering process(es) and exporting
process(es) are co-located at this device, for example at a
router.
That's it for now ;)
Thanks Nick for editing the draft.
See you in Seoul
Regards, Benoit.
|