[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

comments on draft-ietf-psamp-framework-05.txt



Dear all,

Here is a list of comments on the framework version 5 draft.
As always, feel free to start a new thread on a specific topic discussed below, with a new email subject.
I divided the issues in MAJOR and NON-MAJOR (I didn't want to say MINOR as some of them are not minor)
Note that as it was decided that this draft would be a standard track document, this has been considered in the review. And this is the reason why some new issues appeared!
Note also that some of the comments were already discussed in https://ops.ietf.org/lists/psamp/psamp.2003/msg00038.html. If this is the case, a reference to this posting will be provided.

MAJOR
---------

1. References
We should have a set of normative versus informative references.
In normative, we will have [PSAMP-PROTO], [PSAMP-INFO], [PSAMP-MIB], [PSAMP-TECHNIQUE] I guess
Does it mean that we should wait for the 3 drafts above to be completed before submitting this one as RFC?

2. Terminology section
As a standard track document, it must be the same as the PSAMP protocol draft (or almost), which in turn will be the same as the IPFIX protocol draft (for which we just agreed upon)
Does it mean that we should wait for the [PSAMP-PROTO] to be completed before submitting this one as RFC?

3. Abstract
BTW, I fully agree with Tom Petch's comment that the abstract is confusing.
See post https://psg.com/lists/psamp/psamp.2004/msg00023.html
What is the goal of the document? A framework + requirement + architecture?
Two minor comments
    - "describes" must be changed by "specifies".
    - PSAMP acronym must be explained.

4. Draft table of content for the requirements.
      3.   Requirements................................................7 
      3.1  Selection Process Requirements..............................7 
      3.2  Reporting Process Requirements..............................8 
      3.3  Export Process Requirements.................................8
      3.4  Configuration Requirements..................................9  
But there are requirements all over the place in the draft.
For example.
      5.   Reporting Process..........................................15 
      5.1  Mandatory Contents of Packet Reports (MUST)................15 
      5.2  Extended Packet Reports (MAY)..............................16 
or
      7.   Export Process.............................................19 
My confusion comes from the section 3 which seems to be a complete list of requirements but actually it's not...
Just another example of the confusion: Section 4.2 "PSAMP Packet Selection Operations" is full of requirements.
So should we have a new subsection 3 with potential further references to this section?

Some more comments on section 3:
    - Does section 3 define only the generic requirements?
    - Some of the requirement in section 3 don't have a MAY, SHOULD, MUST or lower case may!
    - I don't understand some requirement. For example:
   
      * Transparency: allow transparent interpretation of the report 
        stream, without any need to obtain additional information 
        concerning the observed packet stream. 
       
      * Robustness to Information Loss: allow robust interpretation of 
        the report stream with respect to packet reports missing due to 
        data loss, e.g. in transport, or within the selection, reporting 
        or exporting processes.  Inclusion in reporting of information 
        that enables the accuracy of measurements to be determined. 
        What does it mean: interpretation of the report stream?
        It means everything and nothing? How to determine what is required?
        Maybe we want to distinguish in these subsections what we would like to achieve and
        what the requirements are!

5. Terminology section.
The order (as described by Tom Petch in https://psg.com/lists/psamp/psamp.2004/msg00023.html) is not adequate and intuitive.
For example, some terms are used but defined later: reporting process, selectors, etc...
As noted https://ops.ietf.org/lists/psamp/psamp.2003/msg00038.html point 2 and point 5, some diagrams would be useful.

Observation Point            primitive selector 1       primitive selector 2            
                             (selection process 1)       (selection process 2)
Observed Packet Stream ->           -> Packet Stream        -> Packet Stream

To clarify: 
	- one or more observation points
	- observed packet stream versus packet stream

Another diagram is needed with the selector, composite selection process, primitive section process, composite selector.

6. Upper cases for the terms defined in the terminology section
As agreed in point 6 of https://ops.ietf.org/lists/psamp/psamp.2003/msg00038.html
Specifically with the forward definition in the terminology section.

7. A new architecture section?
Should a new architecture section be developped in this draft around the text below?
Anyway, this paragraph doesn't belong to the terminology section
     Various possibilities for the high level architecture of these 
      elements are as follows. 
       
          MP = Measurement Process, EP = Export Process 
       
         +---------------------+                 +------------------+ 
         |Observation Point(s) |                 | Collector(1)     | 
         |MP(s)--->EP----------+---------------->|                  |     
         |MP(s)--->EP----------+-------+-------->|                  | 
         +---------------------+       |         +------------------+ 
                                       |     
         +---------------------+       |         +------------------+ 
         |Observation Point(s) |       +-------->| Collector(2)     | 
         |MP(s)--->EP----------+---------------->|                  | 
         +---------------------+                 +------------------+ 
                                          
         +---------------------+          
         |Observation Point(s) |          
         |MP(s)--->EP---+      |          
         |              |      |          
         |Collector(3)<-+      | 
         +---------------------+   
8. Section 4.1 "packet selection terminology"
This is exactly (most of the time) the same as the http://www.ietf.org/internet-drafts/draft-ietf-psamp-sample-tech-04.txt
What's the point to duplicate the info in this draft? We should just have a reference to the other draft.
On the top of that, not all the terms are used in this draft.
Anyway some comments (so maybe for Tanja?):
     - Approximative Selection: selection operations in any of the above categories ??
    -  size of the population -> population size
	Attained Selection Frequency: the actual frequency with which 
        packets are selected by a selection process. The attained 
        sampling frequency is calculated as ratio of the size of a sample 
        size to the size of the population from which it was selected.  
    - the paragraph is almost the same as [PSAMP-TECHNIQUE] but not quite -> very confusing

9. Packet Report and Packet Interpretation.
Those 2 terms were nice to describe the PSAMP requirements.
But, as we know that we will use IPFIX and as we know we don't have those two terms in IPFIX, do we need them in this draft?
I explain! In section 5.4, we see
    
      Information for use in report interpretation MUST include  
       
           (i) configuration parameters of the selectors of the packets 
           reported on.  
            
           (ii) format of the packet report; 
            
           (iii) indication of the inherent accuracy of the reported 
           quantities, e.g., of the packet timestamp.  
            
           (iv) identifiers for observation point, measurement process, 
           and export process.  
(i) will use the concept of selector ID
(ii), (iii) will be sent as information element in a template
(iv) observation point, export process will be sent as information element in a template
So it's not like we would send a separate packet with all the report interpretation information. So I'm thinking the concepts of report interpretation doesn't make sense from a [PSAMP-PROTO] point of view and is even confusing. Why have it here?
On the same topic, the following is not true, as described in [PSAMP-PROTO], as I just shown:
      It is not envisaged that all report interpretation be included in 
      every packet report. Many of the quantities listed above are 
      expected to be relatively static; they could be communicated 
      periodically, and upon change. 
10. Section 7.2
- "Note that IPFIX being still developped; this is given only as a possible example"
  Is this appropriate for a standard track doc.?
  Should we wait for IPFIX to be published before this draft?
11. Transport protocol
In section 7.3, 7.5, 7.6, I think we should clearly express the choice from IPFIX.
http://ipfix.doit.wisc.edu/archive/2377.html
And clearly give the options.
Section 7.7 must dissapear!



NON-MAJOR ISSUES
-------------------------

1. Motivation
    - this section must be changed according to the new abstract
    - "describes" must be changed by "specifies".
    - this section speaks of "capabilities of network elements" while the abstract speaks of "deployment in router interfaces"
       Is it network element wide or router interface wide. We must be consistent!
    - "(iii) describe a protocol by which information on sampled packets is reported to applications;"
       not needed as IPFIX is selected
    - "(iv) describe a protocol by which packet selection and reporting are configured."
      We speak of a MIB here. But I guess we don't want to describe SNMP...
      I would just express: specify some requirements for a MIB
    - The following paragraph is not adequate anymore as IPFIX and its transport protocol are selected.
      Conversely, we expect that aspects of this framework not 
      specifically concerned with the central issue of packet selection 
      and report formation may be able to leverage work in other Working 
      Groups. Potential examples are the format and export of reports on 
      selected packets, which may leverage the information model and 
      export protocols of IP Flow Information Export (IPFIX) [QZCZ03], 
      and work in congestion aware unreliable transport in the Datagram 
      Congestion Control Protocol (DCCP) [FHK02], and related work in The 
      Stream Control Transmission Protocol (SCTP) [SCTP] and the SCTP 
      Partial Reliability Extension [SCTP-PR]. 
2.  Selection State definition.
           (ii) a timestamp of observation of the packet at the 
      observation points; 
observation point must be singular

3. Packet Report definition.
It's not obvious at all from the definition that you mean what is the section  5.1. Some examples or clarifications needed.

4. New section "PSAMP and IPFIX interfaction" for the text below.
      The PSAMP measurement process can be viewed as analogous to the 
      IPFIX metering process. The PSAMP measurement process takes an 
      observed packet stream as its input, and produces packet reports as 
      its output. The IPFIX metering process produces flow records as its 
      output. The distinct name “measurement process” has been retained 
      in order to avoid potential confusion in settings where IPFIX and 
      PSAMP coexist, and in order to avoid the implicit requirement that 
      the PSAMP version satisfy the requirements of an IPFIX metering 
      process (at least while these are under development). The relation 
      between PSAMP and IPFIX is further discussed in [QC03]. 
(at least while these are under development)? Not adequate for standard track!
Draft [QC03] is not valid anymore

5. A space to be deleted
      A specific reporting processes meeting these requirements, and th e 
      requirement for ubiquity, is described in Section 5. 
6. References.
Instead of having unreadable references like [QZCZ03], the following references would make more sense, and are used in the IPFIX and PSAMP protocol drafts: IPFIX-REQ, IPFIX-PROTO, IPFIX-INFO, PSAMP-MIB, PSAMP-FRAME, PSAMP-INFO, PSAMP-TECHNIQUE

7. Following the previous point, we would need a "PSAMP Documents Overview".
In IPFIX serie of drafts, we have the following. To be adapted for PSAMP
   IPFIX Documents Overview 
    
   The IPFIX protocol provides network administrators with access to IP 
   flow information. The architecture for the export of measured IP 
   flow information out of an IPFIX exporting process to a collecting 
   processing is defined in [IPFIX-ARCH], per the requirements defined 
   in [IPFIX-REQ]. [IPFIX-PROTO] specifies how IPFIX flow record data, 
   options record data and control information is carried via a 
   congestion-aware transport protocol from IPFIX exporting process to 
   IPFIX collecting process. IPFIX has a formal description of IPFIX 
   information elements (fields), their name, type and additional 
   semantic information, as specified in [IPFIX-INFO]. Finally [IPFIX-
   AS] describes what type of applications can use the IPFIX protocol 
   and how they can use the information provided. It furthermore shows 
   how the IPFIX framework relates to other architectures and 
   frameworks. 
8. section 4.2 "PSAMP Packet Selection Operations.
No upper cases in Spacing and Interval Length.

9. [ZMRD03] not in the reference section

10. The following paragraphs should be in [PSAMP-TECHNIQUE] only, IMHO
        When the hash function is sufficiently good, hash-based selection 
        can be used to approximate uniform random sampling over the hash 
        domain. The target sampling frequency is then the ratio of the 
        size of the selection range to the hash range. 
         
        Applications of hash-based selection include: 
            
           (i) Trajectory Sampling: all routers use the same hash 
           selector; the hash domain includes only portions of the packet 
           that do not change from hop to hop. (For example, in an IP 
           packet, TTL is excluded.) Hence packets are consistently 
           selected in the sense that they are selected at all routers on 
           their path or none. Reports packets also include a second hash 
           (the label hash) that distinguishes different packets. Reports 
           of a given packet reaching the collector from different 
           routers can be used to reconstruct the path taken by the 
           packet. Trajectory sampling is proposed in [DuGr01]; further 
           description is found in [ZMRD03]; some applications are 
           described in Section 10. 

           (ii) Consistent Flow Sampling: the hash domain is a flow key. 
           For a given flow, either all or none of its packets are 
           sampled. This is accomplished without the need to maintain 
           flow state. 
       
           Some applications need to calculate packet hashes for purpose 
           other than selection (e.g. the label hash in trajectory 
           sampling). This can be achieved by placing a calculated hash 
           in the selection state, and setting the selection range to be 
           the whole of the hash range. 
11. As noted in https://ops.ietf.org/lists/psamp/psamp.2003/msg00038.html point 29, I don't think that the section 4.6 "Criteria for choice of selection operations" make sense in this draft. It could make sense in [PSAMP-TECHNIQUE] but not here. And even in [PSAMP-TECHNIQUE], I would put it as an appendix.

12. Section 5.1
- The input sequence number should be part of the packet report or the packet interpretation?
It doesn't match the definition of the packet report! See also point 9 of the MAJOR issues
- "These devices may exercise the option not to provide basic reports. "This sentence should be removed because covered the MAY in the first sentence of section 5.2
- Need references for IPv4, IPv6, MPLS, AToM, BGP AS 
   + acronyms
- "The accuracy of any timestamp reported MUST be supplied in the report interpretation and made available in the MIB."
But we are in the "extended packet reports" section
13. Section 5.3
"If an IPFIX metering process [IPFIX-PROTO] ..."
Otherwise what does "if IPFIX is supported" mean?

14. This is a general comment on the draft. I see a lot of justifications on why we should do this or why we should do that?
Is this really appropriate? Maybe we just want to specify what PSAMP is instead of justifying everything.
Just one example:
      The requirements for robustness and transparency are motivations 
      for including report interpretation in the report stream. Inclusion 
      makes the report stream self-defining.  The PSAMP framework 
      excludes reliance on an alternative model in which interpretation 
      is recovered out of band. This latter approach is not robust with 
      respect to undocumented changes in selector configuration, and may 
      give rise to future architectural problems for network management 
      systems to coherently manage both configuration and data 
      collection. 

15. Section 5.4
"To conserve network bandwidth and resources at the collector, the export packets MAY ..."

16. Section 7.2
- "The report stream MAY ..."

17. section 7.3
")" alone

18. section 7.4
"described in section 5.5"

19. Section 8
"collector-based rate configuration" to be removed

20. Section 10.
           (i) PSAMP aims for ubiquitous deployment of packet 
           measurement, including devices that are not expected to 
           support IPFIX. This offers broader reach for existing 
           applications.   
In practice, it will never happen!
We must support IPFIX because we must export some information elements.

21. Section 10.
MIB-II needs a reference.

22. the reference section is not up-to-date 
D03 and QC03 don't exist anymore

24. The PSAMP device term must be defined, as discussed in https://ops.ietf.org/lists/psamp/psamp.2003/msg00038.html
For reference only, in http://www.ietf.org/internet-drafts/draft-ietf-ipfix-arch-01.txt     * IPFIX Device:
       A device hosting at least an observation point, a metering
       process and an exporting process. Typically, corresponding
       observation point(s), metering process(es) and exporting
       process(es) are co-located at this device, for example at a
       router.



That's it for now ;)
Thanks Nick for editing the draft.
See you in Seoul

Regards, Benoit.