ISPMON BOF - IETF57 - Meeting Minutes (DRAFT)

[Gianluca Iannaccone <gianluca@sprintlabs.com>]

All,

please find attached the draft version of the meeting 
minutes. i am sure i missed/misinterpreted/misspelled 
something. please let me know.

you can also find the slides of the presentations at 
http://www.sprintlabs.com/~gianluca/ietf57. 

thanks,
gianluca



----------------------------------------------------------
Gianluca Iannaccone      '         gianluca@sprintlabs.com
Sprint ATL               '           tel +1 (650) 375-4198
1 Adrian Court           '           fax +1 (650) 375-4079
Burlingame, CA 94010     '    www.sprintlabs.com/~gianluca

Monitoring Infrastructure Deployment (ispmon) BOF
IETF 57, Vienna, July 17, 2003
Meeting Minutes 

Reported by Gianluca Iannaccone (chair) based on notes 
from Matthew Zekauskas.

Agenda bashing

1. Presentations:
  . Deployment of inter-operable and cost-effective 
    monitoring infrastructures in ISP networks 
    (Gianluca Iannaccone, 15')
  . Improving measurement and monitoring for ISPs 
    (Nevil Brownlee, 10’)
  . Sprint’s Continuous Monitoring project 
    (Gianluca Iannaccone, 10’)
  . IPPM Applicability Statement 
    (Henk Uijterwaal, 10’)
  . Multi-domain monitoring across European Research Network 
    (Nicolas Simar, 10’)

2. Scope and candidate charter (15’)

No comments were made on the agenda. 

--
Gianluca Iannaccone presented the proposal draft 
"draft-bhattacharyya-monitoring-deployment-00". 

Three main application areas for monitoring: i) network 
resource usage, ii) traffic accounting, iii) fault diagnosis. 
The three areas have different requirements in terms of 
timescale of interest, granularity of information and 
location of measurement devices. 

The first area has been addressed extensively (OPSTAT, SNMP, RMON). 
The second is currently the focus of IPFIX & PSAMP working group. 
The third is partially addressed by IPPM. Some relevant aspects 
of monitoring (namely storage, aging of data and control plane 
functionalities) are not addressed by any working group.

. It was observed that traffic accounting is also present in the 
  charter of the TEWG but that is has not been mentioned in the draft
  or in the presentation. 

. Matt Mathis observed that a lot of the aspects touched in the draft 
  have already been worked on at the IETF. Previous efforts have failed 
  to converge to a solution because they are really hard problems. 
  For example, the OPSTAT WG addressed the problem of a common format for 
  statistics and ended up arguing for long times about 5 minutes averaging 
  intervals. The same has been experienced in the IPPM WG. The idea was 
  to have an algebra of metrics to be composed, i.e. divide a long path 
  in sections and be able to characterize each section separately and to 
  compose the results to understand the performance on the entire path. 
  This effort has also not converged. 

  A second concern is that some other issues mentioned in the draft seem 
  to belong to the conversations between ISPs and monitoring equipment 
  vendors. Solutions in that area permit vendor differentiation. The IETF 
  does not belong in that space.

. Jon Bennett suggested that the IP Measurement Protocol (IPMP) could be 
  a viable solution for the metric composition problem but that it does 
  not have a WG where it can be discussed. Matt Mathis agreed with Jon 
  Bennett but objects that a group more focused on a specific problem 
  would be able to converge to a solution. 

. Gianluca Iannaccone argued on the second concern (ISP/vendor issues not 
  of interest for IETF) that an ISP would like to standardize the way data
  is collected and stored not the specific analysis or statistical methods
  applied to the data. That area belongs to the IETF and there is interest 
  from ISPs for two reasons: i) ISPs want to have access to multiple vendors, 
  and ii) ISPs want to be able to share part of the collected data. 

  Matt Mathis opposed that sharing statistics between ISPs is something that 
  comes from university world. It is unlikely that ISPs will be willing to 
  share data ("give away maps of wallets"). 

  Gianluca Iannaccone claimed that for network resource usage & traffic 
  accounting there is not much to be shared but for other issues (e.g., 
  fault diagnosis or incident reporting) sharing would be very useful. 

. Rudiger Deib asked what IPFIX, PSAMP are doing wrong that justifies the 
  need for a new WG. Gianluca Iannaccone answered that they are focused on 
  specific applications (traffic accounting area) and do not address some 
  practical problems ISPs currently have (such as storage, aging, control 
  plane). 

--
Nevil Brownlee presented a slide show about "Improving measurement 
and monitoring for ISPs"

ISPs have always monitored their networks. Recently, they also started to 
give near real-time reports of network performance via web pages. However, 
measurement is still an art underpinned by science. An ISPMON WG could 
improve this situation, thereby helping ISPs. Need to find better ways to 
"use" network data. Collection is always easy. Storing, visualizing and 
analyzing data is difficult to do. There are tools (MRTG, Argus, snort, bro, 
...) but can we do better? Would it be worth looking again at something
like the OPSTAT reporting scheme (RFC1857)?

. Nevil Browlee agreed with Matt Mathis that these issues are hard problems
  addressed in the past and that did not get too far. There are a lot of 
  different measurement infrastructures out there but they work only with 
  fairly well specified goals. To build a general infrastructure is really 
  hard. This BOF should define a reasonable number of problems where we can 
  make progress. 

. Nick Duffield asked how the metrics presented in the slides (slides 3/4)
  sit with activities in IPPM WG. 

. Naharito Hakuda (?) declared that the slides are of interest and asked 
  the speaker what he thinks about itrace technology. Nevil Brownlee answered
  that the traceback of packets across providers is an active research area. 
  It is an area that providers should be interested in but currently have not 
  enough experience to say which way is the right way.

. It was observed that the example shown in the slideshow appears to refer to 
  a work addressed in existing WG, IPFIX and IPPM. Gianluca Iannaccone 
  answered that this BOF would not standardize things addressed in other WG 
  but there are two aspects not addressed: i) feedback from ISPs on what 
  applications are most interesting and ii) what to do once the data is 
  collected (storage, aging) is not defined there. 

. Question from the audience: "Why not standardize also those aspects in those 
  groups?" Nevil Brownlee answered that no single WG covers all metrics. So 
  one useful thing ISPMON could do is to create a laundry list. It could be a
  group for sharing experiences. 

. Merike Kaeo, co-chair of IPPM, mentioned that in IPPM and in TEWG there are 
  similar efforts going on. IPPM has not been able to find anyone in ISP 
  community to contribute to the WG. Also the TEWG has a document on 
  requirements for measurement where chairs are trying to get input from 
  ISPs. It seems that this is exactly what ISPMON is trying to do. 

-- 
Gianluca Iannaccone presented a slide show on Sprint's effort in monitoring
the IP backbone network. That effort is the reason behind proposing the BOF 
and the draft on monitoring infrastructure deployment. 

No specific comments. 

-- 
Nicolas Simar presented a slide show on "Multi-domain monitoring across 
European Research Network". Five different research network have built a
team to exchange measurement data across domains and share the monitoring 
infrastructure among providers. See slides for more details. 

. Rudiger Geib asked if the effort is on concatenating measurement done 
  by different tools or also end-to-end measurement as well. Nicolas Simar
  answered that end-to-end is out of scope here. However, if end-to-end 
  is available it could be used to compare it with concatenation of 
  measurements to see where a problem is. 

. Rudiger Geib asked then if in case of different tools you share raw data
  or evaluated data. In case of evaluated that is not addressed in IPFIX or 
  PSAMP. But then it is out of scope for ISPMON as well. In case of raw data
  NOC would have to be aware of every data format. Nicolas Simar suggested 
  that raw data would be better and that providers need to agree on common 
  format per metric. 

--
Henk Uijterwaal presented a slide show about the IPPM Applicability Statement. 
There are many open parameters in the metrics defined in IPPM. Not simple to 
configure. Feedback from operators would be helpful to identify what to 
measure, when to measure and how to configure. So far, it has not been 
successful. 

. Bert Wijnen observed that if operators do not give feedback then there is 
  no reason to do it. Henk Uijterwaal answered that operators always said 
  they are interested. Merike Kaeo noted that privately all operators think 
  it is a good idea but noone would provide input to the mailing list. 
  Rudiger Geib observed that may be it is because there are very few one-way 
  delay systems installed. 

. Gianluca Iannaccone pointed out that one of the problems providers have is 
  that out-of-context questions on applicability are hard to answer. The goal 
  of this BOF would be to provide that context. 

. Dave McDyson confirmed that within MCI there is interest in IPPM but 
  difficult to attend WG there are higher priorities. It is not clear whether 
  a new WG would help at all. It would be valuable instead to have service 
  providers to meet informally in a forum. It is not clear that this is a 
  valid justification for a WG. 

. Randy Bush (AD that gave the OK to this BOF) objected that there is still no 
  driving reason for a WG. None of the issues pointed out by the chair appear 
  to have traction. Underlying the discussion there is the standardization of 
  disk formats and data exchange rather than what measurement metrics are of 
  interest. 

--
Gianluca Iannaccone presented to the room the draft charter for the BOF: 

 1. Provide BCP documents
     . what ISPs need to monitor and what metrics.
     . how to instrument monitoring systems in large-scale provider networks. 
     . describe known-to-work implementations and open issues. 
 2. Specify ways for ISPs to share/compare monitoring data. 
     . common metrics and analysis methods.
     . common formats for collected monitoring data (e.g., packet traces)
 3. Specify components of monitoring infrastructure not addressed in 
    existing WGs
     . storage/aging of collected data
     . statistical analysis of traffic
     . control plane functionalities
 4. Make recommendations to other WGs standardizing different elements of 
    monitoring infrastructure
     . e.g., IPFIX, IPPM, PSAMP, INCH, IDWG, TEWG, etc.,

. Randy Bush asked for the projector to show the table of content of RFC 1857
  where one chapter is dedicated to storage format. That RFC is dated 1995. 
  Gianluca Iannaccone rejected the idea that RFC 1857 is applicable to today's 
  measurement infrastructure. RFC 1857 focus was only on network resource 
  usage metrics (utilization, etc.) not on passive measurements, flow 
  measurements, etc. 

--
The chair closed the meeting getting the sense of the room. 

. Hum if support the BOF: very mild hum
. Hum if oppose the BOF: slightly louder hum 

The majority of the room stays silent. 

The room opposes the BOF.