Re: Preserving established communications (was RE: about draft-nordmark-multi6-noid-00)

[Iljitsch van Beijnum <iljitsch@muada.com>]

On 27 okt 2003, at 13:25, marcelo bagnulo wrote:

> Current studies of bgp convergence time are talking about up to 15 
> minutes
> of BGP convergence when a route is withdrawn. Established 
> communications
> clearly don't survive this.

Ok, but answer me this: how does established communication survive the 
fact that the route to one of the endpoints is withdrawn in the first 
place? No route means no connectivity.

(Un)fortunately these researchers often don't really understand BGP so 
they model it in the wrong way. If you build a huge network, infuse it 
with instability and then revoke a route, you could perhaps end up with 
a 15 minute wait before the route has really disappeared everywhere. 
However in practice it isn't this bad. I've done some tests a while ago 
and what I saw pretty consistently was two minutes.

But apart from that revoking a route at the source isn't a very typical 
condition or one that we need to survive. What we need is to detect a 
broken link and reroute. The first part can take up to three minutes 
because of Cisco's unbelievable default hold time, but it doesn't have 
to: if one end sets a lower hold time, that one is used, and routers 
are by default configured to immediately tear down BGP sessions when 
the associated interface goes down.

Rerouting is a matter of seconds in the absense of prior instability, 
or 30 seconds otherwise.

So many outages don't even lead to a noticable interruption, a good 
number make TCP sessions hang for a relatively long time (30 - 180 
seconds) but don't break them, and only under very unfortunate 
circumstances rerouting takes longer than a TCP timeout.

Note that there are efforts underway to detect a dead neighbor very 
quickly. This should take the edge off of these problems by eliminating 
the 180 second timeout. See 
http://www.ietf.org/internet-drafts/draft-katz-ward-bfd-01.txt

> > have to exchange end-to-end hello packets every 10
> > seconds or so. If you want to do VoIP you need to do them about every 
> > 100
> > milliseconds. Imagine 100 Billion hosts or so communicating across 
> > the Internet doing such e2e hello's. Couldn't that be another form of 
> > scaling problem?

> :-)
> Well, som additional cosndierations about this.
> First, many existing ULP already exchange this type of messages, TCP 
> ack for
> instance, so hellos can be piggybacked into this messages.

If you get an ACK, you know you have reachability so no need to 
piggyback anything. I think it would be a very good idea to dump the 
job of detecting _potential_ failures in the lap of layer 4 (which for 
many real time protocols means the application). TCP has a very good 
idea of how reachable the other end is. For real time protocols, it's 
probably either impossible or too expensive to make rehoming seamless. 
But why would we have to? A short hiccup in the audio/video would be 
bad, but not as bad as 1. sending ACKs ever 100 ms and 2. rehoming 
every time a single packet is lost. It's not like there are outages so 
rehoming is necessary every few minutes.