RE: about draft-nordmark-multi6-noid-00

[Erik Nordmark <Erik.Nordmark@sun.com>]

> The initial packet must be sent with the "rewrite ok" bit not set.
> Suppose now that the internal routing of the site mh1 is such that packets
> sent to PC:: are forwarded through ISPB. This means that the packet
> generated by mh1 described above will be discarded by ingress filtering in
> ISPB because its source address is incompatible and cannot not be rewritten.
> The question is then, how do you deal with this case? (a tunnel between exit
> routers, so that the packet is forwarded through the appropriate isp, an
> icmp back to the initiator?)

One could deal with it any number of ways; same as what has already been
discussed by Christian and others.

I think the easiest way to *avoid* it is to, instead of the setup being
triggered by a data packet as in NOID, have an explicit setup from
the sender (as in SIM) (with ULP packet piggybacked on the setup if need be).
That way the context creation packets can carry the AIDs in the payload
and allow router rewriting of the locators.
In this case of NOID one would have to think carefully about the verification
issues in such a case (since the AID could be different than the source
locator).

> i.ii) Target perspective
> 
> Question: the first reply that the target sends back to the initiator, is
> the "rewrite ok" bit set or not?

For data packets "rewrite ok" can not be set until the context confirm has
been sent.
The context request (which might be the first packet the target sends back)
I think rewrite ok can be set.

> ii) Preserving established communications
> 
> Suppose that there is communication established between mh1 and mh2.
> Suppose that all the locators are working and that all of them were verified
> so they acceptable to be used as valid locators in the communication.
> Currently the communication is using PA:mh1 and PC:mh2 as locators.
> Suddenly a failure in the link between ISPA and its upstream provider
> occurs.
> The idea then is that ISPA detects the situation and then it announces the
> event to mh1. The internal routing system reconverges and packets are no
> longer routed through ISPA but they are routed through ISPB. Since the
> "rewrite ok" would be set at this moment of the communication, then packets
> that are originated by mh1 with PA:mh1 as source address are rewritten by
> the exit router so that PB:mh1 is included as source address. Now mh2 learns
> that it has to use PB:mh1 as destination address because this is the address
> contained in the packets that it is receiving. Is that ok?

ok

> Now my question is how long does this mechanism takes to react? i.e. the
> response time. 

There are two factors: how quickly the site border routers detect 
that a path through an ISP isn't working, and how quickly after that
the peer is aware of the change.
The first factor is independent of any multihoming solution AFAIK; has
to do with stuff like how routes are propagated, routing protocol hello
timers,  etc. 
In NOID the second factor depends on the ULP; if the ULP is sending frequently
enough the peer will notice quite quickly. If the ULP is patiently sitting
waiting for the peer to transmit and not sending anything then the peer
never notices anything using this mechanism and instead has to rely on
ULP hints about retransmissions.
I think this is also rather independent of the multihoming approach;
locator rewriting provides an efficient mechanism when it works.
The only alternative I know is to have all hosts continuosly "ping"
eachother every 10 or so seconds, which is rather inefficient.

> The problem here is that intradomain
> routing is pretty slow detecting outages. Suppose that ISPA is running BGP
> with its upstream provider, then if i am correct, the value recommended in
> the BGP spec to detect an outage between peers is 90 seconds. This means
> that the interruption in the communication will be longer than 90 seconds.
> IMHO this is not good enough to preserve established communications. Am i
> missing something?
> 
> My point is that the usage of the routing system to determine which locator
> to use and to preserve established communication is not a good option
> because:

The routing system seems to be good enough for this purpose when sites are
not multihomed. Why does multihoming make the routing system behave worse
in this respect?

> - It is pretty expensive, since you need to inject the reachability of all
> prefixes from the ISPs to the multi-homed site. 

Things are a bit less black and white than that. Depending on what failures
you  are concerned about you may not need anything but a default route from
each ISP. For example,
 - the link to the ISP failing - just a default handles that
 - the ISPs peerings with other ISPs fail and the ISP has no alternate path;
   maybe you assume that the ISP knows how to handle this. If not you
   might need the ISP to give you the whole routing table.
 - internal failures in the ISP e.g. cutting off your PoP; perhaps just a
   default route can handle this if the routers in the PoP base advertising
   the default on hearing IGP routes from other parts of the ISP's network
 - the ISP having a broken setup where the BGP routes they offer to you
   don't get withdrawn even if they've lost all upstream connectivity 3 days
ago.
   Oops - even receiving the full BGP table from then doesn't handle that
   type of failure.

> It is then my conclusion that path outage detection and locator selection
> has to be performed by the end-host themselves and not by the routing
> system.

This means that every host-pair that communicate (have an existing ULP
connection/association - btw I don't know how to tell this for associations
implemented on top of UDP) have to exchange end-to-end hello packets every 10
seconds or so. If you want to do VoIP you need to do them about every 100 
milliseconds. Imagine 100 Billion hosts or so communicating across the Internet
doing such e2e hello's. Couldn't that be another form of scaling problem?

  Erik