RE: multihoming issues via SCTP

[Iljitsch van Beijnum <iljitsch@muada.com>]

On Tue, 11 Sep 2001, Coene Lode wrote:

> > This means there have to be more paths:

> > A1->X->Y->B1
> > A1->X->B2
> > A2->Y->B1
> > A2->Y->X->B2

> yes,  but does routing allow them to be used? We got the impression that at
> a certain time you can have only one path through the network for a certain
> destination address.

If we define a path as being from a source interface to a destination
interface, then the number of paths is only limited by the product of the
number of interfaces on both sides. Obviously, many of those paths will
share a lot of infrastructure, but they won't be identical.

I think the main problem is that in most cases a default route will be
used, which is tied to one interface. That means the paths originating at
the different interfaces are effectively identical.

> > However, this assumes both that the output interface stays
> > the same over
> > the life time of a connection and that the end host is the
> > one making the
> > decision how packets are routed to the outside. Both are
> > unlikely.

> Well, but that is how it is done with SCTP. SCTP specifies the destination
> address  and IP should try to route based on that IP address to B.

Right. So the decision about the output path is done in IP and SCTP
probably can't control this.

> If B1 is taken as destination address then the source address should be the
> corresponding A1(and that is something done by IP in host A).

I suppose it is possible to do this, although TCP doesn't.

> > If an
> > output interface fails, hosts may simply reroute outgoing
> > packets over the
> > other interface.

> That is a possibility, but that depends on who finds it first. If SCTP
> detects that the interface has gone(via the heartbeats that aren't
> acknowledge), then it will changeover to another destination address(thus
> taking the other interface)

The fact that host B is suddenly sending packets to interface 2 instead of
1 of host A, doesn't influence outgoing packets on host A. Assuming B1 and
B2 are both routed over the same interface (which is likely unless the
host has a full routing table, something that never happens today) host A
will have to detect the failure of interface 1 and reroute over interface
2.

> The question for me is: Does it happen that the host changes it routing
> tables?

Depends on the host. I think most will do this, but it never hurts to
experiment. If this doesn't happen automatically when using defaults, it
is always possible to run RIP.

> > And in most cases, routers will reroute packets over
> > another path if a connection to an ISP fails. So the other
> > side will often
> > still see incoming traffic from a remote address that is no longer
> > reachable for outgoing packets.

> Uhh, doesn't that mean that then the alternate address should be used
> instead of the original addresses?

If the router does this, how would SCTP know a different source address
has to be used?

> And as far as I understand something from routing and routing protocols,
> doesn't that take time?

Of course. Depends on the interface, though: some can detect failures very
fast (less than a second) while others won't see failures at all and there
has to be a timeout in the routing protocol. This takes 30 to 180 seconds.

The former is likely in point-to-point links, the latter in switched
environments (Ethernet, ATM).

> Changing routes in IP networks
> seem to involve first searching for them and waiting for it till the routing
> protocol converges. And it is during that convergence faze that the packets
> will be routed to probably the middle of nowhere(or better said: they are
> dropped)

There are many different routing protocols, all with different properties.
For instance, RIP waits for minutes to decide that a destination is
unreachable and then removes it from the routing table. OSPF and IS-IS on
the other hand, will detect all failures in 30 seconds, and interfaces
that go down immediately. Then the update is flooded through the network
and all routers recompute the topology. In BGP the detection process is
much the same but each router recomputes the routing changes individually.
So in BGP it may take a few moments for routing to stabilize.

Routers do not necessarily drop packets while updating routing
information, but they may route them to unreachable destinations or in
loops for a while.

> > Also, I don't like the idea of a protocol sending keepalives
> > over a path
> > it is otherwise not using. This is a waste of resources and
> > can trigger
> > inefficient use of different kinds of caches.

> Well the alternate routes are not only used for heartbeats, but if the SACK
> don't get back, then the alternative routes are used for the SACK, so they
> are not really idle in the strictest sense of the word.

Well if everything is working, then the other paths aren't used, right?

> What about caches: well caches (in my very humble opinion) shouldn't be
> there in the first place,

Some routers use a "route cache" where the most recently used routing
information is stored. ARP or neighbor discovery are also a form of
cacheing. Why should processing all of this be triggered just to see if a
path is available if you don't intend to use it?

> > Also, it won't do much good
> > unless there are three or more paths of which at least two fail, a
> > situation that is not likely to be very common.

> Murphy law: Failures of links, interfaces and routers will happen.(once upon
> a time, we believed this tale that failures wouldn't happen, but then
> reality and a lot of users thought us otherwise)

I'm not saying failures won't happen, just that in most cases the
heartbeats won't help you. If there are only two paths, you'll have to use
the second one when the first one fails. So why bother to see if the
second path is alive? You'll find out soon enough. If you have three paths
and one fails, then each of the others still work so no need to use
heartbeats again. Only when two paths fail at the same time, it helps to
know which secondary path has also failed so you know to avoid that one
and pick the other.

If doing something and doing nothing yield the same results, the latter is
preferable.

Iljitsch