Router solutions

[Iljitsch van Beijnum <iljitsch@muada.com>]

Hi,

Here some text. It is far from perfect. Send me your comments and it
will improve. This should probably find its way into another document
(roadmap?).

During the last "rebel meeting" I promised I'd write something about the
different ways to do multihoming using currently available routing
mechanisms. In this message I'll list the three ways routing-based
multihoming is done today in IPv4 and discuss mechanisms to adapt them
to IPv6. For simplicity, multihomed networks are assumed to have two
upstream ISPs.

1. Announcing a provider independent prefix

This is the simplest solution, and it is in wide use in IPv4: the
multihomed network advertises a globally visible prefix over two or more
ISPs. This method provides many benefits to the multihomed network and
is simple for ISPs to configure and manage, but it has important
scalability problems as there are no limits on the number of these
prefixes. Also, if and when networks elsewhere decide to filter out the
multihomer's announcement, the multihomer becomes unreachable. In IPv4,
filtering is almost guaranteed for prefixes longer than /24 and some
networks are known to filter on RIR allocation prefix length (usually
/20).

2. Shooting holes in ISP PA block

The multihomed network gets a prefix out of one of its ISP's regular
address blocks. The multihomed network announces its prefix both to the
ISP it got the addresses from (the primary ISP) and the other ISP (the
secondary ISP). As long as there is no filtering further upstream, such
a prefix provides the exact same benefits as a PI block, with one
disadvantage: the multihomer has to renumber when leaving the primary
ISP. The upside is that if there is filtering, the multihomed network is
still reachable over the primary ISP, and possibly through the primary
ISP and then the secondary if the link between the primary ISP and the
multihomed network is down. However, in the presence of filtering, the
multihomed network doesn't enjoy the full benefit of being mulithomed as
it isn't protected against all failure modes. More specifically, if the
primary ISP doesn't accept the multihomer's prefix from the secondary
ISP, there role of the secondary ISP is very limited. If the pirmary ISP
accepts the multihomer's prefix from the secondary ISP, the multihomer
should always have full reachability as long as there aren't any wide
scale problems within the primary ISP's network and peering between both
ISPs is operational.

3. Requesting a PA block

Some multihomed networks act as ISPs and request their own provider
aggregatable prefix / pTLA.

4. Scalability of portable address space solutions

Solutions based on portable addresses (both 1. and 3.) are the most
desirable, since they provide the best possible multihoming, but their
scalability is also the most problematic. It should be possible to
accommodate a larger number of portable prefixes by using an aggregation
mechanism, but aggregation across ISPs is complex. (See geographical
aggregation as explained in draft-van-beijnum-multi6-isp-int-aggr-00.txt,
not further explained here for space reasons.) Also, the presence of
other multihoming solutions will help slow down demand for portable
prefixes. However, it is still likely the number of these prefixes will
grow too large for many networks in the medium or long term. So their
must be either a way to limit the number of portable prefixes, or a
course of action to take when the number of portable prefixes becomes
too large.

Simply creating administrative hurdles or asking a high fee for
assignment of such a prefix is unlikely to have the desired long-term
effect if the need for multihoming grows significantly beyond what is
seen today. Setting a hard limit on either the number of portable
prefixes or their growth creates risks for a landrush and litigation.
Not limiting this number will probably have the effect that some ISPs
may be unwilling to carry the prefixes, even if their number is small at
first, because they are afraid the number of these will get out of hand.
This will lead to a situation similar to the one in IPv4, where the
owner of a portable prefix can't be sure the prefix will be globally
reachable. This situation is undesirable, as in this situation many of
the disadvantages of portable prefixes are present (cost to the
community of carrying them) while at the same time the benefit of having
such a prefix is limited (lack of global connectivity). It has been
proposed to start assigning these prefixes with the understanding they
will be revoked at a speficied time in the future. While this will help
keep the IPv6 global routing table clean in the long run, this doesn't
do anything for the short term.

An alternative could be to reach consensus between ISPs to carry a
certain number of these prefixes, for instance a /36 (4096 /48s) worth
per Regional Internet Registry. It is likely additional /36s will be
allocated in the future. At this time, a new round of consensus building
is required. The advantage of doing this for a larger block of prefixes
would be that it removes much of the uncertainty and risk for both the
ISPs and the multihomed customers. ISPs know they won't be forced to
carry more than a certain number of prefixes, while for customers the
uncertainty of requesting a prefix and then having to wait and see if it
is routable is greatly diminished.

5. Multihoming properties of non-portable prefixes

For shooting holes in pTLA blocks, scalability isn't the biggest
problem, as it is assumed that a good number of ISPs will filter the
individual /48s. If not, this solution is essentially the same as that
using portable prefixes. However, this filtering potentially removes the
multihoming benefits. There are two ways to solve this: the sharing of
blocks of address space between ISPs, and introducing a hierarchy. Both
these solutions have the potential to introduce a significant number of
prefixes, in the order of several ten thousands, in the global routing
table.

5.1 Multihomed address space sharing between two ISPs

Registries could assign blocks of address space to all combinations of
two ISPs that share multihomed customers. Both ISPs then announce this
block to their peers. This solution assumes good interconnection between
the ISPs involved, because otherwise the multihomed customer will be
unreachable from part of the net when one ISP link is down. The main
advantage of this solution over simply shooting holes is that it
protects the multihomer against ISP-wide failure in one ISP.

5.2 Multihomed address space sharing between consortiums of ISPs

This works similar to sharing address space between two ISPs, except
that the number of ISPs is now larger. This means there are always ISPs
in the consortium that must carry traffic for non-customer multihomed
destinations.

5.3 Hierarchy in multihomed address space

IPv6 addresses have enough bits to easily accommodate a two-way
hierarchy in blocks of address space allocated to a single entity. A
three-way hierarchy may also be possible. There are three possible
two-way hierarchies:

1. ISP A - ISP B: blocks are assigned to a primary ISP, sub-blocks to a
   secondary ISP
2. ISP A - IX: blocks are assigned to a primary ISP, sub-blocks to an
   internet exchange
3. IX - ISP A: blocks are assigned to an internet exchange, sub-blocks
   to a primary ISP

Option 1 is useful if the secondary ISP is always preferred. Since this
ISP announces a more specific block, traffic will flow over this ISP as
long as this block is visible. Except for this, option 1 is very similar
to 5.1.

Option 2 can be used together with a "router of last resort" at an
internet exchange. This router announces a more specific route at the IX
and routes the traffic to either the primary or the secondary ISP, and
the primary ISP announces a less specific route to catch all traffic
that doesn't see the route sourced at the IX.

Option 3 makes it possible to use basic geographic aggregation, if a
network so desires. In this case, the network makes sure all traffic
flows to the designated internet exchange, where the /48 routes for
individual multihomed networks are known. If geographic aggregation
isn't desired and/or as a backup, the primary ISP announces a route that
is more specific than the IX one, so traffic will flow over this ISP in
the absense of an individual /48. If the primary ISP goes down, it is
very easy for another ISP operating in the region to take over the
announcement, as this only concerns customers in one region.

6. Conclusion

There seem to be avenues available for improving IPv4-style multihoming
for use in IPv6, but these carry (at least some) complexity and require
basic consensus among operators and Regional Internet Registries.