RE: Regionally aggregatable address space for multihoming

["Tony Hain" <alh-ietf@tndh.net>]

Sean Doran wrote:
> Tony, thanks for the first rev of a draft!  I have some
> comments and questions that I hope may help improve it.
The more the merrier.

>> sites that connect to metropolitan scale exchanges.
>
>Do such exchanges exist?  I'm curious, since I haven't run into any,
>even in Stockholm, where fibre is very close to free.
That wording was directly out of the aggregatable draft and I didn't have a
good reason to change it. In some ways I would consider at least the MSN-UK
& MSN-ASIA sites as directly connected to exchanges. At least the last I
knew they were co-lo with the exchange but may have had a telco fronting for
the physical exchange port.


>> Sites will have the choice to connect to either type of
>> aggregation entity.
>
>Can sites connect to both?  If no, is that a feature or a drawback?
There should be no problem with connecting to both. As I worked through this
their inbound traffic would take whichever path was shorter from any given
source, and it would be up to them which direction they sent the outbound.


>> Any provider announcing a short prefix SHOULD be able to
>> deliver packets to anywhere in the region, either
>> directly or through other arrangements.  In the case
>> where a service provider does not interconnect with
>> others in its region it MUST advertise the longer
>> prefixes associated with its customers.
>
>What if the provider is connected and fully peered with
>all the other networks using addresses from the same
>aggregate, but is partitioned from some or all of them for
>a period of time?
This would require the obvious; either longer prefixes need to be announced,
or there will be black holes. This is no different from today where someone
announces access to a partitioned network.

>(An interesting thinking exercise is trying to figure out
>how networks multihomed at other geographical exchanges,
>and having multiple transit arrangements across those
>exchanges, can determine which long-haul provider should
>be used to reach a network that is ordinarily abstracted
>behind an aggregate covering all the networks in a given
>geographical area/surrounding a given IX.  You have
>discovered one of the hard questions already, viz.:
Yes this is an area that needs much more detail. At a crude level it can be
viewed in the context of the PSTN where the transit provider is 'managed' by
the originator. Basically it would be up to the site network manager to
decide which transit provider he would be sending traffic to for various
parts of the world. If something partitions the result is no different than
today. Either the network that can't get there anymore has alternate
arrangements for transit, or the traffic is black holed.


>> Another issue is the business model being flipped from
>> the current destination's ISP choice determines sources
>> routing; to one of source's ISP choice determines at
>> least its first hop.
>
>In other words, "how does the source make a correct
>choice of next-hop?".
What does 'correct' mean? If I have transit agreements with ISP-ABC &
ISP-XYZ, I am either getting default service from them, or the global
default-free table. If I have the table and they both claim they can get to
the destination region, or I am simply have default, then it is up to me as
the source where I send the bits. With provider-based addresses the routing
system will shunt me to the shortest AS path to that destination, but with
this scheme the provider I hand the bits to would send them along its
preferred path to the region IX before deciding which local provider to
deliver them through.

>Let us know what you come up with, or if someone else has
>properly solved this a la GSE or Ohta-multi-prefix.)
The question I keep coming back to is 'who is deciding what is right, the
service provider or the site manager as the customer?' It clearly has to be
one of those two and not the protocol designer, but individual perspective
has a big impact here. I would argue that the customer would ultimately vote
with his wallet to define 'right and proper', so the service providers will
in turn beat on the protocol designers to get the tools to deliver that.
Minimizing the operational costs of the service provider is certainly one of
the components of 'right', but each scheme has to be balanced against the
costs it imposes back on the customer site.


>> This address allocation mechanism SHOULD NOT be used on a dial access
>> network pop, as scaling routes to sites attached this way are best
>> addressed through provider based allocations consistent with
>> Aggregatable [RFC 2374].
>
>Why not?
For one the location of the endpoint is unknown. For another if a dial-pop
were scaled to handle 10,000 sites it would have to subsume the neighboring
addresses in the 2.5km square. Since dial-pops tend to be located in dense
areas using them with geo addresses would have the maximum negative effect.


>How do multihoming entities who offer both fixed-line and dial access
>accomodate this?
I struggled with this one until I realized that it doesn't matter in
practice. If the dial path were really just an intermittent connection
between routers at the same service provider then it would be reasonable and
expected that the geo prefix would be used on both and the site would simply
announce that as its prefix upstream. Interface metrics would bias the
providers IGP to do the right thing.
If the multi-homed site has both fixed-line and dial to different providers,
they would get one of these prefixes for the fixed-line, and a provider
aggregate prefix for the dial. I would expect local policy to prefer the
fixed-line prefix, but there is no protocol issue with having both. The
place this differs from today is that if one of those paths failed the hosts
would have to initiate mobile IP to maintain any existing connections.



>How far does this reach?  Does this include a dialup pool
>in a site singly-connected to a provider?
I would argue that it applies to anything where the real site location and
its topology is unknown. At first I had mobile systems in this same category
until I decided that the number of active nodes on a cell is low relative to
a dial-pop, and that providers are more likely to allocate a /56 in the
mobile device environment. In that case it would be feasible for the cell to
hand out prefixes based on the tower location, where the large dial-pop it
would not. If my assumption is wrong then I need to move mobile back to the
provider allocation side.


>> for layer-3 exchanges to work there has to be a way to do source
>>  routing within a region to select ixc : rest of the system dst based
>>  routing simple directory lookup on flat 48 bit site id
>> if so lec would dest route within region / src route to ixc
>> ixc would dest route to next region
>> dest lec would dest route within region
>
>IXC and LEC are Americanisms, and moreover are an artefact
>of a regulatory regime which is NOT global.  It's probably
>a good idea to avoid terms like these in a document which
>is designed to work for the whole Internet.  :-)
This is an artifact of the drafty nature of random thoughts. I welcome more
appropriate terms since transit would apply in both the local and long haul
cases. I am not sure how applicable this text is to reality though, as it
resulted from a conversation about layer-3 exchanges. If all the exchanges
are layer-2 there is no magic required, but for a site to influence policy
at a layer-3 IX there would have to be something like source routing.

- --

>If a large U.S. network has a number of peers and
>paying transit customers in W. Europe (for example),
>and each of these networks have customers that are using
>both provider-assigned and IX-assigned addresses,
>how should large U.S. network decide which of the
>customers to send traffic to, if they are announcing only
>an aggregate of:
>
>> W. Europe               080000  :  000000  ->   1080:0000:0000
>
>or
>
>> LHR -    51.47 n    0.350 w             13B7:3B48:4D42
>
>?
This is an artifact of trying to use location info that would be easy for
the layer-1 installer to obtain mixed with text strings that have political
context. There would have to be at least 2 prefix announcements for the
political entity known as W. Europe to cover each side of the meridian.
Clearly the text before that table needs to make it clearer that they are
conceptual examples rather than entries that cover the entire named region.
In fact one of the things I was trying to do in the table preceding that one
was to come up with names for the scopes that had no political meaning.


>Does your apparent (post hoc) answer:
>
>> Routing protocol should only announce a short prefix if the router
>> can get to the entire concatenation, if not it will need to announce
>> specifics.
>
>mean that the U.S. network's x-Atlantic customers must
>announce the individual customer prefixes learned at the
>LHR exchange, and not announce the aggregate for LHR or
>for W. Europe?
I am not sure I understand the question, but for example if a site west of
LHR was accessible via both its geo and provider addresses, the announcement
toward the US would include the prefix for its provider and the prefix that
covered W. Europe west of the meridian (without doing the math, probably
1380::).


>If this U.S. network has several competitors for transit
>sold to networks across the Atlantic, as is the case today,
>what becomes of the idea of announcing 1080:0000:0000 to anywhere?
I need a picture to understand the question. Basically today's business
models are built around the concept that a site works with ISPs to arrange
traffic preferences toward itself. With this scheme the business model would
flip so that a site arranges for best carriage of its outbound traffic into
a region. If the network the site subscribes to does not have direct access
to 1080:: it needs to subscribe to a transit that will get there.


>(Think of what they would do to be able to announce 1080:0000:0000 to
>        -- competitors with no European customers
>        -- customers who are multihomed to competitors
>           with European customers
>        -- customers who are multihomed to European networks
>        -- customers who ARE European networks, but who
>           operate in only one part of W. Europe
>        etc.
>Then think of what happens when the connectivity to one or more of
>their trans-Atlantic customers fails.  This happens sometimes...)
Again I am missing some context here. If a network were providing transit to
1080:: it would need to announce that, but if it only had a specific
customer set and didn't intend to provide access to all of 1080:: then it
would need to announce the specifics. I am getting a feeling that this
question is wrapped tightly around a business model where a site would
arrange for transit services toward itself via a specific provider. There is
nothing wrong with that, but in that case provider based addresses make more
sense. With IPv6 there is no reason a site is restricted to a single prefix
so there is no problem with having prefixes from a US-based and a
European-based provider as well as the location-based one. Address selection
rules will cause the longest match src/dst pair to be used.


>BTW, if you are doing updates to your draft, this is not
>very draft-y-language:  :-)
>> Since 20 bits is nominally what tier-1's do today
>> on a global scale I don't think this is out of line for scale or
>> performance, and it bounds the problem to the set of providers where
>the multi-homing occurs.
More random thoughts that need to be cleaned up before this goes in as an
I-D.