RE: Regionally aggregatable address space for multihoming

[Iljitsch van Beijnum <iljitsch@muada.com>]

On Tue, 12 Jun 2001, Geoff Huston wrote:

> I find it hard to conceive as to what a 'geography' may be when its cheaper 
> to cross the Atlantic by submarine cable, station to station, than it is to 
> do last mile at either end. We had this model in our collective industry 
> heads that "distance = cost" and that local geographies see some cost 
> benefit in locally interconnecting. In the current communications 
> environments the cost of local regulatory regimes appears to be greater 
> than the cost of distance, and a least cost community of common benefit has 
> no geographic counterpart.

I think you are painting too bleak a picture. Let me tell you a story. Five,
six years ago the Internet business was in its infancy here in The
Netherlands. We were connected to the net over British Telecom with a frame
relay line to the UK over a leased line to Amsterdam. So if we wanted to
connect to someone else in Holland, the first hop would be London and then
the packets would take a tour of Europe over Geneva or Stockholm and back to
The Netherlands (99% of the time Amsterdam). Or the traffic would flow over
New York, Washington and/or New Jersey. The reason for this was
simple: everybody needed a good connection to the US and the large networks
wouldn't interconnect with the smaller ones at the local exchange. There
wasn't enough European traffic to justify many links between European
countries. (I did some counting back in those days and 45% of the traffic was
to Dutch destionations, 45% to US destinations and only 10% to other European
destinations.)

Today the situation is completely different. Large networks such as
UUNET/MCI/Worldcom have pan-european networks so in many cases traffic takes
the shortest route to other European destinations. Transatlantic capacity is
very cheap, but not as cheap as housing a router at the Amsterdam Internet
Exchange and exchanging tens or hundreds of megabytes without any traffic
charges. Some of the really large networks still don't want to interconnect
with the smaller ones, but even the traffic between the small networks is
enough to make it worth connecting.

> We are certainly on the horns of some form of dilemma - conventional 
> routing wisdom indicates that some form of hierarchy is one of the few 
> tools that address route scaleability - through hierarchies fine detail is 
> obscured at a distance, and these details only become visible at a local 
> level. However, the cost of circuitry is inexorably dropping - customers 
> find it cost effective to drag two or more local circuits to two or more 
> upstreams as the most cost effective response to resiliency requirements. 
> Providers find it cost effective to drag long haul circuits all over the 
> place as a means of bypassing various components of a locally-provided 
> upstream transit service. At what point in the long haul over-supply market 
> will the long haul providers start to look at end customers as a potential 
> long-haul customers, and at what point will multi-homing become a non-local 
> problem? What appears to be happening is that "geographic" locality and 
> "least cost" locality don't match all that closely. (Yes this "long 
> distance is cheaper" sounds far fetched, but its also far a far fetched, 
> but current, reality which makes a Tokyo - LA circuit cheaper than a Tokyo 
> - Osaka circuit.)

You have convinced me at least to the point where I have to admit that just
assuming that interconnectivity will become more and more local in the future
is a dangerous assumption.

But rather than invalidate the argument for geographic aggregation you merely
weaken it. Even if it's cheaper to connect to LA than to Osaka for someone in
Tokyo, it's still cheaper to connect to LA than to New York or
Johannesburg. To put it more generally: even if long distance is cheap, it
still doesn't make sense to bypass one hub and connect to one further
away.

Also, there are other reasons why long distance isn't fun. First of all,
there is the delay. Some applications just can't tolerate an extra 80 ms
round trip time. Another problem is that if you get traffic for a very large
region together at one point, it is much harder to handle. Today's routers
can maybe do 10 Gbps. So if you have 1 Gbps worth of customers in a city, it
doesn't make much sense to aggregate traffic for 25 cities.

> There may be some other approaches: if you concede that a full routing 
> table at some point in the future will have some considerable cost in terms 
> of specialized support machinery, then the market will sort itself out 
> through the use of different route filters applied to peers, customers and 
> upstreams.

Yes. Combined with the current practice this is a very bad thing: if I run
defaultless and I start to filter out routes to multihomed networks, I can't
reach them at all. If there is a regional aggregate at least there is a
fighting chance.

> The implied route scaling cost of multi-homing would become as 
> explicit cost of protocol scaling, and there would be a solid economic 
> incentive to single-home.

Not really. The people paying for the bigger routers aren't the same ones
that are deciding to multihome. But that doesn't mean we can't create such an
incentive. If a scalable way to do multihoming costs the networks more money,
they can charge their customers more for the service. Just as long as the
people who make the costs are also the people sending the bills, unlike the
current situation.