Re: [RRG] cache issues in LISP and CONS - it's bad . . .

["William Herrin" <bill@herrin.us>]

On 10/20/07, Robin Whittle <rw@firstpr.com.au> wrote:
> Delaying the packet until a mapping response arrives will be better
> than dropping the packet if the mapping arrives early enough that
> the tunneled packet will generate a response which arrives at the
> original sending host before that sending host times out.

Hi Robin,

This is purely a SWAG (it needs careful testing in operation) but my
suspicion is that the 80th percentile TRRP map lookup via DNS will
come in under 200ms. This is because the bulk of the heirarchy will
tend to stick around in the resolver's cache so that the resolver will
already know within a step or two who to ask for the authoritative
map. The map needs a fast timeout but the heirarchy can have a TTL
measured in days.

200ms is a dialup's round trip. For the first packet only. You and I
might notice the extra bump on our SSH connect time if we're paying
attention but Joe Shmoe surfing the web won't.

Anyway, this is clearly an open issue with TRRP that needs to be
addressed with real data rather than educated guesswork.

And as you point out, there may be regional differences in DNS
response time that have to be ironed out. Fortunately this issue has
been addressed before with DNS: the answer is to distribute key
portions of the heirarchy via anycast DNS servers to reduce the cost
of a cache miss. This is currently being deployed for the root zone
and its not outside of expectations for it to be deployed for the
major TLD's as well.

This, by the way, is one advantage of overloading route mapping
operations on an existing distributed map service like DNS: we don't
have to reinvent the wheel. For almost any problem we can think of,
the scalability issues have already been addressed.



> >> Incremental                    RRG                 Yes
> >> deployment via                 messages
> >> "anycast ITRs in               264 & 288
> >> the core"?
> >
> > The answer for TRRP is Yes as well. Note the first sentence of phase 3
> > at http://bill.herrin.us/network/trrp-implementation.html which
> > explicitly envisions this step.
> [...]
> But this still involves "an ITR", which is not anycast.  I am not
> sure if you are saying that your proposal provides for "incremental
> deployment", or "incremental deployment via anycast ITRs in the
> core", so I have separated the two concepts in the chart.

Any deployment (anycast or otherwise) starts with one machine, adds a
second and builds from there.

In the TRRP implementation timeline I tried to envision who these
folks are, why they would deploy an ITR and what scope the ITR would
have. In this context, I think "why?" means "what immediate economic
advantage is derived from the deployment?"

For the first ITR, I create a supply which addresses a previously
unmet demand: micronets. As things stand today, you can't get provider
independent (PI) space unless you can justify a /22. That means
hundreds of Internet-facing machines which eliminates SOHOs and small
businesses from the running. Even if you have a legacy /24, its costly
to get it routed.

So, I create micronets. These are /25 to /29 PI prefixes which can
never be routed in the DFZ. They are only reachable via TRRP.

For these micronets to be reachable by networks without an ITR,
someone has to deploy an ITR and announce a covering "route of last
resort" which leads to an ITR. So, we put it out for bid and one
company wins.

Why would any company bid? Because its almost free money. Until ITRs
are ubiquitously deployed, that company will essentially have a
monopoly on PI micronets.If you want any real inbound bandwidth for
your micronet then until TRRP is ubiquitous, you'll have to pay them.

So, they deploy the first ITR. Global scope limited to the covering
prefix for the micronets.

Who's next? Who deploys the second ITR where the process becomes anycast?

My bet is that it will be the CDNs like Akamai and the route
optimizers like Internap. For one thing, the CDNs make their money by
optimizing delivery. Implementing a local ITR optimizes delivery to
the micronets. For another, they have the technology and are well
positioned to vend optimizing DNS route servers to the micronet
customers. Its a new product line they're uniquely positioned to
offer. Will they jump on the opportunity? You bet they will.

So the CDNs deploy local scope ITRs which take priority over that
first ITR for the Micronets prefix(s). But they also dump the local
default route into the ITR. They read the documentation on TRRP.
That's what it said to do and they lose nothing by doing so.

Who's next? How about peering managers? If your traffic is heavy
outbound and you need to improve your ratio or be depeered, dragging
in the traffic for TRRP can't hurt. Scope: regional default route.

Next? By now micronets are a big enough deal that niche ISPs can
benefit. Deploy an ITR, ETR and DNS route server. Help the customer
get a micronet assignment. Presto: new product. How long has it been
since niche ISPs have had a genuinely new product with which to
freshen their advertising?  Scope: local default route.

Next? None of this is happening in a vacuum. Time marches on and we
close in on IANA free pool exhaustion. IPv6 deployment remains way
behind the curve so the feared address market develops and the DFZ
route table starts to explode. All the backbones will have a bad
couple quarters as they deal with a forklift upgrade cycle.

But not all backbones are created equal: some sell bandwidth cheap on
a razor thin margin. You know who I'm talking about. Those guys won't
have a bad quarter; they'll have a crisis.

Fortunately, a solution presents itself. A dozen TRRP ITRs can be had
for less than the cost of the smallest core router. So, they deploy
ITRs and announce a default (and the Micronet covering prefixes) to
their customers. Then its 60 days notice: Razorthin will discontinue
BGP routing for non-customer /24''s. To retain connectivity, announce
a covering route or establish an ETR.

They will take a hit for this. Even being careful not to exclude
things like the /24 for f-root they'll have a few weeks in which
worldwide connectivity is not complete before everybody gets the
message and deploys an ETR. They'll lose some customers too. But
they'll have avoided a crisis that would have cost tens of millions of
dollars that they didn't have. And the net result is that just about
everybody in the world with a PI /24 will now have a TRRP ETR. Scope:
national default route.

What then? Well, the other backbones like saving money too. Once
Razorthin has taken the hit and caused ETRs to be deployed, they'll
deploy ITRs and reduce their routing tables as well. Scope: global
support of TRRP.

Deployment of ITRs and ETRs continues, of course, but its purpose is
optimization. At this point, TRRP is globally deployed and (knock on
wood) working. At each step along the way here, each organization
which deployed a TRRP component had an immediate economic advantage
for doing so.


> But this still involves "an ITR", which is not anycast.  I am not
> sure if you are saying that your proposal provides for "incremental
> deployment", or "incremental deployment via anycast ITRs in the
> core", so I have separated the two concepts in the chart.

In the sense of ITRs having the same IP address, none of the proposals
are anycast, are they? The ETRs could be anycast but why would you
make the ITRs anycast? Perhaps I misunderstand?

In the sense of multiple distributed ITRs originating the same
prefixes and being able to properly handle any address within the
prefixes, TRRP is anycast.


>   The ISP provides multiple ITR which advertise the BGP prefixes
>   from which the Micronets are allocated.  Each ITR tunnels
>   packets which arrive from non-upgraded networks to the correct
>   [...]
>
> But then your proposal will lose its terseness and start to resemble
> a fulsome Ivip document!

Ha! That's why I put it in hyperlinked web pages instead of trying to
write an internet draft straight off. I can keep the core of the
proposal reasonably concise and still expand on topics that need more
explanation.

Regards,
Bill


-- 
William D. Herrin                  herrin@dirtside.com  bill@herrin.us
3005 Crane Dr.                        Web: <http://bill.herrin.us/>
Falls Church, VA 22042-3004

--
to unsubscribe send a message to rrg-request@psg.com with the
word 'unsubscribe' in a single line as the message text body.
archive: <http://psg.com/lists/rrg/> & ftp://psg.com/pub/lists/rrg