Re: [RRG] 2 billion IP cellphones... IPv6 RIB burden adding to IPv4 burden in the same router?

[Robin Whittle <rw@firstpr.com.au>]

Short version:   To what extent can we consider the routing scaling
                 problem of IPv4 being in some way added to the
                 IPv6 routing scaling problem because in practice
                 the one router with the one set of RAM and CPU
                 resources will often be handling both IPv4 and
                 IPv6?

                 Wesley George suggested they are connected in this
                 way - and I don't recall this being discussed
                 before.

                 Whether to deploy the IPv4 and IPv6 fixes at the
                 same time?

Hi Wes,

I wrote a separate message about whether or not core-edge separation
solutions for the DFZ routing scaling problem might help with the
scaling problems of internal routers.

You wrote, in part:

>> This looks like a fundamental limit on what sort of IP applications
>> can be run on a hand-held device, as long as a connection uses a
>> circuit-switched radio link.
> 
> Well, there's ways around this like using the inbound paging channel
> (the thing that tells the phone to wake up its cellular radio from
> standby to accept an inbound call, and accepts SMS messages) to make
> always on, but mostly in standby, applications work, but you're right,
> it is a limiting factor to simply porting existing (chatty) applications
> over to the mobile space without help from the carrier and their
> infrastructure.

OK.


>> It is hard to imagine why there would be IPv6-only "content".  If
>> someone has something to sell or give away, why only make it
>> available via IPv6?  Perhaps there would be IPv6-only material if it
>> was in a format which is only suitable for 3G cell-phones, and in
>> practice all such cell-phones used IPv6.
> 
> Yes, that is one application. The other is when we actually do run out
> of IPv4 space in some indeterminate number of years. I'm not interested
> in debating whether you believe the projected dates or not, or what you
> think people will do when it happens, but if it does, IPv6-only content
> will crop up by necessity. 

I agree that if there is a substantial number of IPv6-only potential
customers or non-paying viewers, listeners etc. that material will
be made available on IPv6.  I guess it would also be available on
IPv4 unless it was in a specific form which was only compatible with
 particular hosts, such as 3G cellphones, which in fact were found
only with IPv6 connectivity.

> IMO, whether some method will be employed to
> make it reachable via IPv4 is not germaine to the discussion, because it
> will still mean that there is an increase in IPv6 deployment.

Yes.


>>> If they have the stomach for it, they can do some really unholy things
>>> with NATs, where they are either segmenting the network into multiple
>>> NAT regions so that they can reuse the same 1918 space over and over, or
>>> leaving it relatively flat, but coopting large blocks of routable
>>> addresses (that likely do not belong to them, eg 11/8, 12/8, etc) to
>>> assign to devices inside the NAT boundary, taking care to never leak the
>>> routes to the internet.
>>
>> Maybe someone will delurk and provide more details of how 3G
>> operators actually run things today.
> 
> that's sorta why I delurked. I are a 3G operator, and those examples are
> not theoretical. Scary, huh?

Yes!


>> So I don't think widespread IPv6 adoption in cellphones will
>> lead to a scaling problem - assuming anything less than the current
>> IPv4 260k routes is not regarded as a problem.
> 
> Admittedly the problem may be further off in the DFZ. However, I
> don't know why we would design something that only applies to the
> DFZ, since the route scale problem has potential to be much worse
> within a given network than outside it. What incentive do I have
> as an operator to deploy some fantastic new thing for the DFZ if
> I still have to have routers that cost millions to deal with my
> internal network routing table?

I wrote about this in "Solving the routing scaling problem for
internal networks too? Ivip may be able to help".

I will also post a message considering to what extent 3G networks
would contribute to the size of large ISP/carrier networks.
Initially I thought that the number of IP gateways wouldn't be so
large, since some apparently work with up to a million end-users.
However I reconsidered and concluded that the large 3G networks,
fully deployed, are probably going to place a great load on internal
routing systems.


>>> I'm confused as to what we're trying to accomplish by endlessly debating
>>> if and or when the ship will sink instead of acknowledging that it is
>>> indeed taking on water and we might want to agree on a plan to prevent
>>> it from actually going down.
>>
>> The trouble is there are two ships.
> 
> No, I don't think so. I don't disagree that we have a potential problem
> on IPv4, but this isn't an either/or thing, or a phase I/phase II thing,
> because we're going to exist in a dual-stack environment indefinitely,
> so we need a dual-stack solution. Solving IPv4 first simply delays the
> inevitable, and while it may lead to a more efficient solution for
> IPv4-only and for IPv6-only, it's highly unlikely that two separate
> solutions run simultaneously will be as efficient in dual-stack as one
> solution that has some compromises to make it work for both
> simultaneously. Even if there are still separate processes for IPv4 vs
> IPv6, you still can take advantage of efficiences in implementation by
> being able to reuse algorithms and hardware components between the two
> stacks.

In my proposal, the architecture is much the same for IPv4 as for
IPv6.  The overhead costs of encapsulation in IPv6 are higher than
in IPv4 due to the longer header.

My attempts to find an alternative to encapsulation and to Six/One
Router's address rewriting ("translation") approach resulted in two
"Forwarding" approaches, both involving new use of bits in the
existing IPv4 and IPv6 headers - and therefore requiring a hopefully
minor upgrade to existing DFZ and some internal routers:

   ETR Address Forwarding (EAF) - for IPv4
   http://tools.ietf.org/html/draft-whittle-ivip4-etr-addr-forw-01

   Prefix Label Forwarding (PLF) - for IPv6
   http://www.firstpr.com.au/ip/ivip/ivip6/

There are significant differences between these two, but they both
share the quality that there is no encapsulation overhead, and no
PMTUD problems as there are with map-encap.

The EAF for IPv4 approach is the cleanest and most powerful.  It has
30 bits to play with and works for ETRs which need to be on every
fourth IPv4 address, forwarding the packet there directly.

The PLF for IPv6 approach is trickier, since there are only 20 bits
to play with, and ETR addresses are 128 bits.  I use halve the
code-points (2^19 of them) to forward packets from ITRs to ETRs,
with each codepoint matching a specific DFZ prefix from a set of
contiguous prefixes of this number.  The other 2^19 codepoints are
reserved for use in internal networks - a separate namespace defined
and managed for each ISP or end-user network which wishes to use
this system.

So while IPv4 and IPv6 at a high level of abstraction have the same
"architecture", there may be profound differences in what is and is
not possible regarding new architectural additions which must remain
within the constraints of existing headers.

I think that the whole Ivip for IPv4 system should run purely over
IPv4 and likewise that the IPv6 system should rely purely on IPv6.
So I don't foresee any specific synergy in running the same ITR, ETR
or other function's code on the same device - other than convenience
in choosing one device to do the same function for both IPv4 and IPv6.

The common nature and function of ITRs, ETRs, full database query
servers (QSDs) and of the servers which make up the fast push
mapping system means that as long as the participating routers or
hosts are dual-stack with stable native IPv6 addresses, they will be
able to perform these roles for both IPv4 and IPv6.  This is
somewhat simpler than having completely different constructs for the
IPv4 and the IPv6 solution.


> Consider that all routers have a finite amount of space for either IPv4
> routes, IPv6 routes, or some combination of the two. The very fact that
> a router has to exist in dual-stack means that the existence of the IPv6
> table, even if it's smaller, will exacerbate any scaling problems that
> we're experiencing at IPv4. Why not make both as efficient as possible?

Clearly, each implementation should be as efficient as possible.

I hadn't considered how the IPv6 routing table might exacerbate the
problem of IPv4 routers, since they are probably going to be the
same router.   I guess that for a given number of prefixes, the IPv6
RIB information takes more space and may involve some more CPU power
and communications overhead due to the longer addresses.

The IPv6 FIB burden on routers is arguably worse than for IPv4, for
any given number of prefixes, since those prefixes tend to be
longer, such as /48 for IPv6, compared to most traffic flowing
according to IPv4 prefixes which are /24 or shorter.


> Part of the issue we have here is talking about how you implement a
> fundamental change in routing architecture. There's never a good time to
> do it, and most of the debate I've seen focuses on what can and cannot
> change, levels of complexity, etc. Why woudn't we take advantage of
> IPv6's admittedly laggard deployment to ensure that we have a solution
> in place before it becomes mission-critical, deeply entrenched and
> therefore harder to change? Perhaps the necessity of scaling the routing
> table will become its own deployment incentive.


I am not suggesting we should only develop an IPv4 solution.  I am
suggesting one will be needed sooner than for IPv6, because I think
it will be a long time - a decade or two - before the IPv6 DFZ has
more routes in it than the IPv4 DFZ.

Despite thinking that it will be a decade or more before IPv6's DFZ
reaches 250k routes, I have put some effort into solving its routing
scaling problem.

I think we should develop both.  If the IPv6 solution can be
introduced without much prior investment, as is generally assumed
must be the case for IPv4, then I think it would make sense to wait
quite a while before really settling on the IPv6 solution - to gain
experience with the IPv4 solution technically, and in terms of
business models.

But this line of thinking involves some assumptions, including:

1 - That there really are two ships - which you challenge to some
    degree by pointing out that the one router will often be
    required to handle both IPv4 and IPv6 DFZ tasks.  In that
    case, while by some threshold 500k might be regarded as
    a "problem" for IPv4, this doesn't mean that 100k for
    IPv6 is not a problem, since those may chew as much resources
    in a router as 150k or 200k IPv4 prefixes - and on the same
    router this just adds to the resources chewed by IPv4.

2 - That there is no need for long-term preparation of IPv6
    routers etc. before introducing the IPv6 scalable routing
    solution.

Point 2 was my initial thought, since I figured that the IPv4
solution must be "incrementally deployable" (see note below) on the
IPv4 network as it is today.

   "Incrementally deployable" is understood in many ways, but I
   thank that, whatever term is used, we need a system which
   generates direct and immediate benefits to those who invest in
   its adoption, in a way which does not depend much or at all on
   how many other networks have adopted it.

      http://psg.com/lists/rrg/2008/msg01088.html

   I understand the APT model is to appeal to ISPs who will
   implement the solution.  As I wrote here:

      Re: Comparing APT & Ivip - new business models
      http://psg.com/lists/rrg/2008/msg02593.html

   I intend that Ivip appeal directly to end-user networks, who
   are the ones who need to make the decision to adopt it.

   So while it is to the long-term benefit of all ISPs - and
   therefore all Internet users - that the routing scaling
   solution be widely adopted, for Ivip, the plan is that the
   attraction must be in the form of immediate benefits for
   end-user networks, large and small, since it is they which
   must decide to adopt the new addressing system, not their
   ISPs.  The new system makes their network no longer tied to
   their current ISP, or to any particular ISP, as long as there
   are ISPs with ETRs, which there will be.


However, contrary to Point 2, considering the difficulties of
designing and running map-encap due to the PMTUD problems, perhaps
it might be easier to spend some years in background work, upgrading
the DFZ routers to support Forwarding, and then have an easier time
devising and deploying ITRs and ETRs.  In the long term, Forwarding
is the way to go - since it has no encapsulation overhead.

So maybe we should be at least tentatively decide that this IPv6
approach to Forwarding is the best way of handling the traffic
packets between ITR and ETR (actually, for IPv6, just across the DFZ
to the ETR's network's border router).  In that case, it would be
relatively simple to define how the DFZ routers need to be upgraded,
and then it can become part of new router functionality and
hopefully be done as a firmware update for existing routers.

That could take 3 to 5 years, with an investment mainly in some
effort by the major router vendors - hopefully it wouldn't be to
complex, considering it is a small addition to the enormously
complex work the routers already do.  I am sure the changes are not
so serious as to require new silicon or circuit boards in any
substantial modern router which has a multiple CPU-based FIB, rather
than one built of specialised hardware, TCAM etc.

Then - although we could always choose to introduce an IPv6
core-edge separation scheme with map-encap - we might be close
enough to having fully upgraded DFZ routers that it makes sense to
introduce the scheme with Forwarding only.  This would be simpler
and would move us straight to the lasting, long-term, arrangement.


>> One has a scaling problem which will surely accelerate as its most
>> pressing problem - shortage of address space - becomes more pressing
>> and the space is sliced and diced more finely.
>>
>> The other has few passengers and no scaling problem, but a special,
>> growing, class of passenger - a billion or so I guess in 5 years
>> time - will soon be getting on board.
> 
> So if I agreed with your two ships idea, what makes the two so
> fundamentally different that we can't solve the problem common to them
> both simultaneously? 

As I wrote above, in broad principle the two Ivip approaches are the
same, but in their most efficient long-term implementation, based on
Forwarding, rather than map-encap, the IPv4 solution is inevitably
significantly different to the IPv6 solution.

APT, LISP and TRRP are probably much the same in their approaches to
IPv4 and IPv6.  Ivip's map-encap approach is much the same for IPv4
as for IPv6.

Six/One Router only works for IPv6, since it relies on mirroring the
end-user address space with one or more (for multihoming) PA
prefixes from the one or more ISPs.  There is not enough free space
in IPv4 for this to work.


> I don't disagree that the solution will have to be
> tweaked and evolved from its initial deployment as we learn how it works
> in a production environment, but are you actually suggesting that we'll
> have a different solution for IPv4 vs IPv6, rather than one solution
> that we continue to refine for both?

Different in some important low-level details - at least with
Forwarding.


> I'm sure you can speak for IVIP, but because I'm not an expert on any of
> the proposed solutions and variations on them, and don't want to assume,
> I'll risk showing ingorance and ask: Are any of the proposals so
> fundamentally wrapped around IPv4 that their basic implementation
> couldn't work for IPv6 without major alteration?

No - but Six/One Router is only for IPv6.


> If the answer is no, why do we keep coming back to debate whether we
> should do IPv4 first vs IPv4+IPv6 simultanteously?

Even if the same principles are applied to the IPv4 solution and the
IPv6 solution, such as LISP, APT, TRRP or Ivip for both (assuming
map-encap for all), there are going to be two sets of mapping
information, two different PMTUD sets of problems to solve, two sets
of code to write for ITRs and ETRs etc.

IPv4 mapping will have a resolution (AKA "granularity") of one IPv4
address (/32).  IPv6 could have the same (/128), but I think that
would be would be nutty.  Ivip's resolution will be in steps of /64.

Also, because it is necessary to deploy the solutions as separate
entities (because sometimes there will be an IPv4 only device, or an
IPv6 only device), even though they may share a lot of common
principles and some common code, it is not necessary to rush into
deploying the IPv6 solution if this is not an urgent problem, while
the IPv4 problem is.

Because this is such a momentous change to the Internet, I don't
think we should deploy it before we really need to.

However, spending a few years deploying Forwarding upgrades to DFZ
and internal routers while we work out the details of how to use
these capabilities as part of a core-separation scheme such as Ivip
strikes me as a good reason for starting this relatively
non-disruptive phase ASAP.

Having most or all DFZ and many border routers ready to do these
Forwarding techniques doesn't commit us to any particular mapping
system, or to any particular approach to core-edge separation.  If,
for example, some version of APT which is quite different to Ivip
was decided to be best, and it was adapted to Forwarding rather than
encapsulation, then we could use the upgraded routers and deploy it
relatively quickly.  Alternatively, maybe the upgrade wasn't
possible to complete in time, in which case we might decide to
implement a scheme based on map-encap, and upgrade it to Forwarding
in the longer term.



> Given the long-term nature of any redesign of this magnitude, the
> timelines for implementation, challenges to deployment, etc, I can't see
> how any solution that didn't have a method for handling IPv6 would be
> seriously considered viable at this point.

In broad principle, I agree.  However, I think the optimal approach
to getting data from ITR to ETR is not encapsulation, but one of the
Forwarding techniques I mentioned above.  These are bound to differ
significantly in principle and detail between IPv4 and IPv6 because
of the differing address lengths and the differing number of
available bits in the header.

It so happens that the IPv4 approach is much more complete: 30 bits
to specify an ETR address - this is hardly any less useful than
having all 32 bits.

The IPv6 approach is trickier, with only 20 bits to play with - but
I think it is enough to do the job in a rather different way, and
would definitely be worth it, compared to tacking a 40 byte
encapsulating header onto every packet, including especially VoIP
packets with 20 bytes of payload.  That is the overhead for Ivip's
IP-in-IP encapsulation approach.

The overhead is worse for the other systems, which use an IPv6 IP
header, then a UDP header and then their own header.  Also, I think
that SEAL approach to PMTUD management - as would be needed if
Ivip's approach to solving PMTUD problems wasn't used - would
involve extra headers on some or all data packets.


> Maybe we're all affected by a collective hysteria about the necessity of
> IPv6, and we'll realize later that it wasn't as big of a deal as we
> thought. 

If a self-supporting colony of users can be happy paying for
IPv6-only connectivity (plus some kind of HTTP proxy to browse the
IPv4 Web, and/or maybe some system like "Dual Stack Lite") and that
colony can grow to hundreds of millions - enough to attract to IPv6
companies who provide services, freebie stuff, music, video etc. -
then IPv6 usage may grow over the next decade or so into something
quite significant.   Still, I can't foresee a time when the majority
of DSL, DOCSIS, fibre, or wireless laptop end-users will have IPv6
connectivity, which is the precondition for most servers connecting
to IPv6 and for IPv4 users in general being able to do without an
IPv4 address.


> However, the amount of my customers that are asking about IPv6
> connectivity and starting to treat it as table stakes to sell them
> service tells me otherwise in terms of the potential for increase in
> IPv6 deployments over the next few years. The prognostication about IPv4
> exhaust has done a lot to force people to look more seriously at
> deploying IPv6, even if they don't necessarily have anything to worry
> about. In most cases, once the major SPs support IPv6, it's easier for
> all of those companies, schools, etc to simply enable it, rather than
> risking the possibility that some potential customer/user cannot reach
> them, but can reach a competitor. 

This - "some potential customer/user cannot reach them, but can
reach a competitor" - would only happen if there is a way of selling
IPv6-only connectivity to a large number of end-users, in a form in
which IPv4 access to the HTTP or other forms of communication was
either difficult or impossible.

3G cellphones seem to be the most likely place that will occur.  As
I discussed in early August in the "Dual Stack Lite" thread, I think
IPv4 space would have to rise to a really high price before it was
more profitable to try to sell DSL and DOCSIS customers an IPv6 only
(plus Dual Stack Lite) service than to pay the price and to give
each customer their own IPv4 address.  At that price, I am sure a
lot of IPv4 space will become available.


> That may be the closest thing we get
> to a "killer app" on IPv6 anytime soon, but in my mind, that's enough to
> make sure we fix it so that it'll work right long-term.

I fully agree we need to develop a fix for the IPv6 scaling problem.

  - Robin


--
to unsubscribe send a message to rrg-request@psg.com with the
word 'unsubscribe' in a single line as the message text body.
archive: <http://psg.com/lists/rrg/> & ftp://psg.com/pub/lists/rrg