[RRG] Ivip's "Open ITRs in the DFZ" (OITRDs), formerly "Anycast ITRs in the core"

[Robin Whittle <rw@firstpr.com.au>]

"Open ITRs in the DFZ" is the new name for the ITRs previously known
as "Anycast ITRs in the core/DFZ".

As Darrel Lewis has pointed out to me off-list, and as I think other
people have recognised, "anycast" is not the right word for these
ITRs.  Thanks Darrel - it has taken me a while to think of an
alternative name.

Here I describe OITRDs and the business model I expect they would be
deployed under.  The business model and some details of how they
would operate technically is new and somewhat different from what I
have written previously.

Ivip's OITRDs perform the same basic function as LISP's "Proxy
Tunnel Routers" (PTRs) - in the same basic way.  The PTR concept was
first publicly documented in December 2007, with:

  http://tools.ietf.org/html/draft-lewis-lisp-interworking

They handle packets addressed to a micronet (Ivip - EID prefix is
the LISP term) which originate from hosts in networks without ITRs,
and tunnel the packets to the correct ETR.  BTW, the LISP folk admit
"proxy" isn't a very apt term either, but at least it is not wrong,
as "anycast" was.

Ivip's OITRDs and LISP's PTRs are absolutely essential for each
map-encap scheme to be "incrementally deployed" (in my definition).
 Without this, the new mapped address space would only be reachable
for the initially small subset of networks which had ITRs.

LISP has an alternative arrangement called LISP-NAT, but I think it
does not do the job, while PTRs do.  I will add a link to this
critique from http://www.firstpr.com.au/ip/ivip/lisp-links/ .

The more I read on the list about LISP PTRs the less confident I
feel about my understanding of them.  I would really appreciate
someone from the LISP team writing a more expansive description of
PTRs, with several well explained examples showing where they are,
where the various ETRs are for whatever EIDs prefixes they
advertise, how each BGP prefix they advertise is comprised of one or
more LISP EID prefixes etc.


My original conception for OITRDs (June 2007 onwards, as "anycast
ITRs in the core") was that these would be full database ITRs,
located all over the Net, advertising every Mapped Address Block
(MAB) which is managed by Ivip.

Each MAB contains one or more (probably hundreds or thousands) of
User Address Blocks, within which each end-user can define one or
more micronets.  Each micronet has an arbitrary starting point and
length (integer length, not prefix-like power of 2), and all
addresses in the micronet have the same mapping.  Each micronet is
mapped to a single ETR address.

The idea was that wherever the ITR-less networks were, packets
leaving those networks addressed to a micronet address (therefore
within a MAB) would not have to travel far, in general, before they
were forwarded to an OITRD.  In this way, no matter where the ETR
the packets would be tunnelled to, the path lengths would be optimal
or generally close to optimal.

However, I didn't have a business plan for why anyone would deploy
these OITRDs.


Here is the new plan.  Firstly, while it would be technically
possible, and with certain business arrangements commercially
attractive, for an OITRD to advertise ever MAB in the Ivip system,
in the new plan, I will assume that OITRDs are typically only
advertising a subset of all MABs.  This solves some of the scaling
issues, by splitting up the total OITRD task in a given area to
multiple OITRDs.

Secondly, in the past, I assumed the OITRD would be a full database
ITR, this doesn't actually have to be the case - although it often
would be.

There is very little performance difference between a caching ITRC
with reliable, close (say a few metres to a few hundred km), access
to a full database query server (QSD) and a full database ITRD.

So an OITRD could be a caching ITR.  That would involve initial
packet delay for micronets it has no cached mapping for.  Unlike
LISP's ALT network, where the lookup process has long delays and is
subject to packet loss over very long paths, Ivip's hybrid push-pull
architecture enables there to be one or more full database QSD query
servers somewhere close to any caching ITR.


The most likely initial business model for deploying OITRDs is that
there would be a dozen or so RUASes (Root Update Authorisation
Systems), each a separate company or non-profit organisation.

Please see pages 33 to 48 of:

http://www.firstpr.com.au/ip/ivip/draft-whittle-ivip-db-fast-push-00.pdf

for a fuller explanation of RUASes and the fast hybrid push-pull system.

Each RUAS is responsible for the mapping of at least one, but
typically hundreds or thousands of separate MABs.  Each RUAS's MABs
might be scattered all over the address space.  A MAB could be moved
from one RUAS to another, so each MAB is not forever tied forever to
any one RUAS.

The RUASes work together to build and run the Launch servers and the
Replicator network, pushing the mapping updates out to whoever wants
the feed for their ITRDs and QSDs.  Probably the last parts of the
push network would be operated by nearby ISPs and end-user networks
cooperating to run their own Replicators.  Replicators are ordinary
COTS servers, located at Internet exchanges etc.

Each RUAS, directly or via various UAS (Update Authorisation
Systems) has thousands, millions or perhaps billions of separate
end-users, each with their own UAS - each divided into one or more
micronets.   The RUAS receives revenue from these end-users by
charging them directly or indirectly for their address space and the
mapping service (through one or more levels of UASes and potentially
other retail levels).

Each RUAS needs to run a widespread set of OITRDs to handle packets
emerging from ITR-less networks which are addressed to any one of
its MABs.

Therefore, the RUAS is motivated to deploy these OITRDs widely, in
order that their direct and indirect end-user customers are assured
of optimal or generally close to optimal path lengths for packets
sent from anywhere to wherever their chosen ETR is.

The RUAS would therefore provide these services for all the
end-users of the MABs it is responsible for:

  1 - The use of the address space - if the MAB address space
      belonged to the RUAS.  Alternatively, some UAS or some
      retailer might be charging the end-user for using a part of
      the MAB which was assigned by an RIR to the UAS or retailer.

  2 - Sending updates to through the fast-push system (Launch and
      Replicator servers).

  3 - Provision of snapshot files of each MAB's mapping, for the
      purpose of booting ITRDs and QSDs.

  4 - Provision of a global set of OITRDs to handle traffic for the
      end users of all the RUASes MABs.

One way or another, the RUAS would be gaining revenue from end-uses
directly, or via subsidiary UASes, for the services above.

I expect that each end-user's impact on the RUASes costs will be
reflected in some way in what they pay, directly or indirectly to
the RUAS.

For instance:

  A flat, small, fee for each mapping update - 0.5 to 5 or even 10
  cents.  This is for a changed ETR address for a currently defined
  micronet.

  Some monthly or daily fee per micronet, since each micronet adds
  to the size of the database to be stored, and of the snapshot
  files.

  Since end-users can split and merge micronets in their UAB as
  often as they wish, there might be charges for those changes too.

  Since some end-users may cause substantial traffic to flow on the
  OITRDs, there would need to be some sampling of traffic at OITRDs
  and some charge, at least for micronets which carried high volumes
  of traffic.


Each RUAS could deploy its own set of OITRDs.

Alternatively, some or all of them could form a consortium to deploy
a global set of OITRDs, with each one handling all the MABs of the
participant RUASes.

Another approach is that one RUAS which already has one or more
OITRDs in some locations where a second RUAS wants to to have OITRDs
as well, could charge the second RUAS for handling the second RUASes
MABs on the first RUASes OITRDs.

Yet another approach would be one or more companies running OITRDs
and renting their capacity to as many RUASes as wanted to use them.


So a single OITRD could technically be in one of the following states:

 1 - It is advertising all the MABs of just one RUAS.

 2 - It is advertising some subset of the MABs of one RUAS - to
     spread the load in one location of all the RUAS's MABs over
     multiple co-located OITRDs.

 3 - It is advertising the MABs or two or more RUASes, but not of
     every RUAS.

 4 - It is advertising the MABs of all RUASes.

Option 4 was the only one I contemplated in the initial ivip-arch-00 ID.

In all cases, I intend that the OITRD tunnel ever packet it receives
for each MAB.  So it is either a full database ITRD or it behaves as
one, being an ITRC, with one or more nearby, reliable, full database
QSD query servers.

In the ivip-arch-00 ID I contemplate some ITRs passing on packets to
upstream ITRs if the first ITR didn't have mapping for the packet's
micronet.   While this may seem unnecessary, since all ITRCs will
have a nearby QSD to get the mapping from in a few or a few tens of
milliseconds, this arrangement would still make sense if a
particular ITRC ran out of cache space, or was temporarily unable to
get mapping information.

OITRDs do not have this option.  Since they advertise the MAB in
BGP, they can't pass the packet out to the DFZ if they can't tunnel
it - since it would be forwarded straight back to the same OITRD.

So all OITRDs must be able to cope reliably with packets sent to the
MABs they advertise.  This means that if the OITRD looses sync with
the mapping updates (for more than a minute or so, I guess) or if it
can't catch up (via downloading the missing packets) then it needs
to stop advertising the affected MABs.  Then, the BGP system will
forward packets which would have gone to this OITRD to other nearby
OITRDs.


I hope this helps everyone understand this important part of Ivip -
from a technical and a business perspective.

I will be glad to answer questions, discuss criticisms, provide
examples etc.

  - Robin






--
to unsubscribe send a message to rrg-request@psg.com with the
word 'unsubscribe' in a single line as the message text body.
archive: <http://psg.com/lists/rrg/> & ftp://psg.com/pub/lists/rrg