Re: [RRG] Idea for shooting down

[Robin Whittle <rw@firstpr.com.au>]

Hi Brian,

A short version of this message is:

  I think your scheme is not needed - a single BGP system and a
  single ITR-ETR system should do the trick.


You wrote:

> There would not only be a separate instantiation of
> BGP4 in each region, but also a separate instantiation of (e.g.)
> LISP-ALT.

This is very ambitious.

I don't see how an ITR-ETR scheme could be made to work in separate
sections of the Net like this.  If there is one address space for
the whole Internet, there needs to be one ITR-ETR scheme, as far as
I know.

>> Assuming there is an ITR-ETR scheme which successfully limits the
>> number of prefixes in the current single DFZ (to some acceptable
>> number, the same, somewhat higher or lower than the current ~240k),
>> what benefits would your proposal provide?
> 
> None. But historically, routing swamps have a habit of emerging
> spontaneously. This is an idea of how to take the sting out of that.

By creating a global multilevel system (AKA a potential swamp, not
just in horizontal dimensions but also vertically) of smaller
individually routing nascent swamps, each of which is kept small
enough that their swampfulness remains within acceptable bounds!

>>>> I get the impression that the result would be the Internet being
>>>> much less meshed than it is at present. 
>>> Not exactly. The individual regions can be as meshed as one
>>> wants, and they are *not* geographical regions - in fact they
>>> are more likely to evolve as conglomerates of ISPs.
>>
>> I got the impression from the text and diagrams that no ISP in a
>> Level 1 domain could link to any other ISP in any other Level 1
>> domain - that the packets would have to flow via a Level 2 domain,
>> in the simplest instance.
> 
> Correct. But remember it is a virtual topology; it doesn't need
> to add more than one hop to the real topology, if such a connection
> is needed. 

I don't understand how no more than one hop is added, since it could
take multiple hops to get out of an ISP in a Level 1 domain, up to
an ISP in a higher level domain, and then down to another ISP in
another Level 1 domain.

> I do assume that ISPs which happen to exchange a lot
> of traffic will end up in the same region - very likely the regions
> would emerge in existing economic blocs, with the primary driver being
> economic, rather than geographic or topological, nearness.

OK - this lack of meshiness and increased compartmentalisation seems
like a bad idea to me.


>>>> This would have negative
>>>> impacts including:
>>>>
>>>> 1 - Less robustness in the event of link and router failure.
>>> As noted in the draft, there can be multiple links between
>>> layers - as many as desired, in fact.
>>
>> There could be, but since there are now multiple domains, with
>> restrictions on what each ISP in each domain can link to (as best I
>> understand it) then this would surely reduce the opportunities for
>> meshing compared to the current situation where there is only one
>> "domain" and any ISP can link to any other ISP.
> 
> Yes. But that may simply be the price of scalability.

I don't think we need a scheme such as this.  An ITR-ETR scheme can
handle huge numbers of prefixes, whilst keeping the current, single,
BGP system working much as it does today.

>>>> 2 - Often longer path lengths (AKA "stretch") as a packet has to
>>>>     get out of one of the many Level 1 domains (or the one Level 0
>>>>     domain) up to some Level 2 domain in order to get to another
>>>>     Level 1 (or the Level 0) before it could be delivered.  This is
>>>>     true even if the ISP border routers connecting to the sites with
>>>>     the sending and receiving hosts are physically close, but happen
>>>>     to be of ISPs which are in separate domains.  (I assume an ISP
>>>>     can only be in one domain, but perhaps I am wrong.)
>>> Definitely more stretch. But that may be inevitable as we grow towards
>>> ten billion hosts.
>>
>> ITR-ETR schemes with well placed ITRs and ETRs don't necessarily
>> involve longer path lengths.  In many or most cases, once the system
>> is widely adopted, I think there would be no extra distance or
>> number of routers for most packets.
> 
> Well, you don't know that a priori. 

In general, for the current ITR-ETR proposals, I think we do know this.

If an ITR is on the path the packet would ordinarily travel on
(without the ITR-ETR scheme) and so is the ETR, then it is
reasonable to assume the tunnel will follow the same path - so there
are no extra hops, stretch etc.

This will be the typical case for most traffic in the currently
discussed schemes - with the reasonable assumption that the ETR is
on whatever would otherwise be the path.

If the ITRs are full database, all traffic will follow the same path
at it would have without the ITR-ETR scheme.

With only a caching ITR on the path, a few initial packets would
have to travel to a full database ITR.  With Ivip or eFIT-APT, that
is likely to be a local ITR or Default Mapper, which is probably on
the path anyway.  Otherwise, the first packets may have a longer
path to take.  LISP-EMACS involves probably much longer paths for
initial packets, since it is a global system - although it is
intended that one copy of the initial packet winds up at the ETR,
which is where an ITR would send it anyway if it had the mapping
information.

Ivip or any other scheme which supports non-upgraded networks with
"anycast ITRs in the core" may involve sub-optimal path lengths when
all the packets must go to the nearest full database ITR outside the
originating network, which is possibly somewhat (or maybe a long
way) off the shortest path.

But your multilevel system seems to involve frequent deviations to
routers in a higher level just to get the packet down to the same
level, in another domain, even if the destination is physically
close to the source.

> If we use something like GRE
> you actually have no control at the ITR-ETR level over how many
> actual routing hops are involved in the GRE tunnel.

Ivip does its own tunneling in a very straightforward way - IP in IP
- without involving any other protocols.  The inner packet is the
original traffic packet.  The outer destination address is that of
the ETR.  The outer source address is that of the sending host.


>> Your system, as I understand it, involves restrictions on which ISP
>> can connect with other ISPs, so the total system is going to be more
>> hierarchical, branched and compartmentalised than would be the case
>> with a single domain, as exists today.

You responded to questions about ISP connectivity later.

>>>> 3 - Therefore, further dimensions of "Balkanisation" of the
>>>>     Net, where actual packet delivery times and reliability
>>>>     levels depend not just on physical (partly geographic)
>>>>     topology, link capacities and traffic levels but also on
>>>>     which domain each host is in and how far the packet has to
>>>>     travel to go up and then down a level to the destination
>>>>     domain.
>>> I don't see why that is Balkanisation, which implies walled gardens
>>> to me. I would counter this and the previous two arguments with the
>>> assertion that an organised hierarchy of finite meshes is intrinsically
>>> easier to understand and debug than a single unbounded mesh.
>>
>> I didn't mean "walled gardens" - just more like your Step 4 diagram,
>> in which, as I understand it, an ISP in the bottom right Level 1
>> domain can't connect - or finds it more complex and therefore
>> expensive to connect - to an ISP in the bottom middle Level 1 domain.
>>
>> I doubt that conceptually and practically more complex arrangements
>> lead to greater ease of understanding and debugging.  The ITR-ETR
>> schemes all introduce a roughly similar new set of architectural
>> arrangements in terms of address space, ITRs, ETRs etc. which will
>> make the total Internet harder to understand and debug.  These
>> schemes are only being contemplated because we perceive a
>> prohibitive scaling limit in the DFZ if we don't implement such a
>> scheme and the number of end-user networks (and ISPs too) continues
>> to grow as we expect.  These schemes are being contemplated just to
>> keep the Internet running with its current hosts and most of the
>> current BGP routers untouched - not because the result will be
>> simpler or easier to debug.
> 
> Which is worrying in itself.

Here is a list of possible architectures, ordered best to worst in
terms of ease of debugging:

1 - Current BGP system.

2 - Current BGP system plus a Ivip or some similarly effective
    ITR-ETR scheme.

...

? - Current BGP system without an ITR-ETR scheme, some time in the
    future, when the number of prefixes and routers is so great that
    the whole system doesn't work very well - which means it
    can't easily be debugged.
...

? - A system such as yours which involves multiple BGP systems,
    in a layered hierarchy.

? - As above, but with a separate ITR-ETR scheme in each domain
    (which I think is impossible) - which is what you are
    suggesting.

...

? - Any system which involves host changes.


What I meant to say is:

These ITR-ETR schemes are not being contemplated with the express
purpose of being easier to debug than the current system, but
because the current system can't continue, and we need something
more complex - which will no-doubt be harder to debug than the
current system.

I am convinced that any architecture such as yours which creates
multiple BGP systems and/or multiple ITR-ETR systems will be far
harder to debug than a simpler approach.

> I would maintain that a hierarchy is conceptually and practically
> simpler than an ever-growing mesh. There's a systems engineering
> tradeoff here.

It is definitely not conceptually simpler.

I think the ITR-ETR approach enables the BGP system to retain its
current, practical, scale - without any conceptual changes to the
BGP system itself.  The ITR-ETR system is definitely a bunch of
extra concepts and functional elements.  However, the currently
discussed proposals are all for one BGP system with one ITR-ETR
system as the optimal outcome.  (Multiple Ivip systems could operate
in parallel - even on a commercial basis - but this is not what I am
pursuing.)

The trick is to make the ITR-ETR system much more scalable than BGP
is.  This is a challenge, but seems practical.

For instance, Ivip enables each ISP to select where it wants to put
full database ITRs (ITRDs) and Query Servers (QSDs) and where to put
the caching ITRs (ITRCs) and ITFHs (ITR Function in Host).  This
enables them to optimise the arrangement so as not to push all the
mapping data to too many devices, by relying more on local query and
response traffic.

Fanning out a stream of mapping data, fast, to ITRDs and QSCs all
over the world is a challenge, but it is much simpler, faster and
more efficient than trying to get global distribution of a
comparable quantity of information via BGP's slow, awkward,
uncoordinated, one peer to the next approach - which may take ages
to converge on the optimal outcome.

I am proposing we add exactly one (admittedly complex) conceptual
thing to the Internet - and only one instance of that thing.

You seem to be suggesting we need a new multilevel BGP system to
overtake the current one, with a large number of instances of this
new system.  You also wrote that each such instance requires its own
instance of another new concept - an ITR-ETR scheme.


>>>> 4 - Therefore, a lot of fuss as ISPs try to decide which domain
>>>>     to be in.
>>> I don't see this as being that much different from today's fuss
>>> as ISPs decide who to peer with.
>>
>> Can an ISP be in two domains at the same time?  Can an ISP be in two
>> or more Level 1 domains?  Two or more domains of different levels?
>> In the Level 0 (existing DFZ) and in some other domain(s)?
> 
> I think an ISP would have to operate two ASes to be in two domains
> at the same time. I need to think a bit more about shortcut routes
> in such a case (i.e. violating the separation of regions). It seems
> likely that if a packet shows up in the "wrong" place, there's no
> reason it can't be routed further through the system (but some
> thought is needed about spoofing and loop-prevention).

You have complex rules about different levels of BGP system and
then, because this creates obvious inefficiencies and unwanted
complexities, you are tempted to create further arrangements (more
complexity still) to resolve these difficulties.

I just think your scheme is not needed - a single BGP system and a
single ITR-ETR system should do the trick.


  - Robin

  http://shootdowns-R-us.net
  This week, the Big Bang Theory: http://astroneu.com


--
to unsubscribe send a message to rrg-request@psg.com with the
word 'unsubscribe' in a single line as the message text body.
archive: <http://psg.com/lists/rrg/> & ftp://psg.com/pub/lists/rrg