[RRG] Re: Supposed impossibility of scaling for mobility

[Robin Whittle <rw@firstpr.com.au>]

I am replying to 2 messages from Louise and 1 from Scott.

Below is another attempt at explaining Ivip's approach to mobility,
further to previous messages in this thread and to:

   Scaling, Mobility & 228 mapping changes a second
   http://psg.com/lists/rrg/2008/msg00535.html


I describe how this form of mobility it could be done with any other
map-encap scheme (LISP, APT or TRRP) - only not so well as with Ivip.

I discuss how there are 3 layers of mobility system at work:

1 - Within the access network, such as switching between base
    stations, but always retaining the one IP address, which is
    used as a care-of address by the mobile host.

2 - Between the one TTR and the multiple care-of addresses of
    multiple access networks in the one city - really in the one
    1000km area or whatever, but also temporarily at least from any
    access network.

3 - The map-encap scheme using a mapping change to select a new
    TTR, which typically only needs to occur after travelling
    1000km or whatever.



Hi Louise,

Thanks for your messages, in which you wrote:

> Whilst I agree that if you have a new feature, it will aid
> deployment, I am very unsure that global mobility is it.  I think
> this because there is a lack of global mobility within the voice
> world, and I have a suspision that this is partly due to the time
> to build security associations making handover (as opposed to
> portability) technically dificult, and partly due to a business
> model that holds customers to you - the easier you make handover
> the easier it is for someone to start using a different network

The failure of the telcos to provide seamless mobility for voice has
various causes, including difficulties agreeing on air interface,
radio frequencies etc. so a single global handset could be built.
GSM and 3G has security arrangements, involving the SIM talking to
the central server of the home network before the phone can do
anything.  It is impressive how they make this work when roaming on
the other side of the globe.

Just because the telco's couldn't do it with cellphones doesn't mean
it can't be done with IP.

Anyway, this failure makes a genuinely portable IP address or subnet
of IP address space all the more valuable because it would be a
seamless, international, platform for voice communications, without
the fuss of adapting to new IP addresses, fighting through NAT etc.


> What I would really like though is to understand what the
> trade-offs are - if we support mobility what will it add in
> terms of system complexity or cost.

There is no extra architectural complexity for the main Ivip system
which provides scalable multihoming and portability (and a simple
yet real-time controllable form of TE, without explicitly load
sharing).  This architecture involves simple ITRs tunnelling to a
single ETR for each micronet, with no reachability stuff or
multihoming service restoration decision making in the system at
all.  The end-user provides that, with whatever sophistication they
like.  The key to this is fast push, essentially real-time control
of the world's ITRs.

If Oprah is geared up to do a webinar to 800,000 users:

http://event.oprah.com/videochannel/event/event_landing.html?promocode=HP11

and since our fabulous Internet can delivery packets halfway round
the world in 0.18 seconds, I reckon we should be able to use the Net
to achieve close to real-time control of the world's ITRs.  I am
aiming for 5 seconds latency.

It makes no sense to push to all those ITRs (NERD), and a global
query server system is too slow (ALT - or TRRP unless radically
souped up with integrated anycast sites) and plagued by bottlenecks.

So we need a system with some pull, and a "Notify" system to get
changed mapping to ITRs which need it.  So Ivip uses fast push (paid
for by end-users, per update) to wherever operators want to place
their full database query servers (and full database ITRs), with the
operators using pull with Notify from there to however far in their
network they want to place ITRs.

This gives end-users whatever control they want of the ITRs, rather
then forcing them to use the limited options available in LISP, APT
or TRRP - each ITR detecting failure on its own (actually APT is a
little different from this) and each making its own decision about
which of a list of ETRs to use.

This has nothing to do with mobility.

Mobility can be added to any map-encap scheme - LISP, APT, Ivip or
TRRP - by adding Translating Tunnel Routers.  There is no way of
preventing this by the architecture of the map-encap scheme.  To an
ITR, a TTR is indistinguishable from an ETR.

TTRs don't absolutely need to be standardised, other than that they
must behave like IETF-standardised ETRs.  Five different TTR
companies could have five different global networks of TTRs, and
each company could have its customers install a different set of
software for tunnelling to the TTRs, and for enabling the company's
management system to figure out where the mobile device's care-of
address(es) are.

However:

1 - Ivip would be much better for this than the other map-encap
    schemes, because the end user can select a new TTR within
    seconds, while the others have very slow push times (APT and
    NERD), or necessarily slow caching times with no Notify system
    (ALT and TRRP).

2 - I intend that Ivip include standards for the way TTRs and
    mobile devices interact, so there won't need to be a bunch of
    incompatible protocols in wide use between mobile devices
    and TTRs.

The extra bits in Ivip for mobility will all concern TTR <--> mobile
device communication.  That doesn't compromise or burden the rest of
the system with any extra complexity.

In terms of costs, there are some extra costs due to mobility - but
these all arise from the greater use of the mapping system, not due
to the cost per usage being any higher.

Lets say without mobility there are 100M end-users with 200M
micronets, generating 10M updates a day - a small number of
multihoming service restoration changes and the rest due to a small
number of end-users making lots of real-time changes to their load
sharing.  (So a lot of the cost of the fast push system is paid for,
quite happily, by these few intensive end-users who are saving a lot
of money due to the new kind of load-sharing control.)

With mobility, there are 2 billion end-users with 2.2 billion
micronets (most mobile end-users will have cellphone/computers
whatever, and only need a single micronet).  Since they only
generally change their mapping when moving large distances, such as
1000km, they add another 10M changes a day.

So the cost to all networks is that their full database ITRs and
query servers (ITRDs and QSDs) now need to deal with twice the rate
of updates and 22 times the number of micronets.  The storage is not
a prohibitive problem (terabyte hard drive or probably RAM or FLASH
by the time this level of adoption occurs), and the traffic cost of
the updates will be pretty small, by the standards of the day.
(This is 2020 or so.)

But . . . the value of the Internet to those networks which
themselves have no mobile end-users is higher, because the Internet
in general is better due to the ease and utility of being able to
communicate reliably and without fuss with all those mobile devices.

It could be argued that if the entire uptake of these mobile devices
was in a single country, and the people who used them never left
that country or communicated with anyone outside that country, then
the rest of the world would be paying a higher price for their full
database ITRs and query servers just to get all these updates and
micronets which in fact do them no good.

In reality, people all over the world will be using mobility, and
they will be communicating freely with all the other non-mobile
people, so almost everyone benefits from mobility - making these
extra costs not such a problem.


> and also a better definition of mobility - if we are looking at
> several seconds, are we really talking "portability" ie no session
> continuation across the mobility event, even if the IP address
> stays the same some session types will drop with that type of time
> lag - ....

As I have tried to describe before, this kind of mobility is not
like what you are used to with existing mobile IP systems.

The 5 second or whatever lag in tunnelling traffic to a new TTR does
not imply lost connectivity.  The idea is the mobile host sets 2-way
tunnels to two TTRs, changing the mapping to the new one while it
still has a tunnel to the old one.


The global ITR system with its essentially real-time control system
(fast hybrid push-pull with Notify) enables the end-user (or more
likely the company who provides their TTR services) to tunnel
packets from all correspondent hosts to one TTR or another, with a 5
second or so latency.

This tunnelling is generally along optimal paths, since there are
ITRs everywhere (as needed for the basic map-encap functions of
scalable multihoming etc.) and the TTRs are located close to, or
within, access networks all over the world.  "Access network" means
anywhere a mobile device could gain a care-of address, either via
plugging the device in via an Ethernet cable, via 3G, WiFi, WiMax or
whatever.  So "Access network" means pretty much any ISP, university
network, business network etc. as well as any network with 3G, WiFi,
WiMax etc.

No matter what sort of care-of address the mobile host gets, it can
create a 2-way encrypted tunnel to a nearby TTR.  The TTR company's
global control system and the software in the mobile host work
together to determine which close TTRs are best to tunnel to.

The mobile host tunnels to TTR-A from its (Care of Address) CoA-A it
got via a 3G network.

Then, while the 3G link is operating, it gets a WiFi or a cabled
Ethernet link to some other network, and establishes CoA-B.  From
there, it can make a second 2-way tunnel to TTR-A.  This may be a
longer tunnel than from CoA-A, but it works.

The TTR can send incoming packets down both tunnels, so if one fails
(say the 3G system has a glitch, drops out or ends the session,
terminating use of the CoA-A address) the mobile host still get the
packets from TTR-A via the WiFi link.  Likewise, outgoing packets
can be sent up both links for redundancy.  (The TTR and the mobile
host need some software to avoid duplicate packets getting to the IP
stack, or being forwarded out of the TTR to the rest of the Net.)

So the mobile host can move from access network to access network,
keeping up at least one tunnel at all times to the TTR.  This gives
it full connectivity - with no mapping changes.

The mobile host can move to another city, even on the other side of
the world, gaining care of addresses CoA-C, CoA-D etc.  From each of
these it can make a 2-way tunnel to TTR-A, retaining connectivity.

However, now the path lengths are not optimal when the new access
network is in the UK and the original access networks and TTR-A are
in Australia.  So it would be best if the mobile host found a nearby
TTR - in the UK - and set up a 2-way tunnel to that.  Then it would
be best to change the mapping (actually the TTR company would
probably be orchestrating this, rather than the mobile host) to
tunnel packets to the new TTR near the new access network.

Mapping changes are not needed when changing to a new access
network.  They are desirable when the new access network is a long
way from the original access network - because to ensure optimal
path lengths (and therefore low delays, minimal packet losses etc.)
it is necessary to choose a TTR nearby and so change the mapping to
point to that.  "Nearby" probably means within 1000 km, unless
perhaps you are doing a really large amount of data for a long time
in one city, with hosts in that city, and it makes sense to choose a
TTR in that city, rather than one 200km away.

> .... I know I hit reload on a web page if it seems sluggish to
> respond and I suspect I only wait a few seconds; are the mobility
> events being essentially quite rare (crossing borders) ?

I hope the above explanation makes it clear that most of this
mobility stuff happens between the mobile host and the TTRs, which
have nothing at all to do with the main Ivip system.

Overall, there are three levels of mobility at work:

1 - Within a radio access network, layer 1 and 2 (I think) mobility
    concerning different base-stations, access points etc.  The
    result of this is that the mobile host retains a single CoA.

    In the above examples, there are often two or perhaps more
    separate access networks in use at once, each providing a
    different CoA.


2 - Between various CoAs of the various access networks and the one
    TTR.  This is the business of the TTR companies and the software
    in the mobile host.  Ideally this would be IETF standardised,
    but it need not be, since it doesn't really concern the
    map-encap scheme.  The only thing the map-encap scheme sees is
    the TTRs, which must behave the same as ETRs.

    As long as this activity works with the one TTR, then there
    is no need to change mapping.

    To reduce path length, reduce delay, improve performance and
    to reduce packet losses, it is best if the TTR is close to
    wherever the access network is.  "Close" typically means less
    than 1000 km or something like this.


3 - When the mobile node selects a new TTR - which will typically
    be because it has moved to an access network which is more than
    1000km or whatever from where it was when it set up a 2-way
    tunnel with the first TTR - then this is a time to change
    the mapping.  There is no break in connectivity, since the
    incoming packets go to either the old or the new TTR.

    This is the part of the total mobility job which is performed
    by the map-encap scheme.  The map-encap scheme gets packets
    to the current TTR with typically optimal paths, because it
    is already designed to do this for any other kind of ETR -
    for the purposes of scalable multihoming etc.

    Any map-encap scheme could do this, but only one with
    essentially real-time control of mapping (Ivip) can really
    serve the mobile user well.  They reasonably want to change
    their mapping to a new TTR in a few seconds - even if they
    only do it every few weeks.  Even if they are travelling
    internationally by jet, they will probably only want to
    change their mapping every few hours.

    It could be done with LISP-ALT, but only by shortening
    caching times in order to improve response times to
    the end-user's mapping change.  The cost of that, for all
    operators of ITRs which happened to be sending packets
    to that end-user's EID, would be more frequent requests for
    new mapping data.


> The idea of holding your IP address whilst going between home
> and work raises a load of associated questions about security and
> accountability.

Sure - no-one has to do it.

Your PC/phone or whatever could have two IP addresses at all times,
via two micronets, and you could switch from one IP address to the
other with the PC itself.

Actually, your micronet could be for two IP addresses.  Then when
your PC sets up a 2-way tunnel to a TTR it is for both IP addresses.
 The TTR gets packets tunnelled to it from 2 micronets - one is your
personal one and the other is your work one.  So every time you
change TTR, you need two mapping changes rather than one.  But you
only change TTR when you travel 1000km or more.


> Also intersting to ask why the phone networks chose to use "nasty
> routing inefficincies"

Phone companies tend to suck in the way the charge for every
conceivable thing and their marketing and mobile charging plans seem
deliberately designed to confuse.  While their technology has been
bedevilled by incompatible standards, the global phone system does
work and I think it is most impressive.

> Also interesting to wonder why, if mobility is that important to
> you, you dont use existing solutions?

With mobile IPv4, you can't get optimal paths unless you have fancy
new host software in all correspondent hosts.  This will not happen.

Using standard IPv4 hosts, you are bound to send all packets to the
one, fixed, Home Agent router, which may be nowhere near the
correspondent host or the mobile host - so you are stuck with
inefficiencies, delays and higher rates of packet loss.

With the Ivip approach to mobility, correspondent hosts (IPv4 and
IPv6) have absolutely no changes - no mobility software.  The path
lengths are optimal, due to the proximity of ITRs and due to the
mobile host choosing TTRs which are pretty close to whatever access
network they are using.

Likewise with mobile IPv6, you need mobile software in the
correspondent hosts, unless you are happy to send everything on
typically longer paths via the fixed home agent router.  Traditional
mobile IPv6 with optimal path lengths seems more promising due to
the ease with which one can conceive of new OS software in IPv6
hosts in the future - but this is primarily because so few IPv6
hosts are actually used today.

Also, note that the access networks used by the mobile host in the
Ivip model are completely bog standard and have no mobility routers
etc.  No access network which gives you Internet access in any form
- including behind multiple layers of NAT - can prevent the mobile
host from finding a nearby TTR and setting up a 2-way encrypted
tunnel to that TTR.  It can do this to any TTR, to multiple TTRs at
once.  Then the map-encap scheme tunnels packets on generally
optimal paths to the TTRs.

There needs to be no technical or business coordination between the
access network and anyone, other than to the extent that the
end-user gains an IP address.

The TTR company doesn't need any special relationship with the
access networks.

The end-user chooses one of potentially many TTR companies.

The TTR companies don't need any special relationship with the
RUASes which fast push the mapping updates.

The end-user needs a micronet of address space, probably just a
single IPv4 address or a /64 of IPv6 space.

The end-user needs a business relationship with the TTR company, and
probably needs to give the TTR company the username and password
which enable mapping changes for their micronet to be given to the RUAS.

So the end-user needs:

  Mobile device.
  Micronet of address space.
  Relationship with TTR company, and to install their software.

Then, whenever their device can get connectivity, it will find a
nearby TTR (probably the one it has been using for days, unless the
end-user has moved a long distance) and so provide full connectivity
for the end-user's micronet.  This could be completely automated.

The TTRs forward outgoing packets from the mobile host to the Net -
and would typically integrate a TTR within themselves, so there are
no other dependencies when the correspondent host is using Ivip
mapped addresses.  So the access network doesn't have to worry about
forwarding packets with the source address being the end-user's IP
address.  The TTR company's TTRs have that job.


Hi Scott,

You wrote, quoting Bill Herrin:

>> And for the record, I take my laptop back and forth to work every
>> day.  You probably do too. When I do so, it travels between and
>> through networks which are administratively and topologically
>> distant. It would be awfully nice if it could keep its "number"
>> the same way my cell phone does without the nasty routing
>> inefficiencies that had to be introduced into the telephone
>> network... Nice enough that I could see buying service from
>> companies who could do it in preference to those who couldn't.
>
> You elicit a couple of questions here.
>
>   - What your cell phone provides is "session continuity".  That
>     is, you're talking and as you change locations you can
>     continue talking.  I assume that when you go to your office
>     you close your laptop, and you don't have any sessions
>     continuing.  This is sometimes called "nomadicity" as opposed
>     to "mobility".  What exactly are you looking for?

I think Bill wants his SSH sessions and everything else to run
continually, week after week, as he lugs his laptop around.
Ideally, they would continue by some magic even when there is no
connectivity.  Some protocols won't do that, but if the laptop
software knows it "owns" its IP address, then it could be configured
not to clobber that address and all applications which are using it
merely because it has no Ethernet cable, or wireless link to the
outside world.

The OS would simply treat this as 100% packet loss, with a return to
normal operation as soon as the laptop found some access network and
set up a tunnel to the TTR it has been using for weeks.  There are
no mapping changes in this.

As long as he stays around the Virginia, DC, Maryland and probably
PA, NY etc. area, he will be fine with the one TTR.   When he goes
to LA, he will use another TTR, and so generate one mapping change.

(I will write a message about travelling round the globe with a
laptop using an aircraft-based WiFi system and keeping the same IP
address all the way, without loss of connectivity.)


>   - Even if you want full mobility, why do you assume the only way
>     to get it is to have the same IP address the whole time?  See
>     HIP and higher layer session continuity mechanisms.

The Ivip approach is not the only way, but if you require that the
mobility solution involve no new features in the correspondent hosts
(that is, you want to use the Internet as it is today and for the
foreseeable future), and if you require that there be generally
optimal path lengths (no home agent) then as far as I know, the Ivip
approach is the only way.


> Excerpts from Robin Whittle on Thu, Mar 13, 2008 12:35:56AM +1100:

>> Multihoming service restoration and portability both can be done
>> best by the end-user deciding which ETR to use, in their own way,
>
> Why do you believe that?  Most end users only care about their
> subjective experience, and if they have to be involved in the
> complexity of how it's delivered, they feel burdened.

I think end-users now and into the long-term future will have
desires which they can better achieve via real-time control of their
mapping than by whatever pre-ordained options you provide them with
via LISP.  Those options involve simple, fixed, constructs and rely
on each ITR making its own decisions, independently of other ITR and
without actually asking the end-user what they want.

End users might suddenly decide they want their EID mapped someplace
different than what they put into the LISP system half an hour ago.
 Ivip can do whatever the end-user might wants.  LISP, ALT and TRRP
can't.

End users generally won't be doing this stuff manually - they will
have their own software systems to automatically balance incoming
traffic, or their own system (or more likely that of a TTR company,
as described above) to manage their mapping to achieve mobility
and/or multihoming failure detection and service restoration.

>> and having fast control of the world's ITRs to implement that
>> decision.
>
> An ITR will maintain state on millions of flows going through it,
> as to which ETR it wishes to be sent to?

Most ITRs will be caching ITRs.  Even a full database ITR will be a
caching ITR in terms of its FIB, with an integrated full database
query server.  There's little difference between that and a caching
ITR with a direct Ethernet link to a full database query server in a
separate server.

  - Robin


--
to unsubscribe send a message to rrg-request@psg.com with the
word 'unsubscribe' in a single line as the message text body.
archive: <http://psg.com/lists/rrg/> & ftp://psg.com/pub/lists/rrg