Re: [RRG] Re: Six/One Router: why have bilateral mode? Referrals . . .

[Robin Whittle <rw@firstpr.com.au>]

Short version:      The only way I can see of solving several
                    problems with Six/One Router is to implement
                    something like Ivip's OITRDs.  This would
                    provide full multihoming support for packets
                    sent by non-upgraded hosts, remove the
                    problems of referrals and so remove the need
                    for Transit addresses in AAAA records and
                    for Unilateral mode - and therefore for the
                    required flag in the IPv6 header.

                    Also, I have concerns about how each
                    Translation Router can test reachability
                    of the destination network, or destination
                    hosts, via each of the Transit prefixes
                    specified in the mapping reply.  This is
                    essential to multihoming.



Hi Christian,

Thanks for confirming this:

> you are right that Unilateral mode is needed between two upgraded edge
> networks to handle transit address referrals.

My understanding was that these referrals are the only reason that
Unilateral mode is needed - however, your message reveals another
reason:

  Host A in an upgraded network gets an AAAA record for host B
  also in an upgraded network.  The address selection algorithm
  (as you explain it) causes A to choose the Edge address over
  the one or more other addresses, which are all Transit addresses.

  Host A sends a packet to host B on B's Edge address, but for
  some reason, finds B is unreachable on this address.

  Therefore, it tries a Transit address, and happens to find one
  on which B is reachable.

More on this below.

>> Still, I don't see how referrals can be made to
>> work in general, since host A having been referred
>> to host B by B's transit address will (in many
>> higher level protocols) tell B this address, and
>> B will not recognise it, since hosts have no idea
>> about transit addresses.
> 
> B *will* recognize its transit address when being told about it by A,
> based on standard NAT traversal mechanisms such as STUN, ICE (and TURN
> if need be).  As a matter of fact, ICE would even prefer edge
> addresses over transit addresses:  It prioritizes addresses that are
> directly configured on an interface (i.e., the equivalent of edge
> addresses) over addresses on the public side of a NAT (i.e., the
> equivalent of transit addresses).

My impression was that NAT and therefore STUN, ICE and TURN were all
created for IPv4, and that it is widely hoped, expected (or
perfectly reasonably and vehemently demanded) that such things won't
be needed in IPv6.

My understanding of your answer is that any host using the new kind
address space provided by Six/One Router (Edge space) will need to
employ some of these NAT techniques - that is to consider itself
always potentially behind some kind of NAT arrangement - when
communicating with hosts which are on ordinary address space.

Yet the whole idea of router-based core-edge separation schemes
(map-encap, Six/One Router and my two new forwarding based
approaches) is to avoid the need for any changes in hosts.


>> I think the reason why Unilateral mode between two hosts in upgraded
>> networks is necessary is due to the weakness (I think) of the method
>> by which one host finds the address of the other via DNS.  I
>> understand the AAAA record carries the E address and the one or more
>> T addresses.  The host is assumed not to know about Six/One Router,
>> or which addresses are End user addresses or Transit addresses.
>>
>> I think it tries to use the IP addresses in some random order, and
>> as soon as it finds one which works, it sticks with that.
> 
> Actually, the situation where host A in an upgraded edge network
> chooses a destination transit address for host B in another upgraded
> edge network does usually not arise when host A knows about host B's
> complete address set (i.e., the edge address plus all transit
> addresses).  In that case, the standard destination address selection
> algorithm will make host A prefer host B's edge address over any of
> host B's transit addresses.  Again, this is because of the
> longest-prefix match in the destination address selection algorithm,
> and the use of a unique (short) prefix for edge addresses.

OK.

> Said situation rather may occur when host A is being referred to an
> incomplete set of host B's addresses that does not include host B's
> edge address.  Host A in this case has no choice but to select a
> destination transit address.

OK - so this is a referral situation rather than using DNS.

>> (This sounds very slow - how long is the host to wait without a
>> response before it decides one address doesn't work and that it
>> should try the next?)

I meant to take this earlier stuff out of the message because by the
time I wrote the end of it, I realised how the address selection
system worked.

> It's not slow.  Hosts in upgraded edge networks never have to try
> twice because any destination address works for them, be it edge or
> transit address.  Hosts in legacy edge networks may have to retry if
> they happen to select a destination edge address.  

Wouldn't the same principle work in reverse?  If the standard
address selection algorithm enables an Edge network host to
automatically choose the single Edge network address of another
host, over the one or more Transit addresses, wouldn't a host on an
ordinary address (not in the Edge /3 or whatever) choose a Transit
address over an edge address?

If all transit addresses were in 2::/3 and all Edge addresses in
4::/3, my guess is that the system should work nicely in both cases.

> But the time-to-retry should be rather short, because the retry is
> efficiently triggered by an ICMP Destination Unreachable message,
> which the first core router sends when it finds that it doesn't
> have a route to the selected destination edge address.

OK.

>> I think it would be best if the host using DNS AAAA addresses never
>> found that a transit address worked.  

This shouldn't matter if this host A is on an Edge address, since
according to your description above, the standard address selection
algorithm should cause it to choose the one Edge address of B over
the one or more Transit addresses provided in B's AAAA record.
(Except if B is unreachable on this Edge address . . . )

The trouble is with the host A which is not on an Edge address (an
ordinary host, not using Six/One Router address provided address
space) trying addresses of Host B from an AAAA record.  (There is
also trouble if A is on an Edge address and B is unreachable on its
Edge address. )

If my suggestion immediately above would work such as 2::/3 for
ordinary addresses including transit addresses and 4::/3 for Edge
addresses - then that solves the problem, since it would always
choose a Transit address. (Except if B is unreachable on this Edge
address, in which case A will choose a Transit address and maybe
find one which works.)

>> It should only find that the E address works.  If this can be
>> assured, then there would never be a need for Bilateral mode
>> between two hosts in upgraded networks. Then maybe you could get
>> rid of the Bilateral/Unilateral bit, which at present you
>> apparently need, and which can only come from the Flow Label
>> bits.

> Yes, I had thought about this.  One way to realize this would be:  If
> a host in an upgraded edge network attempted to reach a transit
> address from a peer in another upgraded edge network, it should
> receive an ICMP Destination Unreachable message.  This would make the
> host switch over to an edge address of the peer.  

My suggestion:

>> I think it would be best if the host using DNS AAAA addresses
>> never found that a transit address worked.

only helps with A being an upgraded host.  Only such hosts can be
prevented by the Translation Routers in their provider networks from
being able to reach a host B on any one of its transit addresses.

If the host doing this algorithm is an ordinary host, we have no way
of stopping it reaching B via a transit address (in the current
conception of Six/One Router) - even if we wanted to do so with the
Translation Router in B's provider network, since a Transit address
is is the only possible way it could reach B (due to B's Edge
address not being in any advertised BGP prefix).

What if host A is trying to reach host B, with both being in
upgraded networks, but B being currently unreachable via its Edge
address, but is reachable via one or more of its Transit addresses?

Then host A would try a Transit address and if it found that worked,
host A would assume this was the address to be used in the future.
Then host A might give that Transit address in a referral, which is
not what we want, since that means we have to keep Unilateral Mode
and so the Bilateral/Unilateral flag in the header.

Why would a host be unreachable via its Edge address but still be
reachable via its Transit address?

Lets say the remote host B has an Edge address AE and two Transit
addresses AT1 and AT2, which are in two separate Transit prefixes,
advertised by ISPs P1 and P2 respectively.

Host A uses DNS to look up B's FQDN (there could be any number of
FQDNs which refer to B) and gets back three addresses in no
particular order:

   AE   AT1   AT2

Let's say A first tries to use AE, but finds this doesn't work.  How
could this be?  Let's say the Translation Router (TR) in A's network
looks up the mapping for AE and gets a result (or already has it
cached) to the effect that the address should be translated to the
AT1 prefix, unless this is unreachable, in which case it should use
AT2 instead.

Let's say that P1's network is unreachable to A's TR right now, or
that the link from B to P2's TR is down.  Either way, B is
unreachable via AT1, but is still reachable via AT2.

I understand that Six/One Router is resembles the non-Ivip map-encap
schemes in that you don't have a fast mapping system - meaning that
the end-user network does not have real-time control over how their
address space (micronets for Ivip) is mapped to transit prefixes
(ETR addresses for Ivip).  Consequently, like the non-Ivip map-encap
schemes, you need to have mapping which specifies multiple methods
of getting packets to the destination (multiple ETRs for map-encap
and multiple Transit prefixes for Six/One Router) together with
priorities for which one to choose if they all work.

How does A's TR decide whether B can be reached via AT1?  It's not
good enough to test reachability to the TR of P1.  (It is not clear
to me how A's TR could do this anyway, since it doesn't know the
address of P1's TR.  In a map-encap scheme, the ITR does know the
exact address of the ETR it is supposed to test reachability to.)

Even if P1's TR was reachable, how would A's TR test reachability of
B through this prefix?

I think this is a fundamental problem of the non-Ivip map-encap
schemes as well.  I think non-Ivip map-encap schemes rely on getting
an ICMP message when the tunneled packet hits some router somewhere
which can't forward the packet correctly.  That might be OK if the
outage is before the ETR, but it doesn't help the ITR decide whether
the destination network is reachable via the ETR.  If there is an
outage between the ETR and the destination host, any ICMP message
would not go to the ITR, but to the sending host - and in outage
situations, I don't think it can be assumed there would be any ICMP
message anyway.

Ivip doesn't need any of this, since it is the responsibility of the
end-user network to monitor reachability however they choose, and to
use the real-time fast push mapping system to tell all the ITRs a
single ETR address to use for each micronet.

I don't understand how a Six/One Router Translation Router tests
reachability.

But let's say B is unreachable via AT1 right now, and this is where
A's TR is translating the packets to, when A sends the packets to
B's real address: AE.

Nothing comes back.  Or maybe there is an ICMP message to A - this
would come back through A's TR, and be translated to A's Edge
address.  The absence of positive response or the ICMP message would
alert A that this address AE was not working.

   You can't very well have A's TR looking at these ICMP messages to
   A as a means of reachability testing for B.  Firstly, this would
   be difficult to do, looking at every packet to determine it was
   an ICMP message of potential interest.  Secondly, the TR couldn't
   do it securely, since it could only reject spoofed ICMP messages
   if it retained a few second's worth of recently translated
   outgoing packets (or at least the first few hundred bytes of
   them.  That would not scale well.

So A sends a packet to another address, by chance AT2, and finds it
works.  This is not what you want.  You want A to ideally never try
a transit address - or at least to never find that it works.

The standard address selection system apparently will cause it to
choose AE first, every time.  The problem is if AE doesn't work.

You can't very well have A's TR blocking all packets sent by hosts
to any address in the AT2 prefix. (Actually, you could, as I discuss
below, with having a separate /3 purely for Transit addresses, and
by making all translation occur at the border routers, not at any
other routers inside each provider network - the translation router
would reject any packet coming from inside its network addressed to
a transit address.)

Host A can't be required to do anything special because it is on an
Edge address, or because B is on an Edge address too - because if we
require host changes like this, it is going to be very much more
difficult - probably impossible - to get enough people to adopt the
new technology.

I think the only way out of this is ensuring that A's TR always
knows which of the two potential prefixes AT1 or AT2 a host such as
B can currently be reached on.  (Also another way is implementing
something like OITRDs - then the AAAA record only contains the Edge
address and no host ever knows any Transit address.)

I can't see how you could achieve this.  As with the non-Ivip
map-encap systems, you can't very well have your TRs (ITRs in
map-encap) constantly testing reachability to each destination
network, just in case they get a packet from one of their own
network's host addressed to a host in the destination network.


> The reason I did not
> follow this approach is because I wanted both address types to be
> reachable from upgraded edge networks.  But the discussion with you is
> making me reconsider, actually...

If you could ensure an upgraded host could never reach another
upgraded host except on its Edge address, then I think you wouldn't
need the Bilateral/Unilateral stuff, or the extra bit in the IPv6
header.

Perhaps one way of doing this is to split the address space into
three clearly identifiable prefixes.  TRs would know about these.
Hosts would not be required to know about it.

Let's say you have these prefixes:

  2::/3   Ordinary current address space for hosts not using
          the new kind of address space (Edge) provided by
          Six/One Router.

  4::/3   All Transit prefixes are in this prefix.

  6::/3   All Edge addresses are in this prefix.


Now a TR can easily tell when our host A (in an upgraded network
with an Edge address) is sending a packet to some host B also in an
upgraded network with an Edge address - but is sending it to some
Transit prefix by which B could be reached, rather than using B's
real Edge address.

Then the TR can drop the packet and send an ICMP Destination
unreachable address to A.

So A would be physically unable to reach B via any address other
than B's Edge address AE.

Also, with the above arrangement of prefixes, I think the address
selection algorithm (I don't know it in detail myself, just going by
what you wrote about it) should always cause a non-upgraded host
(its address is in 2::/3) to choose a Transit address instead of an
Edge address when considering multiple addresses in an AAAA record.

  (This would not work if the arrangement was:

    2::/3  Ordinary.
    4::/3  Edge.
    6::/3  Transit.)

This arrangement should be - OK to have providers advertise only
2::/3 prefixes for their non-upgraded end-user networks (PI and PA)
and only 4::/3 prefixes for the purpose of accepting packets with
Transit addresses which need to be translated so the packet goes to
the Edge address in an end-user network this provider is connecting
to the Net.

However, the arrangement I suggest still has a problem.

Currently, with Six/One Router as you describe it now, if host A got
a referral to host B (both on Edge addresses) via some host R which
only knew B via one of B's Transit address, then the system would
still work - since A could send a packet to B using the transit
address.  (But still, you have to have B changed to use STUN, ICE,
TURN etc. which I think is a host change which would make Six/One
Router much harder to deploy.)

With the change I suggest, A would not be able to use this Transit
address for B at all.  So this is not a satisfactory outcome at all.

Again, the problem is referrals which provide Transit addresses.


Perhaps if you instituted the equivalent of Ivip's OITRDs (Open ITRs
in the DFZ) - similar to LISP's "Proxy Tunnel Routers", then you
would achieve a number of things:

1 - All non-upgraded hosts would be able to reach upgraded hosts
    via their Edge address.

2 - Therefore, you don't need to put Transit addresses in the DNS.

3 - Only the mapping system and the TRs need to know about these
    Transit addresses.  Hosts never see them.

4 - Then all hosts can reach each other, and the upgraded hosts only
    have their Edge address known to all hosts.

5 - This would overcomes the serious barrier to adoption you have
    at present - as afflicted LISP prior to PTRs: that multihoming
    only works with traffic from hosts in upgraded networks.

6 - Now you don't have a referral problem.  I think you don't need
    the Bilateral/Unilateral bit, since Translation Routers will
    always be translating both source and destination Edge addresses
    to Transit addresses for outgoing packets, and for incoming
    packets, both source and destination Transit addresses to Edge
    addresses.


You would probably still want to keep Edge addresses in one /3
prefix and Transit addresses in another /3, separate from the prefix
used by non-upgraded hosts.  You would certainly want all Transit
addresses in a distinctive /3, so that each TR could easily decide
to drop a packet (and generate an ICMP message) for any packet
coming from inside its network which was addressed to such a prefix.

However, I think my suggestion that TRs block packets sent by hosts
to Transit prefixes precludes doing translation anywhere but at the
border routers.  I think having to do all the translation work at
the border router is a serious bottleneck.  With Ivip and LISP at
least, it is possible to put the ITR function closer to the sending
hosts, inside the networks - not just at the border routers.
Likewise the ETR function.  Ivip enables the ITR function to be in
the sending host (but not if the sending host is behind NAT).  This
enables the use of more ITRs, taking the load off the already busy
border routers.  Also, it may be an essentially zero cost option to
put the ITR function in the sending host.

 - Robin


--
to unsubscribe send a message to rrg-request@psg.com with the
word 'unsubscribe' in a single line as the message text body.
archive: <http://psg.com/lists/rrg/> & ftp://psg.com/pub/lists/rrg