Re: [tcpm] TCP, multiple addresses and soft errors when connecting

[Pekka Savola <pekkas@netcore.fi>]

On Fri, 19 Mar 2004, Fernando Gont wrote:
> At 04:52 05/03/2004 +0200, Pekka Savola wrote:
> >TCP timeouts when connecting to a destination, but when the
> >destination is unreachable (e.g., because you don't have global IPv6
> >routes, your first-hop router is not operational, the packets are sent
> >on-link by default, etc.), TCP does not abort the connectivity (and
> >try the next address quicker -- instead of waiting for the TCP
> >timeout).
> 
> Not sure if I got it right. Do you mean TCP does not abort the conectivity, 
> and thus the *application* cannot try the next address quicker?

Yep, that's what I meant (and by application, I mean both the app and
the APIs/libraries it's using).

> >2.3.1.1 TCP Connection Termination
> >   One solution is for TCP to abort connections in SYN-SENT or
> >    SYN-RECEIVED state when it receives an ICMPv6 Destination Unreachable
> >    message.
> 
> But what if this scenario arises for all the addresses, because of a 
> transient problem?
> The application won't cycle again, and will return an error. And the 
> *transient* problem could disappear perhaps 5 seconds later...

Note that this applies to new connections only.

Yes, this is a trade-off.  The assumption is that with big transient
problems, all the addresses {at least of the same address family} are
unreachable, or none of them are.  

So, you either have to choose between having TCP connect() try
retrying for dozens of seconds (hoping the transient to go away) or
failing completely.

The tradeoff recommended was that it's better to fail completely; this
works better for transient and non-transient failures, except for the
case where the transient failure was so short that the TCP retry
mechanism could be the feasible failure recovery mechanism for that.  
I don't think it is..

Obviously, there could be a flag to indicate to the stack whether you 
want this behaviour or not.

> Does it really make sense to handle the whole connection establishment 
> issue inside the app, instead of inside the API itself?

Those are the core APIs we have today.  We could have better ones, but
IMHO I think we need to fix problems in the current ones as long as we
don't have a widely-deployed alternative.
 
> I think that if we're debating this, we're implicitly assuming that 
> applications want to connect to say www.example.com, and not one specific 
> IP address to which it maps to.
> 
> So why not implement such a "higher-level" connect(), and handle all the 
> relevant issues inside the API?

Defining such APIs is probably beyond the scope of the work we can do, 
and the things we can fix except in the 5+ year timeframe.. but if I 
get you right, you're proposing that instead of:

 1) getaddrinfo() or gethostbyname() and connect for one address, 
[where, arguably, TCP retry timeout could be warranted -- I don't 
think so myself, but you could argue for it]

 2) getaddrinfo() / gethostbyname() loop to connect to one address
[where TCP retry timeout could not be warranted]

You'd be proposing to create a new function to accomplish 2) in such a 
manner that it would also react to ICMP messages, etc?

> >   When [HOSTREQS] was written, most applications would mostly only try
> >    one address when establishing communication with a destination.  Not
> >    aborting a connection was a sane thing to do if re-trying a single
> >    address was a better alternative over quitting the application
> >    altogether. With IPv6, and especially on dual stack systems,
> >    destinations are often assigned multiple addresses (at least one IPv4
> >    and one IPv6 address), and applications iterate through destination
> >    addresses when attempting connections.
> 
> Doesn't the problem really have to do with having applications performing 
> such a low-level action?

At some point, someone has to decide which behaviour is desirable.  
Isn't this only hiding the same policy decision inside an abstraction 
layer -- and the same could be achieved by e.g. a setsockopt() flag?

(I don't deny that better APIs would not hurt, but we still seem to 
require low-level C-like primitives, and unless there are additional 
ones implemented there, we still have to fix the problems somehow..)

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings