Re: Shim6 failure recovery after garbage collection

[Erik Nordmark <erik.nordmark@sun.com>]

Joe Abley wrote:

> One of the more comprehensible objections to shim6 that was raised at 
> NANOG 35 was from large content providers who currently serve many 
> thousands of simultaneous clients through load balancers or other 
> content-aggregation devices (the kind of devices which switch 
> connections to origin servers without having to store any locally).

But when I watched the Nanog movie on that session, there seemed to be a 
prevailing confusion that shim6 would setup state for every TCP 
connection, when in fact its state is per IP address pair.

> I don't remember the precise number of simultaneous sessions the devices 
> were intended to be capable of serving, but it was a lot.

> The observation was that with the amount of (server, client) state being 
> held on those devices, adding what might be an average of (say) 2x128 
> bits + misc overhead per session might present scaling difficulties.

Sure, but each TCP connection in TIME_WAIT state (and there can be a 
large number of them for every client IP address) takes up more memory 
than that.

For active sessions then in addition to the TCP state there is also 
socket level state plus some state in the applications memory, so in 
that case the shim6 state (which is per client IP address and not per 
connection) might disappear completely in the noise.

> Perhaps Igor or Patrick could comment on likely numbers of (server, 
> client) state required (perhaps excluding sessions in TIME_WAIT) in 
> their experience -- with actual numbers, the scale of the issue might be 
> more apparent.

I think having three sets of numbers would be the most helpful:
 - number of active TCP connections (not in TIME_WAIT)
 - number of unique client IP addresses for those connections
 - number of TIME_WAIT TCP connections

I can measure how much memory is used by each one (assuming the 
application state per active connection is zero) to get some ballpark 
numbers of the impact on shim6 for every active client IP address.

   Erik