Re: shim6 @ NANOG (forwarded note from John Payne) (fwd)

[Igor Gashinsky <igor@gashinsky.net>]

:: On Fri, 24 Feb 2006 Iljitsch van Beijnum writes:
:: > >   * Keeping shim6 state on the end host doesn't scale for content
:: > >     providers.  A single server may have 30,000 concurrent TCP
:: > > sessions
:: >
:: > Right. So there is precedent for storing state for 30000 instances of
:: > "something". Servers are getting a lot faster and memory is getting
:: > cheaper so adding a modest amount of extra state for longer lived
:: > associations shouldn't be problematic.
:: >
:: > (Visit a run of the mill content provider and see how many 100 byte
:: > GIFs they send you over HTTP connections that have 700 - 1200 byte
:: > overhead and of course all have high performance extensions turned on
:: > for extra bandwidth wastage.)

Wow.. I realize I'm coming a little late to this party, but have you asked 
anybody who works for a large-ish content provider about their views 
on what you are suggesting? (disclaimer: I do work for a fairly large 
content provider, but my views in this email are my own, and do not 
necessarily officially represent my employer's views in this email)

First of all, yes, content providers at times do waste quite a bit of 
bandwidth with (what sometimes may appear as unnecesery (and sometimes 
really is)) overhead. However, very often that is done not because people 
don't understand what they are doing, but in an effort to reduce overhead 
somewhere else. Case and point: we all know that http/1.1 + keep-alive is 
more "efficient" from a bw perspective, yet there are some large web shops 
which force all transactions into http/1.0 +no-keep-alive in order to use 
"accept-filter" and therefore save on application state. That state may be 
deemed as more "costly" to content providers then bandwidth. Now, since 
they are obviously going to great pains to save on this state, what makes 
you think that adding shim6 state into the picture "shouldn't be 
problematic."?

Also, while we are on it, let me offer you a view on some issues with 
shim6 from a content provider standpoint (some may be a duplication of 
what Jason has wrote previously, but I want to document it for sake of 
completeness):

1) Most connections to content providers (with the exceptions of 
long-lived streaming sessions, but those sessions are fairly "few" per 
server) are very short-lived http (think about 15 packets in each 
direction including the setup/teardown). Since, shim6 (as designed right 
now) does not initiate from the first packet(s), it might not take effect 
for these short-lived sessions, and therefore will not help in case of 
failure, so in effect, *does not work* at all for fast http transactions 

1) In order to "fix" #1, shim6 has the potential to put a sizable (over 
10%) state penalties on our servers (to service end-sites w/ shim6), 
something which is arguably the most painful thing for those servers, 
which can translate into millions of dollars of additional hardware, and 
many more millions of dollars per year to power/cool that hardware.

3) While TE has been discussed at length already, but it is something 
which is absolutely required for a content provider to deploy shim6. There 
has been quite a bit of talk about what TE is used for, but it seems that 
few people recognize it as a way of expressing "business/financial 
policies". For example, in the v4 world, the (multi-homed) end-user maybe 
visible via both a *paid* Transit path (say UUNET), and a *free* peering 
link (say Cogent), and I would wager that most content providers would 
choose the free link (even if performance on that link is (not hugely) 
worse). That capability all but disappears in the v6 world if the Client 
ID was sourced from their UUnet ip address (since that's who they chose 
to use for outbound traffic), and the (web) server does not know that 
that locator also corresponds to a Cogent IP (which they can reach for 
free). This change alone would add millions to the bw bills of said 
content providers, and well, reduce the likelyhood of adoption of the
protocol by them. Now, if the shim6 init takes place in the 3way 
handshake process, then the servers "somewhat" know what all possible 
paths to reach that locator are, but then would need some sort of a 
policy server telling them who to talk to on what ip, and that's something 
which will not simply scale for 100K+ machines.

4) As has also been discussed before, the initial connect time has to be 
*very* low. Anything that takes longer then 4-5 seconds the end-users have 
a funny way of clicking "stop" in their browser, deeming that "X is down, 
let me try Y", which is usually not a very acceptable scenario :-) So, 
whatever methodology we use to do the initial set-up has to account for 
that, and be able to get a connection that is actually starting to do 
something in under 2 seconds, along with figuring out which sourceIP and 
destIP pairs actually can talk to each other. 

I hope this gives people some visibility as to what some content providers 
think about shim6, and why deploying it is, well, not something that 
people will scramble (or very possibly chose) to do, unless those are 
addresses. And, yes, everyone understands that it's all about making 
trade-offs, but if you make the wrong trade-offs, and not enough people 
deploy the protocol, it's simply not going to fly, and people will just go 
back to de-aggregating in v6 and let Moore's Law deal with the issue (and 
anyone who thinks that people will prevent paying customers from 
deagregating has not seen how many hoops ISP's will jump through for that 
extra revenue, or how fast customers will jump to other ISP's which will 
allow them to do just that). I don't know if more work on shim6 is the 
answer, or GSE/8+8 is a better alterntive, but it sure looks like what we 
have in shim6 today (and it's current direction) isn't going to cut it.

Just my $0.02 

Thanks,
-igor