Fwd: Distribution CPG Protocol - Some Thoughts

[Stephen Thomas <stephen.thomas@transnexus.com>]

This was sent originally to the wrong list (my fault, please accept 
apologies). I'll also forward the follow-up that continued on that list. 
(No one has objected to the suggestion that cdn@ops.ietf.org is the right 
place for the thread, so I'm taking a bit of liberty in forwarding everyone 
else's messages as well. The benefit is we can get the discussion moved 
there quickly, and we can maintain the same temporal ordering of the 
discussion. Apologies in advance if I've erred.)

Stephen


----Begin Forwarded Text


On the assumption that the WG goes forward, here are some initial thoughts 
on protocols. Some (most?) of this is possibly obvious, or perhaps some 
(most?) is brain-damaged. I'm interested to hear either way.


Distribution CPG Protocol. This has been likened to BGP several times, so 
it seems like a good place to start is looking at what BGP offers (and what 
it doesn't) that appear to be relevant to CDNs.

First, BGP is an advertising protocol. BGP peers advertise autonomous 
system paths that reach CIDR IP subnets. In our case (again, thinking only 
of distribution), two options are available. Distribution CPGs could 
advertise surrogates, or they could advertise content. If DCPGs advertise 
surrogates, it would be up to the recipient CPG to arrange to have content 
pushed to the surrogates. Alternatively, if DCPGs advertised content, then 
it would be up to the recipient to arrange to have its surrogates pull that 
content. I suppose there's no technical reason to limit the protocol to one 
of these options, but, in the interest of schedule and focus, at least 
picking one to start with seems preferable. My own instinct is that 
advertising content works better. There's sort of a one-to-many 
relationship (one content to many surrogates) that makes it more natural. 
(In the reverse, you have to worry about different content providers 
contending for the same advertised surrogate space.)

Proposal 1: The protocol should advertise the availability of content.


Second BGP provides aggregation. Different IP subnets may be combined in 
the advertisements if they share sufficient address bits in common. I can't 
see how this works with CDNs. Sure, a provider could advertise both 
www.foo.bar/images and www.foo.bar/scripts but how would you aggregate 
them? With BGP, a peer that receives an advertisement for 172.16.1.x knows 
what destinations it applies to; there are a finite number (254 or so, 
depending on how you count broadcasts). A DCPG that received an 
advertisement for www.foo.bar, however, wouldn't know about /images or 
/scripts. Similarly, how would a CPG that learned of /scripts and /images 
know to aggregate? After all, there could be a www.foo.bar/mp3 lurking around.

Proposal 2: The protocol need not support aggregation.


Third, BGP assigns paths to its advertised objects. When a server learns a 
new destination from a peer it adds its own AS to the path on subsequent 
re-advertisements. In the simplest case, I don't think paths are relevant 
to DCPGs. Say CDN-1 advertises content to CDN-2, who, in turn, 
re-advertises it to CDN-3. Now, if CDN-3's surrogates want to retrieve that 
content, is there any need to go through CDN-2? Simplest case says "No"; 
we're on the Internet after all. CDN-3's surrogates can go right to the 
content's source. I can think of two complications that might affect this, 
though. First is security. Suppose CDN-1 only trusts CDN-2; maybe it's 
never even heard of CDN-3. In that case, perhaps it wouldn't want CDN-3's 
surrogates grabbing its content. The second complication might be 
accounting/billing. Perhaps CDN-2 expects to get paid for relaying the 
advertisements from 1 to 3, and maybe it needs to know when that content is 
actually retrieved in order account/bill appropriately. In the interest of 
simplicity, I'd propose to reject both of those arguments. For the case of 
security, CDN-1 can construct its advertisements in such a way that when a 
CDN-3 surrogate retrieves the content, the retrieval request identifies the 
advertisement. (For example, CDN-1 could embed the identity of CDN-2 in the 
URL.) The billing argument might be more persuasive, but I'm loathe to add 
complexity to any protocol solely to accommodate billing. And finally, it's 
quite possible that CDN-2 could implicitly achieve the same effect as a 
path by advertising the content as its own and then proxying to the real 
content.

Proposal 3: The protocol need not explicitly support the notion of paths.


Fourth, BGP accommodates policy. I wouldn't go so far as to say that BGP 
supports policies, as (to me) that seems more of a quality of BGP 
implementations rather than the protocol itself. I do think, however, that 
this same sort of accommodation is critical for CDNs. To as great an extent 
as possible, though, I'd suggest leaving that to implementations rather 
than protocols. For the protocol, therefore, it is only necessary to 
provide sufficient information to allow implementations to carry out policy 
decisions.

Proposal 4: The advertised objects must carry sufficient identifying 
information to allow for implementations to make policy decisions; however, 
there is no need for the protocol itself to carry policy descriptions.


Fifth, BGP works best for relatively stable information. There have been a 
fair number of protocol tweaks and configuration folklore to minimize 
route-flapping, but (as is common with distance-vector and path-vector 
protocols) the protocol behaves better when the information is relatively 
stable. Is this the case for CDNs? (I honestly don't know.) How frequently 
is advertised content expected to change? I don't have a specific proposal 
here, and I'm not sure there's much to learn from BGP. (Route convergence 
is only an issue when you have routes, e.g. paths, and proposal 3, at 
least, has rejected them.)



So there's an initial shot at both requirements and implications for 
protocol design. Comments, of course, appreciated.

Stephen



____________________________________________________________________
Stephen Thomas                                       +1 770 671 1888
TransNexus, Chief Technical Officer    stephen.thomas@transnexus.com