Re: [RRG] Are host-stack modifications allowed or disallowed ?

[Robin Whittle <rw@firstpr.com.au>]

Hi Brian,

You wrote:

> I'm thinking that there may be a class of solutions which will 
> work as far as aggregation and scaling are concerned without any
> host modifications, but would allow host modifications to have
> substantial impact on traffic engineering for the benefit of
> sites hosting major server farms. This type of site cares about
> traffic engineering to improve its own performance and 
> reliability, quite independently of ISP concerns about traffic 
> engineering. I could imagine a server farm dynamically adjusting 
> its loc-id mapping as the work load shifts around between 
> phsyical servers, or as it observes performance changes from one
> ISP or another. I would expect this would need tight interaction
> between host virtualisation software and the routing system.

OK - I will try to imagine a scenario such as this.

A site has a bunch of servers, either its own or rented out to
customers.  It has multiple links to ISPs and wants to ensure none
of these links gets congested and/or do some other kind of inbound
traffic engineering.

At some times a particular server or bunch of servers X is very
active, drawing in traffic from some distribution of correspondent
hosts HX.  At other times, another server or bunch of servers Y does
the same, with its distribution of hosts HY.

I think you are suggesting some host changes to the servers only,
presumably just their operating systems rather than the server
applications themselves, which would enable the site administrators
to achieve their TE goals, without additional changes in routers
etc.  Also, I am assuming you are not placing further burdens on the
routers in the DFZ with more prefixes or more changes.

I can't imagine what those changes would be, but clearly if you
could upgrade only the server operating systems, leaving everything
alone, and achieve these goals with IPv4, then it would be a
valuable thing to develop.


>> 2 - To be incrementally deployable, the benefits would need to
>>     ensue pretty directly and immediately to the people who run
>>     these servers and/or the networks they are in.  Or do you mean
>>     some external incentive scheme to encourage server operators to
>>     change their operating system (and perhaps application?)
>>     software?
>>
>> 3 - Also, I think you would need to show that this subset of
>>     address usage being used in the new way would make a big
>>     enough impact on the routing scaling problem for it to be worth
>>     the trouble - ideally sufficient to solve the problem
>>     indefinitely without having to take other measures.
> 
> No, only that it would make a big enough impact on the major
> hosting sites that they would pay for it.

Yes - if it is only the hosting companies who have to change
something, in a way that doesn't complicate the operations of their
customer's applications - and the hosting company receives direct
benefits which enable or cause it to behave in ways which reduce the
burden on DFZ routers, then any progress at all is worthwhile.


>> 4 - I also think you would need to get consensus that the
>>     dichotomy you are implying about networks with public
>>     server hosts and networks without is valid and will
>>     remain valid for some decades to come.
> 
> My personal feeling is that this will persist and even
> increase, but of course the technology available and the
> business models in use have a circular relationship, so
> prediction is quite uncertain.

My point 4 was really only if the "big, supposedly total" solution
to the routing scalability problem relied on distinctions between
different kinds of end-users, such as large and small, server or
client etc.  I think what you are suggesting is some host OS changes
applicable to servers which could be deployed by the hosting company
to acheive TE without burdening anyone else.  I don't see that as a
total solution, but something which would be helpful and should be
pursued.  To the extent that it only works with a subset of end-user
networks, that is fine, because it is not being put in place as a
"total solution".


> However, the fact that sites use DNS and BGP4 to attract
> traffic to particular prefixes today strongly suggests that
> they will use any deployed loc-id mapping system the same
> way.

I think the "total solution" needs to be about as attractive to
large end-user networks, including server farms, as to small
end-users, new entrants etc.  I feel sure that some kind of
map-encap system will be introduced, but that it will take quite a
few years.  A set of host OS changes such as I think you are
suggesting could work independently of the map-encap scheme, maybe
provide additional benefits with a map-encap scheme compared to the
map-encap scheme on its own or maybe do the same things, without the
need for a map-encap scheme.  I am pretty sure no such set of host
changes could do all the things a good map-encap scheme could do.


>> Do you have any possible approaches which would pass all of these tests?
> 
> No, except to suggest that hosts MAY particpate in updating
> the loc-id map.

Yes.  The control of mapping information for the one or more
micronets in each User Address Block in Ivip is controlled by the
end-user via a username and password.  Maybe there needs to be a way
that the mapping for a specific micronet can be controlled by a
separate username password pair.  That way, a hosting company A can
rent out a server to customer C and give them one or more micronets
of mapped address space.  They can also give customer C the username
and password for those micronets, so the customer's software and/or
software on the servers which is controlled by company A can
automatically change the mapping, to achieve various things.  Prime
amongst these would be the server monitoring its own incoming
traffic and making changes to the mapping of its micronets to
achieve some kind of load balancing amongst various upstream ISPs.

Since Ivip ITRs don't perform explicit load sharing, one way of
doing this would be to have the server known through the DNS to
clients by four separate, and perhaps contiguous IP addresses.  Eahc
one is its own micronet.  Then, to fit in with the traffic flows of
all other servers at the site, the host can control the mapping of
the micronets to make all of them mapped to an ETR which brings
traffic in through ISP-M, or to map some of the micronets to another
ETR which brings the traffic in through ISP-N.

Maybe the host itself isn't the best place to do this.  Maybe
company A could use some software on the host to monitor its
activity, and report this to some centralised mapping software,
which attempts to balance the load of all servers in real-time by
sending mapping updates, steering traffic through various upstream
links to keep things nicely balanced.

I am not sure if this is what you were suggesting, but it is a good
example of where Ivip's rapid response (I am aiming for 5 seconds or
less) would enable fine, realtime, control over incoming traffic
patterns over multiple upstream links, in ways which could not be
achieved with LISP, APT or TRRP, since they only have slow ways of
controlling the behaviour of ITRs, and because the ITRs themselves,
working in isolation, can't make decisions the way company A wants.
 Only company A knows what traffic is on its upstream links at any
point in time, so it needs fast, specific, control over the mapping
of micronets to be able to steer incoming traffic as it desires.

Company A needs to pay a small fee for each mapping change, so it
would only fire them out at a rate where the benefit it derives
exceeds the cost of each change - and that cost will be set by the
operators of the Ivip system (multiple Root Update Authorisation
Systems) to support the real cost of sending those updatesa and
replicating them out across the Net.


>> It is not clear to me how host changes could make these initial
>> packet delays much less of a problem, other than to tweak existing
>> protocols so they didn't make such a mess of themselves when such
>> delays occurred.  
> 
> Yes, teaching transport protocols to tolerate a larger initial
> delay seems possible if it had to be done.
> 
>> The delays will still add up and the end-users
>> will still be affected - making this form of address space suck, or
>> at least be widely regarded as being second-rate and best avoided.
> 
> At the risk of repeating an argument already made, I don't think that
> users would even notice a few hundred ms in addition to currently
> experienced start-up delays, as long as the transport behaves
> gracefully.

A few hundred msec is probably not clearly perceptible to most
people, but if it adds up, with one slowed communication further
delaying the start of others which are also slowed, it would become
more noticable.

Also, delays in sending initial packets in P2P streams could affect
the overall behavior of the P2P application in noticable ways.  For
instance a real-time video streaming application which frequently
depended on one or more packets arriving from hosts it had never had
any packets from so far.

Even if, in general, the slowdown was not clearly noticable to
end-users, and even if the marketing folks didn't use these delays
to turn end-users away from the map-encap space, I still think it is
worth great efforts, including perhaps heroic efforts, to reduce or
eliminate any delays we add to Internet communications in general.
The delay would be repeated trillions of times in the future,
wasting collectively many human lifetimes, just a little at a time
for each individual.

  - Robin


--
to unsubscribe send a message to rrg-request@psg.com with the
word 'unsubscribe' in a single line as the message text body.
archive: <http://psg.com/lists/rrg/> & ftp://psg.com/pub/lists/rrg