[jabley@automagic.org: unscientific multi-homing survey: results]

[Joe Abley <jabley-ietf@automagic.org>]

I sent some questions about multi-homing to the nanog list earlier
this week, and a few people replied. This is a summary.

This is extremely non-rigorous, but might give a tiny bit of
insight into something, maybe.

----- Forwarded message from Joe Abley <jabley@automagic.org> -----

Delivered-To: jabley-nanog@automagic.org
Delivered-To: nanog-outgoing@trapdoor.merit.edu
Delivered-To: nanog@trapdoor.merit.edu
Delivered-To: nanog@merit.edu
Date: Fri, 29 Jun 2001 17:50:09 -0400
From: Joe Abley <jabley@automagic.org>
To: nanog@merit.edu
Subject: unscientific multi-homing survey: results
User-Agent: Mutt/1.2.5i
Precedence: bulk
Errors-To: owner-nanog-outgoing@merit.edu
X-Loop: nanog


Results of my unscientific straw poll for multi-homed operators, as
requested by a few people. Some people gave multiple answers to questions,
and some didn't answer particular questions at all. Figures in brackets
indicate frequency of particular responses.

Like I said, unscientific :)

Thanks to those who sent me mail.

  
Joe
  
On Tue, Jun 26, 2001 at 09:15:09AM -0400, Joe Abley wrote:
> If you currently multi-home between two or more providers using IPv4:
>
>  + why do you do it? What are the high-level goals you are hoping to achieve?
>    Are you finding that you are achieving them?
  
  redundancy (3)
  load-balancing (2)
  "shorter paths", performance, better throughput for users (4)
  geographic redundancy (2)
  provider redundancy (3)
  
  we are achieving these objectives (7)
  
>  + how often does your traffic shift between transit providers due to
>    a failure of some kind triggering a re-homing?

  a couple of times per day (1)
  once per week (1)
  once per month (1)
  a couple of times per month (2)
  we don't notice when it shifts (1)
  rarely (1)
  once per six months per circuit (1)
  
>  + how often do you manually shift traffic, and why? Just inbound traffic?
>    Or outbound too?
  
  never (1)
  normally never (1)
  once per month (1)
  occasionally (1)
  when we get complaints from specific netblock/asns that we can fix (1)
  when we need to due to transient heavy traffic loads (e.g. a promotion) (1)
  rarely (three or four times a year) (1)

  to rebalance costs after some kind of DoS attack (1)
  (never) because we are overprovisioned with capacity and
    traffic shifting isn't necessary (1)
  to optimise use of various peering links (1)
  manually setting best paths for various ASes (where BGP doesn't see
    the best path) (1)
  to avoid some specific poor connectivity on a short path (1)
  
  both inbound and outbound (2)

>  + what impact does a manual or automatic shift in traffic between providers
>    have to your users? Do their TCP sessions break? Or do you think they
>    normally stay alive, maybe after a delay? What makes you think this?
   
  people don't normally notice the transition (1)
  transparent if done right (1)
  stuff breaks a bit, but most TCP sessions remain up (1)
  minimal impact (1)
  for 90% of people the sessions sit for a while and then resume, based
    on feedback in a support forum (1)
  there is a several minute delay as the BGP tables across the internal
    core routers adjust (1)
  VPN users get kicked out when there is a transition, since we
    don't have provider-independent space and the addresses associated
    with the down provider become unreachable (1)
  most traffic is HTTP which is not really affected as long is there
    isn't too much flapping (1)
  assume all drop and take any that don't as a bonus (1)
  no detectable effect (1)

  we have never had a ticket opened which resolved to TCP session
    breakage due to a re-homing transition (1)
  
>  + what is *bad* about the way that multi-homing works in IPv4?
  
  need more control of incoming traffic (1)
  hard to limit incoming via upstream, although outgoing preference is
    easy (1)
  nothing that needs fixing at layer 3 (1)
  withdrawing and advertersing routes socks the network too much (1)
  synchronisation takes too long, and sucks up too much router cpu (1)
  nothing (1)
  bgp is unintelligent, and requires too much manual tweaking to
    find the "best" route (1)
  finding providers which will let you multi-home is hard (1)
  needing provider-independent address space (1)
  it can get ugly (1)
  router vendors need to make more boxes with more memory (1)
  i would like to have confidence in using RRs more (1)
  having to trust and cooperate with your peers (1)
  bgp convergence sucks (1)
  
>  + what is good about it?
    
  it works (4)
  it can be done using lots of devices "besides just cisco" (1)
  it's pretty obvious how it works, and even level-1 ticket takers
    at backbone providers understand it most of the time (1)
  it allows us to increase performance, and we like redundancy (1)
  "The redundancy!   The multiple paths!  The ability to choose." (1)
  reliability and shorter paths (1)


----- End forwarded message -----