[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: failure detection



Hi Erik,

There's one other question to consider, which I've not really had a good answer to (I think I asked Marcelo already ;) ) AFAIR.

How is this going to interact with local site routing policy? The ability to express policy in routing typically depends on routing being able to, by and large, affect source-selection. Ie that applications (or things above the output-IP layer) do not unneccesarily go decide what source to use.

To give an example, here's another case of a PA-multihomed site which I had (the misfortune[1]) to setup:

A site bought an expensive, and slow, fixed circuit, in order to accept inbound SMTP primarily, they also procured a faster DSL line[2], but not suitable for hosting inbound services on. The two links had different costs, one was very expensive compared to the other, in several ways:

- the fixed circuit had much lower bitrate
- the fixed circuit, amazingly, was metered by MB (DSL wasn't)

I had to fix their site when they discovered things didn't "just work" (ingress filtering). They wanted all possible traffic to go over their cheap/fast DSL obviously.

Solution was fairly obvious:

- source route for the fixed-address range via the slow fixed circuit
- default via the DSL[3]

This worked because:

Other than responses to SMTP connections, all outbound services (SMTP outbound, Web proxy, etc.) used unspecified addresses.

Fairly obvious so far :), bear with me.

My question is, how would source-remote n^2 probing shim ever work nicely with such multi-cost setups? If shim just happens on the source that belongs to the expensive link, it literally could cost money, or high-cost for some other definition of cost.

IIRC Marcelo's answer was that shim6 should simply take routing into consideration when picking the source(s) to try send probes with.

But that seems very much (to me) either like reinventing the whole SAS process which already is implemented elsewhere in the kernel. Alternatively, it will require lots of extra layering violations in shim6, with shim asking "what sources do you think I could use?" - at which point, you could surely just leave the source unspecified and let the mechanisms which exist anyway do the work you're going to replicate.

Ie, with shim doing probing using different sources:

- Just picking all sources is going to make shim6 really annoying for
  administrators who have inequal-cost site links.

- Manually configuring shim to prefer certain sources would be
  tedious and simply replicating what the kernel already has in its
  routing table (possibly installed dynamically without manual
  intervention)

- Specifying mechanisms for shim to interact with kernel to obtain
  the list of addresses which fit with routing policy and shim's
  desire seems to me to:

   - be a question that can't be answered precisely other than
     "these are the sources I would use for the /best/ route(s)"

   (shim wants to try more than just that if doing n^2 probing, but
    which ones? If just the best, well that can be achieved simply
    by /not/ trying to pick different sources and just not
    specifying an address and hence letting kernel do what it would
    do anyway. If "the best, plus a few more" - which few more, the
    kernel can't tell you from its routing table, do you want the
    lowest cost and the next highest? but what if that's the
    expensive one? there's no way to know.)

  - will require more hooks and 'APIs' between different layers of
    the networking stack. IMHO, that kind of thing can leads to shim
    being hard to, and taking longer to:

    - specify
    - implement

    (particularly when there's talk of other APIs between shim and
     ULP's. I'd strongly suggest following Spencer Dawkin's advice
     and not requiring such, or even investigating this
     much initially.)

--paulj

1. For reasons we won't go into.

2. The site during procurement were told by their provider that fixed addresses were only available with "business" class fixed-circuits, and not possible with DSL. So they got a quite expensive 128k link just for SMTP. sigh.

3. Failover wasn't needed, because they had two VPN links back to head office, one using the DSL link, the other the fixed circuit, with the former VPN tunnel as the preferred route - keepalives were provided either in the tunnel, or dynamic routing protocol ran over it - dont remember. If DSL went down they still had their email delivered to them, but to the 'expensive' MX for them via the VPN.

regards,
--
Paul Jakma	paul@clubi.ie	paul@jakma.org	Key ID: 64A2FF6A
Fortune:
Hmmm, look at those eyes.  He's trying to hypnotize me, but not in the
good Las Vegas way.

		-- Homer Simpson
		   Mountain of Madness