Re: Extending shim6 to use multiple locator pairs simultaneously

[Scott Leibrand <sleibrand@internap.com>]

On 03/22/06 at 9:18am -0800, Erik Nordmark <erik.nordmark@sun.com> wrote:

> Scott,
>
> This is an interesting idea. I think that the hard parts about this fall
> squarely in the transport protocol,

Yes, probably.

> and that the shim might not need to change at all (You might just need
> to have the transport tell the shim to do forked contexts, and that way
> the shim will automatically verify that the two paths are working.)

If this is the case, great!

I just re-read draft-ietf-shim6-proto-04.txt with this in mind, and I
can't find any reason why this couldn't work, but am not sure it would,
either.

The way I understand forking, the ULP tells the shim, via the
as-yet-undeveloped API, that it wants to fork its context for some reason.
Since the shim operates at layer 3, it doesn't really care why the forking
is occurring, or what traffic goes over each forked context.  It simply
forks the context at the sending end, changes the locators in use if
directed to do so by the API, and then sends each context's packets over
the appropriate locator pair.

At the receiving end, AFAICT the shim is aware of the forking, but when it
de-shims the packets and sends them up to the ULP, it erases all traces of
the forking.  Is that correct?  If so, that satisfies the requirement for
multipath TCP that the receiving TCP/IP stack be shim6 capable, but not be
required to have any knowledge of the multiple paths the packets it
receives are taking.

> I suspect there are other issues than congestion control.
> For instance, you will have some skew between the different paths,
> causing packets to arrive at a different order than they were sent. This
> will cause the receiving TCP to send duplicate ACKs, and after the
> sender has received 3 duplicate ACKs, it will do a fast retransmit.
> Those types of algorithms would need to be make robust against packets
> arriving out of order.

Yes, I had thought of that as well.  For this reason I've decided it's
unwise to attempt any multipath at layer 3, and better to have TCP manage
the multipath using context forking and the shim6 API.  TCP multipath
would probably require SACK, and of course would require modifications to
the multipath sender's fast retransmit algorithm as you describe.

> In terms of thinking about this, it might make sense to start with SCTP,
> since SCTP already supports tracking congestion information for each
> path. Then you can ask yourself what would be needed if both paths were
> used as the same time. Once we know what to do, then applying it to TCP
> would be conceptually simple (even though it might be hard in practice.)
> Note that SCTP only uses one path at a time, and only switches after
> detecting a failure. So AFAIK the SCTP folks didn't explore what would
> happen is both paths are used concurrently.

This part is probably getting OT for this list, but...

I just read the specs on SCTP, and it looks like they only specify one
very limited form of multipath: when congestion occurs and retransmission
is required, "the endpoint should attempt to pick the most divergent
source-destination pair from the original source-destination pair to which
the packet was transmitted," so as to not be held back by (or exacerbate)
the congestion that caused the retransmission in the first place.

According to "SCTP for Beginners" at
http://tdrwww.exp-math.uni-essen.de/inhalt/forschung/sctp_fb/sctp_data.html,
"For each destination address (i.e. each possible path) a discrete set of
flow and congestion control parameters is kept, such that from the point
of view of the network, an SCTP association with a number of paths may
behave similarly as the same number of TCP connections."

So it would seem similar behavior in multipath TCP would be warranted,
along with a SACK prerequisite and additional logic to process duplicate
ACKs and avoid fast retransmit when they're caused by different RTTs and
don't indicate loss.

Thanks,
Scott