Re: RSVP Restart (was Re: update GMPLS signaling documents)

[Eric Gray <eric.gray@sandburst.com>]

Yangguang/Yakov,

    This seems to be getting all blown out of proportion, however it
is probably at least partly my fault - for taking earlier discussions
off of the list.  Therefore, I will try to address these comments on
the list - though I will almost certainly regret the attempt.  :-)

    Back on October 1st, Lou Berger announced availability of new
draft version of the signaling and RSVP-TE documents he had been
working on.  On October 3rd, Yangguang repeated some questions
that had been asked previously on the list and Fong Liaw answered
these questions.  This much can be seen at the archival site -

        ftp://ops.ietf.org/pub/lists/ccamp.current

(look for the subject line - "update GMPLS signaling documents").
This site is (unfortunately) not an HTML friendly site.

    I felt that Fong had answered these questions completely, but I
tried to add a little to this in an off-line reply (I copied Fong as a
courtesy).  Most of my replies to the ccamp mailing list are off line,
because usually replies I send to this list bounce (feel free to ask
the list administrator why that happens).  I did not realize at the
time that (at least one of the) questions that Yangguang posted were
actually asked by other people (the posting didn't include this
information), or I would have included Gopal, at least.

    From this reply ensued an off-list discussion that lasted slightly
less than a day. I will try to summarize this here (named participants,
jump in if you think this summary is fundamentally inaccurate).

========================================================================

Yangguang (on list): There is an ordering issue during resynchronization
since the LSPs would depend on other FA-LSPs the interface IDs for which
would have been generated dynamically. So the mechanism described will
not work, unless this information is preserved across restarts - in
which case we might as well preserve other information as well, and
avoid resynchronization.

Fong (on list):  We only need to consider the FA-LSP ingress and egress
since the transit nodes do not see signaling for the hierarchical LSP.
In the egress reset case, the egress can resolve the ordering based on
the fact that it receives a 'recovery' label instead of a suggested label.
In the ingress case, the ingress LSR will know the required ordering.
You need to resynchronize no matter what because of the possibility of
in-progress connection setup.

Me (off-list from here on): Ordering problems should not arise because
Path messages should not be sent until underlying LSPs are established
and interface IDs are known.  Resynchronization is both simpler and more
likely to work than trying to preserve sufficient state to be able to
avoid it. In any case, no implementation should be allowed to assume
indefinitely that it has correctly preserved state.

Yangguang: For FA-LSPs, are you suggesting the re-start happens between
the ingress/egress nodes, instead of its physically adjacent neighbors?

Me:  'Ingress', 'egress' and 'adjacent neighbors' are misleading when
used with hierarchical LSPs without at least further qualification.
Adjacent neighbors (WRT one LSP) have no state relative to 'higher-level'
LSPs (these would be seen as lower-level labels by these LSRs).  Adjacent
neighbors in a higher-level LSP would be the ingress and egress LSRs for
lower-level LSPs on which they are based.

Fong: No, since the data plane of the FA may still function.  The egress
node of the FA may receive RSVP packets for the LSP that uses the FA. If
the data plane does not function after the restart, we don't even have
the dependency problem at all.

Me:  Yes, higher level LSPs labels can be exchanged before the lower
level data plane is established - but cannot be used until the link has
been re-established.  Moreover, if the signaling for the higher level LSP
depends on knowing in advance the way that the link is identified, then
signaling will be delayed until the information is available even if the
control plane remains active.

Yangguang:  My question and example all assume the data path is working
normally and only the control entity fails.  In the example below:


----A------B..B1....C1..C-----D------E-----

B and C are FAs, if B's control entity fails and reboots, it has to
re-sync with C for LSPs using the B-C tunnel (BC FA-LSP).  B must re-sync
with both its immediate neighbor B1 and FA C.  Then B's re-sync process
does have order issue. It should make sure all FA-LSPs are still working
and then re-sync with FAs (if they have a way to keep these information
and mapping after reboot) for LSPs using the FA-LSPs.

Me:  Using your example, suppose that the link between A and B goes down,
imagine that the labels used by A and B for LSPs across this link are
exchanged out of band, and suppose that a means exists by which both A
and B expect this link to be restored.  Under these circumstances, do not
the exact same issues exist as in your example (between B and C)?  Yet
everyone must know that any labels negotiated between A and B will not be
useful until the link is restored.  Also, everyone must realize that - if
the link IDs are lost as a result of the loss of the link between A & B -
the signaling for LSP restoration MUST wait for the link IDs to be
re-established.

 Me (continued): The key difference between a link failure between A and
B and B's having a control plane restart (as in your example) is that (in
your example) you assume the same technique is used to restore all LSPs
involving B.  It is not valid to assume - in a reasonable implementation
- that all LSPS involving B will be re-established simultaneously.  The
ordering constraint is a naturally occurring one given that - in any
recovery - links must first be re-established, then adjacencies across
those links and then any higher level activity.  The fact that - without
of band signaling - it is possible to signal labels before the data plane
is ready to be used in no way implies that it it makes sense to ignore
the natural ordering of hierarchical link re-establishment.

========================================================================

I think it is not clear that anything further needs to be specified in
the draft with respect to the issue of ordering of LSP re-establishment.

Also, with respect to the backward compatibility issue below: it is not
impossible for an implementation to be configurable to support either
GMPLS based signaling or some existing scheme on a per-interface basis,
if that is what you want to do.  To my knowledge, there is not so much
in the way of current 'lateral' compatibility to be worried that much
about backward compatibility.  Who in particular should we aim to be
backward compatible with, given that few existing solutions are truly
compatible with each other?    :-)

Yangguang wrote:

> Yakov,
>
> Thanks for resenting me the message from Eric.
>
> Indeed, instead of looking for answers from you, my intention was just trying to
> point out some holes in the current proposal and your previous response. And
> thanks for just throwing them back to me.
>
> Again, not trying to add more smoke, just repeat my main point again: there are
> existing solid solutions that can satisfy carriers' requirement of preserving
> data path and re-create data path state after rebooting. I think many of us are
> aware of that. If you are proposing a new mechanism, no matter good or bad, at
> least you could consider some "backward" compatibility issues. Meanwhile, even I
> have confidence that you can make it work well eventually, there are some corner
> cases I'd like to point out.
>
> And also a piece of suggestion, before claiming a solution for a problem. It's
> better to understand the problem well first. There is nothing can replace
> working experience of either operating, designing and developing optical
> transport network and NEs.
>
> Yangguang
>
> Yakov Rekhter wrote:
> >
> > Yangguang,
> >
> > > Indeed, you didn't answer my questions.
> >
> > Perhaps it would be more correct to say that I didn't answer
> > your questions *to your satisfaction*. And if that is the case, then
> > I hope that the attached e-mail from Eric Gray (that he let me repost
> > to the list) would answer your questions to your satisfaction.
> >
> > > Instead of informed readers, more
> > > important is to have better informed writers.
> >
> > See the attached.
> >
> > > BTW, I have no intention to let you design my product at all.
> >
> > Glad to hear this !!!
> >
> > Yakov.
> > ---------------------------------------------------------------------
> > Date:    Fri, 19 Oct 2001 15:38:58 EDT
> > To:      Yangguang Xu <xuyg@lucent.com>
> > cc:      Yakov Rekhter <yakov@juniper.net>
> > From:    Eric Gray <eric.gray@sandburst.com>
> > Subject: Re: RSVP Restart (was Re: update GMPLS signaling documents)
> >
> > Yangguang,
> >
> >     Careful, don't let this discussion descend into the nether regions of
> > name calling and finger pointing. :-)
> >
> >     The fact that the seeming remote neighbors use RSVP Hello is clear
> > once you realize that - for the purposes of hierarchical LSP setup, they
> > are adjacent.  You may recall this from an off-line discussion we had.
> >
> >     Yakov's comment was simply stating that it is not a part of standard's
> > development to make an implementer's cook-book.  He was not (I think)
> > trying to imply that you could not write your own implementation.  He was
> > just saying that the effort of achieving interoperability falls on the people
> > doing the implementation not on the people doing the standard.  The bare
> > information is there, but the work is up to you.
> >
> >     As you know, the standard is not complete until several implementations
> > have achieved interoperability independently.  But the opportunity to get in
> > the market stems from being one of those implementations - and there is
> > zero motivation for any implementer to make this task easy for all others.
> > One of the costs associated with the 'running code' part of IETF mistique
> > is that standards are written by implementers.
> >
> >     Unless and until that changes, you just have to learn to deal with it.  :-)
> >
> > You wrote:
> >
> > > Indeed, you didn't answer my questions. Instead of informed readers, more
> > > important is to have better informed writers.
> > >
> > > BTW, I have no intention to let you design my product at all.
> > >
> > > Yangguang
> > >
> > > Yakov Rekhter wrote:
> > > >
> > > > Yangguang,
> > > >
> > > > > Yakov,
> > > > >
> > > > > Further questions in-line. Regards,
> > > >
> > > > Further responses in-line...
> > > >
> > > > > Yangguang
> > > > >
> > > > > > Couple of points:
> > > > > >
> > > > > > (1) First of all, you do run RSVP Hello between C and F.
> > > > >
> > > > > Is this specified somewhere? Or it's your proposal.
> > > >
> > > > What I mentioned in the above should be abundantly obvious to the
> > > > informed reader of RSVP and LSP Hierarchy specs.
> > > >
> > > > > Then, is there a scalability issue here?
> > > >
> > > > No.
> > > >
> > > > > Also, C and F could be thousands mile away (in the transport
> > > > > network) and the control channel bandwidth is expensive.
> > > >
> > > > GMPLS assumes that control channel is more than just 56Kbits/sec link.
> > > >
> > > > Or to put it differently, control channels used by GMPLS have more than
> > > > enough capacity for RSVP Hellos.
> > > >
> > > > > > (2) Since L1 is advertised as an FA into OSPF/ISIS, F should
> > > > > > be able to recover the Interface ID it assigns to L1 from
> > > > > > a combination of (a) the OSPF/ISIS link state database that
> > > > > > F would recover, and (b) the Forward Interface ID (the one
> > > > > > assigned by C).
> > > > >
> > > > > Can OSPF/ISIS happen to be on the same controller as RSVP-TE? I guess the
> > n it
> > > > > has to recover its FA info first. How?
> > > >
> > > > The answer should be abundantly obvious to the informed reader.
> > > >
> > > > And to make it crystal clear, I have no intention in designing your
> > > > products for you.
> > > >
> > > > > > > Let us say the node can resynchronize its neighbor if
> > > > > > > the neighbor restarts and requests state recovery. But
> > > > > > > the issue is how a node can advertise that it does not
> > > > > > > need recovery since all its state was preserved?
> > > > > >
> > > > > > By treating is the same way as the way the spec handles
> > > > > > control channel fault.
> > > > > >
> > > > >
> > > > > If a node can recover states from NVRAM or OS (this solution is
> > > > > cheaper and have been implemented). It may not implement the spec.
> > > > > How could a neighboring node, which implements the spec, not tear
> > > > > down data connections mistakenly? Similar to a backward compatibility
> > > > > problem, even not.
> > > >
> > > > Stating the obvious: interoperability with a node that doesn't implement
> > > > the spec is outside the scope of the spec.
> > > >
> > > > > > > RECOVERY LABEL does not come into picture unless the node
> > > > > > > that is upstream to the restarting node has already received
> > > > > > > a Resv.
> > > > > >
> > > > > > Wrong. Quoting 9.5.3:
> > > > > >
> > > > > >    Upon detecting a restart with a neighbor that supports state
> > > > > >    recovery, a node SHOULD refresh all Path state shared with that
> > > > > >    neighbor.
> > > > > >
> > > > > > So, as you can hopefully see from the above, the upstream node doesn't
> > > > > > wait until it receives a Resv.
> > > > > >
> > > > >
> > > > > Then this may not meet carriers' requirement. (see IPO carriers' requirem
> > ent)
> > > > > Again, back to the telecom world (sorry about this, yet we are talking ab
> > out
> > > > > GMPLS), typically only the established pathes are preserved. Pending requ
> > ests
> > > > > may be purged/aborted. Then how?
> > > >
> > > > Note the verb is "SHOULD", not "MUST". The rest should be
> > > > obvious to the informed reader.
> > > >
> > > > > > > In case of PSC devices, it may be OK to remove state that
> > > > > > > is not resynchronized at the end of the recovery period,
> > > > > > > and the recovery period advertised might reflect that.
> > > > > > > But for LSPs in transport networks, one might want to
> > > > > > > have a different recovery period to avoid any LSP from
> > > > > > > going down because of recovery timer expiry.
> > > > > >
> > > > > > There is no requirement for a node to advertise exactly the
> > > > > > same Restart_Cap on all the interfaces. So, on PSC interfaces
> > > > > > the node could advertise that it will remove the state that
> > > > > > isn't syncronized at the end of the recovery period, while
> > > > > > on the TDM interface precisely the same node could advertise
> > > > > > that the LSPs would be kept even after the recovery time expires.
> > > > > >
> > > > >
> > > > > So, restart_cap is per interface based?
> > > >
> > > > yes.
> > > >
> > > > > Is the spec going to be enhanced/clarified?
> > > >
> > > > The current spec need not be enhanced/clarified - what I mentioned in the
> > > > above should be abundantly obvious to the informed reader.
> > > >
> > > > > > > The first problems seems to be there still - consider two
> > > > > > > adjacent nodes restarting.  They both act both as the restarting
> > > > > > > node as well as the neighbor to the restarting node. So, once
> > > > > > > they learn the state from the upstream neighbor, do they use
> > > > > > > suggested label or the recovery label when they send the path
> > > > > > > message to the just restarted downstream neighbor?
> > > > > >
> > > > > > The recovery label.
> > > > > >
> > > > > > The following should be added to the existing text from the document:
> > > > > >
> > > > > >    In the special case where a restarting node also has a restating
> > > > > >    downstream neighbor, a Recovery_Label object should be used instead
> > > > > >    of a Suggested_Label object.
> > > > >
> > > > > How does a recovering NE know that its neighbor is also recovering? it ma
> > y
> > > > > lose the instance number totally
> > > >
> > > > It's advertised in the neighbor's hellos.
> > > >
> > > > Yakov.
> >
> > --
> > Eric Gray (mailto:eric.gray@sandburst.com)
> > http://www.mindspring.com/~ewgray

--
Eric Gray (mailto:eric.gray@sandburst.com)
http://www.mindspring.com/~ewgray