RE: unified transaction model proposal

[Andy Bierman <abierman@cisco.com>]

At 07:23 AM 5/3/2004, Harrington, David wrote:
>Hi Andy,
>
>I suspect this is an area you'll not want to get into, but I'm concerned
>with cross-protocol coordination. If NETCONF is going to expect global
>locks to constrain other mgmt interfaces, then I think the impact on
>other interfaces needs to be considered.

I don't mind discussing these issues at all.
There is a fuzzy line between implementation details
and standards details when it comes to locking.
The feature is transaction integrity.  Global database
locking is just a way to implement transaction integrity,
not a feature in itself.  Global locking creates performance
and security problems, so it's not nearly the best way
to implement transaction integrity.


>Comments inline.

ditto

> 
>
>> -----Original Message-----
>> From: owner-netconf@ops.ietf.org 
>> [mailto:owner-netconf@ops.ietf.org] On Behalf Of Andy Bierman
>> Sent: Friday, April 30, 2004 2:04 PM
>> To: j.schoenwaelder@iu-bremen.de
>> Cc: netconf@ops.ietf.org
>> Subject: Re: unified transaction model proposal
>> 
>> At 06:14 AM 4/28/2004, Juergen Schoenwaelder wrote:
>> >On Tue, Apr 27, 2004 at 11:16:11AM -0700, Andy Bierman wrote:
>> >>
>> >> >Just for clarification: I assume correctly that commit/rollback
>> >> >only applies to changes created via <edit-config> and not to any
>> >> >other changes (e.g. <copy-config> or <delete-config) that may be
>> >> >made since the lock was established.
>> >> 
>> >> Good question.  I don't think this is the case though.
>> >> We have global targets (<candidate> and <running>).
>> >> If locking is not used, then the agent cannot
>> >> prevent additional sources from making changes.
>> >> Also, a <discard-changes> applies to the whole database,
>> >> not just the changes from the <edit-config> operations
>> >> from the current session.
>> >
>> >So I have to be able to rollback regardless what set of changes have 
>> >happened. In other words, I have to take a complete snapshot, right?
>> 
>> I'm not sure if that is an implementation detail or a
>> protocol requirement. 
>> 
>> Our notion of a transaction doesn't align very well with
>> a traditional database transaction model.  We have optional-to-use
>> explicit global config locking instead of mandatory-to-use implicit
>> partial config locking.  The optional-to-use aspect creates
>> lots of problems like the one you're describing here.
>
>I agree.
>
>I think it is a bad design to have optional locks. Making locks optional
>will greatly increase the conditional-complexity of the
>applications/scripts needed to use NETCONF, which is not in line with a
>number of operators' requirements.

Some operators and EMS developers have problems with the 
global locking because they want to be able to have multiple 
provisioning transactions in progress at once, which affect 
different device resources (and/or instances).


>We're designing a standard, and we should be able to specify what is
>required to be conformant, and I believe the locks should be mandatory,
>not optional. It will greatly simplify the design process for NETCONF,
>provide better interoperability, and improve predictability in the
>network management process. 
>
>Implementors will complain that locks are hard, and therefore should be
>optional. I agree that locks are hard, but they are hard because
>multi-user and cross-interface coordination is hard, and locks help to
>improve the coordination. They should be mandatory because managing
>effectively without them is really hard; having the locks makes it
>easier.
>
>> 
>> How about forcing the duration of a lock to be a single transaction:
>
>In a cross-protocol setting, how will you define "a single transaction"?
>
>If SNMP sets the global lock, what would a single transaction be? One
>PDU, or one transaction carried in multiple PDUs, such as a
>CreateAndWait transaction?
>Could I SNMP-lock a system and then send multiple messages with
>configuration changes, all part of a single transaction, before
>releasing the lock?

You have to create a common transaction model and map
it to NETCONF and SNMP.  The client signals the start
and end of a transaction.  This WG isn't concerned how
the NETCONF transaction model maps to SNMP.  That can
happen later, if needed.


>If the CLI sets the global lock, what is a single transaction? Some
>systems commit changes at the end of every command, while others use
>scratchpads and commit them all at once using a copy-config type of
>command. Do you lock a complete copy-config transaction to apply a
>candidate config, but lock/unlock each command applied to a running
>config?
>
>If a web page is used to configure a device, say with multiple visual
>"folders" to configure different aspects of the device, what is a single
>transaction?

As I pointed out, the client signals the start and end
of a transaction.  The server can't possibly know.


>Which NETCONF and non-NETCONF transactions cause an update of the
>commit/rollback snapshot? If locking/unlocking forces an update, then
>command-by-command lock/unlock could be expensive to keeping
>commit/rollback snapshots updated, especially for protocols without an
>explicit transaction model like SNMP.

IMO, this is an implementation detail.
Our transaction model is somewhat broken because we have
global targets and optional locking.  Here's what application
developers want from the transaction model:
 - a simple transaction model in which the scope is just 
   the config touched during the 'prepare' phase of the transaction. 
 - the application does not obtain an explicit global lock
 - the agent keeps track of the internal locking, with
   a lock-as-you-go model (return error if lock conflict
   occurs at data object/instance level)
 - locking is an implementation detail. The actual features
   provided by the agent are apply-changes, discard-changes,
   persist-changes, and rollback-changes.  (See below).
   The agent needs to insure these functions can be provided somehow.
 - 2 phase commit for multi-device transactions:

    start-edit
    edit-config +
         |
         +----------------------------+
         V                            V
    apply-changes              discard-changes
         |                     end-edit
         |
         +----------------------------+
         V                            V
    persist-changes            rollback-changes
    end-edit                   end-edit

   I realize that 'apply-changes' is a no-op if target == <running>
   and 'persist-changes' is a no-op if #distinct-startup is false,
   but this general transaction model can be applied to every
   NETCONF device.


>> 2) confirmed commit
>> 
>>    <start-edit>
>>       <with-lock>true</with-lock>
>>       <target><candidate/></target>
>>    </start-edit>
>>    <edit-config> +
>>    <commit>
>>       <confirmed-commit/>
>>    </commit>    // lock NOT released after commit
>>    ...          // lock released after N minutes
>>    <commit>     // or lock released after 2nd commit
>
>Will a time-released unlock be viable with a multi-device transaction?

probably, if N is set correctly

>With an implicit unlock, do you know whether the commit succeeded on
>each of the other devices? If not, what is your rollback state?
>
>> 
>> Notes:
>>   - locking is still optional-to-use, but it is forced
>>     to be a single transaction, instead of free-form.
>>   - bad things can still happen if <with-lock>='false',
>>     but no different than any non-NETCONF scenarios
>>   - #rollback, #rollback-on-error, and #confirmed-commit
>>     capabilities are needed
>>   - <start-edit> replaces <lock> 
>>   - <finish-edit> replaces <unlock>  (needed for <running> 
>> target only)
>
>So on a device with candidate config, I don't need these additional
>commands to script a set of commands, but will need it for my
>copy-config command? If candidate isn't supported I need to add these to
>script the set of commands in the first place, right?
>How does that improve interoperability and simplify the conditional
>programming that needs to be done in the scripts? It doesn't sound like
>it makes things easier to write for programmers or easier to read for
>operators.

<start-edit> is required for both targets
<finish-edit> is required for target==<running>.
I think <rollback-changes> is also required for
both targets, and this is a better name/approach for this
operation than 'confirmed commit'.  An explicit
rollback is better than implicit-rollback-through-timeout.

I would rather have one set of operations to represent the
transaction model, rather than a different model and operations
for each target, but if there isn't any interest in working
on this then I guess that won't happen.


>        
>dbh

Andy


--
to unsubscribe send a message to netconf-request@ops.ietf.org with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/netconf/>