[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Poll for consensus on edit operations
Howdy All,
I would like to present some view-points, proposals, and real-life
examples (I believe some folks were asking for some) on the issues of
filtering, node selection, wildcarding, and using XPath. Hopefully,
this can also help us move closer to consensus on these important
issues.
Mandatory XPath
----------------------------------------------------------------------
On the issue of XPath, I believe quite strongly that mandating
full XPath is not the correct choice. Here are some of the reasons
why:
- With XPath, order matters. There are '<' and '>' operators,
and there is a "give the the n'th child" capability. Order
is defined by XML on values and document order when talking
about children nodes. Some boxes which have their internal
ordering by SNMP will have trouble with the XML ordering.
- With XPath, there is a need to perform arithmetic operations
such as addition, multiplication, and modulo arithmetic,
with precedence and grouping.
- With XPath, you have to deal with 'and' and 'or', with
precedence and grouping.
XPath was written in a syntax compatible for use as the value of
an attribute. It is powerful and somewhat unwieldy. It is also
quite heavyweight and it is not easy to implement the full set of
operations given the types of databases which are typically embedded
on NEs. So full XPath as a requirement rather than capability would,
IMHO, severely limit NetConf.
Embedded Operations Attributes
----------------------------------------------------------------------
The current approach of having operation attributes at nodes in a
tree is the best approach, IMHO. It is what is going to be natural
for a user, since the request closely mimics the reply. Building a
simple XML tree with embedded attributes doesn't require that ol'
"PhD" =D.
Consider some of the scenarios which could happen if we don't
allow operations in the tree. Let us say that we want to make sure an
interface exists, is not disabled, but I do not want to change any
configuration which might already exist. I could do this in a single
document as shown in the example below:
<config>
<interfaces>
<ge operation="set">
<instance-id>ge-7/7/7</instance-id>
<disable operation="delete"/>
</ge>
</interfaces>
</config>
If you avoid the operations in the tree, you would have to do
something more like:
<create>
<interfaces>
<ge>
<instance-id>ge-7/7/7</instance-id>
</ge>
</interfaces>
</create>
<delete>
<interfaces>
<ge>
<instance-id>ge-7/7/7</instance-id>
<disable/>
</ge>
</interfaces>
</delete>
This starts to not only take a lot more text, but also becomes
somewhat confusing for the user by breaking up the hierarchy.
The implementation to achieve the first approach is pretty simple,
and here is why. Generally speaking, there are two types of XML
parsing strategies: document model and event-driven. For arbitrary
tasks, document model is easier to work with as a developer, but what
we are doing with the first approach is rather simple. The
event-driven approach visits nodes in XML document order. When a node
with operations attributes is found, apply or remember the operation.
The only thing you need to do is maintain the stack of open elements.
Event-driven parsers are compact and efficient and is the primary
reason why they exist over document-model parsers.
I can speak from experience that this approach works very well.
Get Commands and "Filtering"
----------------------------------------------------------------------
I think a number of people have come to the same conclusion about
XPath being a bit "expensive" as a mandatory protocol. It has been
identified that we need a two step node-selection and filtering
process. The proposal to use a trimmed-down XPath for filtering is
not the right choice. XPath is both a node selection and filtering
mechanism in one. The suggested 'trimming-down' approach as I have
understood seems to be more based on using XPath for node selection
rather than filtering. Node selection, as I argued above, is best
as a simple XML document.
So, how do we do filtering? We need a way to allow a user to get
a good, close approximation of the exact data they want. The
trade-off for getting close rather than exact is that the
implementation must be simple to implement, and that the
implementation must be lightweight on both memory and CPU. Below is a
suggested approach to filtering.
Depth is one area that is important. We should be able to specify
a depth. Getting the entire subtree could be very expensive. But
sometimes, you don't want to enumerate what you want, you just want
the entire subtree. Or, some arbitrary depth. Here are some examples
where I have used various depths:
- Self-depth, which means just get the selected nodes if they
exist (no children), is helpful when you want to target specific
fields under a given node. For example, you may specifically want
an input statistic and an output statistic for an interface,
together with the administrative status, in a monitoring
application.
- Immediate children depth. This is useful if you want to get
all immediate children of a node. Consider, for example, that
you have a number of different interface types. Each interface
type has its own element name and structure for its subtree.
However, to list all interfaces, I don't want to ask for
each type of interface in turn, I just want all of them.
So, I could query for all the immediate children of an
'interfaces' element.
- All descendants. I've actually used this for two different
scenarios. For one, when I don't care to enumerate a whole bunch
of elements under a given node and getting deep is not too
expensive, or if I just want to archive the information and
grab everything.
Once we have a depth specification, we may still want to do a
little more filtering. In a previous e-mail, I alluded to the notion
that 'config' was not a database, 'stat' was not a database, etc., but
were projections of a database such as 'running' or 'boot-config'.
For example, nodes marked with a predicate 'config' can be filtered
from the running database. Nodes can be marked 'config-all' if we
want to get all config nodes, including default values not set by an
administrator. A '<get-config>' command could be short-hand notation
for using a config predicate for all get operations. In fact, in
addition to NetConf-defined predicates, nodes can be given
enterprise-defined predicates. The goal behind enterprise-defined
predicates is to allow the end-user to get perhaps not exact, but
close, replies which can be further processed off of the box. By
allowing enterprise-defined predicates, a very simple to implement and
use syntax can allow pretty close matches to the data being queried,
when taken in conjunction with the depth specfication.
Instances Versus Wildcards
----------------------------------------------------------------------
Another topic is that of specifying instances versus using
wildcards. I would argue that both are needed, even at the same time.
Even SNMP allows you to perform a sort of 'get' using only a prefix of
indexes, performing a walk to discover everything matching the
specified prefix. So what if we were to allow a single prefix and a
wildcard suffix? The example below gets all ports on all cards in
slot 4 on a box.
<get>
<chassis>
<slot>
<instance-id>4</instance-id>
<card>
<port depth='single' filter='all'/>
</card>
</slot>
</chassis>
</get>
Configuration Example
----------------------------------------------------------------------
Below is a sample request for loading static route information. It is
currently in an existing, proprietary format and does not match the
current NetConf proposal. In all of the examples, we have removed the
namespace declarations for brevity.
<soap:Envelope>
<soap:Header>
<rpc:ctag>Create some static routes</rpc:ctag>
</soap:Header>
<soap:Body>
<rpc:Load>
<rpc:Buffer>ExampleStaticRoute</rpc:Buffer>
<x:root>
<x:protocol>
<x:static>
<x:staticDestPrefix>
<x:value>192.168.10.0/24</x:value>
<x:staticNextHop>141.251.10.1</x:staticNextHop>
<x:staticPref>15</x:staticPref>
</x:staticDestPrefix>
<x:staticDestPrefix>
<x:value>192.168.20.0/24</x:value>
<x:staticNextHop>192.168.20.1</x:staticNextHop>
<x:staticPref>10</x:staticPref>
</x:staticDestPrefix>
<x:staticDestPrefix>
<x:value>192.168.30.0/24</x:value>
<x:staticNextHop>192.168.30.1</x:staticNextHop>
<x:staticPref>10</x:staticPref>
</x:staticDestPrefix>
<x:staticDestPrefix>
<x:value>192.168.40.0/24</x:value>
<x:staticNextHop>192.168.40.1</x:staticNextHop>
<x:staticPref>20</x:staticPref>
</x:staticDestPrefix>
<x:staticDestPrefix>
<x:value>192.168.50.0/24</x:value>
<x:staticNextHop>192.168.50.1</x:staticNextHop>
<x:staticPref>20</x:staticPref>
</x:staticDestPrefix>
<x:staticDestPrefix>
<x:value>192.168.60.0/24</x:value>
<x:staticNextHop>192.168.60.1</x:staticNextHop>
<x:staticPref>20</x:staticPref>
</x:staticDestPrefix>
<x:staticDestPrefix>
<x:value>192.168.70.0/24</x:value>
<x:staticNextHop>192.168.70.1</x:staticNextHop>
<x:staticPref>20</x:staticPref>
</x:staticDestPrefix>
</x:static>
</x:protocol>
</x:root>
</rpc:Load>
</soap:Body>
</soap:Envelope>
Query Example: Get all
----------------------------------------------------------------------
Here is a sample request document and reply for getting all of the
static route information set in the example above.
<soap:Envelope>
<soap:Header>
<rpc:ctag>Get all Static Routes</rpc:ctag>
</soap:Header>
<soap:Body>
<rpc:GetRequest>
<x:root>
<x:protocol>
<x:static x:filter="all" x:depth="deep"/>
</x:protocol>
</x:root>
</rpc:GetRequest>
</soap:Body>
</soap:Envelope>
<xsoap:Envelope>
<xsoap:Header>
<rpc:ctag>Get all Static Routes</rpc:ctag>
</xsoap:Header>
<xsoap:Body>
<rpc:GetRequestResponse>
<x:root>
<x:protocol>
<x:static>
<x:staticDefaults>
<x:staticPref>5</x:staticPref>
</x:staticDefaults>
<x:staticDestPrefix><x:value>192.168.10.0/24</x:value>
<x:staticPref>15</x:staticPref>
<x:staticNextHop>141.251.10.1</x:staticNextHop>
</x:staticDestPrefix>
<x:staticDestPrefix><x:value>192.168.20.0/24</x:value>
<x:staticPref>10</x:staticPref>
<x:staticNextHop>192.168.20.1</x:staticNextHop>
</x:staticDestPrefix>
<x:staticDestPrefix><x:value>192.168.30.0/24</x:value>
<x:staticPref>10</x:staticPref>
<x:staticNextHop>192.168.30.1</x:staticNextHop>
</x:staticDestPrefix>
<x:staticDestPrefix><x:value>192.168.40.0/24</x:value>
<x:staticPref>20</x:staticPref>
<x:staticNextHop>192.168.40.1</x:staticNextHop>
</x:staticDestPrefix>
<x:staticDestPrefix><x:value>192.168.50.0/24</x:value>
<x:staticPref>20</x:staticPref>
<x:staticNextHop>192.168.50.1</x:staticNextHop>
</x:staticDestPrefix>
<x:staticDestPrefix><x:value>192.168.60.0/24</x:value>
<x:staticPref>20</x:staticPref>
<x:staticNextHop>192.168.60.1</x:staticNextHop>
</x:staticDestPrefix>
<x:staticDestPrefix><x:value>192.168.70.0/24</x:value>
<x:staticPref>20</x:staticPref>
<x:staticNextHop>192.168.70.1</x:staticNextHop>
</x:staticDestPrefix>
</x:static>
</x:protocol>
</x:root>
<x:summary>success</x:summary>
</rpc:GetRequestResponse>
</xsoap:Body>
</xsoap:Envelope>
Query All Interfaces Example
----------------------------------------------------------------------
In this example, we get the list of all interfaces on the box. In the
reply, I made the interfaces 'eth' and 'eth100' to avoid accidental
release of proprietary information. This example shows the use of
'shallow' depth.
<soap:Envelope>
<soap:Header>
<rpc:ctag>Get the names of all interfaces</rpc:ctag>
</soap:Header>
<soap:Body>
<rpc:GetRequest>
<x:root>
<x:interfaces x:filter="all" x:depth="shallow"/>
</x:root>
</rpc:GetRequest>
</soap:Body>
</soap:Envelope>
<xsoap:Envelope>
<xsoap:Header>
<rpc:ctag>Get the names of all interfaces</rpc:ctag>
</xsoap:Header>
<xsoap:Body>
<rpc:GetRequestResponse>
<x:root>
<x:interfaces>
<x:eth><x:value>eth-7/7/7</x:value>
</x:eth>
<x:eth><x:value>eth-9/0/0</x:value>
</x:eth>
<x:eth><x:value>eth-9/0/1</x:value>
</x:eth>
<x:eth100><x:value>eth100-9/0/2</x:value>
</x:eth100>
<x:eth100><x:value>eth100-9/0/3</x:value>
</x:eth100>
<x:eth100><x:value>eth100-9/0/4</x:value>
</x:eth100>
</x:interfaces>
</x:root>
<x:summary>success</x:summary>
</rpc:GetRequestResponse>
</xsoap:Body>
</xsoap:Envelope>
Query a Singleton Interface Example and Multiple-Get-ops.
----------------------------------------------------------------------
Finally, in this example, we show a combination of singleton-get and
wildcard get, where we can see the interfaces on a box and the 777
interface in detail. Again, this is sanitized for proprietary
information purposes.
<soap:Envelope>
<soap:Header>
<rpc:ctag>Get the names of all interfaces</rpc:ctag>
</soap:Header>
<soap:Body>
<rpc:GetRequest>
<x:root>
<x:interfaces x:filter="all" x:depth="shallow"/>
<x:interfaces>
<x:eth x:filter="all"
x:depth="deep"><x:value>eth-7/7/7</x:value>
</x:eth>
</x:interfaces>
</x:root>
</rpc:GetRequest>
</soap:Body>
</soap:Envelope>
<xsoap:Envelope>
<xsoap:Header>
<rpc:ctag>Get the names of all interfaces</rpc:ctag>
</xsoap:Header>
<xsoap:Body>
<rpc:GetRequestResponse>
<x:root>
<x:interfaces>
<x:eth><x:value>eth-7/7/7</x:value>
<x:timeOfLastChange>1970-01-01T00:00:00</x:timeOfLastChange>
<x:ifDescr>InterfaceDescription</x:ifDescr>
<x:intfSnmpIndex>6</x:intfSnmpIndex>
<x:intfOperStatus>notPresent</x:intfOperStatus>
</x:eth>
<x:eth><x:value>eth-9/0/0</x:value>
</x:eth>
<x:eth><x:value>eth-9/0/1</x:value>
</x:eth>
<x:eth100><x:value>eth100-9/0/2</x:value>
</x:eth100>
<x:eth100><x:value>eth100-9/0/3</x:value>
</x:eth100>
<x:eth100><x:value>eth100-9/0/4</x:value>
</x:eth100>
</x:interfaces>
</x:root>
<x:summary>success</x:summary>
</rpc:GetRequestResponse>
</xsoap:Body>
</xsoap:Envelope>
Summary
----------------------------------------------------------------------
In summary, NetConf needs to have a default, easy-to-use and
resource-light means of getting and setting data for boxes big and
small. The current draft gets pretty close to such requirements. The
suggested solution of using XPath, though an interesting idea, imposes
too high a burden on the implementation and resource requirements.
The depth and predicate-based filtering appraoch extends the current
draft in a way which should be sufficiently powerful and avoids
overly-large responses.
Thanks, All!
I'd like an
Xpath expert to redo the 'chassis' example I did but use a
different (child element based) naming scheme instead of
a simple attribute. I'd like to see how the path to
"ip-filter.1" is constructed in this case. I suspect the
expression is rather complex, if possible at all.
The attribute vs. element based naming scheme doesn't affect the
XPATH expression all that much. All that's required is to remove
the @ from the predicate. Here's Andy's example with
attributes as instance names.
<modify
start-path="//chassis/slots/slot[instance-id='3']/ports/port[instance-id
='7']/duplex-mode">
<chassis>
<slots>
<slot>
<instance-id>3</instance-id>
<ports>
<port>
<instance-id>7</instance-id>
<duplex-mode>auto</duplex-mode>
</port>
</ports>
</slot>
</slots>
</chassis>
</modify>
<create
start-path="//chassis/slots/slot[instance-id='3']/ports/port[instance-id
='7']/filters/ip-filters/ip-filter[instance-id='1']">
<chassis>
<slots>
<slot>
<instance-id>3</instance-id>
<ports>
<port>
<instance-id>7</instance-id>
<filters>
<ip-filters>
<ip-filter>
<instance-id>1</instance-id>
<action>permit</action>
<direction>both</direction>
<address>
<addr-type>ipv4</addr-type>
<addr>192.168.1.1</addr>
</address>
</ip-filter>
</ip-filters>
</filters>
</port>
</ports>
</slot>
</slots>
</chassis>
</create>
But this is big and unwieldy. Along the lines of one of Andy's
suggestions, this kind of request:
<modify
start-path="//chassis/slots/slot[instance-id='3']/ports/port[instance-id
='7']">
<duplex-mode>auto</duplex-mode>
</modify>
is attractive. However this would make at least a subset of XPATH
a requirement. Also once we've started moving down this road the next
logical place to stop is XSLT. Why stop with just one memory hungry
standard when you can have two?
I'd like NETCONF to be something that does not require a Ph.D.
in XML to understand and use. We should keep it simple. This protocol
aims to replace screen-scraping, after all. It can replace floor wax
and dessert toppings in version 2.0.
Rob
--
to unsubscribe send a message to netconf-request@ops.ietf.org with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/netconf/
--
to unsubscribe send a message to netconf-request@ops.ietf.org with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/netconf/>