Re: comments on draft-bierman-sming-ds-02.txt

[Andy Bierman <abierman@cisco.com>]

At 11:34 AM 3/12/2002, Juergen Schoenwaelder wrote:

>[DISCLAIMER]
>
>We have read the document in detail from page 1 to page 31 and we have
>written down comments/questions in that order. So the answers to some
>of these questions became clear later on but we have left them
>untouched since we are lazy.

thanks for reviewing 31 of the 83 pages. That's a good
start, I guess.  I will try to respond to the technical
questions raised. I think terminology can be finalized
after the technical details are resolved.


>[DISCLAIMER]
>
>SMI-DS is used to refer to Andy's SMI-DS and SMIng is used to refer to
>the SMIng which originated in the NMRG.
>
>
>1. The document sometimes uses terminology without explanations so that
>   the reader has to guess what precisely the meaning is. For example, 
>   page 4: multi-layer data structure
>
>     Is this a nested data structure such as a struct within a struct?
>     And if so, what kind of nested data structures are allowed? Note
>     that it is relatively straight-forward to "flatten" nested structs
>     in SMIv2 but it requires tables if there is a nested array.
>
>2. The distinction between objects and types is unclear. Our understanding
>   is that an object is something which I can instantiate while a type is
>   something which just describes the structure (and perhaps just for the
>   purpose of reusing the type).
>
>3. We prefer the following terms
>
>   o "constructed type/object" for something which contains other types/objects
>   o "base types" for primitive base types provided by the language (Integer32)
>   o "derived types" for types that are derived from other base types or 
>     constructed types.
>
>   We suggest to avoid "complex types/objects" since this term can be
>   mis-leading.
>
>4. We find the terms "MIB Object" and "Data Object" confusing, especially since
>   the meaning of "SMI Data Structure variable declaration" is not explained.
>   What is the difference to a "SMI Data Structure Definition"?
>
>5. We find the difference between "Scalar" and "SCALAR" a bit
>   confusing. We believe that SMI-DS "SCALARS" are what SMIng calls
>   an attribute while "Scalars" are what SMIng calls scalars.
>
>6. We are not sure whether the goal of 100% forward and backward
>   compatibility has been reached or to what extend. The SMI-DS
>   translation to SMIv2 is not really defined in Appendix A and thus
>   it is hard to judge how much is in fact translatable (and what the
>   compatibility implication on the wire will be).

I agree this appendix isn't finished and should have more detail.
Keep in mind that the SMI is a description of object semantics.
It does not specify on-the-wire encoding. SMIv2 semantics can be
transparently converted to SMI-DS and then transparently converted
back to SMIv2, with no on-the-wire changes in either direction.
Just as with SMIv2 -> SMIv1, there are new features for which the
conversion will not be transparent.  Converting UNIONs and STRUCTs
to SMIv2 is simply a matter of adding lots of verbose text to
every affected DESCRIPTION clause, where the phrase "associated instance"
is used quite frequently.


>7. It is unclear whether compatibility on the wire between engines
>   that use SMIv2 style definitions and engines that use SMIng style
>   definitions (and naming) is still a design objective.

Please explain what on-the-wire problems exist


>8. The proposed new naming touches both the language and the protocol,
>   at least in the sense that no existing implementation we are aware
>   of will be able to handle hierarchically named variables in a
>   meaningful way. We would expect that 100% forward/backward
>   compatibility implies that I can use existing tools to look at data
>   implemented by an agent whose implementation is based on an SMI-DS
>   MIB module.

The new hierarchical naming is identical to SMIv2 for all the
constructs currently available in SMIv2.  For generic applications
which have no knowledge of the objects they are manipulating
(e.g. mibwalk), all the protoOps work exactly as before, and
they still work. For applications that need to understand the
objects they use, the semantics of the object and the structure
of the instance portion of the OID need to be known.  This is
no different than the current situation with SMIv2.


>9. We believe that a syntax which looks close to SMIv2 but means
>   sometimes something different is worse than a syntax which looks
>   different to SMIv2 and means something different. People who do not
>   understand the differences will merge SMIv2/SMI-DS in arbitrary
>   ways. This has happened with SMIv1/SMIv2.

Please provide examples where this is the case in SMI-DS.
It was certainly my intent to maintain semantics and syntax
as much as possible.  I think it would be more confusing to
MIB readers and writers to transition to SMIng syntax, in
which absolutely every macro and clause in changed.


>10. We believe it should be a design goal that accessible data objects
>    can be used with existing SNMP applications (which is more than
>    just existing versions of SNMP) - which probably implies no
>    changes in the naming. Note that SMIng does the naming in the
>    protocol specific mappings. Supporting SMI-DS naming in protocols
>    would from this viewpoint be another protocol mapping.

This is a repeat of point 8. See response 8.


>11. We believe that "SCALAR" is a confusing name, but we do not really
>    have a much better proposal (leaf object? leaf variable? leaf
>    member?). In SMIng, we would perhaps call them leaf attributes.
>    Having a name for this would help to make definitions easier to
>    understand.
>
>12. We do not understand what "only SCALAR data objects are actually
>    accessible via a management protocol" actually means. Does it
>    imply that I can not manipulate a constructed type? Do I have to
>    get/set all the leaf attributes of a constructed type? Can I do
>    walks over constructed types or can I only do walks over the leaf
>    attributes that represent parts of a constructed types?

I am hoping the EOS WG will provide new ProtoOps to transfer
aggregates. SNMP has no mechanisms to transfer entire tables,
only accessible objects, and this limitation applies to SMI-DS
as well. Constructs with MAX-ACCESS (explicit or implied) of
not-accessible cannot be transferred on-the-wire. The SMIng
proposal does not provide any mechanisms to transfer entire
classes with SNMP either.


>13. We believe that an array should be similar to an ASN.1 SEQUENCE OF,
>    that is the array feature should work with an arbitrary type. The
>    SMI-DS ARRAY mixes containment and multi-valuedness in one
>    construct.  Programming languages such as C and even ASN.1 do not
>    do this.  Arrays do not yet exist in SMIng (only in the form of
>    tables in the protocol mappings).

Programming languages such as C allow both types of constructs.
You can declare an array in a typedef of a variable declaration.
It is up to the MIB designer to decide which way is most
appropriate for the given task. SMI-DS has examples of both.

>14. We expect that not only base types, but also derived types should
>    be allowed in index clauses (just constructed types are not
>    allowed).

This is rather problematic and a big change from SMIv2.
How do you INDEX an ARRAY with a UNION, or a complex STRUCT
with nested STRUCTs and UNIONs? This capability is not
that important, otherwise it would have shown up in the
requirements document.


>15. We agree with the intention of the UNION, although the definition
>    can be improved. You do not need the discriminator since you
>    encode this information in the SMI-DS names. So in fact, you have
>    some sort of an "implicitly discriminated union." Unions do not
>    yet exist in SMIng.

The descriptors are not present on-the-wire. Unless the discriminator
is present in the OID, the applications and agents have no way of
knowing which node in the UNION is being referenced.


>16. We believe that SMIng achieves a much clearer and thus simpler
>    separation of potentially reusable type definitions and variable
>    declarations.

Can you provide an example of what you mean here? This is
a broad generalization.


>17. The relationships to SMIv2 are interesting. You change
>    NOTIFICATION-TYPE to NOTIFICATION for language consistency
>    reasons, which we have sympathy for. I am sure someone else will
>    see this as a gratitious change. You also claim that lots of other
>    constructs do not change as well, including MODULE-COMPLIANCE.
>    Even if this is true for the syntax (which I am not sure of), I am
>    sure that semantics will have to change.

This is not a gratuitous change.
The syntax structure of SMIv2 is 
    "<descriptor> <construct-type> <construct>".
The syntax structure of SMI-DS is
    "<construct-type> <descriptor> <construct>"
which is much more consistent with programing languages.


>18. Since there are not table/entry definitions anymore, how do you
>    create names for them when translating to SMIv2 and what happens
>    if there are name clashes with other definitions?

This was omitted from the draft. A simple algorithm is needed
that implicitly defines a "fooEntry" descriptor for the .1 
node of a complex type definition for "foo". Naming conventions
(which disallow top-level objects from using the suffix "Entry"
are needed to avoid name collisions. Since these "fooEntry" 
descriptors are only useful in SMIv2 IMPORTS and AUGMENTS
clauses, this should not pose a problem in the future. 
These descriptors have no use in SMI-DS.


>19. There is a need for a mechanism to disambiguate descriptors or
>    identifiers. For example, assume you have an ARRAY which contains
>    two members of the same STRUCT type and you want to refer to one
>    of the STRUCT members in a notification definition. SMIng provides
>    the necessary infrastructure for this and this alone affects
>    several statements/clauses/macros/constructs (whatever you name
>    it).

This refers to IMPORTS I guess. A naming convention is needed
to specify the proper object, e.g., foo.c32 or bar.c32.


>20. It is unclear how much savings the new naming scheme provides on
>    agents. With SMI-DS, you probably have fewer RowStatus objects
>    since you do not need them for "contained" multi-valued
>    attributes. You also can improve code stubs since the compiler now
>    knows that one table is really just used to hold multi-valued
>    attributes of some other object. We fail however to understand why
>    the hierarchical naming simplifies things such as (a) reducing the
>    degrees of ordering freedom we have in SNMP or (b) reducing the
>    need to have createAndWait/notInService states in the first place.

Removing the multiple interacting r/c tables will help reduce
agent complexity because the object ancestry is fully specified
in every object instance. Using one top level RowStatus (or
none at all with new EOS support) simplifies agent code a
great deal.  MIB design dictates whether createAndWait or
notInService are needed. SMI-DS does change that.  It is
important to maintain the hierarchical structure of complex
data types so generic functions can be written to process them,
no matter where they occur for a given usage. The SMIng
proposal will require that the SNMP engine have knowledge
of every usage of every such construct, and rearrange the
OIDs manually to achieve what SMI-DS provides automatically.
Forcing humans to manipulate the OID ordering in protocol
mapping clauses is not workable. It means that every time
a new object is added to an embedded construct, every
MIB that uses that construct has to be updated to assign the
new embedded node to a different OID.  This is not consistent
with the use of TEXTUAL-CONVENTIONs today. 


>21. We agree with the idea to hold compliance information in the
>    agent.  However, we do not understand the value in introducing
>    "pseudo OIDs". Why are conformance definitions not being kept in a
>    properly indexed conformance table? Conformance data is just data
>    and there should be no need for a special mechanism for this
>    purpose. The pseudo OIDs may give funny results when you do
>    getnext walks. (Also, the restriction that you can report
>    conformance to only a single compliance definition might hurt if
>    conformance statements are revised and you need to report multiple
>    compliance statement OIDs for continued interoperation.)

It would be very burdensome to maintain a MIB for the conformance
information, for every MIB object that ever gets defined.
Since the conformance data is always the same for every
data object, it is easier to align the naming.  I suppose
it could be done either way.  A given implementation of a
given object instance can only comply with one MODULE-COMPLIANCE, 
so there is no need to allow for multiple compliance statement
OIDs.


>22. We do not understand what "Non-SCALAR Static Sub-Identifier
>    Naming" is and how it relates to "Non-SCALAR Data Object
>    Naming". We assume that the first refers to the portion of the OID
>    which depends on the data type without the instance information
>    while the second one extends this with instance identifier
>    encodings.

The first is the generic form, e.g. <compatNode> is not specified.
The latter is the specific form, e.g. <compatNode> equals 1.


>23. Example 5.7.1:
>
>    - There is no status associated with the TYPEDEF itself. Why not?

It seemed redundant to me, since the accessible objects are the
ones that are used. I originally had it this way, then took it
out. I can put this back if it is actually useful.


>    - More general: Should access clauses only be used when names are
>      assigned to reusable data types? Specifying that an attribute is
>      read-only in a type definition complicates reuse. This affects
>      both SMIng and SMI-DS.

Good question.  This could be useful. A simple example would
be an InetAddress STRUCT that is read-only in a monitoring MIB
and read-create in a configuration MIB.


>    - Hierarchical naming shows an advantage here because it
>      implicitly provides the discriminator in the name portion of a
>      union varbind element.
>
>    - It is unclear how unions map to SMIv2 if the SCALARs of the
>      union happen to use different base types.

They map by using verbose text in the DESCRIPTION clause, just
like in the EXPRESSION-MIB, which has several versions of
the result object. The text says that only one of the set
of such objects is actually instantiated, based on the value
of the associated discriminator object.



>24. Example 5.7.3:
>
>    - Would the example still work if an interface would first be
>      defined as a constructed type which is than assigned to a
>      variable or would this cause incompatible OIDs?

I actually struggled with this issue for awhile, and tried to
come up with a construct called UNWRAPPED STRUCT, but the
power of automatic hierarchical naming falls apart because
you can't arbitrarily strip the wrapper.  Defining the
ifTable as a STRUCT TYPEDEF would add an extra layer of
naming.  I think it's important that at least one
method for transparent conversion be provided. It isn't
a requirement that every possible way to define a given 
data structure be transparently compatible with SMIv2.


>    - We do not understand why there is a need for a new SIZE clause.

This is a machine-readable indication of the number of instances
that are expected/allowed for that ARRAY INDEX.  I suppose this
should usually only be relevant in VAR declarations, not TYPEDEFs.
The DSMON example of a fixed sized array of 64 DSCP-to-Counter Aggregation
groups is an example where it's useful in a TYPEDEF, but this is 
the exception. 


>    - We think that the introduction of INDEX clauses in parallel to
>      the SCALAR clause complicates things. This leads to several
>      special cases. How is the ordering of index components defined?
>      See for example the rules for MAX-ACCESS in INDEX clauses and
>      the optionality of subid assignments. We think it would be
>      simpler to keep the old notion of a separate index definition.

The order is specified in the document to be the order they
appear in the MIB, just like with SMIv2 INDEX clauses.
I think explicit INDEX clauses are more readable than
hunting around various MIBs looking for the OBJECT-TYPE
macros that match the descriptor identified in an SMIv2
INDEX clause.

>    - Is it a bug or a feature that there are no STATUS clauses in
>      INDEX definitions?

INDEX objects are not accessible, but they are implicitly
present in every OID of the accessible objects. I don't
know of any current SMIv2 tables that use deprecated INDEX
components. A STATUS clause could be added, if needed.


>25. Data structure augmentations:
>
>    - What is the expected value of introducing SPARSE-CONDITION and
>      SPARSE-EXPRESSION?

To provide human and machine readable definitions of the
existence relationship between the augmented and augmenting
table.  SPARSE-AUGMENTS is a concept that has been around
for years, and buried in DESCRIPTION clauses.


>    - SPARSE-CONDITIONS can also go into normal description clauses
>      since it is just free form text. How do we decide that some text
>      is so important that it justifies its own clause?

This existence relationship between tables is used so often
that it is worth formalizing. 


>    - Regarding SPARSE-EXPRESSION, what is the intended usage?  Is it
>      just documentation clarity? Is the intention to support
>      automatic test tools? Do these machine readable expressions
>      significantly simplify/improve code generators? And what after
>      all does the expression language look like?

This is intended to be a machine readable version of the
SPARSE-CONDITION. The syntax is TBD, but I would like
to use a subset of the PolicyScript syntax, found in the PM MIB.
If a simple expression is possible, then tools could take
advantage of it. If a complex expression is needed, then this
clause would be omitted, rather than making the expression
syntax too complicated.


>26. Augments Example:
>
>    - We guess that the SIZE clauses in the definition of inetAddrType
>      and inetAddr are just cut & paste errors.

no. this refers to the number of allowed instances of the INDEX
component, not the number of octets in an individual instance.


>    - We are still undecided what the added value of arrays is. It
>      looks like SMI DS arrays are pretty close to tables except that
>      they can conceptually be embedded due to the hierarchical naming
>      scheme.

The ability to embed any of the complex data types in a
manner consistent with modern programming languages is
important.


>27. The VAR STRUCT format is unclear. It sometimes contains other
>    SCALAR/STRUCT/UNION/ARRAY clauses (see ipXStats example) and
>    sometimes it just contains flat SYNTAX etc. clauses (see myAddress
>    example).  From reading the grammar, it looks like both are
>    allowed? What is the difference between both formats in case I
>    just have a single STRUCT, except that you safe a few lines in
>    some cases?

Data definitions are dictated by the problem being addressed
by the MIB author. This is no different than a C programmer
defining data types for a program. Sometimes they only
contain base types, and sometimes they contain some complex
types. SMI-DS provides the capability, and it's up to the
MIB designer to define appropriate data types for the
problem being addressed.


>28. The ID says that pointers can be used to reference complex data
>    objects. It is unclear how this works. How do I reference
>    entSensorData[17]? Where does the instance identifying portion of
>    the OID value start?

I had to change the OID naming to be compatible with SMIv2.
The old naming would allow for this feature. Currently,
only constructs that would be legal in SMIv2 can be pointed at.


>29. Pointers may refer to derived types which is not yet allowed in
>    SMIng but seems to be a good addition.

This is a way of removing verbose text from DESCRIPTION clauses
and simplifying it in a formal construct.


>30. What is the purpose of the UNIQUE-SCOPE clause and what are the
>    benefits of making this particular piece of information machine
>    readable?

There could be some benefits to making this common attribute
machine readable. This is not really an important feature.
It was an attempt to provide something like the UNIQUENESS
clause in SPPI.



>/js
>
> -frank

Andy