RE: Compressing Large Indices?

[Tim Jenkins <tjenkins@TimeStep.com>]

> -----Original Message-----
> From: Juergen Schoenwaelder [mailto:schoenw@ibr.cs.tu-bs.de]
> Sent: January 24, 2000 6:45 AM
> To: tjenkins@TimeStep.com
> Cc: mibs@psg.com
> Subject: Re: Compressing Large Indices?
> 
> 
> 
> >>>>> Tim Jenkins writes:
> 
> Tim> I have a question related to the IPsec Monitoring MIBs that John
> Tim> Shriver and I are working on.
> 
> Tim> We have a table that has a number of indices. It's actually a
> Tim> helper table to look up specific rows of another table that has
> Tim> an arbitrary integer as index. Unfortunately, two of the indices
> Tim> of the first table are variable length octet strings. (They're
> Tim> actually the phase 1 IDs used in IKE, so have a type as well as a
> Tim> raw octet buffer.)
> 
...

> 
> and the suiteByCreatorsEntry is indexed as 
> 
>     suiteByCreatorsP1LocalIdType  : IpsecDoiIdentType
>     suiteByCreatorsP1LocalId      : IpsecRawId
>     suiteByCreatorsP1RemoteIdType : IpsecDoiIdentType
>     suiteByCreatorsP1RemoteId     : IpsecRawId
>     suiteByCreatorsIndex          : Unsigned32
> 
> where
> 
>     IpsecDoiIdentType ::= INTEGER
>     IpsecRawId ::= OCTET STRING (SIZE (0..255)).
> 
> Obviously, you can't use a raw IpsecRawId in any INDEX clauses since
> its size exceeds the SMIv2 max OID length.
> 

Right.

...

> 
> You are saying that a (IpsecDoiIdentType, IpsecRawId) pair is not
> necessarily unique, even if you have a size of 0..255? Is there any
> architectural reason for the size limitation of 0..255 in the first
> place? (I am just curious.)

Actually, one of the possible types for the IpsecRawId is DER DN. I'm told
they don't really have a limit; we put 255 on just to bound it. We were
hoping that in any practical system, that limit would result in no
truncation.

> 
> Back to your question. I have not read the IPsec MIBs and my comments
> may be stupid. But in general, it is desirable to use unique and
> reasonably short values (or even human readable names) to index MIB
> tables. Since your IpsecRawId is way to long and since it may not be
> unique (if I read your comments above correctly), why do you not
> define secondary names (keys) that are shorter and unique?

That's basically what the hash would be, right?

To answer your suggestion below, I'm not sure we can do that reliably
without some complicated rule set. There are 9 potential ID types, some of
which are not necessarily going to contain any human readable characters.

However, your point about not needing a hash function might be make sense;
perhaps just using the first 4 bytes of the raw ID might be sufficient.
However, without looking further, I'm pretty sure that this will result in
lots of collisions, since one of the ID types is IPv6 address.

> 
> This does not have to be done by a hashing algorithm - you can also
> provide a mapping table which is for example indexed by the first N
> bytes of the raw ID plus the unique name (preferably human readable)
> the agent implementation wants to assign. Then you use the unique name
> throughout the MIBs to index the tables. The read-only column of the
> mapping table would contain the IpsecRawId value. A manager who needs
> to map a raw ID into the unique name used throughout the MIBs would
> walk the subtree with the raw ID prefix to lookup the MIB internal ID.
> 
> As I said, I have no clue about the IPsec MIBs and the suggestion
> above may be non-sense in the context of IPsec. But in general, it is
> worthwhile to have short unique names (or numbers) as table indexes if
> you can't use the natural keys. It may pay off to have mappings from
> more complex external data structures to MIB internal names if that
> simplifies the indexing structure of MIB tables considerably. It is
> possible to speedup lookups by indexing the mapping table(s) with
> portions of the complex data structure so that an intelligent manager
> only needs to retrieve parts of the mapping table.
> 

Hmmm. Yes, I think we can make this work. It requires yet another table, but
reduces the number of occurrences of the ID strings throughout the MIB.

I'll take a detailed look at this.

> /js
> 
> -- 
> Juergen Schoenwaelder      Technical University Braunschweig
> <schoenw@ibr.cs.tu-bs.de>  Dept. Operating Systems & Computer Networks
> Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 
> Braunschweig, Germany
> Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>
> 

Thanks!

Tim