Subject: Re: [nmrg] meetings
From: Luca Deri <[email-masked]>
To: Juergen Schoenwaelder <[email-masked]>
Cc: Network Management Research Group <[email-masked]>
Date: Fri, 30 Apr 1999 15:41:55 +0200

Hi all,
I believe I can't attend most of the meetings below:

>         May 24-28, 1999         IM '99 Boston
>         July 12-16, 1999        45. IETF Oslo
>         October 11-13, 1999     DSOM '99 Z"urich
>         November 8-12, 1999     46. IETF Washington

On May 15th I'll get married hence I'll disappear for a month. I might
be in Zurich, though. Due to budget shortages (deja vu?) I'm almost sure
I cannot attend IETF meetings although I would like to be there.

Concerning IM'99 I have a problem. I should present a poster session (is
about SAMBA+SNMP). Unfortunately when i submitted the paper I didn't
knwo the date of the marriage. Then some colleagues of mine from Finsiel
were supposed to go @ IM'99 but they recently changed their plans. I'm
wondering whether I could ask some of you to help me. I was thinking so
mail you the poster itself and a few copies of the paper, in order not
to cancel my poster. Does some of you want to gimme this favour?

Thanks in advance, Luca.

-- 
Luca Deri		 Finsiel S.p.A.
Via Matteucci 34/B	 56124 Pisa, Italy.
Ph. +39/050/968.639      Fax. +39/050/968.626
Email: [email-masked] WWW: http://www.tlcpi.finsiel.it/~deri/
Software is about stuff, about getting hands dirty - Jim Coplien

Date: Thu, 29 Apr 1999 17:29:32 +0200
From: Juergen Schoenwaelder <[email-masked]>
To: Network Management Research Group <[email-masked]>
Subject: [nmrg] meetings
Sender: [email-masked]

I think its time to talk about face-to-face meetings. I already
mentioned in a previous email the following events:

        May 24-28, 1999         IM '99 Boston
        July 12-16, 1999        45. IETF Oslo
        October 11-13, 1999     DSOM '99 Z"urich
        November 8-12, 1999     46. IETF Washington

I got one (!) reply saying that he generally prefers meetings at IETFs
rather than conferences. Given the silence of the other NMRG members,
I propose to

 (a) hold an official NMRG meeting at the IETF in Oslo

 (b) hold an interim NMRG meeting at IM'99 in Boston (since many of
     us are going to be at IM'99 anyway)

I think we should try to get a meeting room in Oslo during the IETF
week - probably on Friday if everything else if filled up. (But we
actually only need a small room - so there is a good chance that
something is available.) We can also try to meet on Sunday before the
IETF week starts - but this needs to be aligned with our travel plans
(so please raise you voice).

Regarding IM'99, I will arrive in Boston on Saturday 22 and I will
leave on Friday 28. We can either meet on Sunday or on Friday since I
do not plan to attend any of the Friday tutorials. Again, let me know
who will be there so that we can agree where and when to meet.

							Juergen
-- 
Juergen Schoenwaelder  [email-masked] http://www.cs.tu-bs.de/~schoenw
Technical University Braunschweig, Dept. Operating Systems & Computer Networks
Bueltenweg 74/75, 38106 Braunschweig, Germany.        (Tel. +49 531 / 391 3289)

Date: Tue, 27 Apr 1999 14:59:23 +0200
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
CC: [email-masked]
Subject: Re: [nmrg] more thoughts on pdu compression via USM encryption
Sender: [email-masked]

>>>>> Wes Hardaker writes:

Wes> You know, another possibility for compression of data via an
Wes> encryption protocol definition is that we could replace the BER
Wes> encoding with one of the other encoding schemes like CER.  The
Wes> end user would be an idiot to translate it to BER then decode
Wes> that, rather than use the newer encoding scheme directly...

We discussed this point in Lausanne and our final conclusion was that
using other encoding schemes does not seem to be worth the effort.
You can find some of the reasoning in the SimpleTimes article.

Wes> Obviously, a combination might prove the most useful:
Wes> des3-deflate-cer, would be a encrypted, compressed, alternative
Wes> binding for the pdu portion of the packet...

Why? I have doubts that you will have huge savings with your CER
encoding plus deflate compression. BER encoded SNMP data is pretty
regular and I am confident that deflate compression will do a good
job. (Harald once suggested to pre-load the deflate dictionary in
order to tune things even further.) Anyway, I guess we need to make
experiments to find the answers to these questions.

Wes> Actually, I think the best compression scheme would be one that
Wes> is tailored to the data SNMP has to carry in the first place.
Wes> Namely, one that takes into account the various oids in the
Wes> packet and tries to optimize for prefix compression, etc, would
Wes> do better with simple rules over most compression algorithms
Wes> (most likely).

I am not sure. Algorithms like deflate do a reasonable job to
recognize repeated pattern themself. You can give some more hints by
preloading the dictionary. Using a custom encoding to get rid of OID
prefixes is more implementation work and you still need a generic
compression algorithm in order to compress the payload (which can have
arbitrary structure).
							Juergen
-- 
Juergen Schoenwaelder  [email-masked] http://www.cs.tu-bs.de/~schoenw
Technical University Braunschweig, Dept. Operating Systems & Computer Networks
Bueltenweg 74/75, 38106 Braunschweig, Germany.        (Tel. +49 531 / 391 3289)

Date: Mon, 26 Apr 1999 07:41:02 -0700 (PDT)
From: Randy Bush <[email-masked]>
To: Juergen Schoenwaelder <[email-masked]>
Cc: [email-masked]
Subject: Re: [nmrg] draft-irtf-nmrg-snmp-tcp-00.txt
Sender: [email-masked]

> We do not propose to replace UDP as a transport, especially for
> polling engines. We propose to use TCP for bulk data transfers.

our folk reread and think 3.3 is fine warning.

randy

Date: Mon, 26 Apr 1999 13:00:44 +0200
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
CC: [email-masked]
Subject: Re: [nmrg] draft-irtf-nmrg-snmp-tcp-00.txt
Sender: [email-masked]

>>>>> Randy Bush writes:

Randy> chatted to our snmp deployment folk.  assuming that the tcp
Randy> connection would be long lived and allow multiple snmp requests
Randy> to traverse it without being re-established, the agent would to
Randy> need to be able to handle multiple connections at the same
Randy> time.  this could cause the manager to be required to implement
Randy> a very large number of tcp connections simultaneously as well.

We do not propose to replace UDP as a transport, especially for
polling engines. We propose to use TCP for bulk data transfers. There
is text in the ID which specifically addresses the connection
management problem:

: 3.3.  Connection Management
: 
:    The use of TCP connections introduces costs. Connection establishment
:    and shutdown causes additional traffic on the wire. Further,
:    maintaining open connections binds resources in the network layer of
:    the underlying operating system.
: 
:    TCP connections should therefore only be used when the size of the
:    data transferred would otherwise cause large latencies due to small
:    UDP packet sizes and an increased number of interactions.
: 
:    Both, an SNMP entity in the agent role and an SNMP entity in the
:    manager role, are allowed to close the connections at any point in
:    time. This ensures that SNMP entities can control their resource
:    usage and shutdown TCP connections that are not used. Note, SNMP
:    engines are not expected to process any outstanding requests if the
:    underlying TCP connection has been closed. A no response error
:    condition SHOULD be signalled for outstanding requests for command
:    generator applications if the TCP connection is closed.

Do you or your snmp deployment folks think that something is missing
here? Do you think we have to be more verbose and explicit about the
implications of using TCP?
							Juergen

Mail-from: From [email-masked] Sun Apr 25 07:33 MET 1999
Date: Sat, 24 Apr 1999 22:33:09 -0700 (PDT)
From: Randy Bush <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: [nmrg] draft-irtf-nmrg-snmp-tcp-00.txt
Sender: [email-masked]

chatted to our snmp deployment folk.  assuming that the tcp connection would
be long lived and allow multiple snmp requests to traverse it without being
re-established, the agent would to need to be able to handle multiple
connections at the same time.  this could cause the manager to be required
to implement a very large number of tcp connections simultaneously as well.

randy

To: [email-masked]
Subject: [nmrg] more thoughts on pdu compression via USM encryption
Organization: U.C.Davis, Information Technology - D.C.A.S.
From: Wes Hardaker <[email-masked]>
Date: 20 Apr 1999 09:51:24 -0700
Lines: 24
Sender: [email-masked]

You know, another possibility for compression of data via an
encryption protocol definition is that we could replace the BER
encoding with one of the other encoding schemes like CER.  The end
user would be an idiot to translate it to BER then decode that, rather 
than use the newer encoding scheme directly...

Obviously, a combination might prove the most useful:
des3-deflate-cer, would be a encrypted, compressed, alternative
binding for the pdu portion of the packet...

(though if you start generating that many combinations, the number of
oids you need to register will grow quite a bit).

Actually, I think the best compression scheme would be one that is
tailored to the data SNMP has to carry in the first place.  Namely,
one that takes into account the various oids in the packet and tries
to optimize for prefix compression, etc, would do better with simple
rules over most compression algorithms (most likely).

-- 
Wes Hardaker
Distributed Computing Analysis and Support
University of California at Davis

Date: Tue, 20 Apr 1999 11:44:16 +0200
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
CC: [email-masked]
Subject: Re: [nmrg] draft-irtf-nmrg-snmp-tcp-00.txt
Sender: [email-masked]

>>>>> Luca Deri writes:

Luca> [2] Traps In this case the agent (not the manager) has to decide
Luca> what transport has to be used for sending traps. I would say
Luca> that a "TCP-enhanced" agent can choose to use TCP depending on
Luca> the trap size. Nevertheless, if the trapd does not accept TCP
Luca> connections (i.e. the connection establishment fails) then UDP
Luca> is used. This allows implementors to maintain a certain degree
Luca> of backward compatibility.

RFC 2273 defines how a Notification Originator finds the target(s) for
notifications and whether they are send as a trap or an inform. The
MIBs also define which transport is to be used. Hence, I don't think
we need to do anything here.

>> It might be useful to add more text to this section about when to
>> choose TCP and when not. Any proposals?

Luca> UDP MUST be supported whereas TCP is optional (i.e. a)
Luca> applications may support it b) application that support TCP may
Luca> decide to switch back to UDP when necessary [see later]). In
Luca> general originators should be free to use the transport they
Luca> assume is the best for a certain situation.  Nevertheless, TCP
Luca> costs more than UDP in terms of resource usage.  Therefore
Luca> "TCP-enhanced" applications may choose to switch back (i.e.
Luca> refuse new TCP connections) to UDP whenever necessary (i.e. too
Luca> many open connections, resource shortage).

Makes sense. However, I am not really sure whether we should make any
statements about what is mandatory and what not in this statement.
Based on your comment, I have added the following text to section 3:

:  In general, originators of request/response transactions are free to
:  use the transport they assume is the best in a given situation.
:  However, since TCP costs in terms of resource usage are higher than
:  UDP costs, applications using SNMP over TCP may choose to switch back
:  to UDP (i.e. by refusing new TCP connections) whenever necessary
:  (i.e. too many open TCP connections).

(The wordings can probably improved but I think this covers the point
you were trying to make.)

IANA registered a node in the experimental tree for use by the NMRG. I
added a MODULE-IDENTITY which actually turns the MIB fragment into a
valid MIB module. I also added a references section. (I did not add
definitions for IPv6 since we do not have a resolution for SNMP over
UDP/IPv6 yet and I do not consider this important for our current
goal.) Below is the revised document for review.

							Juergen

Network Working Group                           J. Schoenwaelder, Editor
Internet-Draft                                           TU Braunschweig
Expires October 1999                                       20 April 1999

                    SNMP over TCP Transport Mapping

                   <draft-irtf-nmrg-snmp-tcp-01.txt>

Status of this Memo

   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC 2026.  Internet-Drafts are
   working documents of the Internet Engineering Task Force (IETF), its
   areas, and its working groups.  Note that other groups may also
   distribute working documents as Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html

   Distribution of this document is unlimited. Please send comments to
   the Network Management Research Group, <[email-masked]>.

Copyright Notice

   Copyright (C) The Internet Society (1999).  All Rights Reserved.

Abstract

   This memo defines a transport mapping for using the Simple Network
   Management Protocol (SNMP) over TCP. The transport mapping defined in
   this memo can be used with any version of SNMP.

J. Schoenwaelder                                                [Page 1]

Internet-Draft      SNMP over TCP Transport Mapping           April 1999

   Table of Contents

   1 Introduction .................................................    3
   2 Definitions ..................................................    3
   3 SNMP over TCP ................................................    4
   3.1 Serialization ..............................................    4
   3.2 Well-known Values ..........................................    4
   3.3 Connection Management ......................................    5
   4 Acknowledgments ..............................................    5
   5 References ...................................................    6
   6 Editor's Address .............................................    6
   7 Full Copyright Statement .....................................    7

J. Schoenwaelder                                                [Page 2]

Internet-Draft      SNMP over TCP Transport Mapping           April 1999

1.  Introduction

   This memo defines a transport mapping for using the Simple Network
   Management Protocol (SNMP) over TCP. The transport mapping defined in
   this memo can be used with any version of SNMP. This document extends
   the transport mappings defined in RFC 1906 [1].

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [2].

2.  Definitions

   IRTF-NMRG-SNMP-TM DEFINITIONS ::= BEGIN

   IMPORTS
       MODULE-IDENTITY, OBJECT-IDENTITY, experimental
           FROM SNMPv2-SMI
       TEXTUAL-CONVENTION
           FROM SNMPv2-TC;

   nmrgSnmpDomains MODULE-IDENTITY
       LAST-UPDATED "9904201100Z"
       ORGANIZATION "IRTF Network Management Research Group"
       CONTACT-INFO
           "Juergen Schoenwaelder
            TU Braunschweig
            Bueltenweg 74/75
            38106 Braunschweig
            Germany
            Tel: +49 531 391-3283
            E-mail: [email-masked]"
       DESCRIPTION
           "This MIB module defines the SNMP over TCP transport mapping."
       ::= { experimental nmrg(91) 1 }

   -- SNMP over TCP over IPv4

   snmpTCPDomain   OBJECT-IDENTITY
       STATUS      current
       DESCRIPTION
           "The SNMP over TCP/IPv4 transport domain. The corresponding
            transport address is of type SnmpTCPAddress."
       ::= { nmrgSnmpDomains 6 }   -- matches first unused value
                                   -- below snmpDomains

   SnmpTCPAddress ::= TEXTUAL-CONVENTION
       DISPLAY-HINT "1d.1d.1d.1d/2d"
       STATUS       current

J. Schoenwaelder                                                [Page 3]

Internet-Draft      SNMP over TCP Transport Mapping           April 1999

       DESCRIPTION
               "Represents a TCP/IPv4 address:

                  octets   contents        encoding
                   1-4     IP-address      network-byte order
                   5-6     TCP-port        network-byte order
               "
       SYNTAX      OCTET STRING (SIZE (6))
   END

3.  SNMP over TCP

   SNMP over TCP is an optional transport mapping. Implementors are
   encouraged to support SNMP over TCP whenever possible because this
   enables applications to use more efficient bulk MIB data transfers.

   Note, the originator of a request/response transaction chooses the
   transport protocol for the whole transaction. The transport protocol
   MUST NOT change during a transaction.

   In general, originators of request/response transactions are free to
   use the transport they assume is the best in a given situation.
   However, since TCP costs in terms of resource usage are higher than
   UDP costs, applications using SNMP over TCP may choose to switch back
   to UDP (i.e. by refusing new TCP connections) whenever necessary
   (i.e. too many open TCP connections).

3.1.  Serialization

   Each instance of a message is serialized into a single BER encoded
   message, using the algorithm specified in Section 8 of RFC 1906 [1].
   The BER encoded message is then send over a TCP connection.

   Note, it is possible to exchange multiple SNMP request/response pairs
   over a single TCP connection. The length field in the BER encoded
   SNMP message is used to separate multiple requests send over a single
   TCP connection.

3.2.  Well-known Values

   It is suggested that administrators configure their SNMP entities
   acting in an agent role to listen on TCP port 161 for incoming
   connections.  Further, it is suggested that notification sinks be
   configured to listen on TCP port 162.

   When an SNMP entity uses this transport mapping, it must be capable
   of accepting messages that are at least 484 octets in size.

J. Schoenwaelder                                                [Page 4]

Internet-Draft      SNMP over TCP Transport Mapping           April 1999

   Implementation of larger values is encouraged whenever possible.

3.3.  Connection Management

   The use of TCP connections introduces costs. Connection establishment
   and shutdown causes additional traffic on the wire. Further,
   maintaining open connections binds resources in the network layer of
   the underlying operating system.

   TCP connections should therefore only be used when the size of the
   data transferred would otherwise cause large latencies due to small
   UDP packet sizes and an increased number of interactions.

   Both, an SNMP entity in the agent role and an SNMP entity in the
   manager role, are allowed to close the connections at any point in
   time. This ensures that SNMP entities can control their resource
   usage and shutdown TCP connections that are not used. Note, SNMP
   engines are not expected to process any outstanding requests if the
   underlying TCP connection has been closed. A no response error
   condition SHOULD be signalled for outstanding requests for command
   generator applications if the TCP connection is closed.

4.  Acknowledgments

   The definitions in this memo are inspired by definitions found in RFC
   1906 [1]. This document is the result of the Network Management
   Research Group (NMRG). Special thanks go to the following
   participants for their comments and contributions:

   Luca Deri, Jean-Philippe Martin-Flatin, Aiko Pras, Ron Sprenkels,
   Bert Wijnen

J. Schoenwaelder                                                [Page 5]

Internet-Draft      SNMP over TCP Transport Mapping           April 1999

5.  References

[1]  Case, J., McCloghrie, K., Rose, M., and S. Waldbusser, "Transport
     Mappings for Version 2 of the Simple Network Management Protocol
     (SNMPv2)", RFC 1906, SNMP Research, Inc., Cisco Systems, Inc.,
     Dover Beach Consulting, Inc., International Network Services,
     January 1996.

[2]  Bradner, S., "Key words for use in RFCs to Indicate Requirement
     Levels", BCP 14, RFC 2119, Harvard University, March 1997.

6.  Editor's Address

     Juergen Schoenwaelder             Email: [email-masked]
     TU Braunschweig                     Tel: +49 531 391-3283
     Bueltenweg 74/75
     38106 Braunschweig
     Germany

J. Schoenwaelder                                                [Page 6]

Internet-Draft      SNMP over TCP Transport Mapping           April 1999

7.  Full Copyright Statement

   Copyright (C) The Internet Society (1998). All Rights Reserved.

   This document and translations of it may be copied and furnished to
   others, and derivative works that comment on or otherwise explain it
   or assist in its implementation may be prepared, copied, published
   and distributed, in whole or in part, without restriction of any
   kind, provided that the above copyright notice and this paragraph are
   included on all such copies and derivative works.  However, this
   document itself may not be modified in any way, such as by removing
   the copyright notice or references to the Internet Society or other
   Internet organizations, except as needed for the  purpose of
   developing Internet standards in which case the procedures for
   copyrights defined in the Internet Standards process must be
   followed, or as required to translate it into languages other than
   English.

   The limited permissions granted above are perpetual and will not be
   revoked by the Internet Society or its successors or assigns.

   This document and the information contained herein is provided on an
   "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
   TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
   BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
   HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
   MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

J. Schoenwaelder                                                [Page 7]

To: [email-masked]
Cc: Wes Hardaker <[email-masked]>, [email-masked]
Subject: Re: [nmrg] Wildcarding & Query Processing
Organization: U.C.Davis, Information Technology - D.C.A.S.
From: Wes Hardaker <[email-masked]>
Date: 13 Apr 1999 09:58:49 -0700
Lines: 13
Sender: [email-masked]

>>>>> On Wed, 07 Apr 1999 19:51:21 +0100, Luca Deri <[email-masked]> said:

Luca> In addition how would you handle 'get-subtree(1.3.*.161)'?

I'd yell at the person that made such a lousy request...

More seriously, it probably should be implementation dependent what
it's willing to handle?

-- 
Wes Hardaker
Distributed Computing Analysis and Support
University of California at Davis

Date: Fri, 9 Apr 1999 18:47:10 +0200
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
CC: Network Management Research Group <[email-masked]>
Subject: [nmrg] Re: comments on draft-irtf-nmrg-snmp-tcp-00.txt
Sender: [email-masked]

>>>>> David Waitzman writes:

David> cisco implemented SNMP over TCP a long time ago! (probably 1991
David> or 1992) They also added a gettable kind of primitive to it.  I
David> played with it a bit but did not get funding to really exploit
David> it for management.  I don't know when they removed the support
David> for the feature.

I know, this stuff is not really something new. However, I think 
its time now to get this on the table and to specify a reasonable
solution. Our goal is to work out something which we can put into
the major free SNMP implementations. You can find more about this
in the latest SimpleTimes issue.

David> I suggest some security considerations.  How about IPSec and
David> noAuthNoPriv instead of doing "plain" IP and authPriv?  It
David> might be more efficient.

I have been told that host security with IPSec is still not widely
deployed and that there are still open question on how to implement a
deployable key infrastructure. BTW, one of my master students will
look closer ar this option.
							Juergen
-- 
Juergen Schoenwaelder  [email-masked] http://www.cs.tu-bs.de/~schoenw
Technical University Braunschweig, Dept. Operating Systems & Computer Networks
Bueltenweg 74/75, 38106 Braunschweig, Germany.        (Tel. +49 531 / 391 3289)

Date: Fri, 9 Apr 1999 18:37:54 +0200
From: Juergen Schoenwaelder <[email-masked]>
To: Network Management Research Group <[email-masked]>
Subject: [nmrg] [[email-masked]: comments on draft-irtf-nmrg-snmp-tcp-00.txt]
Sender: [email-masked]

I just received this comment.

							Juergen
------- Start of forwarded message -------
Sender: [email-masked]
Date: Fri, 09 Apr 1999 12:11:59 -0400
From: David Waitzman <[email-masked]>
Organization: IronBridgeNetworks
X-Accept-Language: en
To: Juergen Schoenwaelder <[email-masked]>
Subject: comments on draft-irtf-nmrg-snmp-tcp-00.txt

Hi,

Sorry this took so long.

cisco implemented SNMP over TCP a long time ago! (probably 1991 or 1992)
They also added a gettable kind of primitive to it.  I played with it a bit
but did not get funding to really exploit it for management.
I don't know when they removed the support for the feature.

I suggest some security considerations.  How about IPSec and noAuthNoPriv
instead of doing "plain" IP and authPriv?  It might be more efficient.

- -- 
- -david waitzman       work: [email-masked]     play: [email-masked]
IronBridge Networks / 55 Hayden Avenue         Voice: 781 372 8161
Lexington MA  02421                            Fax:   781 372 8090
------- End of forwarded message -------

Date: Thu, 8 Apr 1999 11:21:21 +0200
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
CC: [email-masked]
Subject: Re: [nmrg] Wildcarding & Query Processing
Sender: [email-masked]

>>>>> Wes Hardaker writes:

Wes> I don't think the complexity is as bad as you think (I think its
Wes> hard for other reasons...)

Wes> Internally to your agent, the simplest way to implement it is to
Wes> break the oid at the * and do a getnext for everything beyond it
Wes> and only return the oid/values that match the filter.  This is a
Wes> trivial while() type loop around you're already existing agent
Wes> architecture that can handle getnexts...

This is the simplest way to implement it, but it has poor performance
in the general case. The VACM MIB has the same problem. I think some
toolkit providers are doing quite tricky things in order to skip parts
of the MIB that are known to be outside of a given view. This can be a
big performance gain in subagent environments, as Luca already pointed
out. One would try to do the same for wildcards (which are IMHO very
close to temporary views).

The nasty thing is that you have a temporary view defined by the
wildcards plus the normal view defined by VACM. To really optimize the
number of method functions you have to call, you would need to merge
those views in order to find out which method function needs to be
called. This can be tricky.
							Juergen
-- 
Juergen Schoenwaelder  [email-masked] http://www.cs.tu-bs.de/~schoenw
Technical University Braunschweig, Dept. Operating Systems & Computer Networks
Bueltenweg 74/75, 38106 Braunschweig, Germany.        (Tel. +49 531 / 391 3289)

Date: Wed, 07 Apr 1999 19:51:21 +0100
From: Luca Deri <[email-masked]>
Reply-To: [email-masked]
Organization: Finsiel S.p.A.
To: Wes Hardaker <[email-masked]>
CC: [email-masked]
Subject: Re: [nmrg] Wildcarding & Query Processing
Sender: [email-masked]

Wes,
I agree with you that your solution isn't too complicated to implement
if you have a monolithic agent. Nevertheless you have to cover cases
like multiple SNMP agents connected via SMUX. In addition how would you
handle 'get-subtree(1.3.*.161)'? Don't you think you should return
something like 'complexity limitation' or something else? This implies
that some lmitations to the use of '*' are imposed hence added
complexity to the protocol. That's why I think that isn't worth the
effort.

Cheers, Luca.

-- 
Luca Deri		 Finsiel S.p.A.
Via Matteucci 34/B	 56124 Pisa, Italy.
Ph. +39/050/968.639      Fax. +39/050/968.626
Email: [email-masked] WWW: http://www.tlcpi.finsiel.it/~deri/
Software is about stuff, about getting hands dirty - Jim Coplien

To: [email-masked]
Cc: Wes Hardaker <[email-masked]>, [email-masked]
Subject: Re: [nmrg] Wildcarding & Query Processing
Organization: U.C.Davis, Information Technology - D.C.A.S.
From: Wes Hardaker <[email-masked]>
Date: 07 Apr 1999 10:10:08 -0700
Lines: 27
Sender: [email-masked]

>>>>> On Wed, 07 Apr 1999 10:11:45 +0100, Luca Deri <[email-masked]> said:

>> 2) Wildcarding "in the middle" of OIDs:
>> 
>> In some cases, it might be desirable to do wildcarding somewhere in
>> the middle of an OID (e.g. tcpConnRemPort.10.11.12.13.161.*.161).

Luca> I don't like the idea because in general this costs a lot in
Luca> terms of complexity. In the tcpConnRemPort example you filter on
Luca> adjacent values.  Nevertheless if you shift right the '*' (e.g.
Luca> tcpConnRemPort.*.161) the situation is quite different. If you
Luca> shift the '*' even further you will end up selecting entries
Luca> from different tables/MIBs.

I don't think the complexity is as bad as you think (I think its hard
for other reasons...)

Internally to your agent, the simplest way to implement it is to break 
the oid at the * and do a getnext for everything beyond it and only
return the oid/values that match the filter.  This is a trivial
while() type loop around you're already existing agent architecture
that can handle getnexts...

-- 
Wes Hardaker
Distributed Computing Analysis and Support
University of California at Davis

Date: Wed, 07 Apr 1999 10:11:45 +0100
From: Luca Deri <[email-masked]>
Reply-To: [email-masked]
Organization: Finsiel S.p.A.
To: Wes Hardaker <[email-masked]>
CC: [email-masked]
Subject: Re: [nmrg] Wildcarding & Query Processing
Sender: [email-masked]

Hi all,

> 1) Wildcarding at the end of OIDs:
> 
>    All SNMP frameworks allow to do wildcarding at the end of OIDs. You
>    can retrieve tcpConnRemPort.10.11.12.13.161.* easily with a series
>    of getnext/getbulk operations. Note, the proposed getsubtree will
>    also allow wildcarding at the end of OIDs.
> 
>    Comment: Many MIB tables have been designed so that the indexing
>    order makes sense in the most frequent cases. No action required.
I agree. in any case I believe this is a programming issue and not a
SNMP problem. For instance think about the popular snmpwalk tool. It's
meant for '*' support.

> 2) Wildcarding "in the middle" of OIDs:
> 
>    In some cases, it might be desirable to do wildcarding somewhere in
>    the middle of an OID (e.g. tcpConnRemPort.10.11.12.13.161.*.161).
>    There is no support for such an operation in any SNMP protocol.
>    However, you can use VACM to construct a MIB view which actually
>    filters away everything except those entries that match the
>    wildcard. The downside of this approach is that it is a) pretty
>    complicated to setup a MIB view just to do some filtering and b)
>    that usually write access to VACM tables will be limited to the
>    security administrator. Therefore, this solution is impractical.
> 
>    Comment: Allowing arbitrary wildcarding in the protocol may be
>    useful in some tables. The question is whether there is a simple
>    way to encode the wildcards in the PDU.
I don't like the idea because in general this costs a lot in terms of
complexity. In the tcpConnRemPort example you filter on adjacent values.
Nevertheless if you shift right the '*' (e.g.  tcpConnRemPort.*.161) the
situation is quite different. If you shift the '*' even further you will
end up selecting entries from different tables/MIBs. This is not what we
want I believe hence restrictions have to be introduced. Bottom line: I
think all this isn't worth the effort.

> 3) Query processing on conceptual tables:
> 
>    In some cases, it might be desirable to define queries against
>    arbitrary columns in a conceptual table. Luca described an example
>    where it would be valuable to only retrieve those parts of a table
>    where the value of a particular column satisfies a boolean
>    expression.
> 
>    Comment: How complex do these expressions get? Is it just boolean
>    operations and comparisons? Or do we need additional primitives or
>    even primitives that require state information (like the delta
>    values in the expression MIB)? Further, how do you submit the
>    query? What changes are needed in the protocol and the average
>    agent implementation? Is SNMP the right choice for such a query
>    interface?
As soon as I will have some spare time I will describe my idea in
detail. Stay tuned.

> I think we agreed in Lausanne that we want to solve the bulk transfer
> problem first so that we can build e.g. a query processor on top of
> it. The motivation was to keep the average SNMP agents simple. This
> allows to run the query processor on a resource rich system, talking
> to several resource limited SNMP agents on the (local) network.
This statement is correct. Nevertheless please do not forget that you'll
''compress'' your SNMP traffic much more if you avoid sending/receiving
unnecessary data that compressing OID headers as proposed in the past.

Cheers, Luca.

-- 
Luca Deri		 Finsiel S.p.A.
Via Matteucci 34/B	 56124 Pisa, Italy.
Ph. +39/050/968.639      Fax. +39/050/968.626
Email: [email-masked] WWW: http://www.tlcpi.finsiel.it/~deri/
Software is about stuff, about getting hands dirty - Jim Coplien

To: Juergen Schoenwaelder <[email-masked]>
Cc: [email-masked]
Subject: Re: [nmrg] Wildcarding & Query Processing
Organization: U.C.Davis, Information Technology - D.C.A.S.
From: Wes Hardaker <[email-masked]>
Date: 06 Apr 1999 10:18:19 -0700
Lines: 87
Sender: [email-masked]

>>>>> On Tue, 6 Apr 1999 16:40:44 +0200, Juergen Schoenwaelder <[email-masked]> said:

Wes> get-subtree(tcpConnTable.tcpConnEntry.*.10.11.12.14)

Ron> Now, something totally different would be to try to do this:
Ron> get-subtree(tcpConnRemPort.10.11.12.13.161.*.161).

Those can be done in the same way, pretty much.  The * is filled in
easily using a similar viewmask system as in the vacm tables (ick).
It works here in both these cases because you have a definitive length 
associated with the entry (ok, ipv6 will affect this).

What gets worse is when you want to do (and you will want to at some point):

  get-subtree(usmUserEntry.usmUserName.*.3.119.101.115)

(gets all users named "wes" for all engineIDs)

This problem is larger than just something like the get-subtree
problem.  The VACM suffers badly from the same problem.  I've actually 
gotten about half of a proposal written for a new ACM module that
solves these issues, with a minor jump in complexity, but I'm not sure 
it applies all that easily to arguments to a new PDU type like what
might happen here.  Specifically, it gets really complex to express
variable length pieces in an index.  What I've done in my ACM module
(here out referred to as bacm (b=better)), is actually treat the oid as
a set of sections like it really is (base-oid . index . [index...(s)]).  

To do something similar here, we'd have to come up with a scheme for
representing a variable length mask section as well.  I don't know
that we want to go that far down the line, but I think we should start 
with the hardest problem and draw the line where we want based on what 
we want to solve.

(At this point, I'll say I didn't get enough sleep last night, so
excuse my rambling).

Juergen> 2) Wildcarding "in the middle" of OIDs:

2a) In the middle with a known length (the tcpconns examples)
2b) In the middle with an unknown length (the usmUser example)

Juergen> However, you can use VACM to construct a MIB view which actually
Juergen> filters away everything except those entries that match the
Juergen> wildcard.

I've wondered if the VACM et al should really be using their own table 
for this.  Wouldn't a generic VIEW table be useful for cases like ours 
as well as ACM cases as well as ??? cases?  Again, though, ours is
harder, because it needs to be stored in a request and I don't know if 
you want to include an entire complex table entry in a request?  IE,
if you were using the VACM to select what you wanted in your view,
would you:

1) set the vacm up remotely using sets to create a view that described 
   the data you wanted to retrieve. This would probably entail setting 
   up multiple sections of the viewTreeFamilyTable.

2) create the same view, but include it as a parameter to the
   get-subtree() request?

(1 is better if you're going to access the view repeatedly from a
polling manager, but 2 is better for a single time request.  They are
both large in packet size either way though).

The approach I was suggesting previously was more along the lines of
no-variable length sections, so that:

  get-subtree(usmUserEntry.usmUserName.*.3.119.101.115)

That wouldn't be possible...  But:

  get-subtree(tcpConnRemPort.10.11.12.13.161.*.161)

Would be:

  get-subtree(OID=tcpConnRemPort.10.11.12.13.161.0.0.0.0.161, mask=0xfffe10)

(if I counted right)

Of course, I'd be the first person that wanted the variable length
usmUserName request.  In fact, I know I would...

-- 
Wes Hardaker
Distributed Computing Analysis and Support
University of California at Davis

Date: Tue, 6 Apr 1999 16:40:44 +0200
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
Subject: [nmrg] Wildcarding & Query Processing
Sender: [email-masked]

There have been several suggestions about wildcarding in the last few
days. I am trying to get the key points together so that we can look
at it and see if someone wants to dive deeper into it.

Wes Hardaker <[email-masked]> wrote:

Wes> My one modification to the get-subtree request is that it allow
Wes> for wildcarding somehow (possibly through an optional parameter).
Wes> Much of the time, people want to get an entire row from a table,
Wes> not a column.  Lets say we wanted get all the tcp entries in the
Wes> connection table for a given ip address (10.11.12.13, EG):
Wes>
Wes> get-subtree(tcpConnTable.tcpConnEntry.*.10.11.12.14)

Luca Deri <[email-masked]> wrote:

Luca> What I need is 'get-subtree(dot1dTpFdbPort > 127)' where 127 is
Luca> the latest port ID to which hubs are connected.

[dot1dTpFdbPort is defined in the BRIDGE-MIB (RFC 1493). Note,
 dot1dTpFdbPort is not part of the instance identifier.]

Ron Sprenkels <[email-masked]> writes:

Ron> Now, something totally different would be to try to do this:
Ron> get-subtree(tcpConnRemPort.10.11.12.13.161.*.161).

I think we are talking about three related but different things
here:

1) Wildcarding at the end of OIDs:

   All SNMP frameworks allow to do wildcarding at the end of OIDs. You
   can retrieve tcpConnRemPort.10.11.12.13.161.* easily with a series
   of getnext/getbulk operations. Note, the proposed getsubtree will
   also allow wildcarding at the end of OIDs.

   Comment: Many MIB tables have been designed so that the indexing
   order makes sense in the most frequent cases. No action required.

2) Wildcarding "in the middle" of OIDs:

   In some cases, it might be desirable to do wildcarding somewhere in
   the middle of an OID (e.g. tcpConnRemPort.10.11.12.13.161.*.161).
   There is no support for such an operation in any SNMP protocol.
   However, you can use VACM to construct a MIB view which actually
   filters away everything except those entries that match the
   wildcard. The downside of this approach is that it is a) pretty
   complicated to setup a MIB view just to do some filtering and b)
   that usually write access to VACM tables will be limited to the
   security administrator. Therefore, this solution is impractical.

   Comment: Allowing arbitrary wildcarding in the protocol may be
   useful in some tables. The question is whether there is a simple
   way to encode the wildcards in the PDU.

3) Query processing on conceptual tables:

   In some cases, it might be desirable to define queries against
   arbitrary columns in a conceptual table. Luca described an example
   where it would be valuable to only retrieve those parts of a table
   where the value of a particular column satisfies a boolean
   expression.

   Comment: How complex do these expressions get? Is it just boolean
   operations and comparisons? Or do we need additional primitives or
   even primitives that require state information (like the delta
   values in the expression MIB)? Further, how do you submit the
   query? What changes are needed in the protocol and the average
   agent implementation? Is SNMP the right choice for such a query
   interface?

I think we agreed in Lausanne that we want to solve the bulk transfer
problem first so that we can build e.g. a query processor on top of
it. The motivation was to keep the average SNMP agents simple. This
allows to run the query processor on a resource rich system, talking
to several resource limited SNMP agents on the (local) network.

Anyway, I like to encourage everyone on this list to "think loud" and
to make proposals how these things may work. This is a "research
group" and we are expected to experiment with ideas. If someone has
ideas or even answers to some of the questions raised above, please
let us know.
							Juergen
-- 
Juergen Schoenwaelder  [email-masked] http://www.cs.tu-bs.de/~schoenw
Technical University Braunschweig, Dept. Operating Systems & Computer Networks
Bueltenweg 74/75, 38106 Braunschweig, Germany.        (Tel. +49 531 / 391 3289)

Date: Thu, 1 Apr 1999 13:02:25 +0200
From: Juergen Schoenwaelder <[email-masked]>
To: Network Management Research Group <[email-masked]>
Subject: [nmrg] Wes Hardaker joins the NMRG
Sender: [email-masked]

I have added Wes Hardaker to the NMRG mailing list. I am sure he likes
to comment on the wildcarding thread just started on this list.

Wes also wrote that he might be able to prototype modifications to the
SNMP protocol. I think that is all good news.

							Juergen
-- 
Juergen Schoenwaelder  [email-masked] http://www.cs.tu-bs.de/~schoenw
Technical University Braunschweig, Dept. Operating Systems & Computer Networks
Bueltenweg 74/75, 38106 Braunschweig, Germany.        (Tel. +49 531 / 391 3289)

Date: Thu, 01 Apr 1999 10:48:07 +0200
From: Ron Sprenkels <[email-masked]>
Organization: University of Twente
To: [email-masked]
CC: Network Management Research Group <[email-masked]>
Subject: [nmrg] Re: [beam] get-subtree Issue
Sender: [email-masked]

Luca,

Just a few of my thoughts on get-subtree:

> Wes>    CURRENT: get-subtree(
> Wes> tcpConnTable.tcpConnEntry.tcpConnState.10.11.12.14,
> Wes> tcpConnTable.tcpConnEntry.tcpConnLocalAddress.10.11.12.14,
> Wes> tcpConnTable.tcpConnEntry.tcpConnLocalPort.10.11.12.14,
> Wes> tcpConnTable.tcpConnEntry.tcpConnRemAddress.10.11.12.14,
> Wes> tcpConnTable.tcpConnEntry.tcpConnRemPort.10.11.12.14)
> Wes>    WITH WILDCARDS: get-subtree(
> Wes> tcpConnTable.tcpConnEntry.*.10.11.12.14)

I think that in this particular example from Wes, expanding the wildcard at
the manager side is still possible. From the MIB spec, the manager can find
out that the only possible values for the * in
tcpConnTable.tcpConnEntry.*.10.11.12.14 are those 5, tcpConnState through
tcpConnRemPort. So at the API level, this kind of wildcarding can be
supported, and it can be expanded into a get-subtree protocol operation
with a varbindlist of 5 varbinds: tcpConnTable.tcpConnEntry.tcpConnState
through tcpConnTable.tcpConnEntry.tcpConnRemPort.

In short, I don't think we need wildcards if all of the OID-pieces that
will match it can just be read from the MIB-spec. This is in my view pure a
API matter, not a protocol operation matter.

Now, something totally different would be to try to do this:
get-subtree(tcpConnTable.tcpConnEntry.tcpConnRemPort.10.11.12.13.161.*.161).
In this case, we have a wildcard spec that contains a wildcard (the *)
_not_ at the end of the spec, but in the middle. So
fixed-part.*.fixed-part. 
This get-subtree call should return all tcpConnRemPort varbinds for the TCP
connections from (local address 10.11.12.13, port 161) to (any remote
address, port 161).
The data to be returned for this is _not_ a subtree of the MIB! So this
would really be introducing a operator with semantics that are radically
different from what we have now, and also from the get-subtree discussed so
far.

Yet another case would be a getsubtree on e.g. fixed-part.*.fixed-part.*
or  fixed-part.*.fixed-part.*.fixedpart. You get the idea. What we really
have here is not wildcarding, but _filtering_ on specific portions of the
OID, with the only allowed filter being "OID portion exactly matches
constant". Only in the case of a wildcard of the form fixed-part.* this
reduces to a real subtree of the MIB.

> board 1: hub   (ports 0-63)
> board 2: hub   (ports 64-127)
> board 3: hosts (ports 128-...)
> board 4: hosts (ports ...)
> board 5: hosts (ports ....)
> board 6: hosts (ports ..-383)
> 
> Now, 6 (boards) * 64 (logical sockets) = 384 entries. I don't care of
> hosts connected to hubs, but I just want to know to what port are
> connected hosts directly. As you can see get-subtree is useless because
> I can't use wildcards with entries of the 'dot1dTpFdbPort' table. What I
> need is  'get-subtree(dot1dTpFdbPort > 127)' where 127 is the latest
> port ID to which hubs are connected.

This is indeed a different case, that cannot be solved using any of the
forms of get-subtree or even wildcarding we discussed so far. What you need
here is a logical operator (greater then, or >, in this case) on OID
portions. 

I don't know what the MIB spec is that you are referring to (with
dot1dTpFdbPort in it). In any case, I see to ways to handle this situation:
1: define and implement a protocol operation that does a 'get-subtree',
with a logical operator on a portion of the OID. In your case, the ">"
operator on a single integer (dot1dTpFdbPort). Pro for this is that you
move less data around over the network. Con is that you need to define,
implement and get deployed a operater with in the SNMP mindset
unprecedented semantics.

2: define getsubtree like discussed up to now, so a operation that can
retrieve multiple subtrees of the form fixed-part.* in a single protocol
operation. Make it run over (a efficient form of) TCP, so you have to move
only a couple of big chucks of data over the network. Let the manager throw
away the stuff it is not interested in. Pro: I think it is less dificult to
implement, and less difficult to convice 'IETF/SNMP minded' people/vendors.
Con: you move data over the network that you are not interested in.

My feeling is that 2: is the way to go.

> Bottom line: shall I write a proposal or even a draft internet-draft
> that describes my idea?

Writing down ideas is always good. However, I think it might even be more
productive to discuss your idea yet a bit more over the list; I for one am
still not completely sure I exactly understand what you mean. What do you
think?

Now, related to this, I think there might even be some problems
implementing the 'regular' get-subtree in agents in such a way that it
retrieves exactly on a row-by-row basis the values from the MIB
instrumentation (or from the agent-x subagent, for that matter). But I
think we first need to converge on some defined semantics for our new
operation, before we get into that.

I'm interested to hear what you and all of the other guys here think about
this.

Ron.

--------------------------------------------------------------------------
Ron Sprenkels  [email-masked]   http://www.cs.utwente.nl/~sprenkel
University of Twente, Department of Computer Science, TSS Management group 
P.O. Box 217, 7500 AE Enschede, The Netherlands.    (Tel. +31 53 489 4663)