Subject: [nmrg] Re: please post <draft-schoenw-nrmg-snmp-measure-01.txt>
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
Date: Tue, 21 Mar 2006 17:54:20 +0100
List-Id: Network Management Research Group <nmrg.ibr.cs.tu-bs.de>

On Tue, Mar 21, 2006 at 05:36:10PM +0100, Juergen Schoenwaelder wrote:

> 
>                        SNMP Traffic Measurements
>                  draft-schoenw-nrmg-snmp-measure-01.txt

[...] 
 
> Abstract
> 
>    The Simple Network Management Protocol (SNMP) is widely deployed to
>    monitor, control and configure network elements.  Even though the
>    SNMP technology is well documented, it remains relatively unclear how
>    SNMP is used in practice and what typical SNMP usage patterns are.
>    This document proposes to carry out large scale SNMP traffic
>    measurements in order to develop a better understanding how SNMP is
>    used in real world production networks.  It describes the motivation,
>    the measurement approach, and the tools and data formats needed to
>    carry out such a study.

I have posted an updated ID for the SNMP traffic measurements we are
working on. We have made some progress on the tool chain and we are
collecting the first traces. Note that there is a now a European
project called EMANICS <http://www.emanics.org/> which will help us to
collect more traces and to work on the analysis tools as a joint
effort with other interested parties.

I believe that this work is highly relevant for the NMRG and I hereby
propose to turn this Internet-Draft which is still an individual
submission into an NMRG sponsored Internet-Draft. Please let me know
what you think about this and especially if you have concerns about
doing so.

I am currently considering to organize an NMRG meeting next to the
Summer IETF (July 9-14) in Montreal in order to discuss this activity
with IETF folks. We expect to have first results available by that
time. The meeting duration will depend on the material we have by the
scheduling time, either just a standard two hour time slot or we could
opt for more time on Friday or Sunday. Again, I like to have feedback
on the general idea to discuss this at the next IETF in Montreal
(although it might seem a bit early for some of you to already plan
the next IETF while the current one just started ;-).

/js

-- 
Juergen Schoenwaelder		    International University Bremen
<http://www.eecs.iu-bremen.de/>	    P.O. Box 750 561, 28725 Bremen, Germany

Date: Tue, 21 Mar 2006 17:36:10 +0100
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
Mail-Followup-To: [email-masked], [email-masked]
Cc: [email-masked]
Subject: [nmrg] please post <draft-schoenw-nrmg-snmp-measure-01.txt>
Reply-To: [email-masked]

--Qxx1br4bt0+wmkIi
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

Hi,

please post the attached document as <draft-schoenw-nrmg-snmp-measure-01.txt>.

Thanks,

/js

-- 
Juergen Schoenwaelder		    International University Bremen
<http://www.eecs.iu-bremen.de/>	    P.O. Box 750 561, 28725 Bremen, Germany

--Qxx1br4bt0+wmkIi
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="measure.txt"

NMRG                                                    J. Schoenwaelder
Internet-Draft                           International University Bremen
Expires: September 22, 2006                               March 21, 2006

                       SNMP Traffic Measurements
                 draft-schoenw-nrmg-snmp-measure-01.txt

Status of this Memo

   By submitting this Internet-Draft, each author represents that any
   applicable patent or other IPR claims of which he or she is aware
   have been or will be disclosed, and any of which he or she becomes
   aware will be disclosed, in accordance with Section 6 of BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire on September 22, 2006.

Copyright Notice

   Copyright (C) The Internet Society (2006).

Abstract

   The Simple Network Management Protocol (SNMP) is widely deployed to
   monitor, control and configure network elements.  Even though the
   SNMP technology is well documented, it remains relatively unclear how
   SNMP is used in practice and what typical SNMP usage patterns are.
   This document proposes to carry out large scale SNMP traffic
   measurements in order to develop a better understanding how SNMP is
   used in real world production networks.  It describes the motivation,
   the measurement approach, and the tools and data formats needed to
   carry out such a study.

Schoenwaelder          Expires September 22, 2006               [Page 1]

Internet-Draft          SNMP Traffic Measurements             March 2006

Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
   2.  Measurement Approach . . . . . . . . . . . . . . . . . . . . .  4
     2.1.  Capturing Traffic Traces . . . . . . . . . . . . . . . . .  4
     2.2.  Converting Traffic Traces  . . . . . . . . . . . . . . . .  5
     2.3.  Filtering Traffic Traces . . . . . . . . . . . . . . . . .  6
     2.4.  Storing Traffic Traces . . . . . . . . . . . . . . . . . .  6
     2.5.  Processing Traffic Traces  . . . . . . . . . . . . . . . .  7
   3.  Analysis of Traffic Traces . . . . . . . . . . . . . . . . . .  8
     3.1.  Basic Statistics . . . . . . . . . . . . . . . . . . . . .  8
     3.2.  Periodic vs. Aperiodic Traffic . . . . . . . . . . . . . .  8
     3.3.  Message Size and Latency Distributions . . . . . . . . . .  8
     3.4.  Concurrency Levels . . . . . . . . . . . . . . . . . . . .  8
     3.5.  Table Retrieval Approaches . . . . . . . . . . . . . . . .  9
     3.6.  Trap-Directed Polling - Myths or Reality?  . . . . . . . .  9
     3.7.  Popular MIB Modules  . . . . . . . . . . . . . . . . . . .  9
     3.8.  Usage of Obsolete Objects  . . . . . . . . . . . . . . . .  9
     3.9.  Encoding Length Distributions  . . . . . . . . . . . . . . 10
     3.10. Counters and Discontinuities . . . . . . . . . . . . . . . 10
     3.11. Spin Locks . . . . . . . . . . . . . . . . . . . . . . . . 10
     3.12. Row Creation . . . . . . . . . . . . . . . . . . . . . . . 10
   4.  Security Considerations  . . . . . . . . . . . . . . . . . . . 11
   5.  Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 12
   6.  References . . . . . . . . . . . . . . . . . . . . . . . . . . 13
     6.1.  Normative References . . . . . . . . . . . . . . . . . . . 13
     6.2.  Informative References . . . . . . . . . . . . . . . . . . 13
   Appendix A.  RELAX NG Schema Definition  . . . . . . . . . . . . . 16
   Appendix B.  CSV Format Definition . . . . . . . . . . . . . . . . 19
   Appendix C.  Sample Perl Analysis Script . . . . . . . . . . . . . 20
   Appendix D.  Trace Description Form  . . . . . . . . . . . . . . . 24
   Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 25
   Intellectual Property and Copyright Statements . . . . . . . . . . 26

Schoenwaelder          Expires September 22, 2006               [Page 2]

Internet-Draft          SNMP Traffic Measurements             March 2006

1.  Introduction

   The Simple Network Management Protocol (SNMP) was introduced in the
   late 1980s [RFC1052] and has since then evolved to what is known
   today as the SNMP version 3 Framework (SNMPv3) [RFC3410].  While SNMP
   is widely deployed, it is not clear which features are being used,
   how SNMP usage differs in different types of networks or
   organizations, which information is frequently queried, and what
   typical SNMP interactions patterns are in real world production
   networks.

   There have been several publications in the recent past dealing with
   the performance of SNMP in general [Pat01], the impact of SNMPv3
   security [DSR01][CT04], or the relative performance of SNMP compared
   to Web Services [PDMQ04][PFGL04].  While these papers are generally
   useful to better understand the impact of various design decisions
   and technologies, some of these papers lack a strong foundation
   because authors typically assume certain SNMP interaction patterns
   without having experimental evidence that the assumptions are
   correct.  In fact, there are many speculations how SNMP is being used
   in real world production networks and how it performs, but no
   systematic measurements have been performed and published so far.

   Many authors use the ifTable of the IF-MIB [RFC2863] or the
   tcpConnTable of the TCP-MIB [RFC4022] as a starting point for their
   analysis and comparison.  Despite the fact that there is no evidence
   that operations on these tables dominate SNMP traffic, it is even
   more unclear how these tables are read and which optimizations are
   done (or not done) by real world applications.  It is also unclear
   what the actual traffic trade-off between periodic polling and more
   aperiodic bulk data retrieval is.  Furthermore, we do not generally
   understand how much traffic is devoted to standardized MIB objects
   and how much traffic deals with proprietary MIB objects and whether
   the operation mix differs between these object classes or between
   different operational environments.

   This document describes an effort to collect SNMP traffic traces in
   order to find answers to some of these questions.  It describes the
   tools that have been developed to allow network operators to collect
   traffic traces and to share them with research groups interested in
   analyzing and modeling network management interactions.

Schoenwaelder          Expires September 22, 2006               [Page 3]

Internet-Draft          SNMP Traffic Measurements             March 2006

2.  Measurement Approach

   This section outlines the process of doing SNMP traffic measurements
   and analysis.  The process consists of the following five basic
   steps:

   1.  Capture raw SNMP traffic traces in pcap capture files.

   2.  Convert the raw traffic traces into a structured machine and
       human readable format.  A suitable XML schema has been developed
       for this purpose which captures all SNMP message details.  In
       addition, another more compact comma separated values (CSV)
       format has been developed which only keeps key information about
       SNMP messages.

   3.  Filter the converted traffic traces to hide or anonymize
       sensitive information.  While the filtering is conceptually a
       separate step, filtering may actually be implemented as part of
       the previous data conversion step for efficiency reasons.

   4.  Submit the filtered traffic traces to a repository from where
       they can be retrieved and analyzed.  Such a repository may be
       public, it may be under the control of a research group, or it
       may be under the control of a network operator who commits to run
       analysis scripts on the repository on behalf of researchers.

   5.  Analyze the traces by creating and executing analysis scripts
       which extract and aggregate information.

   Several of the steps listed above require the involvement of network
   operators supporting the SNMP measurement projects.  In many cases,
   the filtered XML and CSV representation of the SNMP traces will be
   the binding interface between the researchers writing analysis
   scripts and the operators involved in the measurement activity.  It
   is therefore important to have a well defined specification of these
   interfaces.

   This section provides some advice and concrete hints how the steps
   listed above can be carried out efficiently.  Some special tools have
   been developed to assist network operators and researchers so that
   the time spent on supporting SNMP traffic measurement projects is
   limited.  The following sections describe the five steps and some
   tools in more detail.

2.1.  Capturing Traffic Traces

   Capturing SNMP traffic traces can be done using packet sniffers such
   as tcpdump [1], ethereal [2], or similar applications.  Somce care

Schoenwaelder          Expires September 22, 2006               [Page 4]

Internet-Draft          SNMP Traffic Measurements             March 2006

   must be taken to specify pcap filter expressions that match the SNMP
   transport endpoints used to carry SNMP traffic (typically 'udp and
   (port 161 or port 162)').  Furthermore, it is necessary to ensure
   that full packets are capture, that is packets are not truncated
   (tcpdump option -s 0).  Finally, it is necessary to carefully select
   the placement of the capturing probe within the network.  Especially
   on bridged LANs, it is important to ensure that all management
   traffic is captured and that the probe has access to all virtual LANs
   carrying management traffic.  This usually requires to place the
   probe(s) close to the management system(s) and to configure dedicated
   monitoring ports on bridged networks.

   It is recommended to capture at least a full week of data.  Operators
   are encouraged to capture traces over even longer periods of time.
   Tools such as tcpslice [1] or pcapmerge [3] can be used to merge or
   split pcap trace files as needed.

   It is important to note that the raw pcap files should be kept in
   stable storage (e.g., compressed and encrypted on a CD ROM or DVD).
   To verify measurements, it might be necessary to go back to the
   original pcap files if for example bugs in the tools described below
   have been detected and fixed.

   For each captured trace, some meta information should be recorded and
   made available.  Appendix D contains a simple ASCII form that is
   suggested to be used to describe some basic meta data associated with
   a traffic trace.

2.2.  Converting Traffic Traces

   Raw traces in pcap format must be converted into a format that is (a)
   human readable and (b) machine readable for efficient post-
   processing.  Human readability makes it easy for an operator to
   verify that no sensitive data is left in a trace while machine
   readability is needed to efficiently extract relevant information.

   The natural choice here is to use an XML format since XML is human as
   well as machine readable and there are many tools and high-level
   scripting language application programming interfaces (APIs) that can
   be used to process XML documents and to extract meaningful
   information.  However, it should be noted that XML is also pretty
   verbose which increases processing overhead.  In particular, the
   usage of XML streaming APIs is strongly suggested since APIs that
   require an in memory representation of XML documents do not handle
   large traces well.

   Appendix A of this document defines a [OASISRNG] schema for
   representing SNMP traffic traces in XML.  The schema captures all

Schoenwaelder          Expires September 22, 2006               [Page 5]

Internet-Draft          SNMP Traffic Measurements             March 2006

   relevant details of an SNMP messages in the XML format.  Note that
   the XML format retains some information about the original ASN.1/BER
   encoding to support message size analysis.

   A lightweight alternative to the full blown XML representation based
   on comma separated values (CSV) is defined in Appendix B.  The CSV
   format only captures the most essential parts of SNMP messages and is
   thus more compact and faster to process.

   The snmpdump [4] package has been developed to convert raw pcap files
   into XML and CSV format.  The snmpdump program reads either pcap
   files or XML files as input and produces XML files or CSV files as
   output.  Specific elements can be filtered if that is required to
   protect sensitive data.  The current snmpdump implementation is able
   to correctly deal with IPv4 fragments.

2.3.  Filtering Traffic Traces

   Filtering sensitive data can be achieved by manipulating the XML
   representation of an SNMP trace.  Standard XSLT processors such as
   xsltproc [5] can be used for this purpose.  People familiar with Perl
   might also be interested in using the XML::LibXML [6] Perl package to
   manipulate XML documents from within Perl.

   The snmpdump program can filter out sensitive information, e.g., by
   deleting or clearing all XML elements whose name matches a regular
   expression.  Work is in progress to provide data type specific
   anonymization transformations that maintain lexicographic ordering
   for values that appear in instance identifiers [HS06].

2.4.  Storing Traffic Traces

   The pcap traces together with the XML / CSV formatted traces should
   be stored in a stable archive or repository.  Such an archive or
   repository might either be maintained by research groups (e.g., the
   NMRG) or by network operators.  It is of key importance that captured
   traces are not lost or modified as they may form the basis of future
   research projects and may also be needed to verify published research
   results.  Access to the archive might be restricted to those who have
   signed some sort of a non-disclosure agreement.

   Lossless compression algorithms embodied in programs such as gzip or
   bzip2 can be used to compress even large trace files down to a size
   where they can be burned on DVDs for cheap longterm storage.

   It must be stressed again that it is important to keep the original
   pcap traces in addition to the XML / CSV formatted traces since the
   pcap traces are the most authentic source of information.

Schoenwaelder          Expires September 22, 2006               [Page 6]

Internet-Draft          SNMP Traffic Measurements             March 2006

   Improvements in the tool chain may require to go back to the original
   pcap traces and to rebuild all intermediate formats from them.

2.5.  Processing Traffic Traces

   Scripts that analyze traffic traces must be verified for correctness.
   Ideally, all scripts used to analyze traffic traces would be
   publically accessible so that third parties can verify them.
   Furthermore, sharing scripts will enable other parties to repeat an
   analysis on other traffic traces and to extend such analysis scripts.
   A common versioned repository for analysis scripts might be useful to
   establish.

   Due to the availability of XML parsers and the simplicity of the CSV
   format, trace files can be processed with tools written in almost any
   programming language.  However, in order to facilitate a common
   vocabulary and to allow operators to easily read scripts they execute
   on trace files, it is suggested that analysis scripts are written in
   the Perl programming language using the XML::LibXML [6] Perl package
   to read the XML format of the trace files.  Using a scripting
   language such as Perl instead of system programming languages such as
   C or C++ has the advantage to reduce development time and to make
   scripts more accessible to operators who may want to verify scripts
   before running them on trace files which potentially contain
   sensitive data.

   Appendix C show a simple Perl script which computes some summary
   statistics.

   It should be noted here that the snmpdump tool provides an API to
   process SNMP messages in C/C++.  While the coding of trace analysis
   programs in C/C++ should in general be avoided for code readability,
   verifiability and portability reasons, using C/C++ might be the only
   option to deal with very large traces efficiently.

Schoenwaelder          Expires September 22, 2006               [Page 7]

Internet-Draft          SNMP Traffic Measurements             March 2006

3.  Analysis of Traffic Traces

   This section discusses several questions that can be answered by
   analyzing SNMP traffic traces.  The questions raised in the following
   subsections are meant to be illustrative and no attempt has been made
   to provide a complete list.

3.1.  Basic Statistics

   Basic statistics cover things such as the SNMP protocol versions used
   or the protocol operations used in a traffic trace.  In addition, a
   rough classification of the data manipulated into 'standardized',
   'proprietary', and 'experimental' can be done.  Appendix C contains a
   simple analysis script deriving some of these very basic statistics
   from a traffic trace.

3.2.  Periodic vs. Aperiodic Traffic

   SNMP is used to periodically poll devices as well as to retrieve
   information on request of an operator or application.  The periodic
   polling leads to periodic traffic patterns while the on demand
   information retrieval causes more aperiodic traffic patterns.  It is
   worthwhile to understand what the relationship is between the amount
   of periodic and aperiodic traffic.  In addition, it will be
   interesting to research whether there are multiple levels of
   periodicity at different time scales.

3.3.  Message Size and Latency Distributions

   SNMP messages are size constrained by the transport mappings used and
   the buffers provided by the SNMP engines.  For the further evolution
   of the SNMP framework, it would be useful to know what the actual
   message size distributions are.  In addition, it would be useful to
   understand the latency distributions, especially the distribution of
   the processing times by SNMP command responders.  Some SNMP
   implementations approximate networking delays by measuring request-
   response times and it would be useful to understand to what extent
   this is a viable approach.

3.4.  Concurrency Levels

   SNMP allows management stations to retrieve information from multiple
   agents concurrently.  It will be interesting to identify what the
   typical concurrency level is that can be observed on production
   networks or whether management applications prefer more sequential
   ways of retrieving data.

Schoenwaelder          Expires September 22, 2006               [Page 8]

Internet-Draft          SNMP Traffic Measurements             March 2006

3.5.  Table Retrieval Approaches

   Tables can be read in several different ways.  The simplest and most
   inefficient approach is to retrieve tables cell-by-cell in column-by-
   column order.  More advanced approaches try to read tables row-by-row
   or even multiple-rows-by-multiple-rows.  In addition, the retrieval
   of index elements can be suppressed in most cases.  It will be useful
   to know which of these approaches are actually used on production
   networks.

3.6.  Trap-Directed Polling - Myths or Reality?

   SNMP is built around a concept called trap-directed polling.
   Management applications are responsible to periodically poll SNMP
   agents to determine their status.  SNMP agents can in addition send
   traps to notify SNMP managers about events so that SNMP managers can
   adopt their polling strategy and basically react faster than normal
   polling would allow to do.

   Analysis of SNMP traffic traces can identity whether trap-directed
   polling is actually deployed.  In particular, the question that
   should be addressed is whether SNMP notifications lead to changes in
   the short-term polling behavior of management stations.  In
   particular, it should be investigated to what extent SNMP managers
   use automated procedures to track down the meaning of the event
   conveyed by an SNMP notification.

3.7.  Popular MIB Modules

   An analysis of object identifier prefixes can identify the most
   popular MIB modules and the most important object types or
   notification types defined by these modules.  Such information would
   be very valuable for the further maintenance and development of these
   and related MIB modules.

3.8.  Usage of Obsolete Objects

   Several objects from the early days have been obsoleted because they
   cannot properly represent today's networks.  A typical example is the
   ipRouteTable which was obsoleted because it was not able to represent
   classless routing, introduced and deployed on the Internet in 1993.
   Some of these obsolete objects are still mentioned in popular
   publications as well as research papers.  It will be interesting to
   find out whether they are also still used by management applications
   or whether management applications have been updated to use the
   replacement objects.

Schoenwaelder          Expires September 22, 2006               [Page 9]

Internet-Draft          SNMP Traffic Measurements             March 2006

3.9.  Encoding Length Distributions

   It will be useful to understand the encoding length distributions for
   various data types.  Assumption about encoding length distributions
   are sometimes used to estimate SNMP message sizes in order to meet
   transport and buffer size constraints.

3.10.  Counters and Discontinuities

   Counters can experience discontinuities [RFC2578].  The default
   discontinuity indicator is the sysUpTime scalar of the SNMPv2-MIB
   [RFC3418], which can also be used to detect counter roll-overs.  Some
   MIB modules introduce more specific discontinuity indicators, e.g.,
   the ifCounterDiscontinuityTime of the IF-MIB [RFC2863].  It will be
   interesting to study to what extent these objects are actually used
   by management applications to handle discontinuity events.

3.11.  Spin Locks

   Cooperating command generators can use advisory locks to coordinate
   their usage of SNMP write operations.  The snmpSetSerialNo scalar of
   the SNMPv2-MIB [RFC3418] is the default course-grain coordination
   object.  It will be interesting to find out whether there are command
   generators which coordinate themselves using these spin locks.

3.12.  Row Creation

   Row creation is an operation not natively supported by the protocol
   operations.  Instead, conceptual tables supporting row creation
   typically provide a control column which uses the RowStatus textual
   convention defined in the SNMPv2-TC module.  The RowStatus itself
   supports different row creation modes, namely createAndWait (dribble-
   mode) and createAndGo (one-shot mode).  In addition, different
   approaches can be used to derive the instance identifier if it does
   not have special semantics associated.  It will be useful to study
   which of the various row creation approaches are actually used by
   management applications on production networks.

Schoenwaelder          Expires September 22, 2006              [Page 10]

Internet-Draft          SNMP Traffic Measurements             March 2006

4.  Security Considerations

   SNMP traffic traces usually contain sensitive information.  It is
   therefore necessary to (a) remove unneeded information and (b) to
   anonymize the remaining necessary information before traces are made
   available for analysis.

   Implementations that generate XML traces from raw pcap files should
   have an option to suppress values.  Note that instance identifiers of
   tables also include values and it might therefore be necessary to
   suppress (parts of) the instance identifiers.  Similarly, the packet
   and message headers typically contain sensitive information about the
   source and destination of SNMP messages as well as authentication
   information (community strings or user names).

   Anonymization techniques can be applied to keep more information in
   traces which could reveal sensitive information.  When using
   anonymization, values should only be added when the underlying data
   type is known and an appropriate anonymization transformation is
   available (filter-in principle).  For values appearing in instance
   identifiers, it is usually desirable to maintain the lexicographic
   order.  Special anonymization transformations which preserve this
   property have been developed, although their anonymization strength
   is usually reduced compared to transformations that do not preserve
   lexicographic ordering [HS06].

Schoenwaelder          Expires September 22, 2006              [Page 11]

Internet-Draft          SNMP Traffic Measurements             March 2006

5.  Acknowledgements

   This document was influenced by discussions within the Network
   Management Research Group (NMRG).  Special thanks to Remco van de
   Meent for writing the initial Perl script that lead to the script in
   Appendix C and Matus Harvan for his work on lexicographic order
   preserving anonymization transformations.  Aiko Pras contributed
   ideas to Section 3 while David Harrington helped to improve the
   readability of this document.

   Part of this work was funded by the European Commission under grant
   FP6-2004-IST-4-EMANICS-026854-NOE.

Schoenwaelder          Expires September 22, 2006              [Page 12]

Internet-Draft          SNMP Traffic Measurements             March 2006

6.  References

6.1.  Normative References

   [RFC2578]  McCloghrie, K., Perkins, D., and J. Schoenwaelder,
              "Structure of Management Information Version 2 (SMIv2)",
              STD 58, RFC 2578, April 1999.

   [RFC3411]  Harrington, D., Presuhn, R., and B. Wijnen, "An
              Architecture for Describing Simple Network Management
              Protocol (SNMP) Management Frameworks", STD 62, RFC 3411,
              December 2002.

   [RFC3416]  Presuhn, R., Case, J., McCloghrie, K., Rose, M., and S.
              Waldbusser, "Version 2 of the Protocol Operations for the
              Simple  Network Management Protocol (SNMP)", STD 62,
              RFC 3416, December 2002.

   [RFC3418]  Presuhn, R., Case, J., McCloghrie, K., Rose, M., and S.
              Waldbusser, "Management Information Base (MIB) for the
              Simple Network Management Protocol (SNMP)", STD 62,
              RFC 3418, December 2002.

   [OASISRNG]
              Clark, J. and M. Makoto, "RELAX NG Specification",
              OASIS Committee Specification, December 2001.

   [OASISRNC]
              Clark, J., "RELAX NG Compact Syntax", OASIS Committee
              Specification, November 2002.

6.2.  Informative References

   [RFC1052]  Cerf, V., "IAB Recommendations for the Development of
              Internet Network Management Standards", RFC 1052,
              April 1998.

   [RFC2011]  McCloghrie, K., "SNMPv2 Management Information Base for
              the Internet  Protocol using SMIv2", RFC 2011,
              November 1996.

   [RFC2863]  McCloghrie, K. and F. Kastenholz, "The Interfaces Group
              MIB", RFC 2863, June 2000.

   [RFC3410]  Case, J., Mundy, R., Partain, D., and B. Stewart,
              "Introduction and Applicability Statements for Internet
              Standard Management Framework", RFC 3410, December 2002.

Schoenwaelder          Expires September 22, 2006              [Page 13]

Internet-Draft          SNMP Traffic Measurements             March 2006

   [RFC3430]  Schoenwaelder, J., "Simple Network Management Protocol
              (SNMP) over Transmission  Control Protocol (TCP) Transport
              Mapping", RFC 3430, December 2002.

   [RFC4022]  Raghunarayan, R., "Management Information Base for the
              Transmission  Control Protocol (TCP)", RFC 4022,
              March 2005.

   [PDMQ04]   Pras, A., Drevers, T., van de Meent, R., and D. Quartel,
              "Comparing the Performance of SNMP and Web Services based
              Management", IEEE electronic Transactions on Network and
              Service Management 1(2), November 2004.

   [Pat01]    Pattinson, C., "A Study of the Behaviour of the Simple
              Network Management Protocol", Proc. 12th IFIP/IEEE
              Workshop on Distributed Systems: Operations and
              Management , October 2001.

   [DSR01]    Du, X., Shayman, M., and M. Rozenblit, "Implementation and
              Performance Analysis of SNMP on a TLS/TCP Base", Proc. 7th
              IFIP/IEEE International Symposium on Integrated Network
              Management , May 2001.

   [CT04]     Corrente, A. and L. Tura, "Security Performance Analysis
              of SNMPv3 with Respect to SNMPv2c", Proc. 2004 IEEE/IFIP
              Network Operations and  Management Symposium , April 2004.

   [PFGL04]   Pavlou, G., Flegkas, P., Gouveris, S., and A. Liotta, "On
              Management Technologies and the Potential of Web
              Services", IEEE Communications Magazine 42(7), July 2004.

   [SM99]     Sprenkels, R. and J. Martin-Flatin, "Bulk Transfers of MIB
              Data", Simple Times 7(1), March 1999.

   [Mal02]    Malowidzki, M., "GetBulk Worth Fixing", Simple
              Times 10(1), December 2002.

   [HS06]     Harvan, M. and J. Schoenwaelder, "Prefix- and
              Lexicographical-order-preserving IP Address
              Anonymization", IEEE/IFIP Network Operations and
              Management Symposium NOMS 2006, April 2006.

Schoenwaelder          Expires September 22, 2006              [Page 14]

Internet-Draft          SNMP Traffic Measurements             March 2006

URIs

   [1]  <http://www.tcpdump.org/>

   [2]  <http://www.ethereal.com/>

   [3]  <http://indev.insu.com/Fwctl/pcapmerge.html>

   [4]  <https://subversion.eecs.iu-bremen.de/svn/schoenw/src/snmpdump>

   [5]  <http://xmlsoft.org/XSLT/>

   [6]  <http://www.cpan.org/>

   [7]  <http://www.relaxng.org/>

Schoenwaelder          Expires September 22, 2006              [Page 15]

Internet-Draft          SNMP Traffic Measurements             March 2006

Appendix A.  RELAX NG Schema Definition

   The XML format has been designed to keep all information associated
   with SNMP messages.  The format is specified in RELAX NG compact
   notation [OASISRNC].  Freely available tools such as trang [7] can be
   used to convert RELAX NG compact syntax to other XML schema
   notations.

   start =
     element snmptrace {
       packet.elem*
     }

   packet.elem =
     element packet {
       attribute sec { xsd:unsignedInt },
       attribute usec { xsd:unsignedInt },
       element src { addr.attrs },
       element dst { addr.attrs },
       snmp.elem
     }

   snmp.elem =
     element snmp {
       length.attrs?,
       message.elem
     }

   message.elem =
     element version { length.attrs, xsd:int },
     element community { length.attrs, xsd:hexBinary },
     pdu.elem

   message.elem |=
     element version { length.attrs, xsd:int },
     element message {
       length.attrs,
       element msg-id { length.attrs, xsd:unsignedInt },
       element max-size { length.attrs, xsd:unsignedInt },
       element flags { length.attrs, xsd:hexBinary },
       element security-model { length.attrs, xsd:unsignedInt }
     },
     usm.elem?,
     element scoped-pdu {
       length.attrs,
       element context-engine-id { length.attrs, xsd:hexBinary },
       element context-name { length.attrs, xsd:string },

Schoenwaelder          Expires September 22, 2006              [Page 16]

Internet-Draft          SNMP Traffic Measurements             March 2006

       pdu.elem
     }

   usm.elem =
     element usm {
       length.attrs,
       element auth-engine-id { length.attrs, xsd:hexBinary },
       element auth-engine-boots { length.attrs, xsd:unsignedInt },
       element auth-engine-time { length.attrs, xsd:unsignedInt },
       element user { length.attrs, xsd:hexBinary },
       element auth-params { length.attrs, xsd:hexBinary },
       element priv-params { length.attrs, xsd:hexBinary }
     }

   pdu.elem =
     element trap {
       length.attrs,
       element enterprise { length.attrs, oid.type },
       element agent-addr { length.attrs, ipaddress.type },
       element generic-trap { length.attrs, xsd:int },
       element specific-trap { length.attrs, xsd:int },
       element time-stamp { length.attrs, xsd:int },
       element variable-bindings { length.attrs, varbind.elem* }
     }

   pdu.elem |=
     element (get-request | get-next-request | get-bulk-request |
              set-request | inform | trap2 | response | report) {
       length.attrs,
       element request-id { length.attrs, xsd:int },
       element error-status { length.attrs, xsd:int },
       element error-index { length.attrs, xsd:int },
       element variable-bindings { length.attrs, varbind.elem* }
     }

   varbind.elem =
     element varbind { length.attrs, name.elem, value.elem }

   name.elem =
     element name { length.attrs, oid.type }

   value.elem =
     element null { length.attrs, empty } |
     element integer32 { length.attrs, xsd:int } |
     element unsigned32 { length.attrs, xsd:unsignedInt } |
     element unsigned64 { length.attrs, xsd:unsignedLong } |
     element ipaddress { length.attrs, ipaddress.type } |
     element octet-string { length.attrs, xsd:hexBinary } |

Schoenwaelder          Expires September 22, 2006              [Page 17]

Internet-Draft          SNMP Traffic Measurements             March 2006

     element object-identifier { length.attrs, oid.type } |
     element (no-such-object | no-such-instance | end-of-mib-view) { empty }

   # The blen attribute indicates the number of bytes used by the BER
   # encoded tag / length / value triple. The vlen attribute indicates
   # the number of bytes used by the BER encoded value alone.

   length.attrs =
     ( attribute blen { xsd:unsignedShort },
       attribute vlen { xsd:unsignedShort } )?

   addr.attrs =
     attribute ip { ipaddress.type },
     attribute port { xsd:unsignedShort }

   oid.type =
     xsd:string {
       pattern =
         """[0-2](\.[0-9]+)+"""
     }

   # [XXX] We should extend the regular expression below to cover also
   # IPv6 addresses (including zone indexes ;-).

   ipaddress.type =
     xsd:string {
       pattern =
         """[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*"""
     }

Schoenwaelder          Expires September 22, 2006              [Page 18]

Internet-Draft          SNMP Traffic Measurements             March 2006

Appendix B.  CSV Format Definition

   The CSV format has been design to capture only the most relevant
   information about an SNMP message.  The CSV format uses the following
   fields:

   1.   Time-stamp in the format seconds.microseconds since 1970, for
        example "1137764769.425484".

   2.   Source IP address in dotted quad notation (IPv4), for example
        "127.0.0.1", or compact hexadecimal notation (IPv6), for example
        "::1".

   3.   Source port number represented as a decimal number, for example
        "4242".

   4.   Destination IP address in dotted quad notation (IPv4), for
        example "127.0.0.1", or compact hexadecimal notation (IPv6), for
        example "::1".

   5.   Destination port number represented as a decimal number, for
        example "161".

   6.   Size of the SNMP message (a decimal number) counted in bytes,
        for example "123".  The size excludes all transport, network,
        and link-layer headers.

   7.   SNMP message version represented as a decimal number.  The
        version 0 stands for SNMPv1, 1 for SNMPv2c, and 3 for SNMPv3,
        for example "3".

   8.   SNMP protocol operation indicated by one of the keywords get-
        request, get-next-request, get-bulk-request, set-request, trap,
        trap2, inform, response, report.

   9.   SNMP request-id in decimal notation, for example "1511411010".

   10.  SNMP error-status in decimal notation, for example "0".

   11.  SNMP error-index in decimal notation, for example "0".

   12.  Number of variable-bindings contained in the varbind-list in
        decimal notation, for example "5".

   13.  Object names given as object identifiers in dotted decimal
        notation, for example "1.3.6.1.2.1.1.3.0".  Object names are
        separated by commas.

Schoenwaelder          Expires September 22, 2006              [Page 19]

Internet-Draft          SNMP Traffic Measurements             March 2006

Appendix C.  Sample Perl Analysis Script

   [XXX] This script probably should go away since it does not scale at
   all.  It seems that we can provide perhaps a series of simple scripts
   that operate on the CSV format to produce something meaningful.

   #!/usr/bin/perl

   # This script computes basic statistics from SNMP packet trace files.
   #
   # To run this script:
   #    snmpstat.pl [<filename>]
   #
   # (c) 2002 Remco van de Meent    <[email-masked]>
   # (c) 2005 Juergen Schoenwaelder <[email-masked]>

   use strict;
   use XML::LibXML;

   sub version_stats {
       my $doc = shift;
       my @cntr;
       my $total = 0;
       foreach my $node ($doc->findnodes('//snmp/version')) {
           my $version = $node->textContent();
           $cntr[$version]++;
           $total++;
       }
       printf "SNMP version statistics:\n\n";
       foreach my $version (0, 1, 2) {
           printf "%18s: %5d  %3d\%\n", $version,
               $cntr[$version], $cntr[$version]/$total*100;
       }
       printf "    ---------------------------\n";
       printf "%18s: %5d  %3d\%\n\n", "total", $total, 100;
   }

   sub operation_stats {
       my $doc = shift;
       my @total = $doc->findnodes('//packet/snmp');
       printf "SNMP PDU type statistics:\n\n";
       foreach my $op ("get-request", "get-next-request", "get-bulk-request",
                       "set-request", "trap", "trap-v2", "inform",
                       "response", "report") {
           my @nodes = $doc->findnodes("//packet/snmp/$op");

Schoenwaelder          Expires September 22, 2006              [Page 20]

Internet-Draft          SNMP Traffic Measurements             March 2006

           printf "%18s: %5d  %3d\%\n", $op, $#nodes + 1,
               ($#nodes+1)/($#total+1)*100;
       }
       printf "    ---------------------------\n";
       printf "%18s: %5d  %3d\%\n\n", "total", $#total + 1, 100;
   }

   sub oid_stats {
       my $doc = shift;
       my $oid_ctr = 0;
       my $transmission_ctr; # 1.3.6.1.2.1.10
       my $mib2_ctr;         # 1.3.6.1.2.1
       my $experiment_ctr;   # 1.3.6.1.3
       my $enterprise_ctr;   # 1.3.6.1.4.1
       foreach my $node ($doc->findnodes('//varbind/name')) {
           my $name = $node->textContent();
           for ($name) {
               if    (/1\.3\.6\.1\.2\.1\.10/) { $transmission_ctr++; }
               elsif (/1\.3\.6\.1\.2\.1/)     { $mib2_ctr++; }
               elsif (/1\.3\.6\.1\.3/)        { $experiment_ctr++; }
               elsif (/1\.3\.6\.1\.4\.1/)     { $enterprise_ctr++; }
           }
           $oid_ctr++;
       }
       printf "SNMP OID prefix statistics:\n\n";
       printf "%18s: %5d  %3d\%\n", "transmission",
           $transmission_ctr, ($transmission_ctr/$oid_ctr*100);
       printf "%18s: %5d  %3d\%\n", "mib-2",
           $mib2_ctr, ($mib2_ctr/$oid_ctr*100);
       printf "%18s: %5d  %3d\%\n", "experimental",
           $experiment_ctr, ($experiment_ctr/$oid_ctr*100);
       printf "%18s: %5d  %3d\%\n", "enterprises",
           $enterprise_ctr, ($enterprise_ctr/$oid_ctr*100);
       printf "    ---------------------------\n";
       printf "%18s: %5d  %3d\%\n\n", "total", $oid_ctr, 100;
   }

   sub size_stats {
       my $doc = shift;
       my @total = $doc->findnodes('//packet/snmp');
       printf "SNMP message size statistics:\n\n";
       foreach my $op ("get-request", "get-next-request", "get-bulk-request",
                       "set-request", "trap", "trap-v2", "inform",
                       "response", "report") {
           my $total_ops = 0;
           my $total_len = 0;

Schoenwaelder          Expires September 22, 2006              [Page 21]

Internet-Draft          SNMP Traffic Measurements             March 2006

           foreach my $node ($doc->findnodes("//packet/snmp/$op")) {
               my @msg_len = $node->find('../@blen');
               $total_ops++;
               # $total_len += $msg_len[0];
               # printf "\t%d\t%d\n", @msg_len, $total_len;
           }
           printf "%18s: %5d %5d %f\n", $op, $total_ops, $total_len, $total_ops ? $total_len/$total_ops : 0;
       }
       printf "\n";
   }

   sub min {
       if ($_[0]>$_[1]) {return $_[1]} else {return $_[0]};
   }

   sub max {
       if ($_[0]<$_[1]) {return $_[1]} else {return $_[0]};
   }

   sub varbind_stats {
       my $doc = shift;
       my @total = $doc->findnodes('//packet/snmp');
       printf "SNMP varbind number statistics:\n\n";
       foreach my $op ("get-request", "get-next-request", "get-bulk-request",
                       "set-request", "trap", "trap-v2", "inform",
                       "response", "report") {
           my ($total_ops, $total_vbs, $total_vbs_min, $total_vbs_max);
           foreach my $node ($doc->findnodes("//packet/snmp/$op")) {
               my @varbinds = $node->findnodes("variable-bindings/varbind");
               $total_ops++;
               $total_vbs += $#varbinds + 1;
               $total_vbs_min = min($total_vbs_min, $#varbinds + 1);
               $total_vbs_max = max($total_vbs_max, $#varbinds + 1);
           }
           printf "%18s: %5d %5d %5.2f %5d %5d\n", $op, $total_ops, $total_vbs, $total_ops ? $total_vbs/$total_ops : 0, $total_vbs_min, $total_vbs_max;
       }
       printf "\n";
   }

   @ARGV = ('-') unless @ARGV;
   while ($ARGV = shift) {
       my $parser = XML::LibXML->new();
       my $tree = $parser->parse_file($ARGV);
       my $doc = $tree->getDocumentElement;

       version_stats($doc);

Schoenwaelder          Expires September 22, 2006              [Page 22]

Internet-Draft          SNMP Traffic Measurements             March 2006

       operation_stats($doc);
       oid_stats($doc);
       size_stats($doc);
       varbind_stats($doc);
   }
   exit(0);

Schoenwaelder          Expires September 22, 2006              [Page 23]

Internet-Draft          SNMP Traffic Measurements             March 2006

Appendix D.  Trace Description Form

   The following ASCII form is suggested to keep track of meta
   information associated with a traffic trace.

   Name:         [name of the trace]
   Network:      [name of the network]
   Organization: [name of the organization operating the network]
   Contact:      [name and email address of a contact person]
   Start-Date:   [date in ISO date format]
   End-Date:     [date in ISO date format]
   Size:         [size of the pcap trace in bytes]
   Description:  [description, multiple lines with white space indentation]

Schoenwaelder          Expires September 22, 2006              [Page 24]

Internet-Draft          SNMP Traffic Measurements             March 2006

Author's Address

   Juergen Schoenwaelder
   International University Bremen
   Campus Ring 1
   28725 Bremen
   Germany

   Phone: +49 421 200-3587
   Email: [email-masked]

Schoenwaelder          Expires September 22, 2006              [Page 25]

Internet-Draft          SNMP Traffic Measurements             March 2006

Intellectual Property Statement

   The IETF takes no position regarding the validity or scope of any
   Intellectual Property Rights or other rights that might be claimed to
   pertain to the implementation or use of the technology described in
   this document or the extent to which any license under such rights
   might or might not be available; nor does it represent that it has
   made any independent effort to identify any such rights.  Information
   on the procedures with respect to rights in RFC documents can be
   found in BCP 78 and BCP 79.

   Copies of IPR disclosures made to the IETF Secretariat and any
   assurances of licenses to be made available, or the result of an
   attempt made to obtain a general license or permission for the use of
   such proprietary rights by implementers or users of this
   specification can be obtained from the IETF on-line IPR repository at
   http://www.ietf.org/ipr.

   The IETF invites any interested party to bring to its attention any
   copyrights, patents or patent applications, or other proprietary
   rights that may cover technology that may be required to implement
   this standard.  Please address the information to the IETF at
   [email-masked].

Disclaimer of Validity

   This document and the information contained herein are provided on an
   "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
   OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
   ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
   INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
   INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
   WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Copyright Statement

   Copyright (C) The Internet Society (2006).  This document is subject
   to the rights, licenses and restrictions contained in BCP 78, and
   except as set forth therein, the authors retain all their rights.

Acknowledgment

   Funding for the RFC Editor function is currently provided by the
   Internet Society.

Schoenwaelder          Expires September 22, 2006              [Page 26]


--Qxx1br4bt0+wmkIi--