Subject: [nmrg] RE: [IRSG] review of draft-irtf-nmrg-snmp-measure-04.txt
From: "Bert Wijnen - IETF" <[email-masked]>
To: "Karen R. Sollins" <[email-masked]>, <[email-masked]>
Cc: Internet Research Steering Group <[email-masked]>, [email-masked]
Date: Mon, 19 May 2008 23:49:00 +0200
List-Id: Network Management Research Group <nmrg.ibr.cs.tu-bs.de>

Karen,
Juergen has posted revision 5 with the fixes/updates as you
and he discussed. If you could quickly check that revision 5 is now OK
with you and let us all know if so, that would be appreciated.

The document is here:
  http://www.ietf.org/internet-drafts/draft-irtf-nmrg-snmp-measure-05.txt

NMRG members, pls check and speak up if you see any issues in the
changes that were made for revision 5 as a result of IRSG review.

Bert Wijnen
document shepherd

> -----Oorspronkelijk bericht-----
> Van: Karen R. Sollins [mailto:[email-masked]]
> Verzonden: maandag 19 mei 2008 19:25
> Aan: [email-masked]; Karen R. Sollins
> CC: [email-masked]; Bert Wijnen - IETF; Internet Research Steering
> Group
> Onderwerp: Re: [IRSG] review of draft-irtf-nmrg-snmp-measure-04.txt
>
>
> HI Juergen,
>
> It is all fine with me.  I think all you need now is concurrence from
> the research group.
>
> 			Cheers,
> 			Karen
>
> At 11:46 AM +0200 5/19/08, Juergen Schoenwaelder wrote:
> >On Sun, May 18, 2008 at 09:05:15PM -0700, Karen R. Sollins wrote:
> >
> >>  Thanks for your thoughtful responses.  I also did not think
> that what I was
> >>  suggesting was a lot of work.   At a high level, think about
> a reader who
> >>  is not part of your group, whom you are trying to convince
> that what you
> >>  are doing is valuable or whom you would like to convince to
> do such a data
> >>  collection exercise. I have interspersed my comments into
> your responses
> >>  below.
> >
> >I will repond to those things that are still "open" with suggestions
> >on how to "close" them.
> >
> >KS>  The second high level concern I have is that there is talk about
> >KS>  specific kinds of information to be collected and an interest in not
> >KS>  only the nature but longer term inferences with perhaps implications
> >KS>  for future redesign efforts in the SNMP context.    I have
> two levels
> >KS>  of concern here.  The most important one is that since network
> >KS>  management and in particular SNMP is NOT the primary
> objective of the
> >KS>  net (the primary objective being the transport of real payload), it
> >KS>  seems to me that the truly critical question with respect to network
> >KS>  management traffic is the impact that it has or does not
> have on that
> >KS>  real job.  To me this implies that the measurements MUST
> also include
> >KS>  contextual information.  As an example, it is probably more
> important
> >KS>  to understand whether  or not the network management traffic is
> >KS>  causing significant congestion for the payload traffic than the
> >KS>  particular mix or frequency pattern within the network management
> >KS>  traffic.  Without out the complementary contextual information, the
> >KS> whole measurement exercise seems to me to be of somewhat narrow
> >KS>  value.
> >
> >JS> The measurement may be of narrow value from your point of view but
> >JS> please keep in mind that this document is coming from the Network
> >JS> Management Research Group and not from a general Network Measurement
> >JS> Research Group. Our goal is to understand how network management
> >JS> protocols are being used because that has impact on their design and
> >JS> implementation strategies. Further note that in many networks, the
> >JS> management traffic is logically and sometimes even physically
> >JS> separated from the normal traffic and perhaps this is the reason why
> >JS> we did not even think about the question whether management traffic
> >JS> has an impact on normal traffic.
> >
> >KS> If you want to leave it as is, then I think it would be
> valuable to say as
> >KS> much.  Be specific about what you are not doing, because
> much of the rest
> >KS> of the world looks at network traffic from a broader perspective.
> >
> >I think the abstract and the introduction are pretty clearly spelling
> >out the scope of the measurement effort. So I am not sure changes are
> >needed, but see below.
> >
> >KS>  1. Section 1: It seems to me that there are TWO key questions with
> >KS>  respect to SNMP.  The first is how it is being used, which in turn
> >KS>  leads to the points made in this section, but the second is the
> >KS>  impact of that traffic.  I think that ought to appear in the
> >KS>  Introduction as well.
> >
> >JS> See my comments above. So far, the NMRG did not consider the
> impact of
> >JS> SNMP on other traffic a target of this activity. I don't want to add
> >JS> such text unless I see support from the NMRG and concrete proposals
> >JS> what should be added.
> >
> >KS> Again, as above, if you want to leave the scope as it is,
> then you should
> >KS> probably be up front about that, clearly leaving that work
> to another time
> >KS> and place.
> >
> >I propose to add the following paragraph just before the last
> >paragraph in section 1:
> >
> >    The measurement approach described in this document is by design
> >    limited to the study of SNMP traffic.  Studies of other management
> >    protocols or the impact of management protocols such as SNMP on
> >    other traffic sharing the same network resources is left to future
> >    efforts.
>
> This is what I was looking for.
>
> >KS>  2. Section 2.1.  The second paragraph begins with, "It is
> recommended
> >KS>  to capture at least a full week of data."  This is never
> justified or
> >KS>  explained.  Is one week really enough?  For what?  Why wouldn't
> >KS>  several weeks be critical, because one week might be anomalous?  Why
> >KS>  isn't a year critical, since we know that there are annual or
> >KS>  seasonal differences in traffic behaviors?  Typically, I find that
> >KS>  one-week data sets often leave me with lots of unanswered questions,
> >KS>  so justify this.
> >
> >JS> The text actually says:
> >JS>
> >JS>    It is recommended to capture at least a full week of
> data.  Operators
> >JS>    are encouraged to capture traces over even longer periods of time.
> >JS>
> >JS> The text tries to establish a lower bound of one week an encourages
> >JS> longer capture periods. I would love to get continuous traces but
> >JS> reality is such that this is not feasible. Our idea is
> simply to catch
> >JS> at least the weekly behaviour. Yes, there is of course also
> monthly or
> >JS> yearly behaviour but I believe it is not useful to set the
> bar so high
> >JS> that nobody gives us appropriate traces. I personally
> believe the text
> >JS> is fine as is.
> >
> >KS> So, what I was really getting at was the question of why one week
> >KS> was the minimum necessary.  So, something like that it is the
> >KS> minimum over which one can see the diurnal patterns in the weekly
> >KS> pattern and it is understood that both for computational and
> >KS> storage reasons the operators may not want to collect more.
> >
> >I have changed the text to the following:
> >
> >    It is recommended to capture at least a full week of data to capture
> >    diurnal patterns and one cycle of weekly behavior.  Operators are
> >    strongly encouraged to capture traces over even longer periods of
> >    time.
>
> OK by me.
>
> >KS>  4. Section 3.3, end of first paragraph:  The sentence reads, "Some
> >KS>  SNMP implementations approximate networking delays by measuring
> >KS>  request-response times and it would be useful to understand to what
> >KS>  extent this is a viable approach."  I agree, but traces
> will not tell
> >KS>  you anything about whether behaviors observed in packet traces are
> >KS>  for this reason or some other reason.  I do not believe you can get
> >KS>  at this question with the data you are collecting.
> >
> >JS> I think it is possible to analyze retransmission behaviour. Depending
> >JS> on the SNMP version used (and the other versions also depending on
> >JS> implementation choices), you can get information whether a
> response is
> >JS> just coming late for the original request or it is actually
> a response
> >JS> to a retranmitted request. We are not talking TCP here; we
> are talking
> >JS> about application layer retransmissions and SNMP has its own
> msgID and
> >JS> requestID fields.
> >
> >KS> The point I was trying to make here is that it is very
> difficult to intuit
> >KS> the reasons behind behaviors seen in the traffic, unless someone or
> >KS> something tells you.  So, you can see what the ends do, but not why.
> >
> >I agree, but this is a rather general observation and not necessarily
> >specific to 3.3 and it is not clear to me what I should do about it.
> >There is already a general remark in the last paragraph of section 2.5
> >that one has to be careful with drawing conclusions that go beyond
> >what you can really get out of traces.
>
> OK - I'll drop it.
>
> >
> >KS>  5. Section 3.4: Please explain why it is "interesting"
> (your word) to
> >KS>  identify whether concurrency or sequentiality is occurring?  What
> >KS>  will you "learn" if both are observed?  And, if one is
> occurring more
> >KS>  frequently or under specific identifiable conditions, what further
> >KS>  does that tell you?  Just knowing that one or the other occurs is
> >KS>  only the tip of the iceberg, and without acknowledging the fact that
> >KS>  these are important and unanswered questions, just learning first
> >KS>  ordered details suggests you are setting the bar too low.
> >
> >JS> The introduction of section 3 says:
> >JS>
> >JS>    The questions raised in the following subsections are meant to be
> >JS>    illustrative and no attempt has been made to provide a complete
> >JS>    list.
> >JS>
> >JS> I believe it is a good idea to first figure out whether there is an
> >JS> iceberg or not (keeping your analogy) and if there is one to ask
> >JS> questions how big the iceberg might be. For SNMP agent
> implementations
> >JS> that tend to do quite some caching, it is useful to know how well
> >JS> caching strategies are working in real-world networks. The
> concurrency
> >JS> level an agent experiences has clear impact on that. Furthermore, it
> >JS> will be useful to know how bursty the traffic tends to be or how well
> >JS> managers spread the traffic over polling intervals and this is again
> >JS> related to the concurrency we can extract from traces.
> >JS>
> >JS> I am not really sure what I should change, perhaps the word
> >JS> "interesting" is the source of the trouble and I should replace this
> >JS> with "valuable"?
> >
> >KS> What I was trying to ask was that you tell the reader a bit about
> >KS> what makes it interesting.  It certainly doesn't have to be
> >KS> complete, but just a> hint.  I don't think you should change the
> >KS> word "interesting", because if it isn't interesting, you probably
> >KS> shouldn't be doing it.
> >
> >I added the following sentence to section 3.4:
> >
> >    The concurrency level and the amount of redundant requests has
> >    implications on caching strategies employed by SNMP agents.
>
> Fine.
>
> >KS>  6. Section 3.5:  Please explain what you would do  with the
> >KS>  information about which approach to table retrieval is used.  Again,
> >KS>  what if the results tell you that both are used?  And, if not, of
> >KS>  what use is it to know which approach is prevalent?  Mighten it be
> >KS>  useful to know the conditions under which one or the other is used
> >KS>  more commonly?
> >
> >JS> This again has direct impact on agent implementation techniques and
> >JS> caching strategies.
> >
> >KS> And again, just explain a little.
> >
> >I changed the last sentence of section 3.5 to read as follows:
> >
> >    [...] It will be useful to know which of
> >    these approaches are used on production networks since this has a
> >    direct implication on agent implementation techniques and caching
> >    strategies.
>
> Fine.
>
> >/js
> >
> >--
> >Juergen Schoenwaelder           Jacobs University Bremen gGmbH
> >Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
> >Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>
>
>
> --
>
> Karen R. Sollins, Ph. D.
> Principal Research Scientist
> M.I.T. CSAIL
> The Stata Center
> 32 Vassar St., 32-G818
> Cambridge, MA 02139
> V: 617/253-6006
> F: 617/253-2673
> E: [email-masked]
>

Date: Mon, 19 May 2008 21:11:33 +0200
From: Juergen Schoenwaelder <[email-masked]>
To: "Karen R. Sollins" <[email-masked]>
Mail-Followup-To: "Karen R. Sollins" <[email-masked]>, [email-masked], Bert Wijnen - IETF <[email-masked]>, Internet Research Steering Group <[email-masked]>
Cc: Internet Research Steering Group <[email-masked]>, [email-masked]
Subject: [nmrg] Re: [IRSG] review of draft-irtf-nmrg-snmp-measure-04.txt
Reply-To: [email-masked]

On Mon, May 12, 2008 at 02:52:02PM -0400, Karen R. Sollins wrote:

> Here is my full review of draft-irtf-nmrg-snmp-measure-04.txt.

[...]

I have posted the -05 version of the document which includes all the
changes discussed here. The -05 version is already online but not yet
on the tools pages (no nice colored diffs yet).

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

Resent-From: [email-masked]
Resent-Date: Mon, 19 May 2008 21:11:21 +0200
Resent-Message-ID: <[email-masked]>
Resent-To: [email-masked]
From: [email-masked]
To: [email-masked]
Date: Mon, 19 May 2008 12:00:02 -0700 (PDT)
Sender: [email-masked]
Subject: [nmrg] I-D Action:draft-irtf-nmrg-snmp-measure-05.txt 
Reply-To: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.

	Title           : SNMP Traffic Measurements and Trace Exchange Formats
	Author(s)       : 
	Filename        : draft-irtf-nmrg-snmp-measure-05.txt
	Pages           : 23
	Date            : 2008-05-19

The Simple Network Management Protocol (SNMP) is widely deployed to
monitor, control and (sometimes also) configure network elements.
Even though the SNMP technology is well documented, it remains
relatively unclear how SNMP is used in practice and what typical SNMP
usage patterns are.

This document describes an approach to carrying out large scale SNMP
traffic measurements in order to develop a better understanding how
SNMP is used in real world production networks.  It describes the
motivation, the measurement approach, and the tools and data formats
needed to carry out such a study.

This document was produced within the IRTF's Network Management
Research Group (NMRG) and represents the consensus of all of the
active contributors to this group.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-irtf-nmrg-snmp-measure-05.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Message/External-body;
	name="draft-irtf-nmrg-snmp-measure-05.txt"; site="ftp.ietf.org";
	access-type="anon-ftp"; directory="internet-drafts"

Content-Type: text/plain
Content-ID: <[email-masked]>

--NextPart
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
I-D-Announce mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/i-d-announce
Internet-Draft directories: http://www.ietf.org/shadow.html
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

--NextPart--

From: "David B Harrington" <[email-masked]>
To: <[email-masked]>
Subject: RE: [nmrg] Re: [IRSG] review of draft-irtf-nmrg-snmp-measure-04.txt
Date: Mon, 19 May 2008 14:43:31 -0400
Cc: 'Internet Research Steering Group' <[email-masked]>, "'Karen R. Sollins'" <[email-masked]>, [email-masked]

Thanks,
dbh 

> -----Original Message-----
> From: Juergen Schoenwaelder 
> [mailto:[email-masked]] 
> Sent: Monday, May 19, 2008 2:31 PM
> To: David B Harrington
> Cc: 'Karen R. Sollins'; 'Internet Research Steering Group'; 
> [email-masked]
> Subject: Re: [nmrg] Re: [IRSG] review of 
> draft-irtf-nmrg-snmp-measure-04.txt
> 
> On Mon, May 19, 2008 at 09:15:49AM -0400, David B Harrington wrote:
>  
> > I think something should be said that this information could be
used
> > by an attacker (especially an attacker internal to the
organization)
> > to decide/pinpoint where and how to attack. This information might
> > need to be anonymized, although that would seem to defeat 
> the purpose
> > of having the information. I don't really know what to suggest
here
> > other than to raise the point in the security 
> considerations that such
> > location information might be sensitive, and could aid an
attacker.
> 
> I suggest to add the following text to the end of the security
> considerations:
> 
>    The meta data associated with traces and in particular
information
>    about the organization owning a network and the description of
the
>    measurement point in the network topology where a trace 
> was collected
>    may be misused to decide/pinpoint where and how to attack 
> a network.
>    Meta data therefore needs to be properly protected.
> 
> In addition, I like to replace "generate XML traces" with "generate
> CSV or XML traces" in the first sentence of the second paragraph of
> the security considerations text since the rest of the paragraph
> applies to both trace formats and not just XML.
> 
> /js
> 
> -- 
> Juergen Schoenwaelder           Jacobs University Bremen gGmbH
> Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
> Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>
> 

Date: Mon, 19 May 2008 20:31:02 +0200
From: Juergen Schoenwaelder <[email-masked]>
To: David B Harrington <[email-masked]>
Subject: Re: [nmrg] Re: [IRSG] review of draft-irtf-nmrg-snmp-measure-04.txt
Mail-Followup-To: David B Harrington <[email-masked]>, "'Karen R. Sollins'" <[email-masked]>, 'Internet Research Steering Group' <[email-masked]>, [email-masked]
Cc: 'Internet Research Steering Group' <[email-masked]>, "'Karen R. Sollins'" <[email-masked]>, [email-masked]
Reply-To: [email-masked]

On Mon, May 19, 2008 at 09:15:49AM -0400, David B Harrington wrote:
 
> I think something should be said that this information could be used
> by an attacker (especially an attacker internal to the organization)
> to decide/pinpoint where and how to attack. This information might
> need to be anonymized, although that would seem to defeat the purpose
> of having the information. I don't really know what to suggest here
> other than to raise the point in the security considerations that such
> location information might be sensitive, and could aid an attacker.

I suggest to add the following text to the end of the security
considerations:

   The meta data associated with traces and in particular information
   about the organization owning a network and the description of the
   measurement point in the network topology where a trace was collected
   may be misused to decide/pinpoint where and how to attack a network.
   Meta data therefore needs to be properly protected.

In addition, I like to replace "generate XML traces" with "generate
CSV or XML traces" in the first sentence of the second paragraph of
the security considerations text since the rest of the paragraph
applies to both trace formats and not just XML.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

Date: Mon, 19 May 2008 10:25:06 -0700
To: [email-masked], "Karen R. Sollins" <[email-masked]>
From: "Karen R. Sollins" <[email-masked]>
Cc: Internet Research Steering Group <[email-masked]>, [email-masked]
Subject: [nmrg] Re: [IRSG] review of draft-irtf-nmrg-snmp-measure-04.txt

HI Juergen,

It is all fine with me.  I think all you need now is concurrence from 
the research group.

			Cheers,
			Karen

At 11:46 AM +0200 5/19/08, Juergen Schoenwaelder wrote:
>On Sun, May 18, 2008 at 09:05:15PM -0700, Karen R. Sollins wrote:
>
>>  Thanks for your thoughtful responses.  I also did not think that what I was
>>  suggesting was a lot of work.   At a high level, think about a reader who
>>  is not part of your group, whom you are trying to convince that what you
>>  are doing is valuable or whom you would like to convince to do such a data
>>  collection exercise. I have interspersed my comments into your responses
>>  below.
>
>I will repond to those things that are still "open" with suggestions
>on how to "close" them.
>
>KS>  The second high level concern I have is that there is talk about
>KS>  specific kinds of information to be collected and an interest in not
>KS>  only the nature but longer term inferences with perhaps implications
>KS>  for future redesign efforts in the SNMP context.    I have two levels
>KS>  of concern here.  The most important one is that since network
>KS>  management and in particular SNMP is NOT the primary objective of the
>KS>  net (the primary objective being the transport of real payload), it
>KS>  seems to me that the truly critical question with respect to network
>KS>  management traffic is the impact that it has or does not have on that
>KS>  real job.  To me this implies that the measurements MUST also include
>KS>  contextual information.  As an example, it is probably more important
>KS>  to understand whether  or not the network management traffic is
>KS>  causing significant congestion for the payload traffic than the
>KS>  particular mix or frequency pattern within the network management
>KS>  traffic.  Without out the complementary contextual information, the
>KS> whole measurement exercise seems to me to be of somewhat narrow
>KS>  value.
>
>JS> The measurement may be of narrow value from your point of view but
>JS> please keep in mind that this document is coming from the Network
>JS> Management Research Group and not from a general Network Measurement
>JS> Research Group. Our goal is to understand how network management
>JS> protocols are being used because that has impact on their design and
>JS> implementation strategies. Further note that in many networks, the
>JS> management traffic is logically and sometimes even physically
>JS> separated from the normal traffic and perhaps this is the reason why
>JS> we did not even think about the question whether management traffic
>JS> has an impact on normal traffic.
>
>KS> If you want to leave it as is, then I think it would be valuable to say as
>KS> much.  Be specific about what you are not doing, because much of the rest
>KS> of the world looks at network traffic from a broader perspective.
>
>I think the abstract and the introduction are pretty clearly spelling
>out the scope of the measurement effort. So I am not sure changes are
>needed, but see below.
>
>KS>  1. Section 1: It seems to me that there are TWO key questions with
>KS>  respect to SNMP.  The first is how it is being used, which in turn
>KS>  leads to the points made in this section, but the second is the
>KS>  impact of that traffic.  I think that ought to appear in the
>KS>  Introduction as well.
>
>JS> See my comments above. So far, the NMRG did not consider the impact of
>JS> SNMP on other traffic a target of this activity. I don't want to add
>JS> such text unless I see support from the NMRG and concrete proposals
>JS> what should be added.
>
>KS> Again, as above, if you want to leave the scope as it is, then you should
>KS> probably be up front about that, clearly leaving that work to another time
>KS> and place.
>
>I propose to add the following paragraph just before the last
>paragraph in section 1:
>
>    The measurement approach described in this document is by design
>    limited to the study of SNMP traffic.  Studies of other management
>    protocols or the impact of management protocols such as SNMP on
>    other traffic sharing the same network resources is left to future
>    efforts.

This is what I was looking for.

>KS>  2. Section 2.1.  The second paragraph begins with, "It is recommended
>KS>  to capture at least a full week of data."  This is never justified or
>KS>  explained.  Is one week really enough?  For what?  Why wouldn't
>KS>  several weeks be critical, because one week might be anomalous?  Why
>KS>  isn't a year critical, since we know that there are annual or
>KS>  seasonal differences in traffic behaviors?  Typically, I find that
>KS>  one-week data sets often leave me with lots of unanswered questions,
>KS>  so justify this.
>
>JS> The text actually says:
>JS>
>JS>    It is recommended to capture at least a full week of data.  Operators
>JS>    are encouraged to capture traces over even longer periods of time.
>JS>
>JS> The text tries to establish a lower bound of one week an encourages
>JS> longer capture periods. I would love to get continuous traces but
>JS> reality is such that this is not feasible. Our idea is simply to catch
>JS> at least the weekly behaviour. Yes, there is of course also monthly or
>JS> yearly behaviour but I believe it is not useful to set the bar so high
>JS> that nobody gives us appropriate traces. I personally believe the text
>JS> is fine as is.
>
>KS> So, what I was really getting at was the question of why one week
>KS> was the minimum necessary.  So, something like that it is the
>KS> minimum over which one can see the diurnal patterns in the weekly
>KS> pattern and it is understood that both for computational and
>KS> storage reasons the operators may not want to collect more.
>
>I have changed the text to the following:
>
>    It is recommended to capture at least a full week of data to capture
>    diurnal patterns and one cycle of weekly behavior.  Operators are
>    strongly encouraged to capture traces over even longer periods of
>    time.

OK by me.

>KS>  4. Section 3.3, end of first paragraph:  The sentence reads, "Some
>KS>  SNMP implementations approximate networking delays by measuring
>KS>  request-response times and it would be useful to understand to what
>KS>  extent this is a viable approach."  I agree, but traces will not tell
>KS>  you anything about whether behaviors observed in packet traces are
>KS>  for this reason or some other reason.  I do not believe you can get
>KS>  at this question with the data you are collecting.
>
>JS> I think it is possible to analyze retransmission behaviour. Depending
>JS> on the SNMP version used (and the other versions also depending on
>JS> implementation choices), you can get information whether a response is
>JS> just coming late for the original request or it is actually a response
>JS> to a retranmitted request. We are not talking TCP here; we are talking
>JS> about application layer retransmissions and SNMP has its own msgID and
>JS> requestID fields.
>
>KS> The point I was trying to make here is that it is very difficult to intuit
>KS> the reasons behind behaviors seen in the traffic, unless someone or
>KS> something tells you.  So, you can see what the ends do, but not why.
>
>I agree, but this is a rather general observation and not necessarily
>specific to 3.3 and it is not clear to me what I should do about it.
>There is already a general remark in the last paragraph of section 2.5
>that one has to be careful with drawing conclusions that go beyond
>what you can really get out of traces.

OK - I'll drop it.

>
>KS>  5. Section 3.4: Please explain why it is "interesting" (your word) to
>KS>  identify whether concurrency or sequentiality is occurring?  What
>KS>  will you "learn" if both are observed?  And, if one is occurring more
>KS>  frequently or under specific identifiable conditions, what further
>KS>  does that tell you?  Just knowing that one or the other occurs is
>KS>  only the tip of the iceberg, and without acknowledging the fact that
>KS>  these are important and unanswered questions, just learning first
>KS>  ordered details suggests you are setting the bar too low.
>
>JS> The introduction of section 3 says:
>JS>
>JS>    The questions raised in the following subsections are meant to be
>JS>    illustrative and no attempt has been made to provide a complete
>JS>    list.
>JS>
>JS> I believe it is a good idea to first figure out whether there is an
>JS> iceberg or not (keeping your analogy) and if there is one to ask
>JS> questions how big the iceberg might be. For SNMP agent implementations
>JS> that tend to do quite some caching, it is useful to know how well
>JS> caching strategies are working in real-world networks. The concurrency
>JS> level an agent experiences has clear impact on that. Furthermore, it
>JS> will be useful to know how bursty the traffic tends to be or how well
>JS> managers spread the traffic over polling intervals and this is again
>JS> related to the concurrency we can extract from traces.
>JS>
>JS> I am not really sure what I should change, perhaps the word
>JS> "interesting" is the source of the trouble and I should replace this
>JS> with "valuable"?
>
>KS> What I was trying to ask was that you tell the reader a bit about
>KS> what makes it interesting.  It certainly doesn't have to be
>KS> complete, but just a> hint.  I don't think you should change the
>KS> word "interesting", because if it isn't interesting, you probably
>KS> shouldn't be doing it.
>
>I added the following sentence to section 3.4:
>
>    The concurrency level and the amount of redundant requests has
>    implications on caching strategies employed by SNMP agents.

Fine.

>KS>  6. Section 3.5:  Please explain what you would do  with the
>KS>  information about which approach to table retrieval is used.  Again,
>KS>  what if the results tell you that both are used?  And, if not, of
>KS>  what use is it to know which approach is prevalent?  Mighten it be
>KS>  useful to know the conditions under which one or the other is used
>KS>  more commonly?
>
>JS> This again has direct impact on agent implementation techniques and
>JS> caching strategies.
>
>KS> And again, just explain a little.
>
>I changed the last sentence of section 3.5 to read as follows:
>
>    [...] It will be useful to know which of
>    these approaches are used on production networks since this has a
>    direct implication on agent implementation techniques and caching
>    strategies.

Fine.

>/js
>
>--
>Juergen Schoenwaelder           Jacobs University Bremen gGmbH
>Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
>Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

-- 

Karen R. Sollins, Ph. D.
Principal Research Scientist
M.I.T. CSAIL
The Stata Center
32 Vassar St., 32-G818
Cambridge, MA 02139
V: 617/253-6006
F: 617/253-2673
E: [email-masked]

From: "David B Harrington" <[email-masked]>
To: "'Karen R. Sollins'" <[email-masked]>, <[email-masked]>
Subject: RE: [nmrg] Re: [IRSG] review of draft-irtf-nmrg-snmp-measure-04.txt
Date: Mon, 19 May 2008 09:15:49 -0400
Cc: 'Internet Research Steering Group' <[email-masked]>, [email-masked]

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Karen R. Sollins
> Sent: Monday, May 19, 2008 12:05 AM
> To: [email-masked]; Karen R. Sollins
> Cc: Internet Research Steering Group; [email-masked]
> Subject: [nmrg] Re: [IRSG] review of 
> draft-irtf-nmrg-snmp-measure-04.txt
> 
[...]
> >>  3. Next paragraph: this is where the location question arises.
> >>  Without some completely standardized and self explanatory 
> capturing
> >>  of location information, any data set will be incomparable to
any
> >>  other.
> >
> >I expanded "where the trace was collected" to "where the trace was
> >collected (name of the network and/or name of the organization
owning
> >the network, description of the measurement point in the network
> >topology where the trace was collected)".
> 
> Good.
> 

I think something should be said that this information could be used
by an attacker (especially an attacker internal to the organization)
to decide/pinpoint where and how to attack. This information might
need to be anonymized, although that would seem to defeat the purpose
of having the information. I don't really know what to suggest here
other than to raise the point in the security considerations that such
location information might be sensitive, and could aid an attacker.

Personally, I do not know that one needs to know the organization and
the network within the organization unless you are planning to do
regression testing or correlating the data with information available
via other means. To compare the data with data sets from different
networks, then I think the measurement point in the network topology
and knowing the network topology is far more important (and unlikely
to be available). Given the dynamic nature of network topologies,
especially in any sort of virtualized environment (Virtual LANs,
routers, servers, etc.), I doubt any data set even from the same
network is likely to be directly comparable over time unless an
attempt is made to make the data collections directly comparable by
deliberately not changing the topology. I think a statement to that
effect might be more useful to those doing analyses than adding
organization/network information.

David Harrington
[email-masked]
[email-masked]
[email-masked]

Date: Mon, 19 May 2008 11:46:39 +0200
From: Juergen Schoenwaelder <[email-masked]>
To: "Karen R. Sollins" <[email-masked]>
Mail-Followup-To: "Karen R. Sollins" <[email-masked]>, [email-masked], Bert Wijnen - IETF <[email-masked]>, Internet Research Steering Group <[email-masked]>
Cc: Internet Research Steering Group <[email-masked]>, [email-masked]
Subject: [nmrg] Re: [IRSG] review of draft-irtf-nmrg-snmp-measure-04.txt
Reply-To: [email-masked]

On Sun, May 18, 2008 at 09:05:15PM -0700, Karen R. Sollins wrote:

> Thanks for your thoughtful responses.  I also did not think that what I was 
> suggesting was a lot of work.   At a high level, think about a reader who 
> is not part of your group, whom you are trying to convince that what you 
> are doing is valuable or whom you would like to convince to do such a data 
> collection exercise. I have interspersed my comments into your responses 
> below.

I will repond to those things that are still "open" with suggestions
on how to "close" them.

KS>  The second high level concern I have is that there is talk about
KS>  specific kinds of information to be collected and an interest in not
KS>  only the nature but longer term inferences with perhaps implications
KS>  for future redesign efforts in the SNMP context.    I have two levels
KS>  of concern here.  The most important one is that since network
KS>  management and in particular SNMP is NOT the primary objective of the
KS>  net (the primary objective being the transport of real payload), it
KS>  seems to me that the truly critical question with respect to network
KS>  management traffic is the impact that it has or does not have on that
KS>  real job.  To me this implies that the measurements MUST also include
KS>  contextual information.  As an example, it is probably more important
KS>  to understand whether  or not the network management traffic is
KS>  causing significant congestion for the payload traffic than the
KS>  particular mix or frequency pattern within the network management
KS>  traffic.  Without out the complementary contextual information, the
KS> whole measurement exercise seems to me to be of somewhat narrow
KS>  value. 

JS> The measurement may be of narrow value from your point of view but
JS> please keep in mind that this document is coming from the Network
JS> Management Research Group and not from a general Network Measurement
JS> Research Group. Our goal is to understand how network management
JS> protocols are being used because that has impact on their design and
JS> implementation strategies. Further note that in many networks, the
JS> management traffic is logically and sometimes even physically
JS> separated from the normal traffic and perhaps this is the reason why
JS> we did not even think about the question whether management traffic
JS> has an impact on normal traffic.

KS> If you want to leave it as is, then I think it would be valuable to say as 
KS> much.  Be specific about what you are not doing, because much of the rest 
KS> of the world looks at network traffic from a broader perspective.

I think the abstract and the introduction are pretty clearly spelling
out the scope of the measurement effort. So I am not sure changes are
needed, but see below.

KS>  1. Section 1: It seems to me that there are TWO key questions with
KS>  respect to SNMP.  The first is how it is being used, which in turn
KS>  leads to the points made in this section, but the second is the
KS>  impact of that traffic.  I think that ought to appear in the
KS>  Introduction as well.

JS> See my comments above. So far, the NMRG did not consider the impact of
JS> SNMP on other traffic a target of this activity. I don't want to add
JS> such text unless I see support from the NMRG and concrete proposals
JS> what should be added.

KS> Again, as above, if you want to leave the scope as it is, then you should 
KS> probably be up front about that, clearly leaving that work to another time 
KS> and place.

I propose to add the following paragraph just before the last
paragraph in section 1:

   The measurement approach described in this document is by design
   limited to the study of SNMP traffic.  Studies of other management
   protocols or the impact of management protocols such as SNMP on
   other traffic sharing the same network resources is left to future
   efforts.

KS>  2. Section 2.1.  The second paragraph begins with, "It is recommended
KS>  to capture at least a full week of data."  This is never justified or
KS>  explained.  Is one week really enough?  For what?  Why wouldn't
KS>  several weeks be critical, because one week might be anomalous?  Why
KS>  isn't a year critical, since we know that there are annual or
KS>  seasonal differences in traffic behaviors?  Typically, I find that
KS>  one-week data sets often leave me with lots of unanswered questions,
KS>  so justify this.

JS> The text actually says:
JS>
JS>    It is recommended to capture at least a full week of data.  Operators
JS>    are encouraged to capture traces over even longer periods of time.
JS>
JS> The text tries to establish a lower bound of one week an encourages
JS> longer capture periods. I would love to get continuous traces but
JS> reality is such that this is not feasible. Our idea is simply to catch
JS> at least the weekly behaviour. Yes, there is of course also monthly or
JS> yearly behaviour but I believe it is not useful to set the bar so high
JS> that nobody gives us appropriate traces. I personally believe the text
JS> is fine as is.

KS> So, what I was really getting at was the question of why one week
KS> was the minimum necessary.  So, something like that it is the
KS> minimum over which one can see the diurnal patterns in the weekly
KS> pattern and it is understood that both for computational and
KS> storage reasons the operators may not want to collect more.

I have changed the text to the following:

   It is recommended to capture at least a full week of data to capture
   diurnal patterns and one cycle of weekly behavior.  Operators are
   strongly encouraged to capture traces over even longer periods of
   time.  

KS>  4. Section 3.3, end of first paragraph:  The sentence reads, "Some
KS>  SNMP implementations approximate networking delays by measuring
KS>  request-response times and it would be useful to understand to what
KS>  extent this is a viable approach."  I agree, but traces will not tell
KS>  you anything about whether behaviors observed in packet traces are
KS>  for this reason or some other reason.  I do not believe you can get
KS>  at this question with the data you are collecting.

JS> I think it is possible to analyze retransmission behaviour. Depending
JS> on the SNMP version used (and the other versions also depending on
JS> implementation choices), you can get information whether a response is
JS> just coming late for the original request or it is actually a response
JS> to a retranmitted request. We are not talking TCP here; we are talking
JS> about application layer retransmissions and SNMP has its own msgID and
JS> requestID fields.

KS> The point I was trying to make here is that it is very difficult to intuit 
KS> the reasons behind behaviors seen in the traffic, unless someone or 
KS> something tells you.  So, you can see what the ends do, but not why.

I agree, but this is a rather general observation and not necessarily
specific to 3.3 and it is not clear to me what I should do about it.
There is already a general remark in the last paragraph of section 2.5
that one has to be careful with drawing conclusions that go beyond
what you can really get out of traces.

KS>  5. Section 3.4: Please explain why it is "interesting" (your word) to
KS>  identify whether concurrency or sequentiality is occurring?  What
KS>  will you "learn" if both are observed?  And, if one is occurring more
KS>  frequently or under specific identifiable conditions, what further
KS>  does that tell you?  Just knowing that one or the other occurs is
KS>  only the tip of the iceberg, and without acknowledging the fact that
KS>  these are important and unanswered questions, just learning first
KS>  ordered details suggests you are setting the bar too low.

JS> The introduction of section 3 says:
JS>
JS>    The questions raised in the following subsections are meant to be
JS>    illustrative and no attempt has been made to provide a complete
JS>    list.
JS>
JS> I believe it is a good idea to first figure out whether there is an
JS> iceberg or not (keeping your analogy) and if there is one to ask
JS> questions how big the iceberg might be. For SNMP agent implementations
JS> that tend to do quite some caching, it is useful to know how well
JS> caching strategies are working in real-world networks. The concurrency
JS> level an agent experiences has clear impact on that. Furthermore, it
JS> will be useful to know how bursty the traffic tends to be or how well
JS> managers spread the traffic over polling intervals and this is again
JS> related to the concurrency we can extract from traces.
JS>
JS> I am not really sure what I should change, perhaps the word
JS> "interesting" is the source of the trouble and I should replace this
JS> with "valuable"?

KS> What I was trying to ask was that you tell the reader a bit about
KS> what makes it interesting.  It certainly doesn't have to be
KS> complete, but just a> hint.  I don't think you should change the
KS> word "interesting", because if it isn't interesting, you probably
KS> shouldn't be doing it.

I added the following sentence to section 3.4:

   The concurrency level and the amount of redundant requests has
   implications on caching strategies employed by SNMP agents.

KS>  6. Section 3.5:  Please explain what you would do  with the
KS>  information about which approach to table retrieval is used.  Again,
KS>  what if the results tell you that both are used?  And, if not, of
KS>  what use is it to know which approach is prevalent?  Mighten it be
KS>  useful to know the conditions under which one or the other is used
KS>  more commonly?

JS> This again has direct impact on agent implementation techniques and
JS> caching strategies.

KS> And again, just explain a little.

I changed the last sentence of section 3.5 to read as follows:

   [...] It will be useful to know which of
   these approaches are used on production networks since this has a
   direct implication on agent implementation techniques and caching
   strategies.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

From: "Bert Wijnen - IETF" <[email-masked]>
To: "Karen R. Sollins" <[email-masked]>, <[email-masked]>
Date: Mon, 19 May 2008 10:11:37 +0200
Importance: Normal
Cc: Internet Research Steering Group <[email-masked]>, [email-masked]
Subject: [nmrg] RE: [IRSG] review of draft-irtf-nmrg-snmp-measure-04.txt

Thanks for the discussion sofar.

>From what I have seen during the development of this document
I believe that the scope of the document was intentionally
limited to what the document describes. In other weords, we
intentionally did not want to include the "impact of SNMP traffic
on otehr network activity".
>From the discussion below, I think what we need to do is to be
somewhat more explicit about the fact that and why we have
limited ourselves to that scope.

Karen, please let me know if you agree with my assesment.

Further Juergen has fixed a few things and made a few more changes
as per below discussion.

I asumme that the NMRG group does agree with this approach.
If not, please speak up quickly/soon.

Bert Wijnen
document shepherd

> -----Oorspronkelijk bericht-----
> Van: Karen R. Sollins [mailto:[email-masked]]
> Verzonden: maandag 19 mei 2008 6:05
> Aan: [email-masked]; Karen R. Sollins
> CC: [email-masked]; Bert Wijnen - IETF; Internet Research Steering
> Group
> Onderwerp: Re: [IRSG] review of draft-irtf-nmrg-snmp-measure-04.txt
>
>
> HI Juergen,
>
> Thanks for your thoughtful responses.  I also did not think that what
> I was suggesting was a lot of work.   At a high level, think about a
> reader who is not part of your group, whom you are trying to convince
> that what you are doing is valuable or whom you would like to
> convince to do such a data collection exercise. I have interspersed
> my comments into your responses below.
>
> 			Cheers,
> 			Karen
>
> At 2:20 PM +0200 5/16/08, Juergen Schoenwaelder wrote:
> >On Mon, May 12, 2008 at 02:52:02PM -0400, Karen R. Sollins wrote:
> >
> >>  Here is my full review of draft-irtf-nmrg-snmp-measure-04.txt.
> >
> >Thank you very much for your substantial review. Below is how I plan
> >to address your comments. For most of your comments, the changes are
> >easy to do. For some of your comments, I suggest to make no changes
> >and I try to explain why. Please check and let me and the RG know
> >where you do not agree with my suggested plan of action.
> >
> >>  I do not think this document is quite ready for publication, but it
> >>  is quite close and I don't think this should be major work.
> >>
> >>  My review falls into two parts.  The first is broad comments and the
> >>  second is nits.  Overall, I agree with the working group that it is
> >>  valuable to spell out a set of details such as this, more or less a
> >>  "protocol" in the sense of biologically based research, describing
> >>  what should be done, what kind of information should be collected and
> >>  why and how to handle it.   I have not reviewed the XML or CSV
> >>  representations nor the reference lists in detail.  With that said, I
> >>  have two high level concerns with this document.
> >>
> >>  The first is that there is confusion about whether this document
> >>  describes a proposed measurement effort or (closer to the reality of
> >>  what is in the document) a methodology by which such measurements
> >>  could and perhaps should be taken.  So, given the bulk of the
> >>  document, I do not believe that it is at all explicit about the
> >>  particular measurement exercise that will be done.   With that said,
> >>  there are a few introductory points that are worth changing.
> >>
> >>  1. The abstract includes the sentence, "This document proposes to
> >>  carry out large scale SNMP traffic measurements."  First, the
> >>  document won't be carrying out anything.  More importantly the
> >>  document describes an approach (not even a pan), so I suggest that
> >>  the sentence read, "This document describes an approach to carrying
> >>  out..."
> >
> >changed as suggested
> >
> >>  2. In Section 1, the 4th paragraph reads, "This document describes an
> >>  effort to collect SNMP traffic traces..."  Instead, I suggest that it
> >>  read, "This document recommends an approach to collecting, codifying,
> >>  and handling SNMP traffic traces..."
> >
> >changed as suggested
> >
> >>  The second high level concern I have is that there is talk about
> >>  specific kinds of information to be collected and an interest in not
> >>  only the nature but longer term inferences with perhaps implications
> >>  for future redesign efforts in the SNMP context.    I have two levels
> >>  of concern here.  The most important one is that since network
> >>  management and in particular SNMP is NOT the primary objective of the
> >>  net (the primary objective being the transport of real payload), it
> >>  seems to me that the truly critical question with respect to network
> >>  management traffic is the impact that it has or does not have on that
> >>  real job.  To me this implies that the measurements MUST also include
> >>  contextual information.  As an example, it is probably more important
> >>  to understand whether  or not the network management traffic is
> >>  causing significant congestion for the payload traffic than the
> >>  particular mix or frequency pattern within the network management
> >>  traffic.  Without out the complementary contextual information, the
> >  > whole measurement exercise seems to me to be of somewhat narrow
> >>  value.
> >
> >The measurement may be of narrow value from your point of view but
> >please keep in mind that this document is coming from the Network
> >Management Research Group and not from a general Network Measurement
> >Research Group. Our goal is to understand how network management
> >protocols are being used because that has impact on their design and
> >implementation strategies. Further note that in many networks, the
> >management traffic is logically and sometimes even physically
> >separated from the normal traffic and perhaps this is the reason why
> >we did not even think about the question whether management traffic
> >has an impact on normal traffic.
>
> If you want to leave it as is, then I think it would be valuable to
> say as much.  Be specific about what you are not doing, because much
> of the rest of the world looks at network traffic from a broader
> perspective.
>
> >
> >>  Secondarily, the document hits on one of my little pet peeves
> >>  - location.  There is mention that the meta-data must include "where
> >>  the trace was collected".  That, in and of itself, is a challenging
> >>  problem.  Is this geographic, topological, or based on other metrics?
> >>  If it is based on something that can change with time (e.g. ip
> >>  addresses or topology), then without keeping track of how those
> >>  metrics may have changed over time, the data set becomes useless
> >>  because the location becomes uninterpretable.  As you all hint, it is
> >>  important that the collected information be archived, in order to be
> >>  useful at significantly later times, and thus, everything importatn
> >>  about the situation must be recorded.  I would like to see more
> >>  specificity with respect to location and clearer acknowledgement of
> >>  its need in order to keep the collected information useful outside
> >>  the current context.
> >
> >Makes sense.
> >
> >>  More generally, there are many cases in which there are statements
> >>  about what kind of information should be collected (e.g.one week
> >>  traces, concurrent vs. sequential requests, etc.) where there is no
> >>  explanation of the criticality of this information or how to go
> >>  beyond the immediately observable facts.
> >>
> >>  I realize that it is generally not in the nature of RFC to provide
> >>  explanation as much as simple unadorned facts, but this is an unusual
> >>  kind of document, because it is describing a potential information
> >>  gathering exercise.  It has some of the explanations of why things
> >>  are important, but it is quite incomplete.  I realize that it is
> >>  important to collect and archive measurement sets exactly because one
> >>  cannot know ahead of time all the potential uses, but there should
> >>  either be a story about collecting everything possible or an initial
> >>  use for each kind of information collected.  In some cases neither
> >>  argument is presented.
> >>
> >>  This all leads to a number of specific questions:
> >>
> >>  1. Section 1: It seems to me that there are TWO key questions with
> >>  respect to SNMP.  The first is how it is being used, which in turn
> >>  leads to the points made in this section, but the second is the
> >>  impact of that traffic.  I think that ought to appear in the
> >>  Introduction as well.
> >
> >See my comments above. So far, the NMRG did not consider the impact of
> >SNMP on other traffic a target of this activity. I don't want to add
> >such text unless I see support from the NMRG and concrete proposals
> >what should be added.
>
> Again, as above, if you want to leave the scope as it is, then you
> should probably be up front about that, clearly leaving that work to
> another time and place.
>
> >
> >>  2. Section 2.1.  The second paragraph begins with, "It is recommended
> >>  to capture at least a full week of data."  This is never justified or
> >>  explained.  Is one week really enough?  For what?  Why wouldn't
> >>  several weeks be critical, because one week might be anomalous?  Why
> >>  isn't a year critical, since we know that there are annual or
> >>  seasonal differences in traffic behaviors?  Typically, I find that
> >>  one-week data sets often leave me with lots of unanswered questions,
> >>  so justify this.
> >
> >The text actually says:
> >
> >    It is recommended to capture at least a full week of data.  Operators
> >    are encouraged to capture traces over even longer periods of time.
> >
> >The text tries to establish a lower bound of one week an encourages
> >longer capture periods. I would love to get continuous traces but
> >reality is such that this is not feasible. Our idea is simply to catch
> >at least the weekly behaviour. Yes, there is of course also monthly or
> >yearly behaviour but I believe it is not useful to set the bar so high
> >that nobody gives us appropriate traces. I personally believe the text
> >is fine as is.
>
> So, what I was really getting at was the question of why one week was
> the minimum necessary.  So, something like that it is the minimum
> over which one can see the diurnal patterns in the weekly pattern and
> it is understood that both for computational and storage reasons the
> operators may not want to collect more.
>
> >
> >>  3. Next paragraph: this is where the location question arises.
> >>  Without some completely standardized and self explanatory capturing
> >>  of location information, any data set will be incomparable to any
> >>  other.
> >
> >I expanded "where the trace was collected" to "where the trace was
> >collected (name of the network and/or name of the organization owning
> >the network, description of the measurement point in the network
> >topology where the trace was collected)".
>
> Good.
>
> >
> >>  4. Section 3.3, end of first paragraph:  The sentence reads, "Some
> >>  SNMP implementations approximate networking delays by measuring
> >>  request-response times and it would be useful to understand to what
> >>  extent this is a viable approach."  I agree, but traces will not tell
> >>  you anything about whether behaviors observed in packet traces are
> >>  for this reason or some other reason.  I do not believe you can get
> >>  at this question with the data you are collecting.
> >
> >I think it is possible to analyze retransmission behaviour. Depending
> >on the SNMP version used (and the other versions also depending on
> >implementation choices), you can get information whether a response is
> >just coming late for the original request or it is actually a response
> >to a retranmitted request. We are not talking TCP here; we are talking
> >about application layer retransmissions and SNMP has its own msgID and
> >requestID fields.
>
> The point I was trying to make here is that it is very difficult to
> intuit the reasons behind behaviors seen in the traffic, unless
> someone or something tells you.  So, you can see what the ends do,
> but not why.
>
> >
> >>  5. Section 3.4: Please explain why it is "interesting" (your word) to
> >>  identify whether concurrency or sequentiality is occurring?  What
> >>  will you "learn" if both are observed?  And, if one is occurring more
> >>  frequently or under specific identifiable conditions, what further
> >>  does that tell you?  Just knowing that one or the other occurs is
> >>  only the tip of the iceberg, and without acknowledging the fact that
> >>  these are important and unanswered questions, just learning first
> >>  ordered details suggests you are setting the bar too low.
> >
> >The introduction of section 3 says:
> >
> >    The questions raised in the following subsections are meant to be
> >    illustrative and no attempt has been made to provide a complete
> >    list.
> >
> >I believe it is a good idea to first figure out whether there is an
> >iceberg or not (keeping your analogy) and if there is one to ask
> >questions how big the iceberg might be. For SNMP agent implementations
> >that tend to do quite some caching, it is useful to know how well
> >caching strategies are working in real-world networks. The concurrency
> >level an agent experiences has clear impact on that. Furthermore, it
> >will be useful to know how bursty the traffic tends to be or how well
> >managers spread the traffic over polling intervals and this is again
> >related to the concurrency we can extract from traces.
> >
> >I am not really sure what I should change, perhaps the word
> >"interesting" is the source of the trouble and I should replace this
> >with "valuable"?
>
> What I was trying to ask was that you tell the reader a bit about
> what makes it interesting.  It certainly doesn't have to be complete,
> but just a hint.  I don't think you should change the word
> "interesting", because if it isn't interesting, you probably
> shouldn't be doing it.
>
> >
> >>  6. Section 3.5:  Please explain what you would do  with the
> >>  information about which approach to table retrieval is used.  Again,
> >>  what if the results tell you that both are used?  And, if not, of
> >>  what use is it to know which approach is prevalent?  Mighten it be
> >>  useful to know the conditions under which one or the other is used
> >  > more commonly?
> >
> >This again has direct impact on agent implementation techniques and
> >caching strategies.
>
> And again, just explain a little.
>
> >
> >>  Now on to the real NITS:
> >>  1. p. 1, Section 1, middle of second paragraph:
> >>  s/In fact, there are many speculations how SNMP/In fact, there are
> >>  many speculations on how SNMP/
> >
> >fixed
> >
> >>  2. p. 6, Section 2.2, 5th paragraph:
> >>  s/for SNMP messages that got fragmented/for SNMP messages that
> >>were fragmented/
> >
> >fixed
> >
> >>  3. p. 7, Section 2.4, last paragraph:
> >>  s/Improvements in the tool chain may require to go back to the
> >>  original pcap traces and to rebuild all intermediate formats from
> >>  them./Improvements in the tool chain may require going back to the
> >>  original pcap traces and rebuilding all intermediate formats from
> >>  them./
> >
> >fixed
> >
> >>  4. p. 7, Section 2.5, first paragraph:
> >>  s/all scripts used to analyze traffic traces would be/all scripts
> >>  used to analyze traffic traces will be/
> >
> >fixed
> >
> >>  5. p. 8, first sentence:
> >>  s/A common versioned repository for analysis of scripts might be
> >>  useful to establish./It might be useful to establish and common,
> >>  versioning repository for analysis scripts./
> >
> >fixed
> >
> >>  6. p. 6, second paragraph (includes several corrections):
> >>  s/it is suggested that analysis scripts are written in scripting
> >>  languages such as Perl using suitable Perl modules to manipulate XML
> >>  documents [8].  Using a scripting language such as Perl instead of
> >>  system programming languages such as C or C++ has the advantage to
> >>  reduce development time and to make scripts more accessible to
> >>  operators who may want to verify scripts before running them on trace
> >>  files which potentially contain sensitive data./it is suggested that
> >>  analysis scripts be written is scripting languages such as Perl using
> >>  suitable Perl modules to manipulate XML documents [8].  Using a
> >>  scripting language such as Perl instead of system programming
> >>  languages such as C or C++ has the advantage of reducing development
> >>  time and making scripts more accessible to operators who may want to
> >>  verify scripts before running them on trace files which may contain
> >>  sensitive data./
> >
> >fixed
> >
> >>  6. p. 6, end of next paragraph:
> >>  s/might be the only option to deal/might be the only option
> in dealing/
> >
> >fixed
> >
> >>  7. p. 9, Section 3.2.  In this case there are enough corrections in
> >>  close proximity that I've grouped them together.  I suggest that the
> >>  following text:
> >>
> >>  "SNMP is used to periodically poll devices as well as to retrieve
> >>  information on request of an operator or application.  The periodic
> >>  polling leads to periodic traffic patterns while the on demand
> >>  information retrieval causes more aperiodic traffic patterns.  It is
> >>  worthwhile to understand what the relationship is between the amount
> >>  of periodic and aperiodic traffic.  It will be interesting to
> >>  research whether there are multiple levels of periodicity at
> >>  different time scales.
> >>
> >>  The periodic polling behavior may dependent on the application and
> >>  the polling engine it uses.  For example, some management platforms
> >>  allow applications to specify how long polled values may be kept in a
> >>  cache before it is polled again."
> >>
> >>  have the following grammatical changes made:
> >>
> >>  "SNMP is used to periodically poll devices as well as to retrieve
> >>  information at the request of an operator or application.  The
> >>  periodic polling leads to periodic traffic patterns while on-demand
> >>  information retrieval causes more aperiodic traffic patterns.  It is
> >>  worthwhile to understand what the relationship is between the amount
> >>  of periodic and aperiodic traffic.  It will be interesting to
> >>  understand whether there are multiple levels of periodicity at
> >>  different time scales.
> >>
> >>  Periodic polling behavior may be dependent on the application and
> >  > polling engine it uses.  For example, some management platforms allow
> >>  applications to specify how long polled values may be kept in a cache
> >>  before they are polled again."
> >
> >fixed
> >
> >>  8. p. 10, Section 3.6, middle first paragraph:
> >>  s/ adopt/adapt/
> >
> >fixed
> >
> >/js
> >
> >--
> >Juergen Schoenwaelder           Jacobs University Bremen gGmbH
> >Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
> >Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>
>
>
> --
>
> Karen R. Sollins, Ph. D.
> Principal Research Scientist
> M.I.T. CSAIL
> The Stata Center
> 32 Vassar St., 32-G818
> Cambridge, MA 02139
> V: 617/253-6006
> F: 617/253-2673
> E: [email-masked]
>

Date: Sun, 18 May 2008 21:05:15 -0700
To: [email-masked], "Karen R. Sollins" <[email-masked]>
From: "Karen R. Sollins" <[email-masked]>
Cc: Internet Research Steering Group <[email-masked]>, [email-masked]
Subject: [nmrg] Re: [IRSG] review of draft-irtf-nmrg-snmp-measure-04.txt

HI Juergen,

Thanks for your thoughtful responses.  I also did not think that what 
I was suggesting was a lot of work.   At a high level, think about a 
reader who is not part of your group, whom you are trying to convince 
that what you are doing is valuable or whom you would like to 
convince to do such a data collection exercise. I have interspersed 
my comments into your responses below.

			Cheers,
			Karen

At 2:20 PM +0200 5/16/08, Juergen Schoenwaelder wrote:
>On Mon, May 12, 2008 at 02:52:02PM -0400, Karen R. Sollins wrote:
>
>>  Here is my full review of draft-irtf-nmrg-snmp-measure-04.txt.
>
>Thank you very much for your substantial review. Below is how I plan
>to address your comments. For most of your comments, the changes are
>easy to do. For some of your comments, I suggest to make no changes
>and I try to explain why. Please check and let me and the RG know
>where you do not agree with my suggested plan of action.
>
>>  I do not think this document is quite ready for publication, but it
>>  is quite close and I don't think this should be major work.
>>
>>  My review falls into two parts.  The first is broad comments and the
>>  second is nits.  Overall, I agree with the working group that it is
>>  valuable to spell out a set of details such as this, more or less a
>>  "protocol" in the sense of biologically based research, describing
>>  what should be done, what kind of information should be collected and
>>  why and how to handle it.   I have not reviewed the XML or CSV
>>  representations nor the reference lists in detail.  With that said, I
>>  have two high level concerns with this document.
>>
>>  The first is that there is confusion about whether this document
>>  describes a proposed measurement effort or (closer to the reality of
>>  what is in the document) a methodology by which such measurements
>>  could and perhaps should be taken.  So, given the bulk of the
>>  document, I do not believe that it is at all explicit about the
>>  particular measurement exercise that will be done.   With that said,
>>  there are a few introductory points that are worth changing.
>>
>>  1. The abstract includes the sentence, "This document proposes to
>>  carry out large scale SNMP traffic measurements."  First, the
>>  document won't be carrying out anything.  More importantly the
>>  document describes an approach (not even a pan), so I suggest that
>>  the sentence read, "This document describes an approach to carrying
>>  out..."
>
>changed as suggested
>
>>  2. In Section 1, the 4th paragraph reads, "This document describes an
>>  effort to collect SNMP traffic traces..."  Instead, I suggest that it
>>  read, "This document recommends an approach to collecting, codifying,
>>  and handling SNMP traffic traces..."
>
>changed as suggested
>
>>  The second high level concern I have is that there is talk about
>>  specific kinds of information to be collected and an interest in not
>>  only the nature but longer term inferences with perhaps implications
>>  for future redesign efforts in the SNMP context.    I have two levels
>>  of concern here.  The most important one is that since network
>>  management and in particular SNMP is NOT the primary objective of the
>>  net (the primary objective being the transport of real payload), it
>>  seems to me that the truly critical question with respect to network
>>  management traffic is the impact that it has or does not have on that
>>  real job.  To me this implies that the measurements MUST also include
>>  contextual information.  As an example, it is probably more important
>>  to understand whether  or not the network management traffic is
>>  causing significant congestion for the payload traffic than the
>>  particular mix or frequency pattern within the network management
>>  traffic.  Without out the complementary contextual information, the
>  > whole measurement exercise seems to me to be of somewhat narrow
>>  value. 
>
>The measurement may be of narrow value from your point of view but
>please keep in mind that this document is coming from the Network
>Management Research Group and not from a general Network Measurement
>Research Group. Our goal is to understand how network management
>protocols are being used because that has impact on their design and
>implementation strategies. Further note that in many networks, the
>management traffic is logically and sometimes even physically
>separated from the normal traffic and perhaps this is the reason why
>we did not even think about the question whether management traffic
>has an impact on normal traffic.

If you want to leave it as is, then I think it would be valuable to 
say as much.  Be specific about what you are not doing, because much 
of the rest of the world looks at network traffic from a broader 
perspective.

>
>>  Secondarily, the document hits on one of my little pet peeves
>>  - location.  There is mention that the meta-data must include "where
>>  the trace was collected".  That, in and of itself, is a challenging
>>  problem.  Is this geographic, topological, or based on other metrics?
>>  If it is based on something that can change with time (e.g. ip
>>  addresses or topology), then without keeping track of how those
>>  metrics may have changed over time, the data set becomes useless
>>  because the location becomes uninterpretable.  As you all hint, it is
>>  important that the collected information be archived, in order to be
>>  useful at significantly later times, and thus, everything importatn
>>  about the situation must be recorded.  I would like to see more
>>  specificity with respect to location and clearer acknowledgement of
>>  its need in order to keep the collected information useful outside
>>  the current context.
>
>Makes sense.
>
>>  More generally, there are many cases in which there are statements
>>  about what kind of information should be collected (e.g.one week
>>  traces, concurrent vs. sequential requests, etc.) where there is no
>>  explanation of the criticality of this information or how to go
>>  beyond the immediately observable facts.
>>
>>  I realize that it is generally not in the nature of RFC to provide
>>  explanation as much as simple unadorned facts, but this is an unusual
>>  kind of document, because it is describing a potential information
>>  gathering exercise.  It has some of the explanations of why things
>>  are important, but it is quite incomplete.  I realize that it is
>>  important to collect and archive measurement sets exactly because one
>>  cannot know ahead of time all the potential uses, but there should
>>  either be a story about collecting everything possible or an initial
>>  use for each kind of information collected.  In some cases neither
>>  argument is presented.
>>
>>  This all leads to a number of specific questions:
>>
>>  1. Section 1: It seems to me that there are TWO key questions with
>>  respect to SNMP.  The first is how it is being used, which in turn
>>  leads to the points made in this section, but the second is the
>>  impact of that traffic.  I think that ought to appear in the
>>  Introduction as well.
>
>See my comments above. So far, the NMRG did not consider the impact of
>SNMP on other traffic a target of this activity. I don't want to add
>such text unless I see support from the NMRG and concrete proposals
>what should be added.

Again, as above, if you want to leave the scope as it is, then you 
should probably be up front about that, clearly leaving that work to 
another time and place.

>
>>  2. Section 2.1.  The second paragraph begins with, "It is recommended
>>  to capture at least a full week of data."  This is never justified or
>>  explained.  Is one week really enough?  For what?  Why wouldn't
>>  several weeks be critical, because one week might be anomalous?  Why
>>  isn't a year critical, since we know that there are annual or
>>  seasonal differences in traffic behaviors?  Typically, I find that
>>  one-week data sets often leave me with lots of unanswered questions,
>>  so justify this.
>
>The text actually says:
>
>    It is recommended to capture at least a full week of data.  Operators
>    are encouraged to capture traces over even longer periods of time.
>
>The text tries to establish a lower bound of one week an encourages
>longer capture periods. I would love to get continuous traces but
>reality is such that this is not feasible. Our idea is simply to catch
>at least the weekly behaviour. Yes, there is of course also monthly or
>yearly behaviour but I believe it is not useful to set the bar so high
>that nobody gives us appropriate traces. I personally believe the text
>is fine as is.

So, what I was really getting at was the question of why one week was 
the minimum necessary.  So, something like that it is the minimum 
over which one can see the diurnal patterns in the weekly pattern and 
it is understood that both for computational and storage reasons the 
operators may not want to collect more.

>
>>  3. Next paragraph: this is where the location question arises.
>>  Without some completely standardized and self explanatory capturing
>>  of location information, any data set will be incomparable to any
>>  other.
>
>I expanded "where the trace was collected" to "where the trace was
>collected (name of the network and/or name of the organization owning
>the network, description of the measurement point in the network
>topology where the trace was collected)".

Good.

>
>>  4. Section 3.3, end of first paragraph:  The sentence reads, "Some
>>  SNMP implementations approximate networking delays by measuring
>>  request-response times and it would be useful to understand to what
>>  extent this is a viable approach."  I agree, but traces will not tell
>>  you anything about whether behaviors observed in packet traces are
>>  for this reason or some other reason.  I do not believe you can get
>>  at this question with the data you are collecting.
>
>I think it is possible to analyze retransmission behaviour. Depending
>on the SNMP version used (and the other versions also depending on
>implementation choices), you can get information whether a response is
>just coming late for the original request or it is actually a response
>to a retranmitted request. We are not talking TCP here; we are talking
>about application layer retransmissions and SNMP has its own msgID and
>requestID fields.

The point I was trying to make here is that it is very difficult to 
intuit the reasons behind behaviors seen in the traffic, unless 
someone or something tells you.  So, you can see what the ends do, 
but not why.

>
>>  5. Section 3.4: Please explain why it is "interesting" (your word) to
>>  identify whether concurrency or sequentiality is occurring?  What
>>  will you "learn" if both are observed?  And, if one is occurring more
>>  frequently or under specific identifiable conditions, what further
>>  does that tell you?  Just knowing that one or the other occurs is
>>  only the tip of the iceberg, and without acknowledging the fact that
>>  these are important and unanswered questions, just learning first
>>  ordered details suggests you are setting the bar too low.
>
>The introduction of section 3 says:
>
>    The questions raised in the following subsections are meant to be
>    illustrative and no attempt has been made to provide a complete
>    list.
>
>I believe it is a good idea to first figure out whether there is an
>iceberg or not (keeping your analogy) and if there is one to ask
>questions how big the iceberg might be. For SNMP agent implementations
>that tend to do quite some caching, it is useful to know how well
>caching strategies are working in real-world networks. The concurrency
>level an agent experiences has clear impact on that. Furthermore, it
>will be useful to know how bursty the traffic tends to be or how well
>managers spread the traffic over polling intervals and this is again
>related to the concurrency we can extract from traces.
>
>I am not really sure what I should change, perhaps the word
>"interesting" is the source of the trouble and I should replace this
>with "valuable"?

What I was trying to ask was that you tell the reader a bit about 
what makes it interesting.  It certainly doesn't have to be complete, 
but just a hint.  I don't think you should change the word 
"interesting", because if it isn't interesting, you probably 
shouldn't be doing it.

>
>>  6. Section 3.5:  Please explain what you would do  with the
>>  information about which approach to table retrieval is used.  Again,
>>  what if the results tell you that both are used?  And, if not, of
>>  what use is it to know which approach is prevalent?  Mighten it be
>>  useful to know the conditions under which one or the other is used
>  > more commonly?
>
>This again has direct impact on agent implementation techniques and
>caching strategies.

And again, just explain a little.

>
>>  Now on to the real NITS:
>>  1. p. 1, Section 1, middle of second paragraph:
>>  s/In fact, there are many speculations how SNMP/In fact, there are
>>  many speculations on how SNMP/
>
>fixed
>
>>  2. p. 6, Section 2.2, 5th paragraph:
>>  s/for SNMP messages that got fragmented/for SNMP messages that 
>>were fragmented/
>
>fixed
>
>>  3. p. 7, Section 2.4, last paragraph:
>>  s/Improvements in the tool chain may require to go back to the
>>  original pcap traces and to rebuild all intermediate formats from
>>  them./Improvements in the tool chain may require going back to the
>>  original pcap traces and rebuilding all intermediate formats from
>>  them./
>
>fixed
>
>>  4. p. 7, Section 2.5, first paragraph:
>>  s/all scripts used to analyze traffic traces would be/all scripts
>>  used to analyze traffic traces will be/
>
>fixed
>
>>  5. p. 8, first sentence:
>>  s/A common versioned repository for analysis of scripts might be
>>  useful to establish./It might be useful to establish and common,
>>  versioning repository for analysis scripts./
>
>fixed
>
>>  6. p. 6, second paragraph (includes several corrections):
>>  s/it is suggested that analysis scripts are written in scripting
>>  languages such as Perl using suitable Perl modules to manipulate XML
>>  documents [8].  Using a scripting language such as Perl instead of
>>  system programming languages such as C or C++ has the advantage to
>>  reduce development time and to make scripts more accessible to
>>  operators who may want to verify scripts before running them on trace
>>  files which potentially contain sensitive data./it is suggested that
>>  analysis scripts be written is scripting languages such as Perl using
>>  suitable Perl modules to manipulate XML documents [8].  Using a
>>  scripting language such as Perl instead of system programming
>>  languages such as C or C++ has the advantage of reducing development
>>  time and making scripts more accessible to operators who may want to
>>  verify scripts before running them on trace files which may contain
>>  sensitive data./
>
>fixed
>
>>  6. p. 6, end of next paragraph:
>>  s/might be the only option to deal/might be the only option in dealing/
>
>fixed
>
>>  7. p. 9, Section 3.2.  In this case there are enough corrections in
>>  close proximity that I've grouped them together.  I suggest that the
>>  following text:
>>
>>  "SNMP is used to periodically poll devices as well as to retrieve
>>  information on request of an operator or application.  The periodic
>>  polling leads to periodic traffic patterns while the on demand
>>  information retrieval causes more aperiodic traffic patterns.  It is
>>  worthwhile to understand what the relationship is between the amount
>>  of periodic and aperiodic traffic.  It will be interesting to
>>  research whether there are multiple levels of periodicity at
>>  different time scales.
>>
>>  The periodic polling behavior may dependent on the application and
>>  the polling engine it uses.  For example, some management platforms
>>  allow applications to specify how long polled values may be kept in a
>>  cache before it is polled again."
>>
>>  have the following grammatical changes made:
>>
>>  "SNMP is used to periodically poll devices as well as to retrieve
>>  information at the request of an operator or application.  The
>>  periodic polling leads to periodic traffic patterns while on-demand
>>  information retrieval causes more aperiodic traffic patterns.  It is
>>  worthwhile to understand what the relationship is between the amount
>>  of periodic and aperiodic traffic.  It will be interesting to
>>  understand whether there are multiple levels of periodicity at
>>  different time scales.
>>
>>  Periodic polling behavior may be dependent on the application and
>  > polling engine it uses.  For example, some management platforms allow
>>  applications to specify how long polled values may be kept in a cache
>>  before they are polled again."
>
>fixed
>
>>  8. p. 10, Section 3.6, middle first paragraph:
>>  s/ adopt/adapt/
>
>fixed
>
>/js
>
>--
>Juergen Schoenwaelder           Jacobs University Bremen gGmbH
>Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
>Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

-- 

Karen R. Sollins, Ph. D.
Principal Research Scientist
M.I.T. CSAIL
The Stata Center
32 Vassar St., 32-G818
Cambridge, MA 02139
V: 617/253-6006
F: 617/253-2673
E: [email-masked]

Content-class: urn:content-classes:message
Subject: RE: [OPSAWG] [nmrg] draft-irtf-nmrg-snmp-measure-04.txt oidxmlpattern change
Date: Fri, 16 May 2008 10:27:05 -0400
From: "Natale, Bob" <[email-masked]>
To: "David Harrington" <[email-masked]>, <[email-masked]>, "Bert Wijnen" <[email-masked]>
Cc: [email-masked], [email-masked]

Hi David,

You are right about the fidelity basis for SNMP to ASN.1/BER, but I
don't think the version diffs come into play on this point.  The rule
for BER encoding of the first two sub-ids (X and Y) probably always has
boiled down to "(X*40)+Y" -- which was expressed as far back as
Steedman's book as "40m + n", where (p. 93) he says:

"This saves an octet from every value, exploiting the fact that 0 <= m
<= 2, and that, except possibly for m = 2 (joint-iso-ccitt), n is a
very small number."

Ahhh, the good old days! :)

There is a registered value for 2.41 (ITU-T Rec. X.1083 | ISO/IEC
24708: BioAPI Interworking Protocol (BIP)), approved in Sep-2007, which
requires Juergen's correction...and one could imagine that it could
need to be carried in an SNMP varbind.

Cheers,
BobN

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On
Behalf Of David Harrington
Sent: Friday, May 16, 2008 10:14 AM
To: Natale, Bob; [email-masked]; 'Bert Wijnen'
Cc: [email-masked]; [email-masked]
Subject: Re: [OPSAWG] [nmrg] draft-irtf-nmrg-snmp-measure-04.txt
oidxmlpattern change

Hi,

SMIv2 is based on ASN.1-1988 not ASN.1-2002.
XSDMI should maintain fidelity to SMIv2.
I would think the snmp-measure document should do so also.

We should be liberal in what we accept, and conservative in what we
send, of course. I think XSDMI should enforce the 1988 syntax, since
one expected use case is validation; it probably will make no
difference to the snmp-research document, whose use case is reporting.

In practice, I have never seen a 2.X in any SMIv2 MIB module.

dbh 

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Natale, Bob
> Sent: Friday, May 16, 2008 9:48 AM
> To: [email-masked]; Bert Wijnen
> Cc: [email-masked]; [email-masked]
> Subject: Re: [OPSAWG] [nmrg] 
> draft-irtf-nmrg-snmp-measure-04.txt oid xmlpattern change
> 
> Hi,
> 
> Juergen is correct (per Sec. 8.19 of ITU-T Recommendation X.690,
> prepared by ITU-T Study Group 17 and approved on 14 July 2002,
> identical text is also published as ISO/IEC 8825-1).
> 
> draft-ietf-opsawg-smi-datatypes-in-xsd-02.txt (to be posted any day
> now!) will contain the fix Juergen provides below.
> 
> Cheers,
> BobN
> 
> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]] On Behalf Of Juergen
> Schoenwaelder
> Sent: Friday, May 16, 2008 8:27 AM
> To: [email-masked]; Bert Wijnen
> Subject: [nmrg] draft-irtf-nmrg-snmp-measure-04.txt oid xml pattern
> change
> 
> Hi,
> 
> there was recently a discussion on the XSD pattern for object
> identifier values and it was pointed out that the pattern we picked
up
> from the SMI-XSD document is too restrictive by not allowing OIDs of
> the format 2.X.* where X is larger than 39. I therefore propose that
> we change the pattern used by oid.type from
> 
>       "[0-2](\.[1-3]?[0-9])(\.(0|([1-9]\d*))){0,126}"
> 
> to
> 
>       "([0-1](\.[1-3]?[0-9]))|(2.(0|([1-9]\d*)))" ~
>       "(\.(0|([1-9]\d*))){0,126}"
> 
> Any objections against this change which is really a bugfix? Bert,
do
> we run into procedural trouble if we fix this or an additional N
weeks
> delay? (In that case, I would be tempted to go along with the buggy
> version of the pattern. ;-)
> 
> /js
> 
> -- 
> Juergen Schoenwaelder           Jacobs University Bremen gGmbH
> Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
> Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>
> -- 
> !! This message is brought to you via the `nmrg' mailing list.
> !! Please do not reply to this message to unsubscribe. To
unsubscribe
> or adjust
> !! your settings, send a mail message to 
> <[email-masked]>
> !! or look at https://mail.ibr.cs.tu-bs.de/mailman/listinfo/nmrg.
> _______________________________________________
> OPSAWG mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/opsawg
> 

_______________________________________________
OPSAWG mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/opsawg

From: "David Harrington" <[email-masked]>
To: "'Natale, Bob'" <[email-masked]>, <[email-masked]>, "'Bert Wijnen'" <[email-masked]>
Subject: RE: [OPSAWG] [nmrg] draft-irtf-nmrg-snmp-measure-04.txt oid xmlpattern change
Date: Fri, 16 May 2008 10:13:40 -0400
Cc: [email-masked], [email-masked]

Hi,

SMIv2 is based on ASN.1-1988 not ASN.1-2002.
XSDMI should maintain fidelity to SMIv2.
I would think the snmp-measure document should do so also.

We should be liberal in what we accept, and conservative in what we
send, of course. I think XSDMI should enforce the 1988 syntax, since
one expected use case is validation; it probably will make no
difference to the snmp-research document, whose use case is reporting.

In practice, I have never seen a 2.X in any SMIv2 MIB module.

dbh 

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Natale, Bob
> Sent: Friday, May 16, 2008 9:48 AM
> To: [email-masked]; Bert Wijnen
> Cc: [email-masked]; [email-masked]
> Subject: Re: [OPSAWG] [nmrg] 
> draft-irtf-nmrg-snmp-measure-04.txt oid xmlpattern change
> 
> Hi,
> 
> Juergen is correct (per Sec. 8.19 of ITU-T Recommendation X.690,
> prepared by ITU-T Study Group 17 and approved on 14 July 2002,
> identical text is also published as ISO/IEC 8825-1).
> 
> draft-ietf-opsawg-smi-datatypes-in-xsd-02.txt (to be posted any day
> now!) will contain the fix Juergen provides below.
> 
> Cheers,
> BobN
> 
> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]] On Behalf Of Juergen
> Schoenwaelder
> Sent: Friday, May 16, 2008 8:27 AM
> To: [email-masked]; Bert Wijnen
> Subject: [nmrg] draft-irtf-nmrg-snmp-measure-04.txt oid xml pattern
> change
> 
> Hi,
> 
> there was recently a discussion on the XSD pattern for object
> identifier values and it was pointed out that the pattern we picked
up
> from the SMI-XSD document is too restrictive by not allowing OIDs of
> the format 2.X.* where X is larger than 39. I therefore propose that
> we change the pattern used by oid.type from
> 
>       "[0-2](\.[1-3]?[0-9])(\.(0|([1-9]\d*))){0,126}"
> 
> to
> 
>       "([0-1](\.[1-3]?[0-9]))|(2.(0|([1-9]\d*)))" ~
>       "(\.(0|([1-9]\d*))){0,126}"
> 
> Any objections against this change which is really a bugfix? Bert,
do
> we run into procedural trouble if we fix this or an additional N
weeks
> delay? (In that case, I would be tempted to go along with the buggy
> version of the pattern. ;-)
> 
> /js
> 
> -- 
> Juergen Schoenwaelder           Jacobs University Bremen gGmbH
> Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
> Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>
> -- 
> !! This message is brought to you via the `nmrg' mailing list.
> !! Please do not reply to this message to unsubscribe. To
unsubscribe
> or adjust
> !! your settings, send a mail message to 
> <[email-masked]>
> !! or look at https://mail.ibr.cs.tu-bs.de/mailman/listinfo/nmrg.
> _______________________________________________
> OPSAWG mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/opsawg
> 

Content-class: urn:content-classes:message
Subject: RE: [nmrg] draft-irtf-nmrg-snmp-measure-04.txt oid xml pattern change
Date: Fri, 16 May 2008 09:48:16 -0400
From: "Natale, Bob" <[email-masked]>
To: <[email-masked]>, "Bert Wijnen" <[email-masked]>
Cc: [email-masked], [email-masked]

Hi,

Juergen is correct (per Sec. 8.19 of ITU-T Recommendation X.690,
prepared by ITU-T Study Group 17 and approved on 14 July 2002,
identical text is also published as ISO/IEC 8825-1).

draft-ietf-opsawg-smi-datatypes-in-xsd-02.txt (to be posted any day
now!) will contain the fix Juergen provides below.

Cheers,
BobN

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]] On Behalf Of Juergen
Schoenwaelder
Sent: Friday, May 16, 2008 8:27 AM
To: [email-masked]; Bert Wijnen
Subject: [nmrg] draft-irtf-nmrg-snmp-measure-04.txt oid xml pattern
change

Hi,

there was recently a discussion on the XSD pattern for object
identifier values and it was pointed out that the pattern we picked up
from the SMI-XSD document is too restrictive by not allowing OIDs of
the format 2.X.* where X is larger than 39. I therefore propose that
we change the pattern used by oid.type from

      "[0-2](\.[1-3]?[0-9])(\.(0|([1-9]\d*))){0,126}"

to

      "([0-1](\.[1-3]?[0-9]))|(2.(0|([1-9]\d*)))" ~
      "(\.(0|([1-9]\d*))){0,126}"

Any objections against this change which is really a bugfix? Bert, do
we run into procedural trouble if we fix this or an additional N weeks
delay? (In that case, I would be tempted to go along with the buggy
version of the pattern. ;-)

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>
-- 
!! This message is brought to you via the `nmrg' mailing list.
!! Please do not reply to this message to unsubscribe. To unsubscribe
or adjust
!! your settings, send a mail message to <[email-masked]>
!! or look at https://mail.ibr.cs.tu-bs.de/mailman/listinfo/nmrg.

Date: Fri, 16 May 2008 14:27:04 +0200
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked], Bert Wijnen <[email-masked]>
Mail-Followup-To: [email-masked], Bert Wijnen <[email-masked]>
Subject: [nmrg] draft-irtf-nmrg-snmp-measure-04.txt oid xml pattern change
Reply-To: [email-masked]

Hi,

there was recently a discussion on the XSD pattern for object
identifier values and it was pointed out that the pattern we picked up
from the SMI-XSD document is too restrictive by not allowing OIDs of
the format 2.X.* where X is larger than 39. I therefore propose that
we change the pattern used by oid.type from

      "[0-2](\.[1-3]?[0-9])(\.(0|([1-9]\d*))){0,126}"

to

      "([0-1](\.[1-3]?[0-9]))|(2.(0|([1-9]\d*)))" ~
      "(\.(0|([1-9]\d*))){0,126}"

Any objections against this change which is really a bugfix? Bert, do
we run into procedural trouble if we fix this or an additional N weeks
delay? (In that case, I would be tempted to go along with the buggy
version of the pattern. ;-)

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

Date: Fri, 16 May 2008 14:20:42 +0200
From: Juergen Schoenwaelder <[email-masked]>
To: "Karen R. Sollins" <[email-masked]>
Mail-Followup-To: "Karen R. Sollins" <[email-masked]>, [email-masked], Bert Wijnen - IETF <[email-masked]>, Internet Research Steering Group <[email-masked]>
Cc: Internet Research Steering Group <[email-masked]>, [email-masked]
Subject: [nmrg] Re: [IRSG] review of draft-irtf-nmrg-snmp-measure-04.txt
Reply-To: [email-masked]

On Mon, May 12, 2008 at 02:52:02PM -0400, Karen R. Sollins wrote:

> Here is my full review of draft-irtf-nmrg-snmp-measure-04.txt.

Thank you very much for your substantial review. Below is how I plan
to address your comments. For most of your comments, the changes are
easy to do. For some of your comments, I suggest to make no changes
and I try to explain why. Please check and let me and the RG know
where you do not agree with my suggested plan of action.

> I do not think this document is quite ready for publication, but it 
> is quite close and I don't think this should be major work.
> 
> My review falls into two parts.  The first is broad comments and the 
> second is nits.  Overall, I agree with the working group that it is 
> valuable to spell out a set of details such as this, more or less a 
> "protocol" in the sense of biologically based research, describing 
> what should be done, what kind of information should be collected and 
> why and how to handle it.   I have not reviewed the XML or CSV 
> representations nor the reference lists in detail.  With that said, I 
> have two high level concerns with this document.
> 
> The first is that there is confusion about whether this document 
> describes a proposed measurement effort or (closer to the reality of 
> what is in the document) a methodology by which such measurements 
> could and perhaps should be taken.  So, given the bulk of the 
> document, I do not believe that it is at all explicit about the 
> particular measurement exercise that will be done.   With that said, 
> there are a few introductory points that are worth changing.
> 
> 1. The abstract includes the sentence, "This document proposes to 
> carry out large scale SNMP traffic measurements."  First, the 
> document won't be carrying out anything.  More importantly the 
> document describes an approach (not even a pan), so I suggest that 
> the sentence read, "This document describes an approach to carrying 
> out..."

changed as suggested

> 2. In Section 1, the 4th paragraph reads, "This document describes an 
> effort to collect SNMP traffic traces..."  Instead, I suggest that it 
> read, "This document recommends an approach to collecting, codifying, 
> and handling SNMP traffic traces..."

changed as suggested

> The second high level concern I have is that there is talk about 
> specific kinds of information to be collected and an interest in not 
> only the nature but longer term inferences with perhaps implications 
> for future redesign efforts in the SNMP context.    I have two levels 
> of concern here.  The most important one is that since network 
> management and in particular SNMP is NOT the primary objective of the 
> net (the primary objective being the transport of real payload), it 
> seems to me that the truly critical question with respect to network 
> management traffic is the impact that it has or does not have on that 
> real job.  To me this implies that the measurements MUST also include 
> contextual information.  As an example, it is probably more important 
> to understand whether  or not the network management traffic is 
> causing significant congestion for the payload traffic than the 
> particular mix or frequency pattern within the network management 
> traffic.  Without out the complementary contextual information, the 
> whole measurement exercise seems to me to be of somewhat narrow 
> value.  

The measurement may be of narrow value from your point of view but
please keep in mind that this document is coming from the Network
Management Research Group and not from a general Network Measurement
Research Group. Our goal is to understand how network management
protocols are being used because that has impact on their design and
implementation strategies. Further note that in many networks, the
management traffic is logically and sometimes even physically
separated from the normal traffic and perhaps this is the reason why
we did not even think about the question whether management traffic
has an impact on normal traffic.

> Secondarily, the document hits on one of my little pet peeves 
> - location.  There is mention that the meta-data must include "where 
> the trace was collected".  That, in and of itself, is a challenging 
> problem.  Is this geographic, topological, or based on other metrics? 
> If it is based on something that can change with time (e.g. ip 
> addresses or topology), then without keeping track of how those 
> metrics may have changed over time, the data set becomes useless 
> because the location becomes uninterpretable.  As you all hint, it is 
> important that the collected information be archived, in order to be 
> useful at significantly later times, and thus, everything importatn 
> about the situation must be recorded.  I would like to see more 
> specificity with respect to location and clearer acknowledgement of 
> its need in order to keep the collected information useful outside 
> the current context.

Makes sense.

> More generally, there are many cases in which there are statements 
> about what kind of information should be collected (e.g.one week 
> traces, concurrent vs. sequential requests, etc.) where there is no 
> explanation of the criticality of this information or how to go 
> beyond the immediately observable facts.
> 
> I realize that it is generally not in the nature of RFC to provide 
> explanation as much as simple unadorned facts, but this is an unusual 
> kind of document, because it is describing a potential information 
> gathering exercise.  It has some of the explanations of why things 
> are important, but it is quite incomplete.  I realize that it is 
> important to collect and archive measurement sets exactly because one 
> cannot know ahead of time all the potential uses, but there should 
> either be a story about collecting everything possible or an initial 
> use for each kind of information collected.  In some cases neither 
> argument is presented.
> 
> This all leads to a number of specific questions:
> 
> 1. Section 1: It seems to me that there are TWO key questions with 
> respect to SNMP.  The first is how it is being used, which in turn 
> leads to the points made in this section, but the second is the 
> impact of that traffic.  I think that ought to appear in the 
> Introduction as well.

See my comments above. So far, the NMRG did not consider the impact of
SNMP on other traffic a target of this activity. I don't want to add
such text unless I see support from the NMRG and concrete proposals
what should be added.

> 2. Section 2.1.  The second paragraph begins with, "It is recommended 
> to capture at least a full week of data."  This is never justified or 
> explained.  Is one week really enough?  For what?  Why wouldn't 
> several weeks be critical, because one week might be anomalous?  Why 
> isn't a year critical, since we know that there are annual or 
> seasonal differences in traffic behaviors?  Typically, I find that 
> one-week data sets often leave me with lots of unanswered questions, 
> so justify this.

The text actually says:

   It is recommended to capture at least a full week of data.  Operators
   are encouraged to capture traces over even longer periods of time.

The text tries to establish a lower bound of one week an encourages
longer capture periods. I would love to get continuous traces but
reality is such that this is not feasible. Our idea is simply to catch
at least the weekly behaviour. Yes, there is of course also monthly or
yearly behaviour but I believe it is not useful to set the bar so high
that nobody gives us appropriate traces. I personally believe the text
is fine as is.

> 3. Next paragraph: this is where the location question arises. 
> Without some completely standardized and self explanatory capturing 
> of location information, any data set will be incomparable to any 
> other.

I expanded "where the trace was collected" to "where the trace was
collected (name of the network and/or name of the organization owning
the network, description of the measurement point in the network
topology where the trace was collected)".

> 4. Section 3.3, end of first paragraph:  The sentence reads, "Some 
> SNMP implementations approximate networking delays by measuring 
> request-response times and it would be useful to understand to what 
> extent this is a viable approach."  I agree, but traces will not tell 
> you anything about whether behaviors observed in packet traces are 
> for this reason or some other reason.  I do not believe you can get 
> at this question with the data you are collecting.

I think it is possible to analyze retransmission behaviour. Depending
on the SNMP version used (and the other versions also depending on
implementation choices), you can get information whether a response is
just coming late for the original request or it is actually a response
to a retranmitted request. We are not talking TCP here; we are talking
about application layer retransmissions and SNMP has its own msgID and
requestID fields.

> 5. Section 3.4: Please explain why it is "interesting" (your word) to 
> identify whether concurrency or sequentiality is occurring?  What 
> will you "learn" if both are observed?  And, if one is occurring more 
> frequently or under specific identifiable conditions, what further 
> does that tell you?  Just knowing that one or the other occurs is 
> only the tip of the iceberg, and without acknowledging the fact that 
> these are important and unanswered questions, just learning first 
> ordered details suggests you are setting the bar too low.

The introduction of section 3 says:

   The questions raised in the following subsections are meant to be
   illustrative and no attempt has been made to provide a complete
   list.

I believe it is a good idea to first figure out whether there is an
iceberg or not (keeping your analogy) and if there is one to ask
questions how big the iceberg might be. For SNMP agent implementations
that tend to do quite some caching, it is useful to know how well
caching strategies are working in real-world networks. The concurrency
level an agent experiences has clear impact on that. Furthermore, it
will be useful to know how bursty the traffic tends to be or how well
managers spread the traffic over polling intervals and this is again
related to the concurrency we can extract from traces.

I am not really sure what I should change, perhaps the word
"interesting" is the source of the trouble and I should replace this
with "valuable"?

> 6. Section 3.5:  Please explain what you would do  with the 
> information about which approach to table retrieval is used.  Again, 
> what if the results tell you that both are used?  And, if not, of 
> what use is it to know which approach is prevalent?  Mighten it be 
> useful to know the conditions under which one or the other is used 
> more commonly?

This again has direct impact on agent implementation techniques and
caching strategies.

> Now on to the real NITS:
> 1. p. 1, Section 1, middle of second paragraph:
> s/In fact, there are many speculations how SNMP/In fact, there are 
> many speculations on how SNMP/

fixed

> 2. p. 6, Section 2.2, 5th paragraph:
> s/for SNMP messages that got fragmented/for SNMP messages that were fragmented/

fixed

> 3. p. 7, Section 2.4, last paragraph:
> s/Improvements in the tool chain may require to go back to the 
> original pcap traces and to rebuild all intermediate formats from 
> them./Improvements in the tool chain may require going back to the 
> original pcap traces and rebuilding all intermediate formats from 
> them./

fixed

> 4. p. 7, Section 2.5, first paragraph:
> s/all scripts used to analyze traffic traces would be/all scripts 
> used to analyze traffic traces will be/

fixed

> 5. p. 8, first sentence:
> s/A common versioned repository for analysis of scripts might be 
> useful to establish./It might be useful to establish and common, 
> versioning repository for analysis scripts./

fixed

> 6. p. 6, second paragraph (includes several corrections):
> s/it is suggested that analysis scripts are written in scripting 
> languages such as Perl using suitable Perl modules to manipulate XML 
> documents [8].  Using a scripting language such as Perl instead of 
> system programming languages such as C or C++ has the advantage to 
> reduce development time and to make scripts more accessible to 
> operators who may want to verify scripts before running them on trace 
> files which potentially contain sensitive data./it is suggested that 
> analysis scripts be written is scripting languages such as Perl using 
> suitable Perl modules to manipulate XML documents [8].  Using a 
> scripting language such as Perl instead of system programming 
> languages such as C or C++ has the advantage of reducing development 
> time and making scripts more accessible to operators who may want to 
> verify scripts before running them on trace files which may contain 
> sensitive data./

fixed

> 6. p. 6, end of next paragraph:
> s/might be the only option to deal/might be the only option in dealing/

fixed

> 7. p. 9, Section 3.2.  In this case there are enough corrections in 
> close proximity that I've grouped them together.  I suggest that the 
> following text:
> 
> "SNMP is used to periodically poll devices as well as to retrieve 
> information on request of an operator or application.  The periodic 
> polling leads to periodic traffic patterns while the on demand 
> information retrieval causes more aperiodic traffic patterns.  It is 
> worthwhile to understand what the relationship is between the amount 
> of periodic and aperiodic traffic.  It will be interesting to 
> research whether there are multiple levels of periodicity at 
> different time scales.
> 
> The periodic polling behavior may dependent on the application and 
> the polling engine it uses.  For example, some management platforms 
> allow applications to specify how long polled values may be kept in a 
> cache before it is polled again."
> 
> have the following grammatical changes made:
> 
> "SNMP is used to periodically poll devices as well as to retrieve 
> information at the request of an operator or application.  The 
> periodic polling leads to periodic traffic patterns while on-demand 
> information retrieval causes more aperiodic traffic patterns.  It is 
> worthwhile to understand what the relationship is between the amount 
> of periodic and aperiodic traffic.  It will be interesting to 
> understand whether there are multiple levels of periodicity at 
> different time scales.
> 
> Periodic polling behavior may be dependent on the application and 
> polling engine it uses.  For example, some management platforms allow 
> applications to specify how long polled values may be kept in a cache 
> before they are polled again."

fixed

> 8. p. 10, Section 3.6, middle first paragraph:
> s/ adopt/adapt/

fixed

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

Date: Mon, 12 May 2008 14:52:02 -0400
To: [email-masked], "Bert Wijnen - IETF" <[email-masked]>, "Internet Research Steering Group" <[email-masked]>
From: "Karen R. Sollins" <[email-masked]>
Subject: [nmrg] review of draft-irtf-nmrg-snmp-measure-04.txt

Here is my full review of draft-irtf-nmrg-snmp-measure-04.txt.

I do not think this document is quite ready for publication, but it 
is quite close and I don't think this should be major work.

My review falls into two parts.  The first is broad comments and the 
second is nits.  Overall, I agree with the working group that it is 
valuable to spell out a set of details such as this, more or less a 
"protocol" in the sense of biologically based research, describing 
what should be done, what kind of information should be collected and 
why and how to handle it.   I have not reviewed the XML or CSV 
representations nor the reference lists in detail.  With that said, I 
have two high level concerns with this document.

The first is that there is confusion about whether this document 
describes a proposed measurement effort or (closer to the reality of 
what is in the document) a methodology by which such measurements 
could and perhaps should be taken.  So, given the bulk of the 
document, I do not believe that it is at all explicit about the 
particular measurement exercise that will be done.   With that said, 
there are a few introductory points that are worth changing.

1. The abstract includes the sentence, "This document proposes to 
carry out large scale SNMP traffic measurements."  First, the 
document won't be carrying out anything.  More importantly the 
document describes an approach (not even a pan), so I suggest that 
the sentence read, "This document describes an approach to carrying 
out..."

2. In Section 1, the 4th paragraph reads, "This document describes an 
effort to collect SNMP traffic traces..."  Instead, I suggest that it 
read, "This document recommends an approach to collecting, codifying, 
and handling SNMP traffic traces..."

The second high level concern I have is that there is talk about 
specific kinds of information to be collected and an interest in not 
only the nature but longer term inferences with perhaps implications 
for future redesign efforts in the SNMP context.    I have two levels 
of concern here.  The most important one is that since network 
management and in particular SNMP is NOT the primary objective of the 
net (the primary objective being the transport of real payload), it 
seems to me that the truly critical question with respect to network 
management traffic is the impact that it has or does not have on that 
real job.  To me this implies that the measurements MUST also include 
contextual information.  As an example, it is probably more important 
to understand whether  or not the network management traffic is 
causing significant congestion for the payload traffic than the 
particular mix or frequency pattern within the network management 
traffic.  Without out the complementary contextual information, the 
whole measurement exercise seems to me to be of somewhat narrow 
value.  Secondarily, the document hits on one of my little pet peeves 
- location.  There is mention that the meta-data must include "where 
the trace was collected".  That, in and of itself, is a challenging 
problem.  Is this geographic, topological, or based on other metrics? 
If it is based on something that can change with time (e.g. ip 
addresses or topology), then without keeping track of how those 
metrics may have changed over time, the data set becomes useless 
because the location becomes uninterpretable.  As you all hint, it is 
important that the collected information be archived, in order to be 
useful at significantly later times, and thus, everything importatn 
about the situation must be recorded.  I would like to see more 
specificity with respect to location and clearer acknowledgement of 
its need in order to keep the collected information useful outside 
the current context.

More generally, there are many cases in which there are statements 
about what kind of information should be collected (e.g.one week 
traces, concurrent vs. sequential requests, etc.) where there is no 
explanation of the criticality of this information or how to go 
beyond the immediately observable facts.

I realize that it is generally not in the nature of RFC to provide 
explanation as much as simple unadorned facts, but this is an unusual 
kind of document, because it is describing a potential information 
gathering exercise.  It has some of the explanations of why things 
are important, but it is quite incomplete.  I realize that it is 
important to collect and archive measurement sets exactly because one 
cannot know ahead of time all the potential uses, but there should 
either be a story about collecting everything possible or an initial 
use for each kind of information collected.  In some cases neither 
argument is presented.

This all leads to a number of specific questions:

1. Section 1: It seems to me that there are TWO key questions with 
respect to SNMP.  The first is how it is being used, which in turn 
leads to the points made in this section, but the second is the 
impact of that traffic.  I think that ought to appear in the 
Introduction as well.

2. Section 2.1.  The second paragraph begins with, "It is recommended 
to capture at least a full week of data."  This is never justified or 
explained.  Is one week really enough?  For what?  Why wouldn't 
several weeks be critical, because one week might be anomalous?  Why 
isn't a year critical, since we know that there are annual or 
seasonal differences in traffic behaviors?  Typically, I find that 
one-week data sets often leave me with lots of unanswered questions, 
so justify this.

3. Next paragraph: this is where the location question arises. 
Without some completely standardized and self explanatory capturing 
of location information, any data set will be incomparable to any 
other.

4. Section 3.3, end of first paragraph:  The sentence reads, "Some 
SNMP implementations approximate networking delays by measuring 
request-response times and it would be useful to understand to what 
extent this is a viable approach."  I agree, but traces will not tell 
you anything about whether behaviors observed in packet traces are 
for this reason or some other reason.  I do not believe you can get 
at this question with the data you are collecting.

5. Section 3.4: Please explain why it is "interesting" (your word) to 
identify whether concurrency or sequentiality is occurring?  What 
will you "learn" if both are observed?  And, if one is occurring more 
frequently or under specific identifiable conditions, what further 
does that tell you?  Just knowing that one or the other occurs is 
only the tip of the iceberg, and without acknowledging the fact that 
these are important and unanswered questions, just learning first 
ordered details suggests you are setting the bar too low.

6. Section 3.5:  Please explain what you would do  with the 
information about which approach to table retrieval is used.  Again, 
what if the results tell you that both are used?  And, if not, of 
what use is it to know which approach is prevalent?  Mighten it be 
useful to know the conditions under which one or the other is used 
more commonly?

Now on to the real NITS:
1. p. 1, Section 1, middle of second paragraph:
s/In fact, there are many speculations how SNMP/In fact, there are 
many speculations on how SNMP/

2. p. 6, Section 2.2, 5th paragraph:
s/for SNMP messages that got fragmented/for SNMP messages that were fragmented/

3. p. 7, Section 2.4, last paragraph:
s/Improvements in the tool chain may require to go back to the 
original pcap traces and to rebuild all intermediate formats from 
them./Improvements in the tool chain may require going back to the 
original pcap traces and rebuilding all intermediate formats from 
them./

4. p. 7, Section 2.5, first paragraph:
s/all scripts used to analyze traffic traces would be/all scripts 
used to analyze traffic traces will be/

5. p. 8, first sentence:
s/A common versioned repository for analysis of scripts might be 
useful to establish./It might be useful to establish and common, 
versioning repository for analysis scripts./

6. p. 6, second paragraph (includes several corrections):
s/it is suggested that analysis scripts are written in scripting 
languages such as Perl using suitable Perl modules to manipulate XML 
documents [8].  Using a scripting language such as Perl instead of 
system programming languages such as C or C++ has the advantage to 
reduce development time and to make scripts more accessible to 
operators who may want to verify scripts before running them on trace 
files which potentially contain sensitive data./it is suggested that 
analysis scripts be written is scripting languages such as Perl using 
suitable Perl modules to manipulate XML documents [8].  Using a 
scripting language such as Perl instead of system programming 
languages such as C or C++ has the advantage of reducing development 
time and making scripts more accessible to operators who may want to 
verify scripts before running them on trace files which may contain 
sensitive data./

6. p. 6, end of next paragraph:
s/might be the only option to deal/might be the only option in dealing/

7. p. 9, Section 3.2.  In this case there are enough corrections in 
close proximity that I've grouped them together.  I suggest that the 
following text:

"SNMP is used to periodically poll devices as well as to retrieve 
information on request of an operator or application.  The periodic 
polling leads to periodic traffic patterns while the on demand 
information retrieval causes more aperiodic traffic patterns.  It is 
worthwhile to understand what the relationship is between the amount 
of periodic and aperiodic traffic.  It will be interesting to 
research whether there are multiple levels of periodicity at 
different time scales.

The periodic polling behavior may dependent on the application and 
the polling engine it uses.  For example, some management platforms 
allow applications to specify how long polled values may be kept in a 
cache before it is polled again."

have the following grammatical changes made:

"SNMP is used to periodically poll devices as well as to retrieve 
information at the request of an operator or application.  The 
periodic polling leads to periodic traffic patterns while on-demand 
information retrieval causes more aperiodic traffic patterns.  It is 
worthwhile to understand what the relationship is between the amount 
of periodic and aperiodic traffic.  It will be interesting to 
understand whether there are multiple levels of periodicity at 
different time scales.

Periodic polling behavior may be dependent on the application and 
polling engine it uses.  For example, some management platforms allow 
applications to specify how long polled values may be kept in a cache 
before they are polled again."

8. p. 10, Section 3.6, middle first paragraph:
s/ adopt/adapt/

			Karen Sollins

-- 
Karen R. Sollins, Ph.D.
Principal Research Scientist
MIT CSAIL, The Stata Center
32 Vassar St., 32-G818
Cambridge, MA 02139, USA
V: +1 617 253 6006
F: +1 617 253 2673
E: [email-masked]