Subject: [Isms] Questions on TLSTM/TSM and RFC 5343

From: Frank Fock <[email-masked]>
Date: Sun, 16 Jan 2011 11:25:12 +0100

Hi,

I am currently implementing RFC 5590, 5591 (TSM),
5953 (TLSTM), and 5343 (Context EngineID Discovery)
for SNMP4J 2.0 (snmp4j.org).

Next to the above, I also want to implement RFC 5592
(SSHTM).

I have not performed any interoperability tests with
other implementations yet, because my implementation
of the SNMP-TLS-TM-MIB is not ready. Within the
next two weeks, I will start interoperability
testing with the NET-SNMP implementation.

Nevertheless, I would like to give an early feedback
and to use the opportunity to ask some questions.

While the implementation of TSM and TLSTM was
straightforward, RFC 5343 imposed some
architectural difficulties:

1. The contextEngineID discovery mechanism of
RFC 5343 cannot be implemented within the SNMPv3
message processing model (MPv3) because the
MPs are not designed to be able to send messages
themselves.
The ASIs do not define a callback to the
Dispatcher for sending PDUs on behalf of the MPs.
Since timeout and retry values are also
not available for the MPs, sending discovery
PDUs has to be done by the Dispatcher.

The Dispatcher however, needs to know if
discovery is needed or not. In order to
be able to decide that, the Dispatcher needs
information of the used Security Model.
Unfortunately, RFC 5343 requires that the
USM discovery has precedence over RFC 5343
contextEngineID discovery. Whether USM
discovery is executed can be decided within
the MPv3/USM only, because the local engine
ID cache needs to be checked therefore.

It was unsatisfying to implement aspects
of the contextEngineID discovery in two
or three different components of the model.

I have not thought about a better solution
yet, but may be my second issue/question
would provide an alternative?

2. The reasoning for the contextEngineID
discovery is not clear to me. If the
TSM would replace the defined magic
local engine ID ’8000000006’H (put in
the scopedPDU by the sender) by the
local engine ID of the SNMP entity
receiving the scopedPDU before handing
it over to the Dispatcher, then discovery
would not be needed and the effect
would be the same, wouldn't it?

(The receiver would replace the local
engine ID in any responses of the TSM
with the magic local engine ID too.)

3. RFC 5591 §5.2.1 should be clarified.
Is the "local engine ID" the magic
engine ID ’8000000006’H or the engine
ID of the SNMP entity processing the
incoming message?

All in all, the RFCs seem to be implementable
as they are, however especially the discovery
mechanism increases the implementation complexity.
Implementations need to be able to send different
PDUs on behalf of a single application level PDU
and support REPORT processing for USM engine ID
and time discovery at the same time as "steeling"
regular RESPONSE PDUs from the incoming PDUs if
they are responses on contextEngineId discovery
requests.

May be someone on the list can explain to me,
why my suggestion (2) above, is not a feasible
replacement of the contextEngineId discovery?

The current (alpha) implementation of TLSTM
and TSM for SNMP4J can be downloaded from:

https://server.oosnmp.net/dist/snapshot/org/snmp4j/snmp4j/2.0-SNAPSHOT/
https://server.oosnmp.net/dist/snapshot/org/snmp4j/snmp4j-agent/2.0-SNAPSHOT/

Regards,
Frank Fock

---

From: "t.petch" <[email-masked]>
Date: Tue, 18 Jan 2011 17:27:10 +0100

<inline>

----- Original Message -----
From: "Frank Fock" <[email-masked]>
To: <[email-masked]>
Sent: Sunday, January 16, 2011 11:25 AM

<tp>
The MPP cannot send a message but does not need to.
The Dispatcher is called by the application, and so knows
the Security Model and whether or not the contextEngineID
is known, so the dispatcher knows all that it needs to
in order to know whether or not to invoke the
discovery.  I am unclear where the problem lies.
</tp>

<tp>
engineID uniquely identifies an engine in a domain, making
object values that would otherwise have the same identifiers
unique.  A Command Generator needs to know the
engineId of the Command Responder.  Originally,
 USM performed the discovery so no more was needed.
With TSM, and perhaps with others, the Security Model
does not perform discovery so another mechanism is
needed; hence RFC5343

I would expect the well-known local value that
all engines implementing this to have to only
be used for discovery, not for any other
PDU.
</tp>

<tp>
Indeed it should, perhaps you should raise an
erratum.  RFC5591 existed as an I-D for many years before
RFC5343 so when the I-D was written, local engineID could only
mean the one that uniquely identified the local engine within
the domain.  RFC5343 introduced an ambiguity that those
of us reviewing RFC5591 did not pick up.

I am sure that many on this list will be very interested in
your further experience and especially in your interoperability
tests.

Tom Petch

</tp>

Regards,
Frank Fock

---

From: Wes Hardaker <[email-masked]>
Date: Tue, 18 Jan 2011 10:52:51 -0800

>>>>> On Sun, 16 Jan 2011 11:25:12 +0100, Frank Fock <[email-masked]> said:

FF> I am currently implementing RFC 5590, 5591 (TSM),
FF> 5953 (TLSTM), and 5343 (Context EngineID Discovery)
FF> for SNMP4J 2.0 (snmp4j.org).

Hi Frank!  I'm certainly glad to hear you're implementing the latest and
greatest of the SNMPv3 RFCs.

FF> While the implementation of TSM and TLSTM was
FF> straightforward

That's certainly good news!

FF> RFC 5343 imposed some architectural difficulties:

That's, of course, less than good news.  But I'm sure we can work
through the issues.

FF> 1. The contextEngineID discovery mechanism of
FF> RFC 5343 cannot be implemented within the SNMPv3
FF> message processing model (MPv3) because the
FF> MPs are not designed to be able to send messages
FF> themselves.

I don't think it actually says this should be the case anywhere in
5343.  It actually implies, in multiple places, that it's done more at
the application level of the SNMPv3 architecture.

Now, real-world library APIs, may wish to hide the discovery process
from the application developers to make it easier to implement an
application.  It's much easier to do this than make each application
handle discovery itself.

Now, having said that technically I think it's the application that
should initiate discovery, I'll also amend this viewpoint with "remember
that 3410 is *An* architecture.  I suspect many stacks may choose to
implement contextEngineID discovery somewhere within their MP or similar
architecture.  As long as the bits on the wire are good, you can do
anything you want.  The pitfall may come at a later date when a future
RFC defining a new architectural block makes you rewrite your
MP-discovery-mechanism because your hack wasn't technically
"block-compliant".  However, I doubt this is of a big concern to most
SNMPv3 stack vendors.

Net-SNMP implements it within the stack somewhere near the "MP" layer.
Since the Net-SNMP architecture pre-dates SNMPv3 by years and years, the
lines between the blocks are a bit fuzzy.

FF> Unfortunately, RFC 5343 requires that the
FF> USM discovery has precedence over RFC 5343
FF> contextEngineID discovery.

Now here's an important missing component of your analysis:  Previously
*no* RFC described how to do contextEngineID discovery.  The *only*
discovery mechanism was defined for USM and it was to discover the
*securityEngineID*.  There is no where that says you should magically
copy the securityEngineID to the contextEngineID if you don't have a
better one to use.  That's just what everyone did because it made sense!
But please do correct me if I'm wrong here (I didn't go back and re-read
every line in the SNMPv3+USM specs to double check; I do know that the
one sentence in RFC3414 that includes "contextEngineID" certainly
doesn't say this).

Rather than make every security module magically do contextEngineID
discovery too, 5343 makes it generic such that security modules that
don't need an engineID at all (like TSM) don't need to create a
discovery mechanism for a value they don't need to make use of.  TSM
doesn't have, or need, a notion of a securityEngineID so the choices
were:

1) make all the engines that wanted to send a DTLS packet first send a
   USM packet to do engineID discovery and continue to use the magic
   "securityEngineID to contextEngineID copying routine".
2) define a new, more generic, contextEngineID discovery mechanism which
   is more securityEngineID agnostic.

The tack taken was #2 which resulted in RFC5343.  In fact, there is
nothing wrong with:

  + application needs to send a SNMPv3/USM message
  + USM does discovery for it's securityEngineID
  + application does it's contextEngineID discovery
  + using both values, sends the resulting mesasge

That should work just fine, and technically might be the right way to do
it considering prior to 5343 there was no notion of a contextEngineID
discovery method.  It's taken for granted by previous implementations
that you could copy the value out of the securityEngineID field or the
contextEngineID field.

FF> 2. The reasoning for the contextEngineID
FF> discovery is not clear to me.

Hopefully the above explains the historical context of why it's needed.

FF> If the TSM would replace the defined magic local engine ID
FF> ’8000000006’H (put in the scopedPDU by the sender) by the local
FF> engine ID of the SNMP entity receiving the scopedPDU before handing
FF> it over to the Dispatcher, then discovery would not be needed and
FF> the effect would be the same, wouldn't it?

I do suspect that there are other ways in which discovery could be done,
including using the above suggestion.  But it'd require that every
defined security model follow this suggestion or some other similar
suggestion.  5343 leaves it a bit more architecturally separated, which
in my opinion is architecturally cleaner.  Plus, I'm not sure the above
is legal either:

   The procedure when a command generator receives a message is as
   follows:

   (1) If the received values of messageProcessingModel, securityModel,
       securityName, contextEngineID, contextName, and pduVersion are
       not all equal to the values used in the original request, the
       response is discarded.

So technically, if a CG sends a response with a 8000000006 value and
gets back a different engineID value it must discard the response.  The
RFC5343 method actually returns with the same special contextEngineID
value (8000000006) but the payload contains the correct value.

FF> 3. RFC 5591 §5.2.1 should be clarified.
FF> Is the "local engine ID" the magic
FF> engine ID ’8000000006’H or the engine
FF> ID of the SNMP entity processing the
FF> incoming message?

(Note: 5.2.1 == 5.2 bullet #1)

It should mean the local SNMP entity processing the request.

FF> May be someone on the list can explain to me,
FF> why my suggestion (2) above, is not a feasible
FF> replacement of the contextEngineId discovery?

I think I've done that because the CG would drop the response with a
different contextEngineID.  (though some might argue that you could let
every agent out there appear to be "8000000006" according to the CG, I
don't think this is how the contextEngineID was perceived to act).

-- 
Wes Hardaker
Cobham Analytic Solutions

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 20 Jan 2011 16:06:50 +0100

> Next to the above, I also want to implement RFC 5592

Very nice to hear that you are working on implementing these
specifications. Please keep us updated and if you manage to do get
things reasonable complete, you may want to get in touch with Wes
Hardaker and Robert Story - perhaps more details can then be added to
the interoperability report.
 
> architectural difficulties:

After reading the document again, I basically agree with Wes' and
Tom's responses saying that there is not really a problem with RFC
5343.
 
> MPs are not designed to be able to send messages

Yep, by design. The contextEngineID is to be known by applications. 
If an application invokes the sendPdu() ASI, it needs to pass the
contextEngineID and an IN parameter. That is why discovery is (at
least conceptually) done by the application.

The common practice to simply use the securityEngineID discovered by
USM as the discovered contextEngineID to be used by applications is
strictly speaking not covered by the RFC3411 architecture and a
pedantic mind might even consider it a layer violation. There is no
formal ASI which allows to pass a discovered securityEngineID as the
contextEngineID to applications (nor is there text in the elements of
the procedure as far as I know). RFC 5343 puts the contextEngineID
discovery where it needs to be, architecturally. Implementation wise,
you likely want to hide this discovery step from application code but
that is an implementation detail.

> with the magic local engine ID too.)

As Wes pointed out, this causes problems with the way the elements of
procedure are written. This also creates a coupling in the sense that
every security model would have to do such magic things; I believe the
RFC 5343 is cleaner.
 
> ID of the SNMP entity processing the

I see that people might misinterpret the sentence when they have RFC
5343 in their mind. Feel free to post an errata proposing a less
ambiguous wording for this so that a clarification is recorded and
will be remembered.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Frank Fock <[email-masked]>
Date: Sun, 23 Jan 2011 18:24:33 +0100

Hi Wes,

Thanks for your reply (and also those of Tom
and Juergen).

I understand that RFC 5343 solves the
contextEngineID discovery problem, but
within the last 10 years with SNMPv3
I learned, that most SNMPv3 users do not
understand the authoritative and context engine
ID concepts.

There are also a couple of companies
creating SNMPv3 devices with identical
authoritative engine ID for all instances
of their device. We API developers are then
asked to work around these problems :-(

That's why I thought about simplifying
the SNMPv3 and making the discovery of
a contextEngineID unnecessary if not
required by the command generator.

Nevertheless, I agree that using a single
magic contextEngineID for the local context,
would interfere with proxy applications,
for example.

I will now first finish my implementation
and then make a complete report and analysis
of the result. This will also include using
contextEngineId discovery with USM.

Best regards,
Frank

---

From: Wes Hardaker <[email-masked]>
Date: Sun, 23 Jan 2011 09:34:17 -0800

>>>>> On Sun, 23 Jan 2011 18:24:33 +0100, Frank Fock <[email-masked]> said:

FF> That's why I thought about simplifying
FF> the SNMPv3 and making the discovery of
FF> a contextEngineID unnecessary if not
FF> required by the command generator.

>From a pure architecture point of view, I agree.  Though I don't think
it would be wise of a SNMP stack to require the "actual" end-application
to think about contextEngineIDs if it didn't have to.  I think most
stacks would better server their users if they had an API that handled
it all, regardless of where the contextEngineID discovery happened.

EG, hypothetical pseudo fake code:

snmpv3_response *
send_snmpv3_request(struct snmpv3_session *session, struct snmpv3_pdu *pdu) {
  if (session->contextEngineID == NULL) {
     snmpv3_response *response =
         send_pdu_to_dispatcher(session, new_contextEngineID_probe_pdu());
     session->contextEngineID = extract_contextEngineID(blob);
  }
  return send_pdu_to_dispatcher(session, pdu);
}

That seems a more helpful API than making each application doing the
steps internally to that.

---

From: "t.petch" <[email-masked]>
Date: Mon, 24 Jan 2011 15:12:24 +0100

----- Original Message ----- 
From: "Frank Fock" <[email-masked]>
To: "Wes Hardaker" <[email-masked]>
Cc: <[email-masked]>
Sent: Sunday, January 23, 2011 6:24 PM

> asked to work around these problems :-(

That surprises me (and sounds like a potential disaster).

To quote RFC3411
" Within an administrative domain, an snmpEngineID is the unique and
   unambiguous identifier of an SNMP engine.  Since there is a one-to-
   one association between SNMP engines and SNMP entities, it also
   uniquely and unambiguously identifies the SNMP entity within that
   administrative domain.  Note that it is possible for SNMP entities in
   different administrative domains to have the same value for
   snmpEngineID.  Federation of administrative domains may necessitate
   assignment of new values."

which seems clear, unless the concept of an engine is unclear.

On the other hand, it is much easier to burn the same identifier into
everything, rather than using a MAC address type approach and
make each device unique. Perhaps we ask too much of those making
devices.

Tom Petch

> _______________________________________________
> Isms mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/isms

---

From: "David Harrington" <[email-masked]>
Date: Mon, 24 Jan 2011 10:46:25 -0500

Hi,

I am a bit surprised that a couple companies are producing
non-compliant devices, and we haven't heard anything on our lists
asking about the uniqueness requirement until now.
Frank, have you contacted the companies' technical support to ask
about this non-compliance?

IIRC, 
snmpEngineID is supposed to be administratively assignable (e.g., via
config file).

                 The initial value for this object may be configured
                 via an operator console entry or via an algorithmic
                 function.  In the latter case, the following
                 example algorithm is recommended.

The vendor might provide a default, but it is up to the deployer to
make sure the value is set to a unique value within the domain.
If what Frank is seeing is a non-changeable value in a device, I think
the implementation is non-compliant.
If what Frank is seeing is duplicate IDs within a deployment, the
admin is preumably at fault for not properly setting the initial
values to be unique.

dbh

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of t.petch
> Sent: Monday, January 24, 2011 9:12 AM
> To: Frank Fock; Wes Hardaker
> Cc: [email-masked]
> Subject: Re: [Isms] IsmsQuestions on TLSTM/TSM and RFC 5343
>

---

From: Frank Fock <[email-masked]>
Date: Mon, 24 Jan 2011 19:06:25 +0100

Hi David,

I personally I have not encountered the
"unique engine ID" issue yet. However,
I get complains from users of SNMP4J
and SNMP++ APIs for several month (if not
years now) who encounter problems if their
network if they try to get values from
more than one device with SNMPv3.

After analysis of dumps and log files,
there are two common errors made by the
manufacturer of the devices (or admins):

1. Using a non-unique hardcoded or unchanged
default value for all deployed devices.
2. The device does not increase its engineBoots
counter when restarted.

The second one is a common error for years
now, whereas the first one got more "popular"
within the last few months.

Both issues above seem to be caused by some
(popular?) SNMP command generator software
out there, that does resynchronize engine ID
and time without asking the user/application
layer. Otherwise this problem would have been
detected by the admins/device creators before
the error had been deployed to the field.

As a consequence of the above problems, I am
asked from time to time to provide also
auto-resynchronization of engine ID and time,
which I refuse, of course.
But asking for such changes in an API shows
how low the security awareness is.
To "get it working" seems the primary driver
:-(

I do not know which companies are creating
the buggy devices or running the incorrect
administered networks, but I could collect
that information starting from now.

Best regards,
Frank

On 24.01.2011 16:46, David Harrington wrote:
>> From: "Frank Fock"<[email-masked]>
>> To: "Wes Hardaker"<[email-masked]>
>> Cc:<[email-masked]>

-- 
AGENT++
http://www.agentpp.com
http://www.snmp4j.com
http://www.mibexplorer.com
http://www.mibdesigner.com