Subject: [Ips] DRAFT Minn. minutes

From: [email-masked]
Date: Tue, 15 Mar 2005 08:35:30 -0500

Sending to the correct list this time ...

Send corrections to me and/or the list.  With the
exception of the administrative/procedural matters
on the iSER over InfiniBand draft, all of the decisions
here reflect the sense of the room in Minneapolis, and
may be objected to or commented upon on the list.  In
the absence of objection, the sense of the room in
Minneapolis will be the rough consensus of the IPS WG.

Thanks,
--David

The IP Storage (ips) WG met 0900-1130 on Tuesday,
March 8 at the IETF meetings in Minneapolis, MN.

MINUTES (DRAFT)
---------------

Administrivia, agenda bashing, draft status review, etc.: 15 min
		David L. Black, EMC (co-chair)
	Blue sheets
	Note Well
	Milestones
		Out of date on web site.  Update discussion postponed to
		end of meeting.

	Draft status
		All non-MIB drafts except iSER and DA are RFCs or in
		RFC Editor's queue.
	
		Elizabeth Rodriguez (co-chair) continues to work with
		authors on resolving expert review comments on remaining
		MIBs.  FC Management MIB has finally made it through this
		process, new versions of iSCSI and iSCSI Authorization MIBs
		coming soon.  iSNS MIB has expired from Internet Draft
servers,
		new version expected shortly.

		iSER and DA status discussion postponed to end of meeting.

iSER and DA: 45min  Mike Ko, IBM
	(draft-ietf-ips-iser-01.txt)
	(draft-ietf-ips-iwarp-da-01.txt)
	iSER = iSCSI Extensions for RDMA
	DA = Datamover Architecture for iSCSI

	No open technical issues on DA draft - it's ready for WG Last Call.

	The open issues on the iSER draft centered on the new
	MaxOutstandingUnexpectedPDUs key.  The key needs to be specified
	so that if the sender violates it (sends too many Unexpected PDUs),
	the receiver is *allowed* to drop the connection, but is *not
	required* to drop it.

	There was a long discussion on when an unsolicited NOP can be
	considered "retired" and its "Unexpected PDU" credit can be
	safely reused by the sender.  Pat Thaler will send detailed text
	to specify this to the list.

	The draft needs to add advice to implementers on how to deal with
	potentially tight target limits on unexpected immediate commands
	- the basic idea is to send non-immediate commands, which aren't
	subject to the limit, and can cause some preceding immediate
commands
	to be considered "retired".

	The details of the specification of the MaxOutstandingUnexpectedPDUs
	key will be:
		Default: "None" (4 letter text string, indicating no limit)
		Minimum allowed value: 2
		Maximum allowed value: 2^32 - 1

	Section 8 of the iSER contains some considerable changes for which
	the details matter - WG members are asked to review it carefully.

	The X# syntax will not be used with keys added by iSER - they
	will be specified by the iSER draft when it becomes a Proposed
Standard
	RFC (as a modification of the iSCSI RFC, 3720), hence IANA does
	not need to register the new iSER keys, and they should not be
	described as "extension keys".

	Schedule discussion on these drafts deferred to after next agenda
item.

iSER over InfiniBand: up to 1 hour 30min  John Hufferd, IBM
	draft-hufferd-ips-iser-sctp-ib-00.{txt,pdf}

	This draft is a proposal to generalize iSER to non-TCP RDMA
transports.
	There are no changes to iSER over TCP.

	The draft requests several changes:
	1) Generalize terms/wording in iSER to allow non-TCP RDMA transports
		such as RDDP/SCTP and InfiniBand's RDMA service (with RC).
		This includes a redefinition of iWARP to encompass SCTP.
	2) Generalize wording in iSER to allow a transport to start in
		native RDMA mode (with Sends for messages) as opposed to
		TCP starting in Stream mode and switching to the RDDP native
		RDMA mode.
	3) Add some sections on how InfiniBand RDMA works as an example.
	4) Extend iSCSI discovery mechanisms to support different
transports.
	5) Exempt non-IP transports (e.g., InfiniBand) from "MUST implement
		IPsec" requirements.

	There were a number of administrative/procedural matters raised by
	these requests that were dealt with the WG co-chair (in consultation
	with the Area Director (Allison Mankin) in some cases:

	a) Item 5) was rejected - the IETF will not approve a blanket
exemption
	of usage of a protocol from security requirements.  The right
approach
	is to refer to RFC 3723 for the security concerns that apply to
iSCSI,
	and draft text to require that they are addressed as appropriate in
	different transport environments.

	b) The authors of this draft have no plans for a draft on iSCSI over
	SCTP without iSER.  Absent such a draft, iSCSI/iSER/SCTP cannot
	be specified, and hence should be removed from the proposal.  NB:
	subsequent list discussion has indicated possible interest in
	writing an iSCSI over SCTP (without iSER) draft, which would make
	it possible to specify iSCSI/iSER/SCTP.

	c) Infiniband-specific issues, such as dealing with possible lack of
	ZBTO support should be dealt with in the InfiniBand Trade
Association,
	not the IETF.

	d) The RDMAP and DDP drafts have passed WG Last Call in the RDDP WG
	with a definition of "iWARP" that is TCP-only (does not include
SCTP).
	The usage of the term "iWARP" in this (ips) WG must respect that
	usage in the RDDP WG, and hence generalizing "iWARP" beyond TCP is
	not appropriate.

	At this point, discussion proceeded to the main issue - whether
	rough consensus exists in the IPS WG to change the iSER draft to
	accommodate to-be-specified usage of iSER over InfiniBand.  Making
	these changes will likely result in delaying iSER while the details
	of the expanded support (e.g., protocol selection information in
	discovery) are worked out.

	After the discussion, based on a show of hands in the room, the
	WG co-chair running the meeting determined that rough consensus
	to make these changes does not exist, and hence the iSER draft will
	proceed to WG Last Call without any changes proposed in this draft.
	During WG Last Call, it will be possible to re-raise these proposed
	changes as WG Last Call comments for further discussion.

	Given this situation, all InfiniBand-specific material for iSER
	should be submitted as a separate individual submission draft (or
	multiple individual submission drafts) that make changes to (update)
	the main iSER draft and the iSCSI discovery mechanism drafts/RFCs as
	necessary. 	Whether and what of these proposals to adopt as
official
	IPS WG work items will be considered at the Paris meeting in early
	August.

	Based on this, the planned schedule is to issue a WG Last Call for
	the DA and iSER drafts in April - authors should prepare versions
	ready for WG Last Call by April 15 (tax day), and the WG Last Call
	will follow the conclusion of the imminent WG Last Call in the RDDP
	WG for the remaining drafts there.

	The IPS WG milestones have been accordingly revised to:

	Jul 05	Submit iSER (iSCSI Extensions for RDMA) and DA
				(Datamover Architecture) drafts to IESG
	Aug 05  	Submit all remaining MIB drafts to IESG
	Sep 05  	Review with ADs what (if any) additional work the
				WG should undertake

	In other words, the intent is to complete the iSER and DA drafts
	on the mailing list before the Paris meeting (first week of August).
	The Paris meeting will be used to resolve any final MIB issues and
	discuss proposed InfiniBand and SCTP extensions to iSCSI and iSER,
	with charter revision to follow (Sep) if any of these extensions
	are added to the IP Storage (ips) WG's program of work.

----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
[email-masked]        Mobile: +1 (978) 394-7754
----------------------------------------------------

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

_______________________________________________
Ips mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ips

---

From: Mike Ko <[email-masked]>
Date: Wed, 23 Mar 2005 11:18:36 -0800

The open issues on iSER from David's minutes are as follows:

1. "The open issues on the iSER draft centered on the new
MaxOutstandingUnexpectedPDUs key.  The key needs to be specified
so that if the sender violates it (sends too many Unexpected PDUs),
the receiver is *allowed* to drop the connection, but is *not
required* to drop it."

Section 6.7 of the current iSER draft (-01) states that "If the
sending side fails to adhere to the declared limit as set by the
receiving side, it is up to the receiving side to determine ways of
handling the overrun, including dropping the connection."

I believe the wording is clear and no changes are required.  If you 
disagree, please post suggested changes to the reflector.

2. "There was a long discussion on when an unsolicited NOP can be
considered "retired" and its "Unexpected PDU" credit can be
safely reused by the sender.  Pat Thaler will send detailed text
to specify this to the list.
The draft needs to add advice to implementers on how to deal with
potentially tight target limits on unexpected immediate commands
- the basic idea is to send non-immediate commands, which aren't
subject to the limit, and can cause some preceding immediate
commands
to be considered "retired"."

The discussion at the Minneapolis meeting was on the handling of 
unidirectional NOP-outs where ITT = TTT = 0xffffffff.  The concern was a 
race condition on the target acknowledging the immediate command 
(NOP-out).  This race condition can happen under the following scenario:

a. There are multiple connections in the session.
b. The initiator sends a number of unidirectional NOP-outs on connection A 
with CmdSN=x.
c. The initiator sends a non-immediate PDU on connection B with CmdSN=x.
d. When the target sends a PDU on connection A with ExpCmdSN=x+1, there 
might still be NOP-outs in flight from the initiator to the target on 
connection A.
e. When the initiator receives the PDU from the target with ExpCmdSN=x+1, 
it can send another MaxOutstandUnexpectedPDUs worth of NOP-outs to the 
target.  So the target is now inundated with a number of NOP-outs with 
CmdSN=x+1 totalling MaxOutstandingUnexpectedPDUs, plus some number of 
NOP-outs with CmdSN=x in flight earlier.

To avoid this scenario, the initiator must send a non-immediate PDU with 
CmdSN=x on the same connection as the NOP-out.  When the target returns a 
PDU with ExpCmdSN=x+1, then all NOP-outs with CmdSN=x will be acknowledged 
and retired.  The initiator must send the non-immediate PDU on the same 
connection as the NOP-outs, since any NOP-outs sent afterwards on the same 
connection (or any other connection, but the ones sent on this connection 
are the ones of interest) will have CmdSN=x+1.  However, the control-type 
PDU from the target with ExpCmdSN=x+1 need not be returned on the same 
connection.

Section 8.3 of the current iSER draft (-01) states that "For a NOP-out PDU 
with ITT = TTT = 0xffffffff and CmdSN = x, the PDU is outstanding until 
the target responds with a control-type PDU on the same connection where 
ExpCmdSN is at least x+1."

This will be changed to the following:

For a NOP-out PDU with ITT = TTT = 0xffffffff and CmdSN = x, the PDU is 
outstanding until the initiator sends a non-immediate control-type PDU on 
the same connection where CmdSN is at least x and the target responds with 
a control-type PDU where ExpCmdSN is at least x+1 on any connection.

There were also discussions on the reflector after the Minneapolis meeting 
concerning the use of the "MUST" directive relating to Operational 
Primitive qualifiers.  I will post the updated iSER draft (-02) to the 
reflector later in the week with changes on the qualifier handling and the 
NOP-out retirement handling described above.

Mike

---

From: Paul Koning <[email-masked]>
Date: Wed, 23 Mar 2005 17:08:03 -0500

>>>>> "Mike" == Mike Ko <[email-masked]> writes:

 Mike> The open issues on iSER from David's minutes are as follows:
 Mike> 1. "The open issues on the iSER draft centered on the new
 Mike> MaxOutstandingUnexpectedPDUs key.  The key needs to be
 Mike> specified so that if the sender violates it (sends too many
 Mike> Unexpected PDUs), the receiver is *allowed* to drop the
 Mike> connection, but is *not required* to drop it."

 Mike> Section 6.7 of the current iSER draft (-01) states that "If the
 Mike> sending side fails to adhere to the declared limit as set by
 Mike> the receiving side, it is up to the receiving side to determine
 Mike> ways of handling the overrun, including dropping the
 Mike> connection."

 Mike> I believe the wording is clear and no changes are required.  If
 Mike> you disagree, please post suggested changes to the reflector.

I'm wondering why there is specific wording on this specific key.  My
view is that a violation of protocol operating constraints defined by
ANY negotiated or declared parameter is a protocol error, and should
be handled by the common rules for protocol errors.  I don't see a
need for a rule that is specific to this one key.

The current spec has somewhat strange text for protocol errors
(10.1.3.5).  Describing the handling of protocol errors as "is similar
to" is not helpful guidance for implementers.  I assume the intent was
"is the same as", i.e., terminate the stream and terminate the
connection.  (The same applies to 10.1.3.3 which has the same
wording.)  If "the same as" is NOT intended, then the spec should
spell out precisely what aspects of the handling are different and in
what way.

So for specific wording, I would delete the entire sentence "If the
sending side..." from 6.7, and instead add to the section 6 text: "If
the sending side acts in a manner not permitted by the negotiated or
declared login/text operational key values, this is a protocol error
and the receiving side MUST (SHOULD?) handle this according to the
rules for protocol errors (section 10.1.3.5)."

If people feel that mandating a disconnect on a protocol error is too
restrictive, then I'd prefer to see the more flexible rule be the
common rule, i.e., the more flexible text should appear in 10.1.3.5
and not in section 6.7.  My own preference is to keep protocol error
handling simple -- terminate the connection and be done with it.

	  paul

---

From: Mike Ko <[email-masked]>
Date: Wed, 23 Mar 2005 15:26:34 -0800

Paul,

The intent in section 10.1.3.5 is indeed "is the same as", as you pointed 
out.  I can make those changes in the -02 version.

As for iSER protocol error, section 10.1.3.5 describes what happens when 
the iSER Hello/HelloReply messages are not sent during connection setup. 
Since iSER Hello/HelloReply messages convey iSER parameters critical to 
the ensuing iSER connection, failure to send them as required would result 
in erroneous operation.  In this case dropping the connection is the right 
thing to do.

On the other hand, the same cannot be said for the new keys introduced in 
section 6.  For RDMAExtensions, the key is negotiated during iSCSI mode 
and so the error handling should be the same as for other iSCSI keys. 
TargetRecvDataSegmentLength, InitiatorRecvDataSegmentLength, and 
MaxOutstandingUnexpectedPDUs are added for buffer management.  If an 
iSCSI/iSER node fails to adhere to the negotiated/declared limit, the 
connection partner can certainly drop the connection.  However, we want to 
be able to leave the flexibility for implementations that can tolerate 
occasional hiccups.  But you are correct that the error handling for these 
new keys (except MaxOutstandingUnexpectedPDUs) are not spelled out.  I am 
fine with removing the sentence "If the sending side..." from 6.7.  But 
instead I would consider adding the following changes to section 10.1.3.5. 
 I will combine the existing text in section 10.1.3.5 into one paragraph 
to describe the handling of iSER Hello/HelloReply exchange error, and add 
a new paragraph which states "If the sending side of an iSER-enabled 
connection acts in a manner not permitted by the negotiated or declared 
login/text operational key values as described in section 6, this is a 
protocol error and the receiving side MAY (capitalize?) handle this the 
same as for iSER format errors as described in section 10.1.3.4."

Mike
To:     [email-masked]
cc:     [email-masked] 
Subject:        Re: [Ips] DRAFT Minn. minutes

paul