Subject: iSCSI-related conclusions from Orlando interim Meeting

From: [email-masked]
Date: Fri, 19 Jan 2001 19:29:36 -0500

This is a summary of the conclusions reached in the iSCSI area
at the interim meeting in Orlando earlier this week.  FCIP-related
conclusions will follow under separate cover.  These conclusions
are stated as preliminary consensus of those in the room in Orlando
and are subject to further discussion and revision on this mailing
list.

(1) Framing
- The iSCSI draft will be revised to add a formal iSCSI interface
	to framing
	- This will support the current marker approach, SCTP, and
		SCTP-like chunking for TCP.
	- The existing description of markers will be moved to an
		appendix as an example.
-The WARP group working on RDMA will write a draft on using SCTP-like
	chunking for iSCSI without RDMA

(2) Error Recovery
- Terminology Changes
	- "Reference Number" and "RN" are used only for SCSI
		-iSCSI uses "Sequence Number" and "SN" instead
	- "AER" is used only for SCSI
		- iSCSI communication of asynchronous events is through a
mechanism
		 	that is now called "Asynchronous Messages" - iSCSI
uses these
			to implement AER
		- If a SCSI initiator has disabled AER, iSCSI does not send
the
			corresponding Asynchronous Messages
- CmdSN (formerly CmdRN) is mandatory in all cases to simplify things.
	- This includes single connection sessions
- SCSI has defined a new (optional) SCSI Command Ref Number
	- iSCSI will use byte #2 in iSCSI header (currently Reserved) to
transport this
	- This is not the ideal solution, but matches the level of support
in FCP.
	- We have to check where mode page is to negotiate this
		- If it's on a transport-specific mode page, iSCSI will
			use a text key to negotiate this instead
- DataSN (formerly DataRN) functionality will be removed.  Error recovery is
now at the
	granularity of commands, not within a command.
- There will be a significant connection recovery write-up, including
details, procedures
	and examples added to the draft.
- Ping/NOP issues
	- A description of intended usage will be added to the draft as
advice to
		implementers.
		- A ping is intended to check whether the corresponding
protocol is alive
	- NOP responses are not permitted to request responses to avoid
loops
- Abort WARNING will be added to the draft.
	- Immediate Delivery of Aborts and similar task management commands
may have
		unexpected results when multiple TCP connections are in use
because
		Abort, Clear Task Set, etc. may bypass command(s) to be
aborted/cleared
		on other TCP connections.
	- Ordered Delivery should be used instead when this is a concern.
(3) CRC
	- iSCSI will use separate CRCs on header and data to make header
modification
		easier for systems that need to do that.
		- Using the same CRC algorithm on header and data is simpler
- in particular
			the idea of using a 16-bit CRC on the header was
discarded.
		- One CRC should cover both the fixed header and optional
extensions
	- Sense of the room on CRC Algorithms
		- CRC-32 is the obvious first/default choice.
		- There is some interest in investigating both weaker
(Adler-32) and
			stronger (CRC-64) CRCs (CRC-64 may not be
appropriate for header,
			and is still a research topic).
	- CRCs are MUST implement
		- Open issue: whether CRC use is negotiable
		- Default: Use CRCs
	- Limiting the amount of data per CRC
		- Probability of undetected error rises with amount of data
covered by
			one CRC.
		- There will be at most one data CRC per PDU
		- This limit SHOULD be enforced limit by fragmenting large
data blocks
			into Multiple Data PDUs
		- Julian Satran will look for the reference that suggests
CRC-32 should
			not be used on data blocks significantly larger than
8-10k.
(4) Security
	- Current iSCSI security digests will be removed in favor of IPsec
and/or TLS
		- Only reason for digests is data integrity (i.e., CRCs)
		- Open issue: How does iSCSI negotiate or detect presence of
lower
			level security?
		- Open issue: What is minimum security required to be used
			(authentication/integrity) by IETF?  David Black
			- SRP and Kerberos login authentication will remain
				in the iSCSI draft pending resolution of
this issue.
(5) Naming
	- UTF-8 will be used instead of ASCII for text strings in iSCSI
login and
		text commands, naming, discovery, etc.
	- Binary values will be encoded in UTF-8 rather than adding
type/length support.
	- Localization of iSCSI text is forbidden on the wire for
interoperability
		reasons (e.g., keys/values in login and text commands are
the same
		byte strings, no matter what the locale).

Ok, comment away ...

Thanks,
--David

---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA  01748
+1 (508) 435-1000 x75140     FAX: +1 (508) 497-8500
[email-masked]       Mobile: +1 (978) 394-7754
---------------------------------------------------

---

From: David Robinson <[email-masked]>
Date: Fri, 19 Jan 2001 16:47:17 -0800

>         - Binary values will be encoded in UTF-8 rather than adding

What does this mean? UTF-8 is an encoding of Unicode character sets, not
arbitrary
binary values. It is still just binary data so what
possible advantage is this?

	-David

---

From: "David Peterson" <[email-masked]>
Date: Fri, 19 Jan 2001 18:38:51 -0600

The SCSI command reference number is negotiated via the logical unit control
page (18h).
Dave

---

From: Santosh Rao <[email-masked]>
Date: Fri, 19 Jan 2001 18:01:19 -0800

I assume the below is referring to targets using NOP-IN as a NOP request in
order to test a connection (?)

If so, I don't see how this can result in a loop. A NOP-IN is sent with an
Initiator Task Tag of 0xFFFFFFFF (assuming the errant usage of 0 as a reserved
value were to be fixed in the spec). An initiator on seeing a NOP-IN with an
I.T.T. of 0xFFFFFFFF will respond by sending NOP-OUT with I.T.T. of 0xFFFFFFFF.
When a target receives a NOP-OUT with an I.T.T. of 0xFFFFFFFF, it is aware that
this is a response to a NOP-IN it had originated earlier with a T.T.T that it
specifies.

How is a loop caused in the above scenario ?

However, there is the other issue of the convoluted and non-intuitive logic in
the above approach. There exist benefits of simplicity of understanding by the
usage of NOP-OUT [or NOP Command] for the request and NOP-IN [or NOP response]
for the response.

Regards,
Santosh

---

From: Santosh Rao <[email-masked]>
Date: Fri, 19 Jan 2001 17:49:30 -0800

All,

I have raised this question on the reflector earlier without getting any
response.

If the target's only means for a logout is to "request a logout" through an AEN
(now called Asynch Message) and host SCSI stacks disable AER by default (which
is the behaviour today), then, targets DO NOT have a reliable means of logging
initiators out, since they cannot send AE messages.

Targets MUST be allowed to send logouts or another reliable means be defined in
the spec that allow targets to logout initiators. This peer-to-peer model
[wherein a target can originate logout] is not a violation of SAM and allowing
targets to send logouts is the most expeditious form of error recovery at the
target end.

Such a peer-to-peer model will also rid the spec of its convoluted way of using
NOP-IN [which is a NOP response] as a NOP request when targets wish to check
the  connection.

Targets MUST be allowed to originate Logout and NOP-OUT.

Regards,
Santosh

> mechanism that is now called "Asynchronous Messages" - iSCSI
> uses these to implement AER

> the corresponding Asynchronous Messages

---

From: Santosh Rao <[email-masked]>
Date: Sun, 21 Jan 2001 16:05:22 -0800 (PST)

>   may have unexpected results when multiple TCP connections are in use
>   because Abort, Clear Task Set, etc. may bypass command(s) to be
>   aborted/cleared on other TCP connections.

Is the above referring to ordered delivery as in CmdSN based ordering or
the use of the Ordered Task Tag ?

I see the following issues with this :

a) CmdSN defines the order of delivery from the iSCSI layer to the SCSI 
   ULP at the target. 
   
   Abortion of active tasks will involve the invalidation of the 
   SCSI State information [being accessed by either adapter micro-code 
   or the FPGA/ASIC] maintained for these tasks. 
   This process is kicked off in the iSCSI layer and this abort 
   functionality in the iSCSI layer will also need to be subject to the 
   ordering constraints of CmdSN. (in addition to the ordering of delivery 
   to the SCSI ULP).

b) A received Task Management command [on, say, connection 'x'] 
   cannot be performed due to previous CmdSNs not yet
   having arrived. (In the case where another connection 'y' in the 
   session is faulty and CmdSNs sent on connection 'y' have'nt arrived).

c) Ordered Task Tags should not be used with error recovery Task
   Management commands, since this can result in a deadlock of the task
   mgmt command in case the older tasks are'nt completing in the SCSI ULP
   due to some ULP problem. ("An Ordered Task cannot enter the enabled
   state until all older tasks have completed.")

d) Given that Immediate command delivery (CmdSN of 0) is un-usable for
   task management commands, is there any other application for this 
   feature? If not, all references to it can be removed from the draft and
   CmdSN can start from 0.

Regards,
Santosh

-- 
#################################
Santosh Rao
Software Design Engineer,
HP, Cupertino.
email : [email-masked]
Phone : 408-447-3751
#################################