Subject: Framing Formats

From: [email-masked]
Date: Tue, 10 Jul 2001 11:06:39 -0400

NOTE: iSCSI tag removed because there's FCIP-related
material in this message.

> We don't want this thread to be started again in this forum.
> For anything related to framing go to the TSWG or to the RDMA mailing
list.

I think Julian may have jumped for that conclusion a little too quickly.
Certainly, there are topics that belong on those mailing lists:

- TCP-aware framing mechanisms are in the domain of TSVWG.  Use
	of URG and PSH for framing is not going to happen.
- RDMA mechanism discussions belong on the RDMA list - this
	is not an official IETF list.

Beyond these, there are a couple of topics that are fair game
for the IPS list:

- TCP-unaware framing mechanisms intended for specification as
	part of an IPS protocol; this includes markers, as well as
	the synchronization recovery mechanism proposed for FCIP.
- Discussion of appropriate requirements (e.g., should markers
	be specified as "MUST implement" with some additional
	rules about which end of a connection controls their
	usage on what traffic?).

The requirements issue is definitely an open issue at this point
in time.  As for alternatives to markers, a fairly high level of rigor
will be expected of approaches that scan data looking for some
distinctive pattern - it basically needs to be the case that this
sort of scan and detect algorithm cannot get confused by any
pattern appearing in a data payload (read or write data for
iSCSI).  A motivating example to think about is where trace
data that includes the distinctive pattern (e.g., from a sniffer)
is being written to or read from a file.
Byte and word stuffing mechanisms to ensure that
the pattern the algorithm is looking for can't appear in data
are one way to accomplish this, but there are others.

While I'm on this topic, let me note that the algorithm in
Annex A of the -03 FCIP draft does not meet this "cannot
get confused" criterion.  For FCIP, it is possible to meet
this criterion without resorting to stuffing techniques - the
basic idea is to scan enough incoming data so that there
have to be valid headers in there, note every possible
header found, and their relationships based on length
fields.  If the resulting situation is ambiguous/confusing,
restart and/or abandon the attempt to re-establish
synchronization.  A considerably improved algorithm
should appear in the -04 version of the FCIP draft based
on some offline discussion with some of the draft authors.

Thanks,
--David

---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA  01748
+1 (508) 435-1000 x75140     FAX: +1 (508) 497-8500
[email-masked]       Mobile: +1 (978) 394-7754
---------------------------------------------------

---

From: "Mike Parkhurst" <[email-masked]>
Date: Tue, 10 Jul 2001 10:23:52 -0700

In my previous posting I was referring to the TCP-unaware variety. At
the heart of the matter is whether we trust the underlying transport
mechanism or not. If we don't, then there has to be some sort of
error-recovery procedure for when information/data is lost in transit.
If we do trust it, then the iSCSI specification is simplified and we
assume that the protocol is riding on an error free link. 

But to go back to the point of this thread....

Byte and word stuffing mechanisms have been proven to work for many
years. Picking a 32 bit number reduces the chances of it showing up in
the data stream. Even if it does the stuffing mechanism would mask it. 

Mike

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
[email-masked]
Sent: Tuesday, July 10, 2001 7:07 AM
To: [email-masked]
Cc: [email-masked]
Subject: Framing Formats

list.

Thanks,
--David

---

From: "Rod Harrison" <[email-masked]>
Date: Tue, 10 Jul 2001 20:42:11 +0100

Byte stuffing will kill the performance of software
implementations.

	Do we really believe re-synching in this way is necessary?
Remember that the host SCSI layer will retry failed
commands. I believe we will see error rates low enough to
make pushing recovery back onto the host a viable solution.

	- Rod

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On
Behalf Of
Mike Parkhurst
Sent: Tuesday, July 10, 2001 6:24 PM
To: [email-masked]
Cc: [email-masked]
Subject: RE: Framing Formats

In my previous posting I was referring to the TCP-unaware
variety. At
the heart of the matter is whether we trust the underlying
transport
mechanism or not. If we don't, then there has to be some
sort of
error-recovery procedure for when information/data is lost
in transit.
If we do trust it, then the iSCSI specification is
simplified and we
assume that the protocol is riding on an error free link.

Byte and word stuffing mechanisms have been proven to work
for many
years. Picking a 32 bit number reduces the chances of it
showing up in
the data stream. Even if it does the stuffing mechanism
would mask it.

Mike

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
On Behalf Of
[email-masked]
Sent: Tuesday, July 10, 2001 7:07 AM
To: [email-masked]
Cc: [email-masked]
Subject: Framing Formats

list.

I think Julian may have jumped for that conclusion a little
too quickly.
Certainly, there are topics that belong on those mailing
lists:

- TCP-aware framing mechanisms are in the domain of TSVWG.
Use
	of URG and PSH for framing is not going to happen.
- RDMA mechanism discussions belong on the RDMA list - this
	is not an official IETF list.

Beyond these, there are a couple of topics that are fair
game
for the IPS list:

- TCP-unaware framing mechanisms intended for specification
as
	part of an IPS protocol; this includes markers, as well as
	the synchronization recovery mechanism proposed for FCIP.
- Discussion of appropriate requirements (e.g., should
markers
	be specified as "MUST implement" with some additional
	rules about which end of a connection controls their
	usage on what traffic?).

The requirements issue is definitely an open issue at this
point
in time.  As for alternatives to markers, a fairly high
level of rigor
will be expected of approaches that scan data looking for
some
distinctive pattern - it basically needs to be the case that
this
sort of scan and detect algorithm cannot get confused by any
pattern appearing in a data payload (read or write data for
iSCSI).  A motivating example to think about is where trace
data that includes the distinctive pattern (e.g., from a
sniffer)
is being written to or read from a file.
Byte and word stuffing mechanisms to ensure that
the pattern the algorithm is looking for can't appear in
data
are one way to accomplish this, but there are others.

this criterion without resorting to stuffing techniques -
the

Thanks,
--David

---

From: "Nate Lawson" <[email-masked]>
Date: Tue, 10 Jul 2001 13:56:03 -0700

> From: [email-masked] On Behalf Of Mike Parkhurst

With TCP you get reliable delivery and sending rate adaptability for free.
Some posts on this thread have expressed sentiments doubting the reliability
of the transport which is surprising.

> the data stream. Even if it does the stuffing mechanism would mask it.

At least in the most naive case, this hurts performance since you have to
analyze more data looking for the marker than you would otherwise
(type/length/value or fixed size headers).  I certainly hope that I wouldn't
have to parse the data of a read looking for autosense data separated by a
marker.  If you have N bytes of data, here is a summary of the amount you'd
have to examine to retrieve a header:

Naive case (say SLIP over a serial link): N bytes
Fixed length packet size: 0 bytes
Fixed length header, variable data: ~4 bytes (length field)
Type/Length: 1 byte type, 4 byte length

The problem with fixed headers is that they're not expandable for future
protocols.  To be fair, TLV has a problem where if the length field gets
corrupted, you have to read up to the maximum number of bytes before
throwing away the PDU.  But I thought iSCSI will be depending on a reliable
transport.

In any case, TLV should be familiar since this is how SCSI cdbs work.

-Nate

---

From: "Mike Parkhurst" <[email-masked]>
Date: Tue, 10 Jul 2001 15:06:57 -0700

If the iSCSI protocol is going to take on the responsibility of insuring
a reliable data stream then there are only two alternatives.

	1) Fixed length commands with padding.
	or
	2) Pattern markers (and the required byte stuffing) that allow
the 
         detection of a command frame boundary. (I agree that this
approach
         is oriented more towards those with hardware solutions)

Personally I would rather drop the whole issue of Syncing and Steering
(section 1.2.8) from the specification all together. In paragraph two of
section 1.2.8.1 there are reasons given for not allowing TCP to do the
re-ordering needed when packets are either dropped or show up out of
order. Yet in paragraph three of the very same section we are told to
use the same TCP techniques to handle dropped or up out of order
packets.

Mike

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Rod Harrison
Sent: Tuesday, July 10, 2001 11:42 AM
To: [email-masked]
Subject: RE: Framing Formats

	- Rod

-----Original Message-----

Mike

-----Original Message-----

list.

Thanks,
--David

---

From: "Rod Harrison" <[email-masked]>
Date: Tue, 10 Jul 2001 23:31:43 +0100

Mike,

	We have (1) now.

	We have a marker scheme for (2) that doesn't require a
search of the data to re-synch.

	Implementations that want to rely on TCP can do so, and
push back onto the host if and when that fails. This should
be very rare, I believe there is a discussion about exactly
how rare somewhere in the archives.

	The only reason to not allow TCP to re-order out of order
packets is to reduce memory footprint. Using the marker
scheme we already have allows an implementation to always
find a header without having to resort to scanning the data
for sentinel patterns. The option to push back onto the
hosts error recovery is always there if this fails.

	However, where I think we agree is on whether iSCSI needs
to provide more integrity than TCP itself. I personally
don't believe so, but only some mileage on the clock will
really tell us.

	- Rod

-----Original Message-----
From: Mike Parkhurst [mailto:[email-masked]]
Sent: Tuesday, July 10, 2001 11:07 PM
To: [email-masked]
Cc: 'Rod Harrison'
Subject: RE: Framing Formats

If the iSCSI protocol is going to take on the responsibility
of insuring
a reliable data stream then there are only two alternatives.

	1) Fixed length commands with padding.
	or
	2) Pattern markers (and the required byte stuffing) that
allow
the
         detection of a command frame boundary. (I agree
that this
approach
         is oriented more towards those with hardware
solutions)

Personally I would rather drop the whole issue of Syncing
and Steering
(section 1.2.8) from the specification all together. In
paragraph two of
section 1.2.8.1 there are reasons given for not allowing TCP
to do the
re-ordering needed when packets are either dropped or show
up out of
order. Yet in paragraph three of the very same section we
are told to
use the same TCP techniques to handle dropped or up out of
order
packets.

Mike

-----Original Message-----

	- Rod

-----Original Message-----

Mike

-----Original Message-----

list.

Thanks,
--David

---

From: "Matt Wakeley" <[email-masked]>
Date: Wed, 18 Jul 2001 12:24:38 -0700

>         We have (1) now.

No we don't.  iSCSI PDUs are variable in length.

<snip>

>         1) Fixed length commands with padding.

[ I'm assuming Mike really meant fixed length PDUs - what is a fixed length
command? ]

-Matt