Subject: iSCSI Markers and Framing, a recomendation

From: "John Hufferd" <[email-masked]>
Date: Mon, 14 Jan 2002 12:37:20 -0800

Now that I have punished you all with the notes defining what is Framing
and COWS.  I will lobby for one.

Here again are the choices

1. FIM now, and Bare Framing later
2. FIM now, and COWS Framing later
3. COWS now, and Bare Framing later
4. COWS now, and COWS Framing Later
5. Nothing now, and some kind of Framing Later
6. FIM now, and some kind of Framing Later

Two arguments one for FIM now and one for Bare Framing later:

Let me start with "Framing Later":

The more I have seen of the work going on defining RDMA/iWARP, and the work
that is going into the Framing Experimental Status, the more I believe, we
can NOT now know how this will work out.  Let me give you an example.  The
only significant concern, with Bare Framing,  is the case of a false
positive indicator only when the network has re-segmented the TCP Segment.
The Bare Framing protocol has a 48-bit Random number and a 16 bit length
field to detect this.  I believe this has a lower probability of false
positives then passing an undetected error through the 16 bit TCP/IP XOR
Checksum, and possibly even the 32-bit CRC.  So the Experimental work was
started to prove that this was not an issue.  Now, even though Bare Framing
is the easiest to implement, and probably statically just fine (at least in
my opinion), some folks are attempting to come up with a fool proof
approach, which probably causes more implementation costs. (You can bet
that will be a major shoot out.)  No one yet has been able to do the needed
math to prove that Bare Framing is a problem or not a problem (volunteers
anyone?).  But sooner or later some one will do that.  There is even a
chance that another word could be added to the Random Number to make it
even stronger.    In the mean time we have two Different COWS proposals,
and no one knows if something now unknown will bleed over from the RDMA
work.

We simply do not know how the Framing will work out.

iSCSI choosing a Marking approach to match one of the perceived Framing
approaches, will not cause that approach to happen, and it is probably
going to cause turf wars, and other non productive interactions.

Therefore, I have come to the conclusion that Markers and Framing are
disjoint issues.  Further, we have no control over the direction of
framing.

Now my arguments for "FIM now":

FIM is the lowest overhead Marker approach we have been able to come up
with.  I think we need something that can easily be placed into SW ( In
this context I am talking about SW that does not  "OWN" the Host TCP/IP
stack.).  As I have stated before, FIM is simple to implement and can
greatly help some HW.   It needs to be Sent only if requested, and not
other wise.   It has been in the spec for some time, is therefore well
understood  and I think it has been scrubbed my a number of different
vendors.  Some of which have told me that they have either placed it in
their product or are working on that.  This is NOT to say that we can not
change it, but that because of its longevity in the spec, has undergone a
lot of study.  And we should have important reasons not to do FIM (such as
if it doesn't work).
Now, Jonathan Stone, has stated that "transatlantic can regularly show 50%
drop rate".   I would like the vendors to have some  approach to enable a
reasonable solution, to this and other long haul requirements, which keeps
the Adapter cost as low as possible.  Also, that would mean that it can NOT
wait until 10 Gig.

Now to the issue of MUST or SHOULD implement.

I have been persuaded that "SHOULD implement on Send" but optional to use,
would be a more acceptable position, since vendors with good and just
reasons would not have to implement it, yet the customer could find
solutions if they needed them.

Therefore, based on the above reasoning, I am recommending:

Choice 6, with "SHOULD implement on Send".  That is,

Leave FIM in the Spec, and add an enabling statement about "SHOULD
implement on Send if requested by the receiving side".

.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Home Office (408) 997-6136, Cell: (408) 499-9702
Internet address: [email-masked]

---

From: "Douglas Otis" <[email-masked]>
Date: Mon, 14 Jan 2002 19:24:23 -0800

John,

One other possible consideration would be to use a framing transport that
also solves:
 - Blind or Flood Attacks
 - Single State Fast Recovery Multi-Homing
 - Non-Blocking Service Level Resolution
 - Improved Data Integrity with Framing and Alignment
 - Standardized Path Management
 - TLV Bundling for Differential Path MTUs
 - Unordered Delivery
 - Payload Identification

These features are extremely important for any mission critical application.

The AAA WG already recommends this new transport for authentication,
authorization, and accounting services.
IPS should review current RSERPOOL efforts for Association aggregation as
well.

A transatlantic link with a 50% loss rate is not of any concern with respect
to bandwidth or system overhead.  This would represent almost no traffic and
would be best handled by a non-modified TCP implementation.

One consideration not in this debate would be to exclude modifications to
TCP and not incorporate these extensions by means of experimental techniques
and instead consider improvements found by a sanctioned  transport.  This
would go a long way in restoring trust.

A random number could be a string of 20 hex bytes or all zeros and the
header found by resegmentation may have a high likelihood of presenting that
value.  In other words, not all random numbers offer significant protection.
Once the wrong header is used, there is no recovery possible even if such an
error is later uncovered, as the intent is to de-encapsulate data below the
transport and place it directly into user buffers.

Techniques specific to iSCSI are not likely to find support in the future as
features not possible by these patch methods will force iSCSI to remain
inferior to ongoing work.  Should you wish to find a technique that will be
used by a broad range of applications, then a search for this technique
should not be conducted within this workgroup, as this is clearly not its
charter nor its expertise.  Your choice of Should use Fixed Interval Markers
is perhaps innocuous, but as this represents changing TCP to make use of
this feature; would not this recommendation of Should be a bit too strong?

Doug

---

From: James Smart <[email-masked]>
Date: Wed, 30 Jan 2002 12:46:06 -0500

As you were looking for more votes from those on the reflector...

I would vote for either 5 or 6, where 6 is defined as - FIM optional to 
use, separately enabled/disabled for tx and rx paths, and should 
implement on the tx path.

-- James

---

From: "Amir Shalit" <[email-masked]>
Date: Wed, 30 Jan 2002 11:42:59 -0800

I would vote for 6.

Amir

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of
James Smart
Sent: Wednesday, January 30, 2002 9:46 AM
To: [email-masked]
Subject: Re: iSCSI Markers and Framing

-- James