From mailnull@www1.ietf.org  Thu Apr  3 18:30:26 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA20023
	for <rddp-archive@odin.ietf.org>; Thu, 3 Apr 2003 18:30:26 -0500 (EST)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h33NWth10565
	for rddp-archive@odin.ietf.org; Thu, 3 Apr 2003 18:32:55 -0500
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h33NWtK10562
	for <rddp-web-archive@optimus.ietf.org>; Thu, 3 Apr 2003 18:32:55 -0500
Received: from www1.ietf.org (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA20019
	for <rddp-web-archive@ietf.org>; Thu, 3 Apr 2003 18:29:54 -0500 (EST)
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h33NWGK10537;
	Thu, 3 Apr 2003 18:32:16 -0500
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h33NVUK10501
	for <rddp@optimus.ietf.org>; Thu, 3 Apr 2003 18:31:30 -0500
Received: from mxic1.corp.emc.com (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA19976
	for <rddp@ietf.org>; Thu, 3 Apr 2003 18:28:29 -0500 (EST)
From: Black_David@emc.com
Received: by mxic1.corp.emc.com with Internet Mail Service (5.5.2653.19)
	id <GLPWZXQA>; Thu, 3 Apr 2003 18:30:59 -0500
Message-ID: <277DD60FB639D511AC0400B0D068B71E0A64182F@corpmx14.us.dg.com>
To: rddp@ietf.org
Date: Thu, 3 Apr 2003 18:26:29 -0500 
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: [rddp] Please send presentations
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

Everyone who used slides in the RDDP meeting in San
Francisco - please send me a copy of your slides if
you have not already done so.

Thanks,
--David

----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
black_david@emc.com        Mobile: +1 (978) 394-7754
----------------------------------------------------
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr  3 21:07:28 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA22759
	for <rddp-archive@odin.ietf.org>; Thu, 3 Apr 2003 21:07:27 -0500 (EST)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3429xd21629
	for rddp-archive@odin.ietf.org; Thu, 3 Apr 2003 21:09:59 -0500
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3429xK21626
	for <rddp-web-archive@optimus.ietf.org>; Thu, 3 Apr 2003 21:09:59 -0500
Received: from www1.ietf.org (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA22752
	for <rddp-web-archive@ietf.org>; Thu, 3 Apr 2003 21:06:56 -0500 (EST)
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3429CK21594;
	Thu, 3 Apr 2003 21:09:12 -0500
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h34287K21555
	for <rddp@optimus.ietf.org>; Thu, 3 Apr 2003 21:08:07 -0500
Received: from mxic1.corp.emc.com (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA22729
	for <rddp@ietf.org>; Thu, 3 Apr 2003 21:05:04 -0500 (EST)
From: Black_David@emc.com
Received: by mxic1.corp.emc.com with Internet Mail Service (5.5.2653.19)
	id <GLPWZ6RD>; Thu, 3 Apr 2003 21:07:33 -0500
Message-ID: <277DD60FB639D511AC0400B0D068B71E0564CABD@corpmx14.us.dg.com>
To: rddp@ietf.org
Date: Thu, 3 Apr 2003 21:03:02 -0500 
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: [rddp] RDDP DRAFT San Francisco Minutes
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

Thanks to Allyn for taking notes.  Send
corrections to me and/or the list please.

Thanks,
--David



RDDP WG Meeting Minutes **DRAFT**
56th IETF
March 17 & 20 2003
------------------

-- TCP Mapping Requirements (David Black)

This is a reminder from the WG chair in combination with the Area Directors
of what is expected of the RDDP WG's TCP mapping work.  

(1) The "proof" requirement that two communicating stacks can't get confused
about the format of communication applies to the entire RDDP protocol stack
running over TCP, not just the issue of whether a modified vs. unmodified
TCP
is being used.  32 bits of zeroes (first marker) to distinguish what is
going
on is not enough, 128 bits randomly generated for the connection is enough
(sufficient, but may be more than needed).  Relying on ULP to do this is
considerably weaker than solving the problem in RDDP.  Negotiation is not
required - a declaration mechanism like the SCTP adaption indication
exchange
is sufficient (no additional round trip is required).

(2) The nfsv4 WG is starting work on application of RDDP to NFSv4.  NFSv4's
current level of data integrity does not require a CRC; if this remains
the case for NFSv4 over RDDP, the RDDP CRC will need to be optional to use.

(3) Fixed interval markers are a potential problem for software
implementations
(see below, as a great deal of time was spent on this issue later).

-- Administrivia

Reminder that the problem statement and architecture drafts are in WG
Last Call.  Please read and comment on the list.

-- SCTP Mapping Draft (Randy Stewart)

See slides (will be posted).  Draft is relatively far along.
Note that SCTP uses (completely) unordered delivery when carrying RDDP
	traffic.  This requires DDP to track its operation ordering
	in order to enforce any delivery order requirements.

-- Applicability Statement Draft (Randy Stewart for Caitlin Bestler)

Plan has changed to a 3 draft approach - (1) TCP mapping, (2) SCTP mapping,
and (3) an applicability statement draft covering both mappings.  Lode Coene
(editor of SCTP's applicability statement, which set a good example) has
agreed to help out with the RDDP applicability statement (many thanks).

Advice from the WG chairs and ADs: A good applicability statement is
guidance on utility, not advertising to promote usage.  An important
target audience is ULP designers who are looking advice on whether
to use RDDP and with which transport.

Both the SCTP mapping draft and applicability statement draft were approved
to be official RDDP WG drafts - next versions will have draft-ietf-rddp-*
names.

-- Allyn Romanow - NICELI Workshop reminder and security update

For NICELI, see:
http://www.acm.org/sigs/sigcomm/sigcomm2003/workshop/niceli/

Security update - folks working on this issue are making good progress
with Catherine Meadows (security advisor).  Initial draft will be posted
for discussion in the next 2-3 weeks.

-- DDP Update -  Hemal Shah
couple of small changes.  

-- RDMAP Update - Paul Culley (for Renato Recio)
a couple of small changes

-- TCP mapping
- MPA Update - Paul Culley

Packing of multiple ULPDUS into single TCP segment is now allowed.
MPA is now specified as an "adaptation layer" for RDDP that must
be explicitly enabled by higher-level protocol (i.e., DDP/RDMA or
ULP further up the stack).

- IFT update - Jim Williams

Added adaption indication, based to some extent on the one used
with SCTP.  CRC is optional and negotiable
Markers are negotiable, with some details to be fleshed out.
Speculative placement proposed for use when markers not provided -
	this received a rather negative reaction in the meeting.

-- TCP mapping issues

David Black worked through several major outstanding issues for the WG:

(1) Alignment and TCP Mapping

This is about aligning transmitted ULPDUS (w/RDDP headers) to TCP segment
boundaries.  

David Black summarized from the mailing list discussion and made a proposal
Alignment is not required for interoperability, the RDDP receiver
cannot rely on it, but it is a potentially useful optimization.

Therefore, the proposal is to treat alignment as an implementation
	optimization only, and hence not use any MUST or SHOULD (in the
	RFC 2119 sense) for alignment, but instead use (at most) lower
	case "should" along with a description of the circumstances
	in which alignment is a useful optimization.

This represents rough consensus of the Working Group participants in the
room.

Additional note from WG chair: While nothing is certain, this approach is
	very likely to be acceptable to the tsvwg (Transport Area) WG, where
	past alignment work has generated controversy.

(2) Markers - this generated a *lot* of discussion, mostly focused on marker
interval.

Markers are motivated by a  need to regain synchronization for placement
after a packet drop or reordering without having to waiting for retransmit.
Markers allow finding where the header is in byte stream.
MPA currently requires markers at 512 byte fixed intervals

A potential problem with markers is the overhead for existing stacks
For example, for an 8KB transfer, 1 send operation becomes 16 sends
(Some people suggest fewer than 16 because can use gather technique.)
This overhead is a potential problem for a software implementation.

Markers have been motivated by flow through designs that may be overly
optimized, especially given above rough consensus on alignment.  The
discussion focused on buffering requirements, and whether a link
frame per connection is required - it clearly is in the worst case,
but there are enough shorter packets/frames (e.g., about 500 bytes
on Ethernet) that on average across multiple connections, one can
use less, and hence a marker interval shorter than the max link
frame size (e.g., about 1500 bytes for Ethernet without jumbo frames)
does save implementation resources.

There was at least one comment that Markers should be optional.

There is no rough consensus, but David notes that there is a lot of
support for markers and the 512 byte interval in particular.

(3) NFSV4 and CRCs
The nfsv4 WG charter now includes NFSV4 over RDDP problem statement
and requirements.

NFS does not require stronger data integrity than TCP/IP checksums, so
a mandatory RDDP CRC may be excessive although optional RDDP CRC could
be useful.

The argument in the other direction is that RDDP is new and wants to
provide a better level of service (data integrity), so demanding that
all users of RDDP step up to that level of service is within reason.

After much discussion the conclusion is to defer the issue of whether
to require CRC usage for further input from the nfsv4 WG's requirements
work.

-David concludes:

alignment - there is rough consensus in the room, will recheck on list.
marker interval - there is clear support for a 512B interval
CRC - will defer "MUST use" issue until nfsv4 WG is further along in
	considering requirements for RDDP with NFSv4.

Hope to close remaining issues in this space by summer IETF (Vienna)
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Mon Apr  7 19:42:15 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA10773
	for <rddp-archive@odin.ietf.org>; Mon, 7 Apr 2003 19:42:15 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h37Nkf623832
	for rddp-archive@odin.ietf.org; Mon, 7 Apr 2003 19:46:41 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h37Nkf823829
	for <rddp-web-archive@optimus.ietf.org>; Mon, 7 Apr 2003 19:46:41 -0400
Received: from www1.ietf.org (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA10741
	for <rddp-web-archive@ietf.org>; Mon, 7 Apr 2003 19:41:04 -0400 (EDT)
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h37NjK823774;
	Mon, 7 Apr 2003 19:45:20 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h37NiB823732
	for <rddp@optimus.ietf.org>; Mon, 7 Apr 2003 19:44:11 -0400
Received: from sandmail.sandburst.com (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA10709
	for <rddp@ietf.org>; Mon, 7 Apr 2003 19:39:15 -0400 (EDT)
To: rddp@ietf.org
Date: Mon, 07 Apr 2003 19:41:43 -0400
From: Stephen Bailey <steph@cs.uchicago.edu>
Message-Id: <20030407234145.DD7244EBF@sandmail.sandburst.com>
Subject: [rddp] Option or mandatory CRC in MPA
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

Hi,

It seems to me that one significant items holding up RDDP and
specifically MPA, is whether CRC should be optional or mandatory.

At the last IETF David said he would be uncomfortable requiring CRC
protection if a key current client protocol (NFS) is not going to
require the protection when not running on RDDP.  The primary argument
for this is to allow RDDP to cross the transition hurdle more easily
by not punishing software clients.

A majority of implementors want CRC, so clearly many implementations
will support it. There's incentive to implement it, and the penalty in
hardware implementations is negligable.

Implementations that do CRC can generate always and check optionally,
which is easy.

Rather than waiting for NFSv4 to settle the argument, it seems we
could push ahead with a reserved spot in the payload, and negotiable
checking.

Steph
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Tue Apr  8 08:43:59 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA08546
	for <rddp-archive@odin.ietf.org>; Tue, 8 Apr 2003 08:43:59 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h38Cmf821754
	for rddp-archive@odin.ietf.org; Tue, 8 Apr 2003 08:48:41 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h38Cmf821751
	for <rddp-web-archive@optimus.ietf.org>; Tue, 8 Apr 2003 08:48:41 -0400
Received: from www1.ietf.org (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA08540
	for <rddp-web-archive@ietf.org>; Tue, 8 Apr 2003 08:43:28 -0400 (EDT)
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h38Clx821683;
	Tue, 8 Apr 2003 08:47:59 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h38Cks821538
	for <rddp@optimus.ietf.org>; Tue, 8 Apr 2003 08:46:54 -0400
Received: from palrel13.hp.com (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA08453
	for <rddp@ietf.org>; Tue, 8 Apr 2003 08:41:41 -0400 (EDT)
Received: from esmail.cup.hp.com (esmail.cup.hp.com [15.0.65.164])
	by palrel13.hp.com (Postfix) with ESMTP id B27C21C0101B
	for <rddp@ietf.org>; Tue,  8 Apr 2003 05:44:13 -0700 (PDT)
Received: from mk731915.cup.hp.com ([15.113.161.2])
	by esmail.cup.hp.com (8.11.1/8.8.6) with ESMTP id h38Cedl12202
	for <rddp@ietf.org>; Tue, 8 Apr 2003 05:40:39 -0700 (PDT)
Message-Id: <5.1.0.14.2.20030408053557.0205bd40@15.13.130.73>
X-Sender: krause@15.13.130.73
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Tue, 08 Apr 2003 05:41:19 -0700
To: rddp@ietf.org
From: Michael Krause <krause@cup.hp.com>
Subject: Re: [rddp] Option or mandatory CRC in MPA
In-Reply-To: <20030407234145.DD7244EBF@sandmail.sandburst.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

At 07:41 PM 4/7/2003 -0400, Stephen Bailey wrote:
>Hi,
>
>It seems to me that one significant items holding up RDDP and
>specifically MPA, is whether CRC should be optional or mandatory.
>
>At the last IETF David said he would be uncomfortable requiring CRC
>protection if a key current client protocol (NFS) is not going to
>require the protection when not running on RDDP.  The primary argument
>for this is to allow RDDP to cross the transition hurdle more easily
>by not punishing software clients.
>
>A majority of implementors want CRC, so clearly many implementations
>will support it. There's incentive to implement it, and the penalty in
>hardware implementations is negligable.
>
>Implementations that do CRC can generate always and check optionally,
>which is easy.
>
>Rather than waiting for NFSv4 to settle the argument, it seems we
>could push ahead with a reserved spot in the payload, and negotiable
>checking.

In some discussions after the meeting with people in NFSv4, it seemed they 
may be fine with RDDP doing the CRC and letting it be optional at their 
layer.  Agree that NFSv4 workgroup needs to make a recommendation here but 
it is not clear to me that having it required is a major issue at all.  The 
only issue raised was why do it in both layers?   If NFSv4 can make it 
optional similar to how iSCSI can make digests optional then this can be 
resolved as a ULP issue allowing the RDDP implementations to be simplified 
by eliminating the need for negotiation.  Perhaps the workgroup chairs 
could organize a discussion to help get this issue resolved.  I would hate 
to make a change that may not be required at the end of the day in this regard.

Mike

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Tue Apr  8 15:02:01 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA26786
	for <rddp-archive@odin.ietf.org>; Tue, 8 Apr 2003 15:02:01 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h38J6pe20271
	for rddp-archive@odin.ietf.org; Tue, 8 Apr 2003 15:06:51 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h38J6p820268
	for <rddp-web-archive@optimus.ietf.org>; Tue, 8 Apr 2003 15:06:51 -0400
Received: from www1.ietf.org (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA26774
	for <rddp-web-archive@ietf.org>; Tue, 8 Apr 2003 15:01:30 -0400 (EDT)
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h38J6B820233;
	Tue, 8 Apr 2003 15:06:11 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h38J57820156
	for <rddp@optimus.ietf.org>; Tue, 8 Apr 2003 15:05:07 -0400
Received: from atlrel8.hp.com (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA26678
	for <rddp@ietf.org>; Tue, 8 Apr 2003 14:59:45 -0400 (EDT)
Received: from rosemail.rose.hp.com (rosemail.rose.hp.com [15.96.64.26])
	by atlrel8.hp.com (Postfix) with ESMTP id 94B751C01626
	for <rddp@ietf.org>; Tue,  8 Apr 2003 15:02:18 -0400 (EDT)
Received: from swathi (dhcp-rosefl-bba009.rose.hp.com [15.3.110.9]) by rosemail.rose.hp.com with SMTP (8.7.1/8.7.3 SMKit7.02) id MAA20671 for <rddp@ietf.org>; Tue, 8 Apr 2003 12:02:17 -0700 (PDT)
Message-ID: <002801c2fe01$5fa7f3a0$096e030f@rose.hp.com>
From: "Mallikarjun C." <cbm@rose.hp.com>
To: <rddp@ietf.org>
References: <20030407234145.DD7244EBF@sandmail.sandburst.com>
Subject: Re: [rddp] Option or mandatory CRC in MPA
Date: Tue, 8 Apr 2003 12:02:16 -0700
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2600.0000
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

I see some issues in doing the CRC checking as a negotiable feature 
at the MPA level.

Apart from the optionality aspect, I don't quite see how ULP-level
CRC negotiation (such as for iSCSI) can harmoniously work with
an MPA-level negotiation of the same.  There's no issue when it's
negotiated to "No".  But when it's negotiated to "Yes" at the iSCSI-level,
as an example, iSCSI wants to generate and check it - not delegate 
the checking/generation part to MPA even if MPA could optionally
do it.  I suspect this issue is not unique to one ULP - other comments,
particularly from NFS folks, would help here.

AFAICT, the only clean solution is to have the ULP always negotiate 
the usage to "No" (with the knowledge that the transport substrate is
doing it), and have the MPA layer always generate/check CRC.
--
Mallikarjun

Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions
Hewlett-Packard MS 5668 
Roseville CA 95747
cbm@rose.hp.com


----- Original Message ----- 
From: "Stephen Bailey" <steph@cs.uchicago.edu>
To: <rddp@ietf.org>
Sent: Monday, April 07, 2003 4:41 PM
Subject: [rddp] Option or mandatory CRC in MPA


> Hi,
> 
> It seems to me that one significant items holding up RDDP and
> specifically MPA, is whether CRC should be optional or mandatory.
> 
> At the last IETF David said he would be uncomfortable requiring CRC
> protection if a key current client protocol (NFS) is not going to
> require the protection when not running on RDDP.  The primary argument
> for this is to allow RDDP to cross the transition hurdle more easily
> by not punishing software clients.
> 
> A majority of implementors want CRC, so clearly many implementations
> will support it. There's incentive to implement it, and the penalty in
> hardware implementations is negligable.
> 
> Implementations that do CRC can generate always and check optionally,
> which is easy.
> 
> Rather than waiting for NFSv4 to settle the argument, it seems we
> could push ahead with a reserved spot in the payload, and negotiable
> checking.
> 
> Steph
> _______________________________________________
> rddp mailing list
> rddp@ietf.org
> https://www1.ietf.org/mailman/listinfo/rddp
> 
>  
> 

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Tue Apr  8 20:16:58 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA09280
	for <rddp-archive@odin.ietf.org>; Tue, 8 Apr 2003 20:16:58 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h390LrQ13608
	for rddp-archive@odin.ietf.org; Tue, 8 Apr 2003 20:21:53 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h390Lr813605
	for <rddp-web-archive@optimus.ietf.org>; Tue, 8 Apr 2003 20:21:53 -0400
Received: from www1.ietf.org (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA09274
	for <rddp-web-archive@ietf.org>; Tue, 8 Apr 2003 20:16:27 -0400 (EDT)
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h390LD813555;
	Tue, 8 Apr 2003 20:21:13 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h390KJ813516
	for <rddp@optimus.ietf.org>; Tue, 8 Apr 2003 20:20:19 -0400
Received: from emulex.emulex.com (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA09207
	for <rddp@ietf.org>; Tue, 8 Apr 2003 20:14:52 -0400 (EDT)
Received: from xbl.ad.emulex.com (xbl.ma.emulex.com [172.16.12.11])
	by emulex.emulex.com (8.12.9/8.12.8) with ESMTP id h390HOLW006878
	for <rddp@ietf.org>; Tue, 8 Apr 2003 17:17:24 -0700 (PDT)
Received: by xbl.ma.emulex.com with Internet Mail Service (5.5.2653.19)
	id <HN1WWKWX>; Tue, 8 Apr 2003 20:17:24 -0400
Message-ID: <3356669BBE90C448AD4645C843E2BF289B904B@xbl.ma.emulex.com>
From: "Williams, Jim" <Jim.Williams@Emulex.com>
To: "'rddp@ietf.org'" <rddp@ietf.org>
Subject: RE: [rddp] Option or mandatory CRC in MPA
Date: Tue, 8 Apr 2003 20:17:17 -0400 
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="ISO-8859-1"
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

The argument for making the CRC optional
in the RDDP over TCP mapping is the following.

  A.  A large percentage of usage will be on
      highly reliable private networks where
      the TCP checksum is more that sufficient
      the achieve the desired level or reliability.

  B.  Optimizing the performance of software 
      implementations is important, and computing
      an unneeded CRC is too high a price to pay
      for avoiding the need to negotiate an option.


The input from NFS is important because it takes
into account years of field experience.  If they
determine that for most all usage, even on private
networks, the TCP checksum is NOT adequate, then
I would regard that as a refutation of point A.
If they determine that for a large enough percentage
of implementations the TCP checksum IS adequate,
then I would regard that as support of point A.

Regarding point B, if one envisions that all 
implementations of interest are hardware customized
for RDDP, then the point does not carry much weight.
However, I would argue that this may be too narrow
a view for the IETF to take.
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Tue Apr  8 20:45:44 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA09997
	for <rddp-archive@odin.ietf.org>; Tue, 8 Apr 2003 20:45:44 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h390oeU15589
	for rddp-archive@odin.ietf.org; Tue, 8 Apr 2003 20:50:40 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h390oe815586
	for <rddp-web-archive@optimus.ietf.org>; Tue, 8 Apr 2003 20:50:40 -0400
Received: from www1.ietf.org (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA09959
	for <rddp-web-archive@ietf.org>; Tue, 8 Apr 2003 20:45:12 -0400 (EDT)
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h390o2815555;
	Tue, 8 Apr 2003 20:50:02 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h38LLE832010
	for <rddp@optimus.ietf.org>; Tue, 8 Apr 2003 17:21:14 -0400
Received: from pheriche.sun.com (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA02748
	for <rddp@ietf.org>; Tue, 8 Apr 2003 17:15:50 -0400 (EDT)
Received: from jurassic.eng.sun.com ([129.146.17.55])
	by pheriche.sun.com (8.9.3p2+Sun/8.9.3) with ESMTP id PAA21524
	for <rddp@ietf.org>; Tue, 8 Apr 2003 15:18:23 -0600 (MDT)
Received: from eng.sun.com (terra.Eng.Sun.COM [129.146.86.99])
	by jurassic.eng.sun.com (8.12.9+Sun/8.12.9) with ESMTP id h38LINmp232352
	for <rddp@ietf.org>; Tue, 8 Apr 2003 14:18:23 -0700 (PDT)
Message-ID: <3E933CA7.1080509@eng.sun.com>
Date: Tue, 08 Apr 2003 14:18:31 -0700
From: Brent Callaghan <brent@eng.sun.com>
Reply-To: Brent.Callaghan@Sun.COM
Organization: Sun Microsystems
User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.0.1) Gecko/20020920 Netscape/7.0
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: rddp@ietf.org
Subject: Re: [rddp] Option or mandatory CRC in MPA
References: <20030407234145.DD7244EBF@sandmail.sandburst.com> <002801c2fe01$5fa7f3a0$096e030f@rose.hp.com>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit


NFS can protect its data with an RPCSEC_GSS checksum, though
it's somewhat expensive to do in the host stack, so it's not
commonly used.

Instead, most NFS mounts today rely on the TCP checksum and
link CRC to protect data payload.

I doubt that many in the NFS community are completely
comfortable relying on the TCP checksum.  If RDDP provides
a 32 bit CRC that's implemented in hardware, then I'm sure
NFS would use it.

But given that NFS is already widely used over regular TCP,
I'm not sure there's enough support to make an RDDP CRC mandatory,
and a performance hurdle for host stack RDDP implementations.

	Brent


Mallikarjun C. wrote:
> I see some issues in doing the CRC checking as a negotiable feature 
> at the MPA level.
> 
> Apart from the optionality aspect, I don't quite see how ULP-level
> CRC negotiation (such as for iSCSI) can harmoniously work with
> an MPA-level negotiation of the same.  There's no issue when it's
> negotiated to "No".  But when it's negotiated to "Yes" at the iSCSI-level,
> as an example, iSCSI wants to generate and check it - not delegate 
> the checking/generation part to MPA even if MPA could optionally
> do it.  I suspect this issue is not unique to one ULP - other comments,
> particularly from NFS folks, would help here.
> 
> AFAICT, the only clean solution is to have the ULP always negotiate 
> the usage to "No" (with the knowledge that the transport substrate is
> doing it), and have the MPA layer always generate/check CRC.
> --
> Mallikarjun
> 
> Mallikarjun Chadalapaka
> Networked Storage Architecture
> Network Storage Solutions
> Hewlett-Packard MS 5668 
> Roseville CA 95747
> cbm@rose.hp.com
> 
> 
> ----- Original Message ----- 
> From: "Stephen Bailey" <steph@cs.uchicago.edu>
> To: <rddp@ietf.org>
> Sent: Monday, April 07, 2003 4:41 PM
> Subject: [rddp] Option or mandatory CRC in MPA
> 
> 
> 
>>Hi,
>>
>>It seems to me that one significant items holding up RDDP and
>>specifically MPA, is whether CRC should be optional or mandatory.
>>
>>At the last IETF David said he would be uncomfortable requiring CRC
>>protection if a key current client protocol (NFS) is not going to
>>require the protection when not running on RDDP.  The primary argument
>>for this is to allow RDDP to cross the transition hurdle more easily
>>by not punishing software clients.
>>
>>A majority of implementors want CRC, so clearly many implementations
>>will support it. There's incentive to implement it, and the penalty in
>>hardware implementations is negligable.
>>
>>Implementations that do CRC can generate always and check optionally,
>>which is easy.
>>
>>Rather than waiting for NFSv4 to settle the argument, it seems we
>>could push ahead with a reserved spot in the payload, and negotiable
>>checking.
>>
>>Steph
>>_______________________________________________
>>rddp mailing list
>>rddp@ietf.org
>>https://www1.ietf.org/mailman/listinfo/rddp
>>
>> 
>>
> 
> 
> _______________________________________________
> rddp mailing list
> rddp@ietf.org
> https://www1.ietf.org/mailman/listinfo/rddp


_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Tue Apr  8 21:06:48 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA10529
	for <rddp-archive@odin.ietf.org>; Tue, 8 Apr 2003 21:06:47 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h391BiQ17220
	for rddp-archive@odin.ietf.org; Tue, 8 Apr 2003 21:11:44 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h391Bi817217
	for <rddp-web-archive@optimus.ietf.org>; Tue, 8 Apr 2003 21:11:44 -0400
Received: from www1.ietf.org (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA10497
	for <rddp-web-archive@ietf.org>; Tue, 8 Apr 2003 21:06:16 -0400 (EDT)
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h391B3817188;
	Tue, 8 Apr 2003 21:11:03 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h391AB817156
	for <rddp@optimus.ietf.org>; Tue, 8 Apr 2003 21:10:11 -0400
Received: from e35.co.us.ibm.com (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA10423;
	Tue, 8 Apr 2003 21:04:42 -0400 (EDT)
Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.17.195.11])
	by e35.co.us.ibm.com (8.12.9/8.12.2) with ESMTP id h3917Es5070088;
	Tue, 8 Apr 2003 21:07:14 -0400
Received: from d03nm115.boulder.ibm.com (d03av02.boulder.ibm.com [9.17.193.82])
	by westrelay02.boulder.ibm.com (8.12.8/NCO/VER6.5) with ESMTP id h3917Dm6104050;
	Tue, 8 Apr 2003 19:07:14 -0600
To: "'rddp@ietf.org'" <rddp@ietf.org>, rddp-admin@ietf.org
MIME-Version: 1.0
Subject: RE: [rddp] Option or mandatory CRC in MPA
X-Mailer: Lotus Notes Release 5.0.11   July 24, 2002
From: John Hufferd <hufferd@us.ibm.com>
Message-ID: <OFAFF652D3.1550E844-ON88256D03.00039F6D-88256D03.00061FD7@us.ibm.com>
Date: Tue, 8 Apr 2003 18:04:17 -0700
X-MIMETrack: Serialize by Router on D03NM115/03/M/IBM(Release 6.0 [IBM]|December 16, 2002) at
 04/08/2003 19:03:22,
	Serialize complete at 04/08/2003 19:03:22
Content-Type: multipart/alternative; boundary="=_alternative 00061EFB88256D03_="
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

This is a multipart message in MIME format.
--=_alternative 00061EFB88256D03_=
Content-Type: text/plain; charset="us-ascii"

I find some of these arguments somewhat strange.  We have all entered into 
this effort to bring optimized processing with Remote Direct Data 
Placement, yet we continue to talk about, and give serious weight to, the 
argument concerning an all software implementation.  It seems that we 
might think about a complete software implementation, in passing, but this 
is NOT the goal of RDMA or this RDDP workgroup (in my opinion). Therefore, 
I do not think it is important to do things that will make the protocol 
more complicated for this small segment of the market, assuming that it 
exist at all other then for a testing platform. 

Most of the applications that will be using the RDMA protocols already 
have existing TCP/IP implementations, and they are not enhanced or 
accelerated by a software implementation of RDMA.  If systems only have a 
standard NIC, then the applications should continue to use TCP/IP. 

Expanding on this point further, the work being done in various places on 
Socket Direct Protocols, will permit the application to continue to use 
the Socket APIs, and the OS will convert that to RDMA where possible and 
reasonable.  I submit that will be when there is an RDMA chip set 
available.

There are other applications that I am aware of that are "adding"  the 
RDMA capability, not taking away the TCP/IP  implementation.  They check 
to see which one they can use and act according. 

So again, being overly concerned about a software implementation of RDMA 
is not something on which we should spend much time. 
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Home Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: hufferd@us.ibm.com




"Williams, Jim" <Jim.Williams@emulex.com>
Sent by: rddp-admin@ietf.org
04/08/2003 05:17 PM

 
        To:     "'rddp@ietf.org'" <rddp@ietf.org>
        cc: 
        Subject:        RE: [rddp] Option or mandatory CRC in MPA



The argument for making the CRC optional
in the RDDP over TCP mapping is the following.

  A.  A large percentage of usage will be on
      highly reliable private networks where
      the TCP checksum is more that sufficient
      the achieve the desired level or reliability.

  B.  Optimizing the performance of software 
      implementations is important, and computing
      an unneeded CRC is too high a price to pay
      for avoiding the need to negotiate an option.


The input from NFS is important because it takes
into account years of field experience.  If they
determine that for most all usage, even on private
networks, the TCP checksum is NOT adequate, then
I would regard that as a refutation of point A.
If they determine that for a large enough percentage
of implementations the TCP checksum IS adequate,
then I would regard that as support of point A.

Regarding point B, if one envisions that all 
implementations of interest are hardware customized
for RDDP, then the point does not carry much weight.
However, I would argue that this may be too narrow
a view for the IETF to take.
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



--=_alternative 00061EFB88256D03_=
Content-Type: text/html; charset="us-ascii"


<br><font size=2 face="sans-serif">I find some of these arguments somewhat strange. &nbsp;We have all entered into this effort to bring optimized processing with Remote Direct Data Placement, yet we continue to talk about, and give serious weight to, the argument concerning an all software implementation. &nbsp;It seems that we might think about a complete software implementation, in passing, but this is NOT the goal of RDMA or this RDDP workgroup (in my opinion). &nbsp;Therefore, I do not think it is important to do things that will make the protocol more complicated for this small segment of the market, assuming that it exist at all other then for a testing platform. &nbsp;</font>
<br>
<br><font size=2 face="sans-serif">Most of the applications that will be using the RDMA protocols already have existing TCP/IP implementations, and they are not enhanced or accelerated by a software implementation of RDMA. &nbsp;If systems only have a standard NIC, then the applications should continue to use TCP/IP. &nbsp;</font>
<br>
<br><font size=2 face="sans-serif">Expanding on this point further, the work being done in various places on Socket Direct Protocols, will permit the application to continue to use the Socket APIs, and the OS will convert that to RDMA where possible and reasonable. &nbsp;I submit that will be when there is an RDMA chip set available.</font>
<br>
<br><font size=2 face="sans-serif">There are other applications that I am aware of that are &quot;adding&quot; &nbsp;the RDMA capability, not taking away the TCP/IP &nbsp;implementation. &nbsp;They check to see which one they can use and act according. &nbsp;</font>
<br>
<br><font size=2 face="sans-serif">So again, being overly concerned about a software implementation of RDMA is not something on which we should spend much time. </font>
<br><font size=2 face="sans-serif">.<br>
.<br>
John L. Hufferd<br>
Senior Technical Staff Member (STSM)<br>
IBM/System Group, San Jose CA<br>
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688<br>
Home Office: (408) 997-6136, Cell: (408) 499-9702<br>
Internet Address: hufferd@us.ibm.com</font>
<br>
<br>
<br>
<table width=100%>
<tr valign=top>
<td>
<td><font size=1 face="sans-serif"><b>&quot;Williams, Jim&quot; &lt;Jim.Williams@emulex.com&gt;</b></font>
<br><font size=1 face="sans-serif">Sent by: rddp-admin@ietf.org</font>
<p><font size=1 face="sans-serif">04/08/2003 05:17 PM</font>
<br>
<td><font size=1 face="Arial">&nbsp; &nbsp; &nbsp; &nbsp; </font>
<br><font size=1 face="sans-serif">&nbsp; &nbsp; &nbsp; &nbsp; To: &nbsp; &nbsp; &nbsp; &nbsp;&quot;'rddp@ietf.org'&quot; &lt;rddp@ietf.org&gt;</font>
<br><font size=1 face="sans-serif">&nbsp; &nbsp; &nbsp; &nbsp; cc: &nbsp; &nbsp; &nbsp; &nbsp;</font>
<br><font size=1 face="sans-serif">&nbsp; &nbsp; &nbsp; &nbsp; Subject: &nbsp; &nbsp; &nbsp; &nbsp;RE: [rddp] Option or mandatory CRC in MPA</font>
<br></table>
<br>
<br>
<br><font size=2 face="Courier New">The argument for making the CRC optional<br>
in the RDDP over TCP mapping is the following.<br>
<br>
 &nbsp;A. &nbsp;A large percentage of usage will be on<br>
 &nbsp; &nbsp; &nbsp;highly reliable private networks where<br>
 &nbsp; &nbsp; &nbsp;the TCP checksum is more that sufficient<br>
 &nbsp; &nbsp; &nbsp;the achieve the desired level or reliability.<br>
<br>
 &nbsp;B. &nbsp;Optimizing the performance of software <br>
 &nbsp; &nbsp; &nbsp;implementations is important, and computing<br>
 &nbsp; &nbsp; &nbsp;an unneeded CRC is too high a price to pay<br>
 &nbsp; &nbsp; &nbsp;for avoiding the need to negotiate an option.<br>
<br>
<br>
The input from NFS is important because it takes<br>
into account years of field experience. &nbsp;If they<br>
determine that for most all usage, even on private<br>
networks, the TCP checksum is NOT adequate, then<br>
I would regard that as a refutation of point A.<br>
If they determine that for a large enough percentage<br>
of implementations the TCP checksum IS adequate,<br>
then I would regard that as support of point A.<br>
<br>
Regarding point B, if one envisions that all <br>
implementations of interest are hardware customized<br>
for RDDP, then the point does not carry much weight.<br>
However, I would argue that this may be too narrow<br>
a view for the IETF to take.<br>
_______________________________________________<br>
rddp mailing list<br>
rddp@ietf.org<br>
https://www1.ietf.org/mailman/listinfo/rddp<br>
</font>
<br>
<br>
--=_alternative 00061EFB88256D03_=--
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Wed Apr  9 09:57:41 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA28064
	for <rddp-archive@odin.ietf.org>; Wed, 9 Apr 2003 09:57:41 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h39E2sI15146
	for rddp-archive@odin.ietf.org; Wed, 9 Apr 2003 10:02:54 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h39E2r815143
	for <rddp-web-archive@optimus.ietf.org>; Wed, 9 Apr 2003 10:02:53 -0400
Received: from www1.ietf.org (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA28047
	for <rddp-web-archive@ietf.org>; Wed, 9 Apr 2003 09:57:10 -0400 (EDT)
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h39E2E815127;
	Wed, 9 Apr 2003 10:02:14 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h39E1r815073
	for <rddp@optimus.ietf.org>; Wed, 9 Apr 2003 10:01:53 -0400
Received: from palrel12.hp.com (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA28029
	for <rddp@ietf.org>; Wed, 9 Apr 2003 09:56:09 -0400 (EDT)
Received: from esmail.cup.hp.com (esmail.cup.hp.com [15.0.65.164])
	by palrel12.hp.com (Postfix) with ESMTP id C562C1C01F9E
	for <rddp@ietf.org>; Wed,  9 Apr 2003 06:58:42 -0700 (PDT)
Received: from mk731915.cup.hp.com ([15.113.161.2])
	by esmail.cup.hp.com (8.11.1/8.8.6) with ESMTP id h39DtEl10345
	for <rddp@ietf.org>; Wed, 9 Apr 2003 06:55:14 -0700 (PDT)
Message-Id: <5.1.0.14.2.20030409061929.02076f70@15.13.130.73>
X-Sender: krause@15.13.130.73
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Wed, 09 Apr 2003 06:28:40 -0700
To: rddp@ietf.org
From: Michael Krause <krause@cup.hp.com>
Subject: Re: [rddp] Option or mandatory CRC in MPA
In-Reply-To: <3E933CA7.1080509@eng.sun.com>
References: <20030407234145.DD7244EBF@sandmail.sandburst.com>
 <002801c2fe01$5fa7f3a0$096e030f@rose.hp.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

At 02:18 PM 4/8/2003 -0700, Brent Callaghan wrote:

>NFS can protect its data with an RPCSEC_GSS checksum, though
>it's somewhat expensive to do in the host stack, so it's not
>commonly used.
>
>Instead, most NFS mounts today rely on the TCP checksum and
>link CRC to protect data payload.
>
>I doubt that many in the NFS community are completely
>comfortable relying on the TCP checksum.  If RDDP provides
>a 32 bit CRC that's implemented in hardware, then I'm sure
>NFS would use it.
>
>But given that NFS is already widely used over regular TCP,
>I'm not sure there's enough support to make an RDDP CRC mandatory,
>and a performance hurdle for host stack RDDP implementations.

- The hardware cost is viewed as rather trivial - easier to always 
implement than to have optional.

- Data integrity is a major issue for customers these days.  Having the 
added CRC protection is critical as illustrated by various past discussions 
on the IPS reflector.

- If I recall, doesn't SCTP require a mandatory CRC?  If that is the case, 
wouldn't it be best to have a single "method" be used by all protocols to 
allow ULPs to migrate between the two with minimum functional deltas where 
possible thus mandatory CRC?

Mike


>         Brent
>
>
>Mallikarjun C. wrote:
>>I see some issues in doing the CRC checking as a negotiable feature at 
>>the MPA level.
>>Apart from the optionality aspect, I don't quite see how ULP-level
>>CRC negotiation (such as for iSCSI) can harmoniously work with
>>an MPA-level negotiation of the same.  There's no issue when it's
>>negotiated to "No".  But when it's negotiated to "Yes" at the iSCSI-level,
>>as an example, iSCSI wants to generate and check it - not delegate the 
>>checking/generation part to MPA even if MPA could optionally
>>do it.  I suspect this issue is not unique to one ULP - other comments,
>>particularly from NFS folks, would help here.
>>AFAICT, the only clean solution is to have the ULP always negotiate the 
>>usage to "No" (with the knowledge that the transport substrate is
>>doing it), and have the MPA layer always generate/check CRC.
>>--
>>Mallikarjun
>>Mallikarjun Chadalapaka
>>Networked Storage Architecture
>>Network Storage Solutions
>>Hewlett-Packard MS 5668 Roseville CA 95747
>>cbm@rose.hp.com
>>
>>----- Original Message ----- From: "Stephen Bailey" <steph@cs.uchicago.edu>
>>To: <rddp@ietf.org>
>>Sent: Monday, April 07, 2003 4:41 PM
>>Subject: [rddp] Option or mandatory CRC in MPA
>>
>>
>>>Hi,
>>>
>>>It seems to me that one significant items holding up RDDP and
>>>specifically MPA, is whether CRC should be optional or mandatory.
>>>
>>>At the last IETF David said he would be uncomfortable requiring CRC
>>>protection if a key current client protocol (NFS) is not going to
>>>require the protection when not running on RDDP.  The primary argument
>>>for this is to allow RDDP to cross the transition hurdle more easily
>>>by not punishing software clients.
>>>
>>>A majority of implementors want CRC, so clearly many implementations
>>>will support it. There's incentive to implement it, and the penalty in
>>>hardware implementations is negligable.
>>>
>>>Implementations that do CRC can generate always and check optionally,
>>>which is easy.
>>>
>>>Rather than waiting for NFSv4 to settle the argument, it seems we
>>>could push ahead with a reserved spot in the payload, and negotiable
>>>checking.
>>>
>>>Steph
>>>_______________________________________________
>>>rddp mailing list
>>>rddp@ietf.org
>>>https://www1.ietf.org/mailman/listinfo/rddp
>>>
>>
>>_______________________________________________
>>rddp mailing list
>>rddp@ietf.org
>>https://www1.ietf.org/mailman/listinfo/rddp
>
>
>_______________________________________________
>rddp mailing list
>rddp@ietf.org
>https://www1.ietf.org/mailman/listinfo/rddp

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Wed Apr  9 12:40:36 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA05495
	for <rddp-archive@odin.ietf.org>; Wed, 9 Apr 2003 12:40:36 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h39Fteo24951
	for rddp-archive@odin.ietf.org; Wed, 9 Apr 2003 11:55:40 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h39Fte824948
	for <rddp-web-archive@optimus.ietf.org>; Wed, 9 Apr 2003 11:55:40 -0400
Received: from www1.ietf.org (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA03586
	for <rddp-web-archive@ietf.org>; Wed, 9 Apr 2003 11:49:54 -0400 (EDT)
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h39Ft2824912;
	Wed, 9 Apr 2003 11:55:02 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h39FsE824868
	for <rddp@optimus.ietf.org>; Wed, 9 Apr 2003 11:54:14 -0400
Received: from thebe.your-site.com (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA03564
	for <rddp@ietf.org>; Wed, 9 Apr 2003 11:48:28 -0400 (EDT)
Received: from 192.168.0.2 (64-144-5-25.client.dsl.net [64.144.5.25])
	by thebe.your-site.com (Postfix) with ESMTP
	id 6E4A624571A; Wed,  9 Apr 2003 11:51:10 -0400 (EDT)
Date: Wed,  9 Apr 2003 10:53:19 -0500
From: Caitlin Bestler <cait@asomi.com>
Subject: RE: [rddp] Option or mandatory CRC in MPA
To: John Hufferd <hufferd@us.ibm.com>, rddp@ietf.org
X-Priority: 3
In-Reply-To: <OFAFF652D3.1550E844-ON88256D03.00039F6D-88256D03.00061FD7@us.ibm.com>
Message-ID: <r01050300-1024-6277A91E6AA311D78DE4003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

On 4/8/03, John Hufferd wrote:

>Most of the applications that will be using the RDMA
>protocols already have existing TCP/IP implementations,
>and they are not enhanced or accelerated by a software
>implementation of RDMA.  If systems only have a standard
>NIC, then the applications should continue to use TCP/IP. 

There are two sides to every client/server relationship.

A software implementation of RDDP on the client side allows
the server side to have fully optimized RDDP. Presumably
with hardware optimization, but even without hardware
optimization there are major advantages to storage servers
in the freedom to deliver material in any order convenient.

Review the draft applicability statement, many are covered
there. These are important. If RDDP was solely of use to
enable a hardware justification then it would not merit
IETF consideration. Motherboards should be changed before
the Internet. The more important point is that the protocol
enables more efficient collaboration, especially for storage
applications, no matter how either end is implemented.

That said, I don't think we need to make the protocol more
complex to save CPU cycles for software implementations.
Calculating CRCs is not difficult. The fact that it is more
easily done in hardware does not mean that it is hard to do
in software.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Wed Apr  9 12:41:36 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA05552
	for <rddp-archive@odin.ietf.org>; Wed, 9 Apr 2003 12:41:36 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h39Fmpn24603
	for rddp-archive@odin.ietf.org; Wed, 9 Apr 2003 11:48:51 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h39Fmp824600
	for <rddp-web-archive@optimus.ietf.org>; Wed, 9 Apr 2003 11:48:51 -0400
Received: from www1.ietf.org (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA03423
	for <rddp-web-archive@ietf.org>; Wed, 9 Apr 2003 11:43:04 -0400 (EDT)
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h39Fm1824536;
	Wed, 9 Apr 2003 11:48:01 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h39FhP824305
	for <rddp@optimus.ietf.org>; Wed, 9 Apr 2003 11:43:25 -0400
Received: from patan.sun.com (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA03321
	for <rddp@ietf.org>; Wed, 9 Apr 2003 11:37:38 -0400 (EDT)
Received: from centralmail1brm.Central.Sun.COM ([129.147.62.1])
	by patan.sun.com (8.9.3p2+Sun/8.9.3) with ESMTP id JAA17981
	for <rddp@ietf.org>; Wed, 9 Apr 2003 09:40:12 -0600 (MDT)
Received: from Sun.COM (jetsun.Central.Sun.COM [129.153.128.60])
	by centralmail1brm.Central.Sun.COM (8.12.8+Sun/8.12.8/ENSMAIL,v2.2) with ESMTP id h39FeB5i018517
	for <rddp@ietf.org>; Wed, 9 Apr 2003 09:40:11 -0600 (MDT)
Message-ID: <3E943EDA.8040609@Sun.COM>
Date: Wed, 09 Apr 2003 10:40:10 -0500
From: David Robinson <David.Robinson@Sun.COM>
Reply-To: David.Robinson@Sun.COM
Organization: Sun Microsystems, Inc.
User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.1) Gecko/20020827
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: "'rddp@ietf.org'" <rddp@ietf.org>
Subject: Re: [rddp] Option or mandatory CRC in MPA
References: <3356669BBE90C448AD4645C843E2BF289B904B@xbl.ma.emulex.com>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

Williams, Jim wrote:

> The input from NFS is important because it takes
> into account years of field experience.  If they
> determine that for most all usage, even on private
> networks, the TCP checksum is NOT adequate, then
> I would regard that as a refutation of point A.
> If they determine that for a large enough percentage
> of implementations the TCP checksum IS adequate,
> then I would regard that as support of point A.

The argument can be made both ways from an NFS perspective.
In the last almost 20 years, there have been very few
complaints that have been attributed to checksum failure.
By far the dominant cause of corruption is simply
buggy software, generally the areas of caching and locking.

This can then be argued that it is not a concern
for NFS, further bolstered by the fact that for the
first 5+ years most deployments were relying only
on the link level CRC. But to counter that, given the
numbers for the failure rates determined by the research
in the area of TCP checksums and the fact that for the
first 10+ years the link speed was only 10Mbps, and only
the last couple years solidly at 100Mbps, statistically the
frequency of errors that would have gotten past the
checksum would be extremely small. At a bit rate of 10G
it could be argued that what has not been a factor in
the past will now be something we have to deal with.

 From a philosophical perspective NFS (and RPC) has always
been for strict layering.  If the underlying transport's
checksum is inadequate, just use a different transport. There
has also been great strides to make the choice of transport
automatically negotiated and transparent to both the user
and administrator. Most customers didn't even notice that they
were quietly switched from UDP to TCP over the last 5 years.
This would argue from an NFS purest perspective those that
are concerned about integrity will force use of SCTP or
other transport that has a stronger checksum. From a vendor
perspective I would likely try to negotiate SCTP first.

The other mitigating factor for NFS is its layered security
architecture which can provide added integrity checking. As
others have said, using a security integrity mechanism to
cover up a transport deficency is a 24ct gold plated solution
to a brass problem. But it is available today.  There is also
an effort to cleanly accomidate a lower layer connection
based security (IPSEC) using the existing RPCSEC_GSS framework
(CCM). This would allow hardware accelerated integity checking
through the security layers.

Overall, in my personal opinion, I would not use NFSv4 as an
argument for making CRC's in RDDP mandatory. NFS today already
has mechanisms that can mitigate any underlying transport
checksum problems. But this is not to say that NFS would not
use such a facility if available, of all the options available
it would probably be the most efficient.

At the core of this entire argument is the simple question
of is RDDP a transport to the ULP providing defined semantics
or is it just layer on top of an existing transport exposing
its semantics? Begging the question, if you don't like TCP
checksums why not just use a "real" transport? (He ducks....) :-)

	-David

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Wed Apr  9 12:46:46 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA05757
	for <rddp-archive@odin.ietf.org>; Wed, 9 Apr 2003 12:46:46 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h39FKfh22426
	for rddp-archive@odin.ietf.org; Wed, 9 Apr 2003 11:20:41 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h39FKf822423
	for <rddp-web-archive@optimus.ietf.org>; Wed, 9 Apr 2003 11:20:41 -0400
Received: from www1.ietf.org (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA02303
	for <rddp-web-archive@ietf.org>; Wed, 9 Apr 2003 11:14:56 -0400 (EDT)
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h39FK2822391;
	Wed, 9 Apr 2003 11:20:02 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h391S7817959
	for <rddp@optimus.ietf.org>; Tue, 8 Apr 2003 21:28:07 -0400
Received: from patan.sun.com (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA11022
	for <rddp@ietf.org>; Tue, 8 Apr 2003 21:22:39 -0400 (EDT)
Received: from jurassic.eng.sun.com ([129.146.17.55])
	by patan.sun.com (8.9.3p2+Sun/8.9.3) with ESMTP id TAA16008
	for <rddp@ietf.org>; Tue, 8 Apr 2003 19:25:11 -0600 (MDT)
Received: from eng.sun.com (terra.Eng.Sun.COM [129.146.86.99])
	by jurassic.eng.sun.com (8.12.9+Sun/8.12.9) with ESMTP id h391PBmp296163
	for <rddp@ietf.org>; Tue, 8 Apr 2003 18:25:11 -0700 (PDT)
Message-ID: <3E93767F.2040404@eng.sun.com>
Date: Tue, 08 Apr 2003 18:25:19 -0700
From: Brent Callaghan <brent@eng.sun.com>
Reply-To: Brent.Callaghan@Sun.COM
Organization: Sun Microsystems
User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.0.1) Gecko/20020920 Netscape/7.0
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: "'rddp@ietf.org'" <rddp@ietf.org>
Subject: Re: [rddp] Option or mandatory CRC in MPA
References: <OFAFF652D3.1550E844-ON88256D03.00039F6D-88256D03.00061FD7@us.ibm.com>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit


John Hufferd wrote:
> :
> Most of the applications that will be using the RDMA protocols already 
> have existing TCP/IP implementations, and they are not enhanced or 
> accelerated by a software implementation of RDMA.  If systems only have 
> a standard NIC, then the applications should continue to use TCP/IP. 

Perhaps.  Though a credible scenario is a large number of
cheap and/or legacy clients doing RDMA to a large, heavily
loaded server (not uncommon).

In that environment, it might be attractive to have the clients
implement RDMA in software, and equip the server with the fancy
RDMA hardware to relieve its suffering.

	Brent


_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Wed Apr  9 12:49:17 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA05912
	for <rddp-archive@odin.ietf.org>; Wed, 9 Apr 2003 12:49:17 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h39FGGs22216
	for rddp-archive@odin.ietf.org; Wed, 9 Apr 2003 11:16:16 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h39FGG822213
	for <rddp-web-archive@optimus.ietf.org>; Wed, 9 Apr 2003 11:16:16 -0400
Received: from www1.ietf.org (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA02218
	for <rddp-web-archive@ietf.org>; Wed, 9 Apr 2003 11:10:29 -0400 (EDT)
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h39FFP822172;
	Wed, 9 Apr 2003 11:15:25 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h39FET822132
	for <rddp@optimus.ietf.org>; Wed, 9 Apr 2003 11:14:29 -0400
Received: from caduceus.fm.intel.com (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA02185
	for <rddp@ietf.org>; Wed, 9 Apr 2003 11:08:41 -0400 (EDT)
Received: from petasus.fm.intel.com (petasus.fm.intel.com [10.1.192.37])
	by caduceus.fm.intel.com (8.11.6p2/8.11.6/d: outer.mc,v 1.51 2002/09/23 20:43:23 dmccart Exp $) with ESMTP id h39F4Ho00164
	for <rddp@ietf.org>; Wed, 9 Apr 2003 15:04:18 GMT
Received: from fmsmsxv040-1.fm.intel.com (fmsmsxvs040.fm.intel.com [132.233.42.124])
	by petasus.fm.intel.com (8.11.6p2/8.11.6/d: inner.mc,v 1.28 2003/01/13 19:44:39 dmccart Exp $) with SMTP id h39F5FF28029
	for <rddp@ietf.org>; Wed, 9 Apr 2003 15:05:15 GMT
Received: from fmsmsx331-2.fm.intel.com ([132.233.42.135])
 by fmsmsxv040-1.fm.intel.com (NAVGW 2.5.2.11) with SMTP id M2003040908112406321
 for <rddp@ietf.org>; Wed, 09 Apr 2003 08:11:24 -0700
Received: from fmsmsx402.amr.corp.intel.com ([132.233.42.200]) by fmsmsx331-2.fm.intel.com with Microsoft SMTPSVC(5.0.2195.5329);
	 Wed, 9 Apr 2003 08:11:10 -0700
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----_=_NextPart_001_01C2FEAA.40E09C34"
X-MimeOLE: Produced By Microsoft Exchange V6.0.6375.0
Subject: RE: [rddp] Option or mandatory CRC in MPA
Date: Wed, 9 Apr 2003 08:11:10 -0700
Message-ID: <6315617889C99D4BA7C14687DEC8DB4E02F95033@fmsmsx402.fm.intel.com>
Thread-Topic: [rddp] Option or mandatory CRC in MPA
Thread-Index: AcL+NPi79KRBaL9FQq+j0E5phX0M7AAcLcug
From: "Shah, Hemal" <hemal.shah@intel.com>
To: <rddp@ietf.org>
X-OriginalArrivalTime: 09 Apr 2003 15:11:10.0883 (UTC) FILETIME=[41097730:01C2FEAA]
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

This is a multi-part message in MIME format.

------_=_NextPart_001_01C2FEAA.40E09C34
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

I also find myself in the same situation as John pointed out below. RDMA
over TCP/IP is optimized to be implemented on RDMA enabled NIC (RNIC). A
host SW implementation of RDMA over TCP/IP realizing real benefits of
RDMA is questionable. Thus, I also find it hard to justify some of the
arguments concerning a complete software implementation of RDMA over
TCP/IP. If some of the features that are optimized for HW can not be
implemented efficiently in SW, then we should not be changing features
that optimize SW implementation at the cost of added HW complexity.

=20

As others have pointed out, the cost of computing/verifying CRC in the
RNIC hardware is not significant compared to the total RNIC
functionality. By not making CRC verification optional simplifies the
RNIC implementation. So, I do not think it is justified to make CRC
verification optional to optimize SW implementation of RDMA over TCP/IP.

=20

Hemal

=20

-----Original Message-----
From: John Hufferd [mailto:hufferd@us.ibm.com]=20
Sent: Tuesday, April 08, 2003 8:04 PM
To: 'rddp@ietf.org'; rddp-admin@ietf.org
Subject: RE: [rddp] Option or mandatory CRC in MPA

=20


I find some of these arguments somewhat strange.  We have all entered
into this effort to bring optimized processing with Remote Direct Data
Placement, yet we continue to talk about, and give serious weight to,
the argument concerning an all software implementation.  It seems that
we might think about a complete software implementation, in passing, but
this is NOT the goal of RDMA or this RDDP workgroup (in my opinion).
Therefore, I do not think it is important to do things that will make
the protocol more complicated for this small segment of the market,
assuming that it exist at all other then for a testing platform.  =20

Most of the applications that will be using the RDMA protocols already
have existing TCP/IP implementations, and they are not enhanced or
accelerated by a software implementation of RDMA.  If systems only have
a standard NIC, then the applications should continue to use TCP/IP.  =20

Expanding on this point further, the work being done in various places
on Socket Direct Protocols, will permit the application to continue to
use the Socket APIs, and the OS will convert that to RDMA where possible
and reasonable.  I submit that will be when there is an RDMA chip set
available.=20

There are other applications that I am aware of that are "adding"  the
RDMA capability, not taking away the TCP/IP  implementation.  They check
to see which one they can use and act according.  =20

So again, being overly concerned about a software implementation of RDMA
is not something on which we should spend much time.=20
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Home Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: hufferd@us.ibm.com=20




=20

"Williams, Jim" <Jim.Williams@emulex.com>=20
Sent by: rddp-admin@ietf.org=20

04/08/2003 05:17 PM=20

       =20
        To:        "'rddp@ietf.org'" <rddp@ietf.org>=20
        cc:        =20
        Subject:        RE: [rddp] Option or mandatory CRC in MPA=20




The argument for making the CRC optional
in the RDDP over TCP mapping is the following.

 A.  A large percentage of usage will be on
     highly reliable private networks where
     the TCP checksum is more that sufficient
     the achieve the desired level or reliability.

 B.  Optimizing the performance of software=20
     implementations is important, and computing
     an unneeded CRC is too high a price to pay
     for avoiding the need to negotiate an option.


The input from NFS is important because it takes
into account years of field experience.  If they
determine that for most all usage, even on private
networks, the TCP checksum is NOT adequate, then
I would regard that as a refutation of point A.
If they determine that for a large enough percentage
of implementations the TCP checksum IS adequate,
then I would regard that as support of point A.

Regarding point B, if one envisions that all=20
implementations of interest are hardware customized
for RDDP, then the point does not carry much weight.
However, I would argue that this may be too narrow
a view for the IETF to take.
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp




------_=_NextPart_001_01C2FEAA.40E09C34
Content-Type: text/html;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html>

<head>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Dus-ascii">


<meta name=3DGenerator content=3D"Microsoft Word 10 (filtered)">

<style>
<!--
 /* Font Definitions */
 @font-face
	{font-family:Tahoma;
	panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
	{font-family:sans-serif;
	panose-1:0 0 0 0 0 0 0 0 0 0;}
 /* Style Definitions */
 p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	margin-bottom:.0001pt;
	font-size:12.0pt;
	font-family:"Times New Roman";}
a:link, span.MsoHyperlink
	{color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{color:purple;
	text-decoration:underline;}
p
	{margin-right:0in;
	margin-left:0in;
	font-size:12.0pt;
	font-family:"Times New Roman";}
span.EmailStyle18
	{font-family:Arial;
	color:navy;}
@page Section1
	{size:8.5in 11.0in;
	margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
	{page:Section1;}
-->
</style>

</head>

<body lang=3DEN-US link=3Dblue vlink=3Dpurple>

<div class=3DSection1>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>I also find myself in the same =
situation
as John pointed out below. RDMA over TCP/IP is optimized to be =
implemented on RDMA
enabled NIC (RNIC). A host SW implementation of RDMA over TCP/IP =
realizing real
benefits of RDMA is questionable. Thus, I also find it hard to justify =
some of
the arguments concerning a complete software implementation of RDMA over
TCP/IP. If some of the features that are optimized for HW can not be
implemented efficiently in SW, then we should not be changing features =
that
optimize SW implementation at the cost of added HW =
complexity.</span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>&nbsp;</span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>As others have pointed out, the =
cost of
computing/verifying CRC in the RNIC hardware is not significant compared =
to the
total RNIC functionality. By not making CRC verification optional =
simplifies
the RNIC implementation. So, I do not think it is justified to make CRC
verification optional to optimize SW implementation of RDMA over =
TCP/IP.</span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>&nbsp;</span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>Hemal</span></font></p>

<p class=3DMsoNormal><font size=3D2 color=3Dnavy face=3DArial><span =
style=3D'font-size:
10.0pt;font-family:Arial;color:navy'>&nbsp;</span></font></p>

<p class=3DMsoNormal style=3D'margin-left:.5in'><font size=3D2 =
face=3DTahoma><span
style=3D'font-size:10.0pt;font-family:Tahoma'>-----Original =
Message-----<br>
<b><span style=3D'font-weight:bold'>From:</span></b> </span></font><font =
size=3D2
 face=3DTahoma><span style=3D'font-size:10.0pt;font-family:Tahoma'>John =
Hufferd</span></font><font
size=3D2 face=3DTahoma><span =
style=3D'font-size:10.0pt;font-family:Tahoma'>
[mailto:hufferd@us.ibm.com] <br>
<b><span style=3D'font-weight:bold'>Sent:</span></b> </span></font><font =
size=3D2 face=3DTahoma><span =
style=3D'font-size:10.0pt;font-family:Tahoma'>Tuesday, April
 08, 2003</span></font><font size=3D2 face=3DTahoma><span =
style=3D'font-size:10.0pt;
font-family:Tahoma'> </span></font><font size=3D2 face=3DTahoma><span
 style=3D'font-size:10.0pt;font-family:Tahoma'>8:04 =
PM</span></font><font size=3D2
face=3DTahoma><span style=3D'font-size:10.0pt;font-family:Tahoma'><br>
<b><span style=3D'font-weight:bold'>To:</span></b> '</span></font><font =
size=3D2
 face=3DTahoma><span =
style=3D'font-size:10.0pt;font-family:Tahoma'>rddp@ietf.org</span></font>=
<font
size=3D2 face=3DTahoma><span =
style=3D'font-size:10.0pt;font-family:Tahoma'>';
rddp-admin@ietf.org<br>
<b><span style=3D'font-weight:bold'>Subject:</span></b> RE: [rddp] =
Option or
mandatory CRC in MPA</span></font></p>

<p class=3DMsoNormal style=3D'margin-left:.5in'><font size=3D3 =
face=3D"Times New Roman"><span
style=3D'font-size:12.0pt'>&nbsp;</span></font></p>

<p class=3DMsoNormal =
style=3D'margin-right:0in;margin-bottom:12.0pt;margin-left:
.5in'><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:12.0pt'><br>
</span></font><font size=3D2 face=3Dsans-serif><span =
style=3D'font-size:10.0pt;
font-family:sans-serif'>I find some of these arguments somewhat strange. =
&nbsp;We
have all entered into this effort to bring optimized processing with =
Remote
Direct Data Placement, yet we continue to talk about, and give serious =
weight
to, the argument concerning an all software implementation. &nbsp;It =
seems that
we might think about a complete software implementation, in passing, but =
this
is NOT the goal of RDMA or this RDDP workgroup (in my opinion). =
&nbsp;Therefore,
I do not think it is important to do things that will make the protocol =
more
complicated for this small segment of the market, assuming that it exist =
at all
other then for a testing platform. &nbsp;</span></font> <br>
<br>
<font size=3D2 face=3Dsans-serif><span =
style=3D'font-size:10.0pt;font-family:sans-serif'>Most
of the applications that will be using the RDMA protocols already have =
existing
TCP/IP implementations, and they are not enhanced or accelerated by a =
software
implementation of RDMA. &nbsp;If systems only have a standard NIC, then =
the
applications should continue to use TCP/IP. &nbsp;</span></font> <br>
<br>
<font size=3D2 face=3Dsans-serif><span =
style=3D'font-size:10.0pt;font-family:sans-serif'>Expanding
on this point further, the work being done in various places on Socket =
Direct
Protocols, will permit the application to continue to use the Socket =
APIs, and
the OS will convert that to RDMA where possible and reasonable. &nbsp;I =
submit
that will be when there is an RDMA chip set available.</span></font> =
<br>
<br>
<font size=3D2 face=3Dsans-serif><span =
style=3D'font-size:10.0pt;font-family:sans-serif'>There
are other applications that I am aware of that are &quot;adding&quot; =
&nbsp;the
RDMA capability, not taking away the TCP/IP &nbsp;implementation. =
&nbsp;They
check to see which one they can use and act according. =
&nbsp;</span></font> <br>
<br>
<font size=3D2 face=3Dsans-serif><span =
style=3D'font-size:10.0pt;font-family:sans-serif'>So
again, being overly concerned about a software implementation of RDMA is =
not
something on which we should spend much time. </span></font><br>
<font size=3D2 face=3Dsans-serif><span =
style=3D'font-size:10.0pt;font-family:sans-serif'>.<br>
.<br>
John L. Hufferd<br>
Senior Technical Staff Member (STSM)<br>
IBM/System Group, </span></font><font size=3D2 face=3Dsans-serif><span
  style=3D'font-size:10.0pt;font-family:sans-serif'>San =
Jose</span></font><font
 size=3D2 face=3Dsans-serif><span =
style=3D'font-size:10.0pt;font-family:sans-serif'> </span></font><font
  size=3D2 face=3Dsans-serif><span =
style=3D'font-size:10.0pt;font-family:sans-serif'>CA</span></font><font
size=3D2 face=3Dsans-serif><span =
style=3D'font-size:10.0pt;font-family:sans-serif'><br>
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688<br>
Home Office: (408) 997-6136, Cell: (408) 499-9702<br>
Internet Address: hufferd@us.ibm.com</span></font> <br>
<br>
</p>

<table class=3DMsoNormalTable border=3D0 cellpadding=3D0 width=3D"100%"
 style=3D'width:100.0%;margin-left:.5in'>
 <tr>
  <td valign=3Dtop style=3D'padding:.75pt .75pt .75pt .75pt'>
  <p class=3DMsoNormal><font size=3D3 face=3D"Times New Roman"><span
  style=3D'font-size:12.0pt'>&nbsp;</span></font></p>
  </td>
  <td valign=3Dtop style=3D'padding:.75pt .75pt .75pt .75pt'>
  <p class=3DMsoNormal><b><font size=3D1 face=3Dsans-serif><span =
style=3D'font-size:
  7.5pt;font-family:sans-serif;font-weight:bold'>&quot;Williams, =
Jim&quot;
  &lt;Jim.Williams@emulex.com&gt;</span></font></b> <br>
  <font size=3D1 face=3Dsans-serif><span =
style=3D'font-size:7.5pt;font-family:sans-serif'>Sent
  by: rddp-admin@ietf.org</span></font> </p>
  <p><font size=3D1 face=3Dsans-serif><span =
style=3D'font-size:7.5pt;font-family:
   sans-serif'>04/08/2003</span></font><font size=3D1 =
face=3Dsans-serif><span
  style=3D'font-size:7.5pt;font-family:sans-serif'> </span></font><font =
size=3D1 face=3Dsans-serif><span =
style=3D'font-size:7.5pt;font-family:sans-serif'>05:17 PM</span></font>
  </p>
  </td>
  <td valign=3Dtop style=3D'padding:.75pt .75pt .75pt .75pt'>
  <p class=3DMsoNormal><font size=3D1 face=3DArial><span =
style=3D'font-size:7.5pt;
  font-family:Arial'>&nbsp; &nbsp; &nbsp; &nbsp; </span></font><br>
  <font size=3D1 face=3Dsans-serif><span =
style=3D'font-size:7.5pt;font-family:sans-serif'>&nbsp;
  &nbsp; &nbsp; &nbsp; To: &nbsp; &nbsp; &nbsp; =
&nbsp;&quot;'rddp@ietf.org'&quot;
  &lt;rddp@ietf.org&gt;</span></font> <br>
  <font size=3D1 face=3Dsans-serif><span =
style=3D'font-size:7.5pt;font-family:sans-serif'>&nbsp;
  &nbsp; &nbsp; &nbsp; cc: &nbsp; &nbsp; &nbsp; &nbsp;</span></font> =
<br>
  <font size=3D1 face=3Dsans-serif><span =
style=3D'font-size:7.5pt;font-family:sans-serif'>&nbsp;
  &nbsp; &nbsp; &nbsp; Subject: &nbsp; &nbsp; &nbsp; &nbsp;RE: [rddp] =
Option or
  mandatory CRC in MPA</span></font> </p>
  </td>
 </tr>
</table>

<p class=3DMsoNormal =
style=3D'margin-right:0in;margin-bottom:12.0pt;margin-left:
.5in'><font size=3D3 face=3D"Times New Roman"><span =
style=3D'font-size:12.0pt'><br>
<br>
<br>
</span></font><font size=3D2 face=3D"Courier New"><span =
style=3D'font-size:10.0pt;
font-family:"Courier New"'>The argument for making the CRC optional<br>
in the RDDP over TCP mapping is the following.<br>
<br>
&nbsp;A. &nbsp;A large percentage of usage will be on<br>
&nbsp; &nbsp; &nbsp;highly reliable private networks where<br>
&nbsp; &nbsp; &nbsp;the TCP checksum is more that sufficient<br>
&nbsp; &nbsp; &nbsp;the achieve the desired level or reliability.<br>
<br>
&nbsp;B. &nbsp;Optimizing the performance of software <br>
&nbsp; &nbsp; &nbsp;implementations is important, and computing<br>
&nbsp; &nbsp; &nbsp;an unneeded CRC is too high a price to pay<br>
&nbsp; &nbsp; &nbsp;for avoiding the need to negotiate an option.<br>
<br>
<br>
The input from NFS is important because it takes<br>
into account years of field experience. &nbsp;If they<br>
determine that for most all usage, even on private<br>
networks, the TCP checksum is NOT adequate, then<br>
I would regard that as a refutation of point A.<br>
If they determine that for a large enough percentage<br>
of implementations the TCP checksum IS adequate,<br>
then I would regard that as support of point A.<br>
<br>
Regarding point B, if one envisions that all <br>
implementations of interest are hardware customized<br>
for RDDP, then the point does not carry much weight.<br>
However, I would argue that this may be too narrow<br>
a view for the IETF to take.<br>
_______________________________________________<br>
rddp mailing list<br>
rddp@ietf.org<br>
https://www1.ietf.org/mailman/listinfo/rddp<br>
<br>
</span></font></p>

</div>

</body>

</html>

------_=_NextPart_001_01C2FEAA.40E09C34--
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Wed Apr  9 13:09:08 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA06581
	for <rddp-archive@odin.ietf.org>; Wed, 9 Apr 2003 13:09:08 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h39HEQn32329
	for rddp-archive@odin.ietf.org; Wed, 9 Apr 2003 13:14:26 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h39HEQ832326
	for <rddp-web-archive@optimus.ietf.org>; Wed, 9 Apr 2003 13:14:26 -0400
Received: from www1.ietf.org (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA06571
	for <rddp-web-archive@ietf.org>; Wed, 9 Apr 2003 13:08:37 -0400 (EDT)
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h39HDW832282;
	Wed, 9 Apr 2003 13:13:33 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h39HCt832248
	for <rddp@optimus.ietf.org>; Wed, 9 Apr 2003 13:12:55 -0400
Received: from ztxmail03.ztx.compaq.com (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA06521
	for <rddp@ietf.org>; Wed, 9 Apr 2003 13:07:06 -0400 (EDT)
Received: from cceexg12.americas.cpqcorp.net (cceexg12.americas.cpqcorp.net [16.110.250.124])
	by ztxmail03.ztx.compaq.com (Postfix) with ESMTP
	id 3F8C88B26; Wed,  9 Apr 2003 12:09:41 -0500 (CDT)
Received: from cceexc16.americas.cpqcorp.net ([16.110.250.58]) by cceexg12.americas.cpqcorp.net with Microsoft SMTPSVC(5.0.2195.2966);
	 Wed, 9 Apr 2003 12:09:41 -0500
X-MimeOLE: Produced By Microsoft Exchange V6.0.6375.0
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: RE: [rddp] Option or mandatory CRC in MPA
Date: Wed, 9 Apr 2003 12:09:40 -0500
Message-ID: <01CA1B1F74A4EC40A9C887F7B58F872403143129@cceexc16.americas.cpqcorp.net>
Thread-Topic: [rddp] Option or mandatory CRC in MPA
Thread-Index: AcL+Mk1SzK86ye5xSe+QoWUc/QxpPgAe+JdQ
From: "Barron, Dwight" <Dwight.Barron@hp.com>
To: "Williams, Jim" <Jim.Williams@emulex.com>, <rddp@ietf.org>
X-OriginalArrivalTime: 09 Apr 2003 17:09:41.0093 (UTC) FILETIME=[CF0D6950:01C2FEBA]
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by www1.ietf.org id h39HCt832249
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 8bit
Content-Transfer-Encoding: 8bit

I think Jim's argument boils down to:

- There are scenarios where a software CRC may be deemed unnecessary overhead and should be optionally eliminated at both the RDDP and ULP layers.

To achieve this efficiency one must accept the following in their implementation:

- Discrimination of the appropriate scenario, eg. identify highly reliable private networks and then determine which traffic solely traverses those highly reliable private networks vs. that which doesn't
- Design the ULP error handling scenarios to deal with the subtle differences in TCP checksum vs. CRC-32

I contrast this with:

- The opportunity to provide all ULPs designed for RDDP the same data integrity for RDDP/TCP and RDDP/SCTP, with roughly equal performance impact for both implementations
- The architectural encouragement to move CRC-32 into hardware at one layer in the protocol stack and be done with it
- Many engineers are uncomfortable operating an RDDP/RDMA protocol without the strength of CRC-32
- The fact that when existing ULPs are recast for RDDP, they can define a negotiation to stay in their current sockets mode and operate without CRC overhead (per John Hufferd's comment)
- The fact that software CRC, while not trivially cheap, is not the performance killer it is perceived to be. Some internal tests indicate that systems of a vintage that are, ahem, challenged to keep up with wire speed on GbE can perform CRC at rates that would support wire speed. Modern systems can perform CRC at rates that would support multiple GbE interfaces

I believe the RDDP group highly values the input of the NFS team's requirements and look forward to reviewing them. However, I think the field experience will have to be tempered with the following forward-looking fudge factors; increase in frequency of errors due to bandwidth increases and quantification of the number of NFS errors that routinely escape detection in the field. Otherwise, how do we know they are not hovering just below some acceptable threshold today (similar to the TCP error escapes), only waiting to surface once better detection is available and/or higher bandwidths push the error rate.

Regards,
Dwight

> -----Original Message-----
> From: Williams, Jim [mailto:Jim.Williams@emulex.com]
> Sent: Tuesday, April 08, 2003 7:17 PM
> To: 'rddp@ietf.org'
> Subject: RE: [rddp] Option or mandatory CRC in MPA
> 
> 
> The argument for making the CRC optional
> in the RDDP over TCP mapping is the following.
> 
>   A.  A large percentage of usage will be on
>       highly reliable private networks where
>       the TCP checksum is more that sufficient
>       the achieve the desired level or reliability.
> 
>   B.  Optimizing the performance of software 
>       implementations is important, and computing
>       an unneeded CRC is too high a price to pay
>       for avoiding the need to negotiate an option.
> 
> 
> The input from NFS is important because it takes
> into account years of field experience.  If they
> determine that for most all usage, even on private
> networks, the TCP checksum is NOT adequate, then
> I would regard that as a refutation of point A.
> If they determine that for a large enough percentage
> of implementations the TCP checksum IS adequate,
> then I would regard that as support of point A.
> 
> Regarding point B, if one envisions that all 
> implementations of interest are hardware customized
> for RDDP, then the point does not carry much weight.
> However, I would argue that this may be too narrow
> a view for the IETF to take.
> _______________________________________________
> rddp mailing list
> rddp@ietf.org
> https://www1.ietf.org/mailman/listinfo/rddp
> 
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Wed Apr  9 16:33:38 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA13495
	for <rddp-archive@odin.ietf.org>; Wed, 9 Apr 2003 16:33:38 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h39Kcxa26602
	for rddp-archive@odin.ietf.org; Wed, 9 Apr 2003 16:38:59 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h39Kcx826599
	for <rddp-web-archive@optimus.ietf.org>; Wed, 9 Apr 2003 16:38:59 -0400
Received: from www1.ietf.org (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA13478
	for <rddp-web-archive@ietf.org>; Wed, 9 Apr 2003 16:33:07 -0400 (EDT)
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h39Kc8826575;
	Wed, 9 Apr 2003 16:38:08 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h39KbW826422
	for <rddp@optimus.ietf.org>; Wed, 9 Apr 2003 16:37:32 -0400
Received: from sandmail.sandburst.com (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA13412
	for <rddp@ietf.org>; Wed, 9 Apr 2003 16:31:41 -0400 (EDT)
To: rddp@ietf.org
Subject: Re: [rddp] Option or mandatory CRC in MPA 
In-Reply-To: Message from "Shah, Hemal" <hemal.shah@intel.com> 
   of "Wed, 09 Apr 2003 08:11:10 PDT." <6315617889C99D4BA7C14687DEC8DB4E02F95033@fmsmsx402.fm.intel.com> 
References: <6315617889C99D4BA7C14687DEC8DB4E02F95033@fmsmsx402.fm.intel.com> 
Date: Wed, 09 Apr 2003 16:34:11 -0400
From: Stephen Bailey <steph@cs.uchicago.edu>
Message-Id: <20030409203413.D8A094EBC@sandmail.sandburst.com>
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

> So, I do not think it is justified to make CRC verification optional
> to optimize SW implementation of RDMA over TCP/IP.

The reason to provide CRC is widespread demand.

The reason to make it optional is that we can not conclusively
evaluate the impact/return tradeoff with data we have now.

The traditional reason for mandating a feature like data integrity is
not demand, but rather because it is broadly unsafe not to implement
it.  Absent experience with implementations and applications, all we
have is `hunch' (albeit strong and widely held), which probably isn't
enough.

The answer will be clear when MPA goes from Proposed to Draft.  I
don't see anything short of a miracle (or a failure by the working
group to advance the proposal at all) making it clear going from ID to
Proposed.

Steph
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Wed Apr  9 20:04:25 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA20812
	for <rddp-archive@odin.ietf.org>; Wed, 9 Apr 2003 20:04:25 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3A09pb08977
	for rddp-archive@odin.ietf.org; Wed, 9 Apr 2003 20:09:51 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3A09p808974
	for <rddp-web-archive@optimus.ietf.org>; Wed, 9 Apr 2003 20:09:51 -0400
Received: from www1.ietf.org (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA20801
	for <rddp-web-archive@ietf.org>; Wed, 9 Apr 2003 20:03:53 -0400 (EDT)
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3A09A808952;
	Wed, 9 Apr 2003 20:09:10 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3A08L808926
	for <rddp@optimus.ietf.org>; Wed, 9 Apr 2003 20:08:21 -0400
Received: from msgbas1x.cos.agilent.com (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA20774
	for <rddp@ietf.org>; Wed, 9 Apr 2003 20:02:24 -0400 (EDT)
From: pat_thaler@agilent.com
Received: from relcos1.cos.agilent.com (relcos1.cos.agilent.com [130.29.152.239])
	by msgbas1x.cos.agilent.com (Postfix) with ESMTP
	id D3FF417F99; Wed,  9 Apr 2003 18:04:58 -0600 (MDT)
Received: from axcsbh1.cos.agilent.com (axcsbh1.cos.agilent.com [130.29.152.143])
	by relcos1.cos.agilent.com (Postfix) with SMTP
	id B52ACD08; Wed,  9 Apr 2003 18:04:58 -0600 (MDT)
Received: from 130.29.152.143 by axcsbh1.cos.agilent.com (InterScan E-Mail VirusWall NT); Wed, 09 Apr 2003 18:04:57 -0600
Received: by axcsbh1.cos.agilent.com with Internet Mail Service (5.5.2653.19)
	id <GRMZLHHL>; Wed, 9 Apr 2003 18:04:57 -0600
Message-ID: <1BEBA5E8600DD4119A50009027AF54A00131C971@axcs04.cos.agilent.com>
To: Jim.Williams@emulex.com, rddp@ietf.org
Subject: RE: [rddp] Option or mandatory CRC in MPA
Date: Wed, 9 Apr 2003 18:04:57 -0600 
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

Jim,

One of the reasons for having end-to-end error detection is to protect against errors that occur in intermediate devices (middle boxes and switches) on the network where the data is not protected by the layer 2 CRC. Such errors can occur on private networks as well as on public networks. Unfortunately, we've seen intermediate devices that introduce errors into the data stream.

Pat

-----Original Message-----
From: Williams, Jim [mailto:Jim.Williams@emulex.com]
Sent: Tuesday, April 08, 2003 5:17 PM
To: 'rddp@ietf.org'
Subject: RE: [rddp] Option or mandatory CRC in MPA


The argument for making the CRC optional
in the RDDP over TCP mapping is the following.

  A.  A large percentage of usage will be on
      highly reliable private networks where
      the TCP checksum is more that sufficient
      the achieve the desired level or reliability.

  B.  Optimizing the performance of software 
      implementations is important, and computing
      an unneeded CRC is too high a price to pay
      for avoiding the need to negotiate an option.


The input from NFS is important because it takes
into account years of field experience.  If they
determine that for most all usage, even on private
networks, the TCP checksum is NOT adequate, then
I would regard that as a refutation of point A.
If they determine that for a large enough percentage
of implementations the TCP checksum IS adequate,
then I would regard that as support of point A.

Regarding point B, if one envisions that all 
implementations of interest are hardware customized
for RDDP, then the point does not carry much weight.
However, I would argue that this may be too narrow
a view for the IETF to take.
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Wed Apr  9 20:26:12 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA21229
	for <rddp-archive@odin.ietf.org>; Wed, 9 Apr 2003 20:26:12 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3A0VdG09887
	for rddp-archive@odin.ietf.org; Wed, 9 Apr 2003 20:31:39 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3A0Vd809884
	for <rddp-web-archive@optimus.ietf.org>; Wed, 9 Apr 2003 20:31:39 -0400
Received: from www1.ietf.org (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA21218
	for <rddp-web-archive@ietf.org>; Wed, 9 Apr 2003 20:25:41 -0400 (EDT)
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3A0V1809865;
	Wed, 9 Apr 2003 20:31:01 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3A0UW809833
	for <rddp@optimus.ietf.org>; Wed, 9 Apr 2003 20:30:32 -0400
Received: from mx01.netapp.com (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA21171
	for <rddp@ietf.org>; Wed, 9 Apr 2003 20:24:34 -0400 (EDT)
Received: from frejya.corp.netapp.com (frejya [10.10.20.91])
	by mx01.netapp.com (8.12.9/8.12.9/NTAP-1.4) with ESMTP id h3A0R8FB010710;
	Wed, 9 Apr 2003 17:27:08 -0700 (PDT)
Received: from tahoe.corp.netapp.com (tahoe.corp.netapp.com [10.10.22.112])
	by frejya.corp.netapp.com (8.12.9/8.12.9/NTAP-1.4) with ESMTP id h3A0R8Ab024685;
	Wed, 9 Apr 2003 17:27:08 -0700 (PDT)
Received: from eisler.com (10.58.48.133 [10.58.48.133]) by tahoe.corp.netapp.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21)
	id 2C1TWYVA; Wed, 9 Apr 2003 17:26:12 -0700
Message-ID: <3E94BA4F.2090300@eisler.com>
Date: Wed, 09 Apr 2003 17:26:55 -0700
From: Mike Eisler <mike@eisler.com>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3
X-Accept-Language: en-us
MIME-Version: 1.0
To: Brent.Callaghan@sun.com
CC: rddp@ietf.org
Subject: Re: [rddp] Option or mandatory CRC in MPA
References: <20030407234145.DD7244EBF@sandmail.sandburst.com> <002801c2fe01$5fa7f3a0$096e030f@rose.hp.com> <3E933CA7.1080509@eng.sun.com>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit



Brent Callaghan wrote:

>
> NFS can protect its data with an RPCSEC_GSS checksum, though
> it's somewhat expensive to do in the host stack, so it's not
> commonly used. 


The RPCSEC_GSS checksum is for protection from malicious
tampering by attackers. Whereas a CRC is more protection from
non-malicious tampering.  They are solving two different problems.

>
>
> Instead, most NFS mounts today rely on the TCP checksum and
> link CRC to protect data payload.
>
> I doubt that many in the NFS community are completely
> comfortable relying on the TCP checksum.  If RDDP provides
> a 32 bit CRC that's implemented in hardware, then I'm sure
> NFS would use it. 


Agreed. Also the analysis done by the iSCSI group is pretty convincing that
the 16 bit TCP checksum is an accident waiting to happen.
It's amazing that IETF has not added a better CRC to TCP.

>
>
> But given that NFS is already widely used over regular TCP, 


Sure but widely used over slower media than what RDDP is
trying to enable. Increase the speed, and the odds of corruption
increase.


_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Wed Apr  9 20:43:15 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA21666
	for <rddp-archive@odin.ietf.org>; Wed, 9 Apr 2003 20:43:15 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3A0mev11396
	for rddp-archive@odin.ietf.org; Wed, 9 Apr 2003 20:48:40 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3A0me811393
	for <rddp-web-archive@optimus.ietf.org>; Wed, 9 Apr 2003 20:48:40 -0400
Received: from www1.ietf.org (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA21642
	for <rddp-web-archive@ietf.org>; Wed, 9 Apr 2003 20:42:44 -0400 (EDT)
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3A0m2811350;
	Wed, 9 Apr 2003 20:48:02 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3A0le811333
	for <rddp@optimus.ietf.org>; Wed, 9 Apr 2003 20:47:40 -0400
Received: from palrel11.hp.com (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA21617
	for <rddp@ietf.org>; Wed, 9 Apr 2003 20:41:39 -0400 (EDT)
Received: from esmail.cup.hp.com (esmail.cup.hp.com [15.0.65.164])
	by palrel11.hp.com (Postfix) with ESMTP id E4C8B1C0174F
	for <rddp@ietf.org>; Wed,  9 Apr 2003 17:44:11 -0700 (PDT)
Received: from mk731915.cup.hp.com ([15.113.161.2])
	by esmail.cup.hp.com (8.11.1/8.8.6) with ESMTP id h3A0ehl22733
	for <rddp@ietf.org>; Wed, 9 Apr 2003 17:40:43 -0700 (PDT)
Message-Id: <5.1.0.14.2.20030409173334.02085df0@15.13.130.73>
X-Sender: krause@15.13.130.73
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Wed, 09 Apr 2003 17:44:09 -0700
To: rddp@ietf.org
From: Michael Krause <krause@cup.hp.com>
Subject: RE: [rddp] Option or mandatory CRC in MPA
In-Reply-To: <1BEBA5E8600DD4119A50009027AF54A00131C971@axcs04.cos.agilen
 t.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

At 06:04 PM 4/9/2003 -0600, pat_thaler@agilent.com wrote:
>Jim,
>
>One of the reasons for having end-to-end error detection is to protect 
>against errors that occur in intermediate devices (middle boxes and 
>switches) on the network where the data is not protected by the layer 2 
>CRC. Such errors can occur on private networks as well as on public 
>networks. Unfortunately, we've seen intermediate devices that introduce 
>errors into the data stream.

Another data point to consider is local I/O technology which is a much 
smaller fabric diameter than a data center and yet still has strong data 
integrity requirements:

- PCI-X 2.0 requires ECC protection

- PCI Express provides an end-to-end CRC which is optional - primary focus 
for optional was graphics which generally do not care about data integrity; 
for servers, we it is basically a MUST to remain consistent with PCI-X 2.0 
ECC protection coverage.

There was analysis done during the creation of these technologies focused 
on preventing intermediate device failures / corruption being undetected 
(as Pat points out) and we felt this was a worthwhile thing to include even 
though the fabric diameter is very small (to us, it isn't a function of 
fabric diameter but one of process technology getting so small that atomic 
hits are more of a problem than people realize).  In any case, the whole 
point was to plug as many potential areas of problems as possible without 
creating undue performance / cost burdens and these were the paths we chose 
and there is strong industry support about having this capability present.

Just something to consider as to what is being provided in new local I/O 
technology with the objective of preventing problems before they occur in 
customer solutions.  Having a mandatory ECRC via MPA or SCTP seems like the 
right thing to do and the software burden to support isn't that high to 
make this impractical - there are plenty of ways to implement this and 
deliver good performance independent of the type of endnode being used to 
generate the RDMA message.

Mike


>Pat
>
>-----Original Message-----
>From: Williams, Jim [mailto:Jim.Williams@emulex.com]
>Sent: Tuesday, April 08, 2003 5:17 PM
>To: 'rddp@ietf.org'
>Subject: RE: [rddp] Option or mandatory CRC in MPA
>
>
>The argument for making the CRC optional
>in the RDDP over TCP mapping is the following.
>
>   A.  A large percentage of usage will be on
>       highly reliable private networks where
>       the TCP checksum is more that sufficient
>       the achieve the desired level or reliability.
>
>   B.  Optimizing the performance of software
>       implementations is important, and computing
>       an unneeded CRC is too high a price to pay
>       for avoiding the need to negotiate an option.
>
>
>The input from NFS is important because it takes
>into account years of field experience.  If they
>determine that for most all usage, even on private
>networks, the TCP checksum is NOT adequate, then
>I would regard that as a refutation of point A.
>If they determine that for a large enough percentage
>of implementations the TCP checksum IS adequate,
>then I would regard that as support of point A.
>
>Regarding point B, if one envisions that all
>implementations of interest are hardware customized
>for RDDP, then the point does not carry much weight.
>However, I would argue that this may be too narrow
>a view for the IETF to take.
>_______________________________________________
>rddp mailing list
>rddp@ietf.org
>https://www1.ietf.org/mailman/listinfo/rddp
>_______________________________________________
>rddp mailing list
>rddp@ietf.org
>https://www1.ietf.org/mailman/listinfo/rddp

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr 10 13:54:29 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA28120
	for <rddp-archive@odin.ietf.org>; Thu, 10 Apr 2003 13:54:29 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3AI0H431157
	for rddp-archive@odin.ietf.org; Thu, 10 Apr 2003 14:00:17 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3AI0H831154
	for <rddp-web-archive@optimus.ietf.org>; Thu, 10 Apr 2003 14:00:17 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA28096
	for <rddp-web-archive@ietf.org>; Thu, 10 Apr 2003 13:53:59 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 193fzF-00036U-00
	for rddp-web-archive@ietf.org; Thu, 10 Apr 2003 13:37:21 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 193fzF-00036P-00
	for rddp-web-archive@ietf.org; Thu, 10 Apr 2003 13:37:21 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3AHxM831040;
	Thu, 10 Apr 2003 13:59:22 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3AHwb830953
	for <rddp@optimus.ietf.org>; Thu, 10 Apr 2003 13:58:37 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA28008
	for <rddp@ietf.org>; Thu, 10 Apr 2003 13:52:19 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 193fxd-00035D-00
	for rddp@ietf.org; Thu, 10 Apr 2003 13:35:41 -0400
Received: from atlrel6.hp.com ([156.153.255.205])
	by ietf-mx with esmtp (Exim 4.12)
	id 193fxd-00035A-00
	for rddp@ietf.org; Thu, 10 Apr 2003 13:35:41 -0400
Received: from rosemail.rose.hp.com (rosemail.rose.hp.com [15.96.64.26])
	by atlrel6.hp.com (Postfix) with ESMTP id 8069F1C02294
	for <rddp@ietf.org>; Thu, 10 Apr 2003 13:54:51 -0400 (EDT)
Received: from swathi (dhcp-rosefl-bba009.rose.hp.com [15.3.110.9]) by rosemail.rose.hp.com with SMTP (8.7.1/8.7.3 SMKit7.02) id KAA01110 for <rddp@ietf.org>; Thu, 10 Apr 2003 10:54:50 -0700 (PDT)
Message-ID: <002201c2ff8a$484d9950$096e030f@rose.hp.com>
From: "Mallikarjun C." <cbm@rose.hp.com>
To: <rddp@ietf.org>
References: <OFAFF652D3.1550E844-ON88256D03.00039F6D-88256D03.00061FD7@us.ibm.com> <3E93767F.2040404@eng.sun.com>
Subject: Re: [rddp] Option or mandatory CRC in MPA
Date: Thu, 10 Apr 2003 10:54:49 -0700
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2600.0000
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

I can see that the software-based client scenario is perhaps
credible for the first-generation of products.  But let me note
that the performance % delta incurred by supporting a mandatory
CRC is likely to be insignificant even for that first generation,
given all the RDMAP/DDP/MPA/TCP/IP protocol processing in 
software that must happen.

From the first-hand experience of being very closely involved in 
iSCSI error recovery design largely predicated on iSCSI digest
failures (which in turn are dependent on TCP checksum "escapes"), 
the single biggest attraction for me in a mandatory MPA CRC is
the deterministic QoS of the transport substrate from the ULP 
perspective.  That leads to obvious ULP protocol optimizations.

As someone has earlier noted, data integrity at high speeds is
an extremely important consideration for storage (and FC satisfies
this today).  IMHO, we would be doing a disservice to the RDDP
technology by designing it to be "optionally deterministic" wrt QoS.
--
Mallikarjun

Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions
Hewlett-Packard MS 5668 
Roseville CA 95747
cbm@rose.hp.com

----- Original Message ----- 
From: "Brent Callaghan" <brent@eng.sun.com>
To: <rddp@ietf.org>
Sent: Tuesday, April 08, 2003 6:25 PM
Subject: Re: [rddp] Option or mandatory CRC in MPA


> 
> John Hufferd wrote:
> > :
> > Most of the applications that will be using the RDMA protocols already 
> > have existing TCP/IP implementations, and they are not enhanced or 
> > accelerated by a software implementation of RDMA.  If systems only have 
> > a standard NIC, then the applications should continue to use TCP/IP. 
> 
> Perhaps.  Though a credible scenario is a large number of
> cheap and/or legacy clients doing RDMA to a large, heavily
> loaded server (not uncommon).
> 
> In that environment, it might be attractive to have the clients
> implement RDMA in software, and equip the server with the fancy
> RDMA hardware to relieve its suffering.
> 
> Brent
> 
> 
> _______________________________________________
> rddp mailing list
> rddp@ietf.org
> https://www1.ietf.org/mailman/listinfo/rddp
> 

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr 10 15:56:45 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04538
	for <rddp-archive@odin.ietf.org>; Thu, 10 Apr 2003 15:56:45 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3AK2Xr08420
	for rddp-archive@odin.ietf.org; Thu, 10 Apr 2003 16:02:33 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3AK2X808417
	for <rddp-web-archive@optimus.ietf.org>; Thu, 10 Apr 2003 16:02:33 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04516
	for <rddp-web-archive@ietf.org>; Thu, 10 Apr 2003 15:56:14 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 193htX-0003xL-00
	for rddp-web-archive@ietf.org; Thu, 10 Apr 2003 15:39:35 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 193htX-0003xI-00
	for rddp-web-archive@ietf.org; Thu, 10 Apr 2003 15:39:35 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3AK1C808354;
	Thu, 10 Apr 2003 16:01:12 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3AK0T808313
	for <rddp@optimus.ietf.org>; Thu, 10 Apr 2003 16:00:29 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04467
	for <rddp@ietf.org>; Thu, 10 Apr 2003 15:54:10 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 193hrX-0003wo-00
	for rddp@ietf.org; Thu, 10 Apr 2003 15:37:31 -0400
Received: from dmz1.silverbacksystems.com ([65.172.158.82])
	by ietf-mx with esmtp (Exim 4.12)
	id 193hrW-0003wN-00
	for rddp@ietf.org; Thu, 10 Apr 2003 15:37:30 -0400
Received: from ns.silverbacksystems.com (gate-camp-hme0.silverbacksystems.com [65.172.158.93])
	by dmz1.silverbacksystems.com (Postfix on SuSE Linux 7.3 (i386)) with ESMTP id 2D57AD2DE
	for <rddp@ietf.org>; Thu, 10 Apr 2003 12:55:55 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1])
	by ns.silverbacksystems.com (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 0B6631974C
	for <rddp@ietf.org>; Thu, 10 Apr 2003 12:55:55 -0700 (PDT)
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
	charset="iso-8859-1"
Date: Thu, 10 Apr 2003 12:55:24 -0700
From: "Somesh Gupta" <somesh_gupta@silverbacksystems.com>
Importance: Normal
In-Reply-To: <002201c2ff8a$484d9950$096e030f@rose.hp.com>
Message-ID: <NMEALCLOIBCHBDHLCMIJAEGKDDAA.somesh_gupta@silverbacksystems.com>
MIME-Version: 1.0
Received: from ns.silverbacksystems.com (localhost [127.0.0.1])
	by localhost (AvMailGate-2.0.0.6) id 04082-3BE572A6;
	Thu, 10 Apr 2003 12:55:11 -0700
Received: from somesh (somesh.corp.silverbacksystems.com [10.0.16.35])
	by ns.silverbacksystems.com (Postfix on SuSE Linux eMail Server 3.0) with SMTP id 3F793A906
	for <rddp@ietf.org>; Thu, 10 Apr 2003 12:55:11 -0700 (PDT)
Subject: RE: [rddp] Option or mandatory CRC in MPA
To: <rddp@ietf.org>
X-AntiVirus: OK! AntiVir MailGate Version 2.0.0.6
	 at ns.silverbacksystems.com has not found any known virus in this email.
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700
X-MSMail-Priority: Normal
X-Priority: 3 (Normal)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

It is difficult for people to definitively say what will or
what will not happen in the marketplace. As an example, we
went thru all of these debates during iSCSI protocol development.

As it turns out, bulk of the products are and may continue
to be "software based".

1. Most initiators at this time are in the category and may
continue to be software based

2. Many (if not majority) of targets are "software based" and
may continue to be.

Interesting, on the target side the processors used are not
these high end P-4s, but slower embedded processors with
smaller caches (32KB or whatever).

In these instances, everything that is done (CRC and mandatory
markers even on receive side where they provide no benefit)
solely from the perspective of a protocol optimized for hardware
implementations is likely to have a significant performance
impact.

I think we should be very clear and explicit about whether
we allow software based products to play in this market on
an equal footing or not.

On one hand a protocol enabling software based products will
have a wider acceptability and shorter time to market/revenue.
It may also have a greater chance as a standards track document.

On the other hand, a protocol focused on hardware based solns
will allow for greater optimization in silicon. There also seems
to be a greater meeting of minds in a certain group on this model.

Somesh
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr 10 16:43:34 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA06163
	for <rddp-archive@odin.ietf.org>; Thu, 10 Apr 2003 16:43:34 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3AKnOX12542
	for rddp-archive@odin.ietf.org; Thu, 10 Apr 2003 16:49:24 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3AKnO812539
	for <rddp-web-archive@optimus.ietf.org>; Thu, 10 Apr 2003 16:49:24 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA06144
	for <rddp-web-archive@ietf.org>; Thu, 10 Apr 2003 16:43:04 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 193icr-0004Gm-00
	for rddp-web-archive@ietf.org; Thu, 10 Apr 2003 16:26:25 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 193icq-0004Gj-00
	for rddp-web-archive@ietf.org; Thu, 10 Apr 2003 16:26:24 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3AKl2812352;
	Thu, 10 Apr 2003 16:47:02 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3AKkQ812325
	for <rddp@optimus.ietf.org>; Thu, 10 Apr 2003 16:46:26 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA05980
	for <rddp@ietf.org>; Thu, 10 Apr 2003 16:40:06 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 193iZz-0004FU-00
	for rddp@ietf.org; Thu, 10 Apr 2003 16:23:27 -0400
Received: from palrel10.hp.com ([156.153.255.245])
	by ietf-mx with esmtp (Exim 4.12)
	id 193iZz-0004FR-00
	for rddp@ietf.org; Thu, 10 Apr 2003 16:23:27 -0400
Received: from esmail.cup.hp.com (esmail.cup.hp.com [15.0.65.164])
	by palrel10.hp.com (Postfix) with ESMTP id 0C20D1C02D88
	for <rddp@ietf.org>; Thu, 10 Apr 2003 13:42:40 -0700 (PDT)
Received: from mk731915.cup.hp.com ([15.113.161.2])
	by esmail.cup.hp.com (8.11.1/8.8.6) with ESMTP id h3AKdAl29880
	for <rddp@ietf.org>; Thu, 10 Apr 2003 13:39:10 -0700 (PDT)
Message-Id: <5.1.0.14.2.20030410133602.01ffe130@15.13.130.73>
X-Sender: krause@15.13.130.73
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Thu, 10 Apr 2003 13:42:34 -0700
To: <rddp@ietf.org>
From: Michael Krause <krause@cup.hp.com>
Subject: RE: [rddp] Option or mandatory CRC in MPA
In-Reply-To: <NMEALCLOIBCHBDHLCMIJAEGKDDAA.somesh_gupta@silverbacksystem
 s.com>
References: <002201c2ff8a$484d9950$096e030f@rose.hp.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

At 12:55 PM 4/10/2003 -0700, Somesh Gupta wrote:
>It is difficult for people to definitively say what will or
>what will not happen in the marketplace. As an example, we
>went thru all of these debates during iSCSI protocol development.
>
>As it turns out, bulk of the products are and may continue
>to be "software based".
>
>1. Most initiators at this time are in the category and may
>continue to be software based
>
>2. Many (if not majority) of targets are "software based" and
>may continue to be.


Given the iSCSI spec just became a standard, it is understandable that 
there is a predominance of soft implementations.  However, not that it is a 
standard, there are people finalizing their hardware implementations thus 
the current state is clearly subject to change.


>Interesting, on the target side the processors used are not
>these high end P-4s, but slower embedded processors with
>smaller caches (32KB or whatever).
>
>In these instances, everything that is done (CRC and mandatory
>markers even on receive side where they provide no benefit)
>solely from the perspective of a protocol optimized for hardware
>implementations is likely to have a significant performance
>impact.

Not clear.  Whether one walks memory to do a checksum calculation, a CRC 
calculation, or a copy operation, the net effect is the processor data 
cache is made hot with these cache lines.  Given applications typically ask 
for data once it has arrived, the fact that is in cache mitigates the cache 
load latency thus the subsequent performance impact.   I do not see this as 
creating a significant performance impact.  Do you have data to show that 
there is such an impact?

>I think we should be very clear and explicit about whether
>we allow software based products to play in this market on
>an equal footing or not.

I thought there was general consensus that the focus of RDMA is hardware 
acceleration and that it must be able to be implemented in software.  There 
were no requirements put forth that I'm aware of that require software and 
hardware to have an equal footing especially as some operations such as a 
RDMA Read will always outperform a software implementation as no processor 
interactions are required when implemented in hardware.

>On one hand a protocol enabling software based products will
>have a wider acceptability and shorter time to market/revenue.
>It may also have a greater chance as a standards track document.
>
>On the other hand, a protocol focused on hardware based solns
>will allow for greater optimization in silicon. There also seems
>to be a greater meeting of minds in a certain group on this model.

Mike

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr 10 17:02:23 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA06859
	for <rddp-archive@odin.ietf.org>; Thu, 10 Apr 2003 17:02:23 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3AL8ET14829
	for rddp-archive@odin.ietf.org; Thu, 10 Apr 2003 17:08:14 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3AL8E814826
	for <rddp-web-archive@optimus.ietf.org>; Thu, 10 Apr 2003 17:08:14 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA06842
	for <rddp-web-archive@ietf.org>; Thu, 10 Apr 2003 17:01:53 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 193iv4-0004SF-00
	for rddp-web-archive@ietf.org; Thu, 10 Apr 2003 16:45:14 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 193iv4-0004SC-00
	for rddp-web-archive@ietf.org; Thu, 10 Apr 2003 16:45:14 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3AL7B814115;
	Thu, 10 Apr 2003 17:07:11 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3AL5h813847
	for <rddp@optimus.ietf.org>; Thu, 10 Apr 2003 17:05:43 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA06693
	for <rddp@ietf.org>; Thu, 10 Apr 2003 16:59:22 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 193isd-0004Pa-00
	for rddp@ietf.org; Thu, 10 Apr 2003 16:42:43 -0400
Received: from thebe.your-site.com ([140.186.45.26])
	by ietf-mx with esmtp (Exim 4.12)
	id 193isd-0004Ow-00
	for rddp@ietf.org; Thu, 10 Apr 2003 16:42:43 -0400
Received: from 192.168.0.2 (64-144-5-25.client.dsl.net [64.144.5.25])
	by thebe.your-site.com (Postfix) with ESMTP
	id 8BC81244995; Thu, 10 Apr 2003 17:01:35 -0400 (EDT)
Date: Thu, 10 Apr 2003 16:03:45 -0500
From: Caitlin Bestler <cait@asomi.com>
Subject: RE: [rddp] Option or mandatory CRC in MPA
To: Michael Krause <krause@cup.hp.com>, rddp@ietf.org
X-Priority: 3
In-Reply-To: <5.1.0.14.2.20030410133602.01ffe130@15.13.130.73>
Message-ID: <r01050300-1024-EB404D596B9711D78DE4003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

On 4/10/03, Michael Krause wrote:

>Not clear.  Whether one walks memory to do a checksum
>calculation, a CRC calculation, or a copy operation, the
>net effect is the processor data cache is made hot with
>these cache lines.  Given applications typically ask for
>data once it has arrived, the fact that is in cache
>mitigates the cache load latency thus the subsequent
>performance impact.   I do not see this as creating a
>significant performance impact.  Do you have data to show
>that there is such an impact?
>
More importantly, there are no additional memory accesses
required that would not have been required to remove
markers.

Admittedly, you could have an adequate DMA engine to avoid
this problem. But then we're worrying about making things
too hard on implementations that have half of the problem
solved in hardware.

Efficient CRC algorithms are well known, even for software.
As you point out, it is *at worst* a pass through the
buffer. If that is unacceptable, then buy an RNIC.

Meanwhile, optionally performing the check will be a
complexity that *all* implementations will have to bear.

Keep in mind that the header MUST be checked. So an
"optional" CRC is really just a *shortened* CRC that
only covers the header. Absolutely no code savings,
only a reduction in memory touches for *some*
implementations.


>>I think we should be very clear and explicit about whether
>>we allow software based products to play in this market on
>>an equal footing or not.
>
>I thought there was general consensus that the focus of
>RDMA is hardware acceleration and that it must be able to
>be implemented in software.  There were no requirements
>put forth that I'm aware of that require software and
>hardware to have an equal footing especially as some
>operations such as a RDMA Read will always outperform a
>software implementation as no processor interactions are
>required when implemented in hardware.

It would also be impossible. Any fixed algorithm can
ultimately be performed with at least a slight edge in
hardware.

The goal is to enable software stacks mostly for quick
adoption by clients. I haven't seen arguments for mandating
that software be able to handle 10 Gb NICs on its own.
I think most people would be very happy with a software
stack that handled a GigE NIC.


Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr 10 18:08:08 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA08994
	for <rddp-archive@odin.ietf.org>; Thu, 10 Apr 2003 18:08:08 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3AME0K19675
	for rddp-archive@odin.ietf.org; Thu, 10 Apr 2003 18:14:00 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3AME0819672
	for <rddp-web-archive@optimus.ietf.org>; Thu, 10 Apr 2003 18:14:00 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA08918
	for <rddp-web-archive@ietf.org>; Thu, 10 Apr 2003 18:07:37 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 193jwh-0004oh-00
	for rddp-web-archive@ietf.org; Thu, 10 Apr 2003 17:50:59 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 193jwh-0004oe-00
	for rddp-web-archive@ietf.org; Thu, 10 Apr 2003 17:50:59 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3AMDC819653;
	Thu, 10 Apr 2003 18:13:12 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3AMCx819632
	for <rddp@optimus.ietf.org>; Thu, 10 Apr 2003 18:12:59 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA08812
	for <rddp@ietf.org>; Thu, 10 Apr 2003 18:06:36 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 193jvi-0004oP-00
	for rddp@ietf.org; Thu, 10 Apr 2003 17:49:58 -0400
Received: from dmz1.silverbacksystems.com ([65.172.158.82])
	by ietf-mx with esmtp (Exim 4.12)
	id 193jvh-0004oK-00
	for rddp@ietf.org; Thu, 10 Apr 2003 17:49:57 -0400
Received: from ns.silverbacksystems.com (gate-camp-hme0.silverbacksystems.com [65.172.158.93])
	by dmz1.silverbacksystems.com (Postfix on SuSE Linux 7.3 (i386)) with ESMTP id 786AED3A0
	for <rddp@ietf.org>; Thu, 10 Apr 2003 15:08:27 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1])
	by ns.silverbacksystems.com (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 5F063DC53
	for <rddp@ietf.org>; Thu, 10 Apr 2003 15:08:27 -0700 (PDT)
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
	charset="us-ascii"
Date: Thu, 10 Apr 2003 15:07:45 -0700
From: "Somesh Gupta" <somesh_gupta@silverbacksystems.com>
Importance: Normal
In-Reply-To: <5.1.0.14.2.20030410133602.01ffe130@15.13.130.73>
Message-ID: <NMEALCLOIBCHBDHLCMIJAEGMDDAA.somesh_gupta@silverbacksystems.com>
MIME-Version: 1.0
Received: from ns.silverbacksystems.com (localhost [127.0.0.1])
	by localhost (AvMailGate-2.0.0.6) id 08638-0C2335B5;
	Thu, 10 Apr 2003 15:07:33 -0700
Received: from somesh (somesh.corp.silverbacksystems.com [10.0.16.35])
	by ns.silverbacksystems.com (Postfix on SuSE Linux eMail Server 3.0) with SMTP id 46D6D9C36
	for <rddp@ietf.org>; Thu, 10 Apr 2003 15:07:33 -0700 (PDT)
Subject: RE: [rddp] Option or mandatory CRC in MPA
To: <rddp@ietf.org>
X-AntiVirus: OK! AntiVir MailGate Version 2.0.0.6
	 at ns.silverbacksystems.com has not found any known virus in this email.
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700
X-MSMail-Priority: Normal
X-Priority: 3 (Normal)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

Mike,

> -----Original Message-----
> From: rddp-admin@ietf.org [mailto:rddp-admin@ietf.org]On Behalf Of
> Michael Krause
> Sent: Thursday, April 10, 2003 1:43 PM
> To: rddp@ietf.org
> Subject: RE: [rddp] Option or mandatory CRC in MPA
>
>
> At 12:55 PM 4/10/2003 -0700, Somesh Gupta wrote:
> >It is difficult for people to definitively say what will or
> >what will not happen in the marketplace. As an example, we
> >went thru all of these debates during iSCSI protocol development.
> >
> >As it turns out, bulk of the products are and may continue
> >to be "software based".
> >
> >1. Most initiators at this time are in the category and may
> >continue to be software based
> >
> >2. Many (if not majority) of targets are "software based" and
> >may continue to be.
>
>
> Given the iSCSI spec just became a standard, it is understandable that
> there is a predominance of soft implementations.  However, not
> that it is a
> standard, there are people finalizing their hardware implementations thus
> the current state is clearly subject to change.

  I am fairly certain there will continue to be a range of implementations.

>
>
> >Interesting, on the target side the processors used are not
> >these high end P-4s, but slower embedded processors with
> >smaller caches (32KB or whatever).
> >
> >In these instances, everything that is done (CRC and mandatory
> >markers even on receive side where they provide no benefit)
> >solely from the perspective of a protocol optimized for hardware
> >implementations is likely to have a significant performance
> >impact.
>
> Not clear.  Whether one walks memory to do a checksum calculation, a CRC
> calculation, or a copy operation, the net effect is the processor data
> cache is made hot with these cache lines.  Given applications
> typically ask
> for data once it has arrived, the fact that is in cache mitigates
> the cache
> load latency thus the subsequent performance impact.   I do not
> see this as
> creating a significant performance impact.  Do you have data to show that
> there is such an impact?

  I think this brings up a number of interesting points:

1. We should assume checksum is offloaded as fairly all NICs do
   it now.
2. There are a certain class of apps that do not touch the data
   but just shunt it from one place to another. A classic example
   is any kind of storage server. These are actually the ones
   that benefit from zero copy the most and may use means other
   than RDDP to get the benefit (i.e. handle SGLs)
3. Obviously there are many other apps that chew on the data.
   In those cases, there is just a whole range of factors that
   come into play. The rate at which data can be burst vs the
   apps consumption of data, the size of the data caches etc
   etc. In a number of cases, it does not matter how much
   network acceleration you provide - the processing of the data
   consumes bulk of the cycles anyway.

>
> >I think we should be very clear and explicit about whether
> >we allow software based products to play in this market on
> >an equal footing or not.
>
> I thought there was general consensus that the focus of RDMA is hardware
> acceleration and that it must be able to be implemented in
> software.  There
> were no requirements put forth that I'm aware of that require
> software and
> hardware to have an equal footing especially as some operations such as a
> RDMA Read will always outperform a software implementation as no
> processor
> interactions are required when implemented in hardware.

  The statement of requirements should be very explicit on this.
  We can all agree that hardware acceleration will provide higher
  performance under all cases, but that is not the point.

>
> >On one hand a protocol enabling software based products will
> >have a wider acceptability and shorter time to market/revenue.
> >It may also have a greater chance as a standards track document.
> >
> >On the other hand, a protocol focused on hardware based solns
> >will allow for greater optimization in silicon. There also seems
> >to be a greater meeting of minds in a certain group on this model.
>
> Mike

  Somesh
>
> _______________________________________________
> rddp mailing list
> rddp@ietf.org
> https://www1.ietf.org/mailman/listinfo/rddp

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr 10 21:02:59 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA14005
	for <rddp-archive@odin.ietf.org>; Thu, 10 Apr 2003 21:02:59 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3B18u731592
	for rddp-archive@odin.ietf.org; Thu, 10 Apr 2003 21:08:56 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3B18u831589
	for <rddp-web-archive@optimus.ietf.org>; Thu, 10 Apr 2003 21:08:56 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA13983
	for <rddp-web-archive@ietf.org>; Thu, 10 Apr 2003 21:02:28 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 193mfv-0005lL-00
	for rddp-web-archive@ietf.org; Thu, 10 Apr 2003 20:45:51 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 193mfu-0005lI-00
	for rddp-web-archive@ietf.org; Thu, 10 Apr 2003 20:45:50 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3B188831507;
	Thu, 10 Apr 2003 21:08:08 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3B17X831487
	for <rddp@optimus.ietf.org>; Thu, 10 Apr 2003 21:07:33 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA13927;
	Thu, 10 Apr 2003 21:01:06 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 193mea-0005k6-00; Thu, 10 Apr 2003 20:44:28 -0400
Received: from e35.co.us.ibm.com ([32.97.110.133])
	by ietf-mx with esmtp (Exim 4.12)
	id 193meZ-0005k2-00; Thu, 10 Apr 2003 20:44:27 -0400
Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.17.195.11])
	by e35.co.us.ibm.com (8.12.9/8.12.2) with ESMTP id h3B13AuT025902;
	Thu, 10 Apr 2003 21:03:10 -0400
Received: from d03nm115.boulder.ibm.com (d03av02.boulder.ibm.com [9.17.193.82])
	by westrelay02.boulder.ibm.com (8.12.8/NCO/VER6.5) with ESMTP id h3B139hW056062;
	Thu, 10 Apr 2003 19:03:09 -0600
To: rddp@ietf.org, rddp-admin@ietf.org
MIME-Version: 1.0
Subject: RE: [rddp] Option or mandatory CRC in MPA
X-Mailer: Lotus Notes Release 5.0.11   July 24, 2002
From: John Hufferd <hufferd@us.ibm.com>
Message-ID: <OFC3E84CCF.9972D99A-ON88256D05.000004A4-88256D05.0005C17A@us.ibm.com>
Date: Thu, 10 Apr 2003 17:59:04 -0700
X-MIMETrack: Serialize by Router on D03NM115/03/M/IBM(Release 6.0 [IBM]|December 16, 2002) at
 04/10/2003 18:58:56,
	Serialize complete at 04/10/2003 18:58:56
Content-Type: multipart/alternative; boundary="=_alternative 0005C08788256D05_="
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

This is a multipart message in MIME format.
--=_alternative 0005C08788256D05_=
Content-Type: text/plain; charset="us-ascii"

As you probably know by now, I believe that software implementations are 
not what  we defined as a problem which needed to be solved.  Having said 
that, lets see where the software implementations might make sense.  First 
off, I categorically reject, out of hand, that a Storage Server 
(controller) will be implementing RDMA in software.  I know of no one that 
is even thinking about that.  So lets move off that topic, and look at the 
Host systems.

The point has been made, and I guess I can accept, that some Client 
systems will be "required" by an "IT Staff" to help the Server System that 
has HW RDMA.  That is, the Clients (desktops) will have to use RDMA mode 
software so that the load on the server is kept low.  I do not think one 
will see this as a large situation, but it may exist.

I do not believe that it is reasonable to talk about Servers running a 
software version of RDMA for all the reasons I stated in my last note.

So lets revisit the Client systems.  Project if you will, to the point in 
time that there are servers that have RDMA chips.  Lets say servers will 
start to have meaningful numbers of systems with RDMA chip (RNIC) sometime 
in 2004 (probably late 2004, but pick your own date since that will not 
change the argument much).  Now, pick the time that you think there will 
be OSes that will support  the RNIC via Socket Direct Protocol (SDP) -- 
which is a way to make RDMA transparent to socket users -- in numbers that 
matter.  Or pick the time that applications will be rewritten to use the 
RDMA features (without SDP),  with or without the Host OS directly 
supporting them (could be supported via the RNIC vendor's special SW). 

Now most of the servers application vendors that I know will not rewrite 
their software to work with RDMA, unless they have a very large market 
that supports the required RDMA software.  There are, of course, some 
bleeding edge vendors and end-users which will, but ask your self when 
your favorite application will do that.

So now lets look at the desktop again, and ask yourself when you think 
those OSes will support (in meaningful numbers) a software version of 
RDMA.

OK, now with what ever dates you have come up with, consider the 
following:

Desktop are shipping with 2-3Mhz this year, probably 1.5 - 2 times that 
next year or so, and even more than that the following year.  We are 
talking about 3-6 Mhz next year, and maybe 6 - 10 Mhz the following year. 
All this at your personal desktop.

For us to be debating the overhead of such a desktop when it is computing 
the CRC is, in my opinion, not reasonable. 

This situation is quite different on the servers, which want every MIP 
they can get in order to serve the desktops; they will need and require HW 
RDMA implementations.

So again, lets not attempt to optimize for the software version, which 
will only be on desktops which do not need the optimization.


.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Home Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: hufferd@us.ibm.com




"Somesh Gupta" <somesh_gupta@silverbacksystems.com>
Sent by: rddp-admin@ietf.org
04/10/2003 03:07 PM

 
        To:     <rddp@ietf.org>
        cc: 
        Subject:        RE: [rddp] Option or mandatory CRC in MPA



Mike,

> -----Original Message-----
> From: rddp-admin@ietf.org [mailto:rddp-admin@ietf.org]On Behalf Of
> Michael Krause
> Sent: Thursday, April 10, 2003 1:43 PM
> To: rddp@ietf.org
> Subject: RE: [rddp] Option or mandatory CRC in MPA
>
>
> At 12:55 PM 4/10/2003 -0700, Somesh Gupta wrote:
> >It is difficult for people to definitively say what will or
> >what will not happen in the marketplace. As an example, we
> >went thru all of these debates during iSCSI protocol development.
> >
> >As it turns out, bulk of the products are and may continue
> >to be "software based".
> >
> >1. Most initiators at this time are in the category and may
> >continue to be software based
> >
> >2. Many (if not majority) of targets are "software based" and
> >may continue to be.
>
>
> Given the iSCSI spec just became a standard, it is understandable that
> there is a predominance of soft implementations.  However, not
> that it is a
> standard, there are people finalizing their hardware implementations 
thus
> the current state is clearly subject to change.

  I am fairly certain there will continue to be a range of 
implementations.

>
>
> >Interesting, on the target side the processors used are not
> >these high end P-4s, but slower embedded processors with
> >smaller caches (32KB or whatever).
> >
> >In these instances, everything that is done (CRC and mandatory
> >markers even on receive side where they provide no benefit)
> >solely from the perspective of a protocol optimized for hardware
> >implementations is likely to have a significant performance
> >impact.
>
> Not clear.  Whether one walks memory to do a checksum calculation, a CRC
> calculation, or a copy operation, the net effect is the processor data
> cache is made hot with these cache lines.  Given applications
> typically ask
> for data once it has arrived, the fact that is in cache mitigates
> the cache
> load latency thus the subsequent performance impact.   I do not
> see this as
> creating a significant performance impact.  Do you have data to show 
that
> there is such an impact?

  I think this brings up a number of interesting points:

1. We should assume checksum is offloaded as fairly all NICs do
   it now.
2. There are a certain class of apps that do not touch the data
   but just shunt it from one place to another. A classic example
   is any kind of storage server. These are actually the ones
   that benefit from zero copy the most and may use means other
   than RDDP to get the benefit (i.e. handle SGLs)
3. Obviously there are many other apps that chew on the data.
   In those cases, there is just a whole range of factors that
   come into play. The rate at which data can be burst vs the
   apps consumption of data, the size of the data caches etc
   etc. In a number of cases, it does not matter how much
   network acceleration you provide - the processing of the data
   consumes bulk of the cycles anyway.

>
> >I think we should be very clear and explicit about whether
> >we allow software based products to play in this market on
> >an equal footing or not.
>
> I thought there was general consensus that the focus of RDMA is hardware
> acceleration and that it must be able to be implemented in
> software.  There
> were no requirements put forth that I'm aware of that require
> software and
> hardware to have an equal footing especially as some operations such as 
a
> RDMA Read will always outperform a software implementation as no
> processor
> interactions are required when implemented in hardware.

  The statement of requirements should be very explicit on this.
  We can all agree that hardware acceleration will provide higher
  performance under all cases, but that is not the point.

>
> >On one hand a protocol enabling software based products will
> >have a wider acceptability and shorter time to market/revenue.
> >It may also have a greater chance as a standards track document.
> >
> >On the other hand, a protocol focused on hardware based solns
> >will allow for greater optimization in silicon. There also seems
> >to be a greater meeting of minds in a certain group on this model.
>
> Mike

  Somesh
>
> _______________________________________________
> rddp mailing list
> rddp@ietf.org
> https://www1.ietf.org/mailman/listinfo/rddp

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



--=_alternative 0005C08788256D05_=
Content-Type: text/html; charset="us-ascii"


<br><font size=2 face="sans-serif">As you probably know by now, I believe that software implementations are not what &nbsp;we defined as a problem which needed to be solved. &nbsp;Having said that, lets see where the software implementations might make sense. &nbsp;First off, I categorically reject, out of hand, that a Storage Server (controller) will be implementing RDMA in software. &nbsp;I know of no one that is even thinking about that. &nbsp;So lets move off that topic, and look at the Host systems.</font>
<br>
<br><font size=2 face="sans-serif">The point has been made, and I guess I can accept, that some Client systems will be &quot;required&quot; by an &quot;IT Staff&quot; to help the Server System that has HW RDMA. &nbsp;That is, the Clients (desktops) will have to use RDMA mode software so that the load on the server is kept low. &nbsp;I do not think one will see this as a large situation, but it may exist.</font>
<br>
<br><font size=2 face="sans-serif">I do not believe that it is reasonable to talk about Servers running a software version of RDMA for all the reasons I stated in my last note.</font>
<br>
<br><font size=2 face="sans-serif">So lets revisit the Client systems. &nbsp;Project if you will, to the point in time that there are servers that have RDMA chips. &nbsp;Lets say servers will start to have meaningful numbers of systems with RDMA chip (RNIC) sometime in 2004 (probably late 2004, but pick your own date since that will not change the argument much). &nbsp;Now, pick the time that you think there will be OSes that will support &nbsp;the RNIC via Socket Direct Protocol (SDP) -- which is a way to make RDMA transparent to socket users -- in numbers that matter. &nbsp;Or pick the time that applications will be rewritten to use the RDMA features (without SDP), &nbsp;with or without the Host OS directly supporting them (could be supported via the RNIC vendor's special SW). </font>
<br>
<br><font size=2 face="sans-serif">Now most of the servers application vendors that I know will not rewrite their software to work with RDMA, unless they have a very large market that supports the required RDMA software. &nbsp;There are, of course, some bleeding edge vendors and end-users which will, but ask your self when your favorite application will do that.</font>
<br>
<br><font size=2 face="sans-serif">So now lets look at the desktop again, and ask yourself when you think those OSes will support (in meaningful numbers) a software version of RDMA.</font>
<br>
<br><font size=2 face="sans-serif">OK, now with what ever dates you have come up with, consider the following:</font>
<br>
<br><font size=2 face="sans-serif">Desktop are shipping with 2-3Mhz this year, probably 1.5 - 2 times that next year or so, and even more than that the following year. &nbsp;We are talking about 3-6 Mhz next year, and maybe 6 - 10 Mhz the following year. &nbsp;All this at your personal desktop.</font>
<br>
<br><font size=2 face="sans-serif">For us to be debating the overhead of such a desktop when it is computing the CRC is, in my opinion, not reasonable. &nbsp;</font>
<br>
<br><font size=2 face="sans-serif">This situation is quite different on the servers, which want every MIP they can get in order to serve the desktops; they will need and require HW RDMA implementations.</font>
<br>
<br><font size=2 face="sans-serif">So again, lets not attempt to optimize for the software version, which will only be on desktops which do not need the optimization.</font>
<br>
<br>
<br><font size=2 face="sans-serif">.<br>
.<br>
John L. Hufferd<br>
Senior Technical Staff Member (STSM)<br>
IBM/System Group, San Jose CA<br>
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688<br>
Home Office: (408) 997-6136, Cell: (408) 499-9702<br>
Internet Address: hufferd@us.ibm.com</font>
<br>
<br>
<br>
<table width=100%>
<tr valign=top>
<td>
<td><font size=1 face="sans-serif"><b>&quot;Somesh Gupta&quot; &lt;somesh_gupta@silverbacksystems.com&gt;</b></font>
<br><font size=1 face="sans-serif">Sent by: rddp-admin@ietf.org</font>
<p><font size=1 face="sans-serif">04/10/2003 03:07 PM</font>
<br>
<td><font size=1 face="Arial">&nbsp; &nbsp; &nbsp; &nbsp; </font>
<br><font size=1 face="sans-serif">&nbsp; &nbsp; &nbsp; &nbsp; To: &nbsp; &nbsp; &nbsp; &nbsp;&lt;rddp@ietf.org&gt;</font>
<br><font size=1 face="sans-serif">&nbsp; &nbsp; &nbsp; &nbsp; cc: &nbsp; &nbsp; &nbsp; &nbsp;</font>
<br><font size=1 face="sans-serif">&nbsp; &nbsp; &nbsp; &nbsp; Subject: &nbsp; &nbsp; &nbsp; &nbsp;RE: [rddp] Option or mandatory CRC in MPA</font>
<br></table>
<br>
<br>
<br><font size=2 face="Courier New">Mike,<br>
<br>
&gt; -----Original Message-----<br>
&gt; From: rddp-admin@ietf.org [mailto:rddp-admin@ietf.org]On Behalf Of<br>
&gt; Michael Krause<br>
&gt; Sent: Thursday, April 10, 2003 1:43 PM<br>
&gt; To: rddp@ietf.org<br>
&gt; Subject: RE: [rddp] Option or mandatory CRC in MPA<br>
&gt;<br>
&gt;<br>
&gt; At 12:55 PM 4/10/2003 -0700, Somesh Gupta wrote:<br>
&gt; &gt;It is difficult for people to definitively say what will or<br>
&gt; &gt;what will not happen in the marketplace. As an example, we<br>
&gt; &gt;went thru all of these debates during iSCSI protocol development.<br>
&gt; &gt;<br>
&gt; &gt;As it turns out, bulk of the products are and may continue<br>
&gt; &gt;to be &quot;software based&quot;.<br>
&gt; &gt;<br>
&gt; &gt;1. Most initiators at this time are in the category and may<br>
&gt; &gt;continue to be software based<br>
&gt; &gt;<br>
&gt; &gt;2. Many (if not majority) of targets are &quot;software based&quot; and<br>
&gt; &gt;may continue to be.<br>
&gt;<br>
&gt;<br>
&gt; Given the iSCSI spec just became a standard, it is understandable that<br>
&gt; there is a predominance of soft implementations. &nbsp;However, not<br>
&gt; that it is a<br>
&gt; standard, there are people finalizing their hardware implementations thus<br>
&gt; the current state is clearly subject to change.<br>
<br>
 &nbsp;I am fairly certain there will continue to be a range of implementations.<br>
<br>
&gt;<br>
&gt;<br>
&gt; &gt;Interesting, on the target side the processors used are not<br>
&gt; &gt;these high end P-4s, but slower embedded processors with<br>
&gt; &gt;smaller caches (32KB or whatever).<br>
&gt; &gt;<br>
&gt; &gt;In these instances, everything that is done (CRC and mandatory<br>
&gt; &gt;markers even on receive side where they provide no benefit)<br>
&gt; &gt;solely from the perspective of a protocol optimized for hardware<br>
&gt; &gt;implementations is likely to have a significant performance<br>
&gt; &gt;impact.<br>
&gt;<br>
&gt; Not clear. &nbsp;Whether one walks memory to do a checksum calculation, a CRC<br>
&gt; calculation, or a copy operation, the net effect is the processor data<br>
&gt; cache is made hot with these cache lines. &nbsp;Given applications<br>
&gt; typically ask<br>
&gt; for data once it has arrived, the fact that is in cache mitigates<br>
&gt; the cache<br>
&gt; load latency thus the subsequent performance impact. &nbsp; I do not<br>
&gt; see this as<br>
&gt; creating a significant performance impact. &nbsp;Do you have data to show that<br>
&gt; there is such an impact?<br>
<br>
 &nbsp;I think this brings up a number of interesting points:<br>
<br>
1. We should assume checksum is offloaded as fairly all NICs do<br>
 &nbsp; it now.<br>
2. There are a certain class of apps that do not touch the data<br>
 &nbsp; but just shunt it from one place to another. A classic example<br>
 &nbsp; is any kind of storage server. These are actually the ones<br>
 &nbsp; that benefit from zero copy the most and may use means other<br>
 &nbsp; than RDDP to get the benefit (i.e. handle SGLs)<br>
3. Obviously there are many other apps that chew on the data.<br>
 &nbsp; In those cases, there is just a whole range of factors that<br>
 &nbsp; come into play. The rate at which data can be burst vs the<br>
 &nbsp; apps consumption of data, the size of the data caches etc<br>
 &nbsp; etc. In a number of cases, it does not matter how much<br>
 &nbsp; network acceleration you provide - the processing of the data<br>
 &nbsp; consumes bulk of the cycles anyway.<br>
<br>
&gt;<br>
&gt; &gt;I think we should be very clear and explicit about whether<br>
&gt; &gt;we allow software based products to play in this market on<br>
&gt; &gt;an equal footing or not.<br>
&gt;<br>
&gt; I thought there was general consensus that the focus of RDMA is hardware<br>
&gt; acceleration and that it must be able to be implemented in<br>
&gt; software. &nbsp;There<br>
&gt; were no requirements put forth that I'm aware of that require<br>
&gt; software and<br>
&gt; hardware to have an equal footing especially as some operations such as a<br>
&gt; RDMA Read will always outperform a software implementation as no<br>
&gt; processor<br>
&gt; interactions are required when implemented in hardware.<br>
<br>
 &nbsp;The statement of requirements should be very explicit on this.<br>
 &nbsp;We can all agree that hardware acceleration will provide higher<br>
 &nbsp;performance under all cases, but that is not the point.<br>
<br>
&gt;<br>
&gt; &gt;On one hand a protocol enabling software based products will<br>
&gt; &gt;have a wider acceptability and shorter time to market/revenue.<br>
&gt; &gt;It may also have a greater chance as a standards track document.<br>
&gt; &gt;<br>
&gt; &gt;On the other hand, a protocol focused on hardware based solns<br>
&gt; &gt;will allow for greater optimization in silicon. There also seems<br>
&gt; &gt;to be a greater meeting of minds in a certain group on this model.<br>
&gt;<br>
&gt; Mike</font>
<br><font size=2 face="Courier New"><br>
 &nbsp;Somesh<br>
&gt;<br>
&gt; _______________________________________________<br>
&gt; rddp mailing list<br>
&gt; rddp@ietf.org<br>
&gt; https://www1.ietf.org/mailman/listinfo/rddp<br>
<br>
_______________________________________________<br>
rddp mailing list<br>
rddp@ietf.org<br>
https://www1.ietf.org/mailman/listinfo/rddp<br>
</font>
<br>
<br>
--=_alternative 0005C08788256D05_=--
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr 10 21:21:59 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA14419
	for <rddp-archive@odin.ietf.org>; Thu, 10 Apr 2003 21:21:59 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3B1Rua32598
	for rddp-archive@odin.ietf.org; Thu, 10 Apr 2003 21:27:56 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3B1Ru832595
	for <rddp-web-archive@optimus.ietf.org>; Thu, 10 Apr 2003 21:27:56 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA14410
	for <rddp-web-archive@ietf.org>; Thu, 10 Apr 2003 21:21:28 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 193myJ-0005rr-00
	for rddp-web-archive@ietf.org; Thu, 10 Apr 2003 21:04:51 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 193myI-0005ro-00
	for rddp-web-archive@ietf.org; Thu, 10 Apr 2003 21:04:50 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3B1O2832379;
	Thu, 10 Apr 2003 21:24:02 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3B1Nt832365
	for <rddp@optimus.ietf.org>; Thu, 10 Apr 2003 21:23:55 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA14277
	for <rddp@ietf.org>; Thu, 10 Apr 2003 21:17:28 -0400 (EDT)
From: Black_David@emc.com
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 193muQ-0005pp-00
	for rddp@ietf.org; Thu, 10 Apr 2003 21:00:50 -0400
Received: from maho3msx2.corp.emc.com ([128.221.11.32])
	by ietf-mx with esmtp (Exim 4.12)
	id 193muQ-0005pm-00
	for rddp@ietf.org; Thu, 10 Apr 2003 21:00:50 -0400
Received: by maho3msx2.corp.emc.com with Internet Mail Service (5.5.2653.19)
	id <2PLNJ33B>; Thu, 10 Apr 2003 21:19:28 -0400
Message-ID: <277DD60FB639D511AC0400B0D068B71E0564CB0A@corpmx14.us.dg.com>
To: hufferd@us.ibm.com, rddp@ietf.org
Subject: RE: [rddp] Option or mandatory CRC in MPA - WG Chair speaks
Date: Thu, 10 Apr 2003 21:14:35 -0400
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

> I find some of these arguments somewhat strange.  We have all entered
> into this effort to bring optimized processing with Remote Direct Data
> Placement, yet we continue to talk about, and give serious weight to,
> the argument concerning an all software implementation.  It seems that
> we might think about a complete software implementation, in passing,
> but this is NOT the goal of RDMA or this RDDP workgroup (in my opinion).

I need to say some fairly strong words on this.  That statement and
the implication that only hardware implementations need apply are at
direct odds with the discussion leading to WG rough consensus on the
ULPDU alignment with TCP segment issue, and the resulting rough consensus.
It is very clear that the discussion leading to that rough consensus:
- Emphasized the ability to run over existing unmodified TCP stacks.
- Emphasized the importance of fan-in scenarios in which a number
	of software clients communicated with a hardware-accelerated
	server.
I believe both of those represent the rough consensus of the RDDP
WG.  To the extent that they do not, we're back in the alignment
morass, as those were crucial to settling the alignment issue, so ...

I am putting the RDDP WG on notice that if this line of discussion
leads to a need to reopen the alignment issue, I may have no choice
but to conclude that the technical issues involved in RDDP over TCP
are of sufficient intractability that I need to recommend to the ADs
that the pending RDDP recharter should delete all TCP work.  

> Most of the applications that will be using the RDMA protocols already
> have existing TCP/IP implementations, and they are not enhanced or
> accelerated by a software implementation of RDMA.  If systems only
> have a standard NIC, then the applications should continue to use TCP/IP.

That needs a lot more discussion, as among other things it excludes
most NFS clients (by count of systems, not count of independently
developed implementations).  This reminds me of the early days of iSCSI
work when the "conventional wisdom" was that iSCSI was going to need
accelerated hardware - at the moment software iSCSI implementations
are coming in with acceptable performance, albeit not as good as could
be achieved in hardware.

It is fine to discuss whether the CRC should be optional, but that
discussion MUST respect the WG rough consensus that *finally* closed
the alignment issue if any progress is going to be made.  Consideration
of all-software clients and their requirements is necessitated by the
current state of the RDDP WG, unless people want to reopen the alignment
issue (see above if you want to do that).

Thanks,
--David
----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
black_david@emc.com        Mobile: +1 (978) 394-7754
----------------------------------------------------
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr 10 23:02:56 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA16202
	for <rddp-archive@odin.ietf.org>; Thu, 10 Apr 2003 23:02:56 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3B38rV07335
	for rddp-archive@odin.ietf.org; Thu, 10 Apr 2003 23:08:53 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3B38p807330
	for <rddp-web-archive@optimus.ietf.org>; Thu, 10 Apr 2003 23:08:51 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA16196
	for <rddp-web-archive@ietf.org>; Thu, 10 Apr 2003 23:02:23 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 193oXv-0006HC-00
	for rddp-web-archive@ietf.org; Thu, 10 Apr 2003 22:45:43 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 193oXv-0006H9-00
	for rddp-web-archive@ietf.org; Thu, 10 Apr 2003 22:45:43 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3B387807306;
	Thu, 10 Apr 2003 23:08:07 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3B37b807286
	for <rddp@optimus.ietf.org>; Thu, 10 Apr 2003 23:07:37 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA16186;
	Thu, 10 Apr 2003 23:01:09 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 193oWk-0006Gs-00; Thu, 10 Apr 2003 22:44:30 -0400
Received: from e33.co.us.ibm.com ([32.97.110.131])
	by ietf-mx with esmtp (Exim 4.12)
	id 193oWj-0006Gp-00; Thu, 10 Apr 2003 22:44:29 -0400
Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.17.195.11])
	by e33.co.us.ibm.com (8.12.9/8.12.2) with ESMTP id h3B334ft154172;
	Thu, 10 Apr 2003 23:03:04 -0400
Received: from d03nm115.boulder.ibm.com (d03av02.boulder.ibm.com [9.17.193.82])
	by westrelay02.boulder.ibm.com (8.12.8/NCO/VER6.5) with ESMTP id h3B333hW133414;
	Thu, 10 Apr 2003 21:03:03 -0600
To: Black_David@emc.com
Cc: rddp@ietf.org, rddp-admin@ietf.org
MIME-Version: 1.0
Subject: RE: [rddp] Option or mandatory CRC in MPA - WG Chair speaks
X-Mailer: Lotus Notes Release 5.0.11   July 24, 2002
From: John Hufferd <hufferd@us.ibm.com>
Message-ID: <OFF84E4CCE.2340B031-ON88256D05.000FA127-88256D05.0010B969@us.ibm.com>
Date: Thu, 10 Apr 2003 19:58:53 -0700
X-MIMETrack: Serialize by Router on D03NM115/03/M/IBM(Release 6.0 [IBM]|December 16, 2002) at
 04/10/2003 20:58:49,
	Serialize complete at 04/10/2003 20:58:49
Content-Type: multipart/alternative; boundary="=_alternative 0010845688256D05_="
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

This is a multipart message in MIME format.
--=_alternative 0010845688256D05_=
Content-Type: text/plain; charset="us-ascii"

David,
I think you read to much into this.  It had nothing to do with the 
alignment issues.  That discussion had valid points independent of the 
arguments being made about CRCs.  We have folks debating the CPU overhead 
with CRC,  and trying to add things which OPTIMIZE software implementation 
for handling CRCs (like not doing it).  That was the issue and what was 
talked about in this note, NOT alignment!  I know that SW will exist, but 
trying to optimize for them is another issue.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Home Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: hufferd@us.ibm.com




Black_David@emc.com
Sent by: rddp-admin@ietf.org
04/10/2003 06:14 PM

 
        To:     John Hufferd/San Jose/IBM@IBMUS, rddp@ietf.org
        cc: 
        Subject:        RE: [rddp] Option or mandatory CRC in MPA - WG Chair speaks



> I find some of these arguments somewhat strange.  We have all entered
> into this effort to bring optimized processing with Remote Direct Data
> Placement, yet we continue to talk about, and give serious weight to,
> the argument concerning an all software implementation.  It seems that
> we might think about a complete software implementation, in passing,
> but this is NOT the goal of RDMA or this RDDP workgroup (in my opinion).

I need to say some fairly strong words on this.  That statement and
the implication that only hardware implementations need apply are at
direct odds with the discussion leading to WG rough consensus on the
ULPDU alignment with TCP segment issue, and the resulting rough consensus.
It is very clear that the discussion leading to that rough consensus:
- Emphasized the ability to run over existing unmodified TCP stacks.
- Emphasized the importance of fan-in scenarios in which a number
                 of software clients communicated with a 
hardware-accelerated
                 server.
I believe both of those represent the rough consensus of the RDDP
WG.  To the extent that they do not, we're back in the alignment
morass, as those were crucial to settling the alignment issue, so ...

I am putting the RDDP WG on notice that if this line of discussion
leads to a need to reopen the alignment issue, I may have no choice
but to conclude that the technical issues involved in RDDP over TCP
are of sufficient intractability that I need to recommend to the ADs
that the pending RDDP recharter should delete all TCP work. 

> Most of the applications that will be using the RDMA protocols already
> have existing TCP/IP implementations, and they are not enhanced or
> accelerated by a software implementation of RDMA.  If systems only
> have a standard NIC, then the applications should continue to use 
TCP/IP.

That needs a lot more discussion, as among other things it excludes
most NFS clients (by count of systems, not count of independently
developed implementations).  This reminds me of the early days of iSCSI
work when the "conventional wisdom" was that iSCSI was going to need
accelerated hardware - at the moment software iSCSI implementations
are coming in with acceptable performance, albeit not as good as could
be achieved in hardware.

It is fine to discuss whether the CRC should be optional, but that
discussion MUST respect the WG rough consensus that *finally* closed
the alignment issue if any progress is going to be made.  Consideration
of all-software clients and their requirements is necessitated by the
current state of the RDDP WG, unless people want to reopen the alignment
issue (see above if you want to do that).

Thanks,
--David
----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
black_david@emc.com        Mobile: +1 (978) 394-7754
----------------------------------------------------
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



--=_alternative 0010845688256D05_=
Content-Type: text/html; charset="us-ascii"


<br><font size=2 face="sans-serif">David,</font>
<br><font size=2 face="sans-serif">I think you read to much into this. &nbsp;It had nothing to do with the alignment issues. &nbsp;That discussion had valid points independent of the arguments being made about CRCs. &nbsp;We have folks debating the CPU overhead with CRC, &nbsp;and trying to add things which OPTIMIZE software implementation for handling CRCs (like not doing it). &nbsp;That was the issue and what was talked about in this note, NOT alignment! &nbsp;I know that SW will exist, but trying to optimize for them is another issue.</font>
<br><font size=2 face="sans-serif">.<br>
.<br>
John L. Hufferd<br>
Senior Technical Staff Member (STSM)<br>
IBM/System Group, San Jose CA<br>
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688<br>
Home Office: (408) 997-6136, Cell: (408) 499-9702<br>
Internet Address: hufferd@us.ibm.com</font>
<br>
<br>
<br>
<table width=100%>
<tr valign=top>
<td>
<td><font size=1 face="sans-serif"><b>Black_David@emc.com</b></font>
<br><font size=1 face="sans-serif">Sent by: rddp-admin@ietf.org</font>
<p><font size=1 face="sans-serif">04/10/2003 06:14 PM</font>
<br>
<td><font size=1 face="Arial">&nbsp; &nbsp; &nbsp; &nbsp; </font>
<br><font size=1 face="sans-serif">&nbsp; &nbsp; &nbsp; &nbsp; To: &nbsp; &nbsp; &nbsp; &nbsp;John Hufferd/San Jose/IBM@IBMUS, rddp@ietf.org</font>
<br><font size=1 face="sans-serif">&nbsp; &nbsp; &nbsp; &nbsp; cc: &nbsp; &nbsp; &nbsp; &nbsp;</font>
<br><font size=1 face="sans-serif">&nbsp; &nbsp; &nbsp; &nbsp; Subject: &nbsp; &nbsp; &nbsp; &nbsp;RE: [rddp] Option or mandatory CRC in MPA - WG Chair speaks</font>
<br></table>
<br>
<br>
<br><font size=2 face="Courier New">&gt; I find some of these arguments somewhat strange. &nbsp;We have all entered<br>
&gt; into this effort to bring optimized processing with Remote Direct Data<br>
&gt; Placement, yet we continue to talk about, and give serious weight to,<br>
&gt; the argument concerning an all software implementation. &nbsp;It seems that<br>
&gt; we might think about a complete software implementation, in passing,<br>
&gt; but this is NOT the goal of RDMA or this RDDP workgroup (in my opinion).<br>
<br>
I need to say some fairly strong words on this. &nbsp;That statement and<br>
the implication that only hardware implementations need apply are at<br>
direct odds with the discussion leading to WG rough consensus on the<br>
ULPDU alignment with TCP segment issue, and the resulting rough consensus.<br>
It is very clear that the discussion leading to that rough consensus:<br>
- Emphasized the ability to run over existing unmodified TCP stacks.<br>
- Emphasized the importance of fan-in scenarios in which a number<br>
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; of software clients communicated with a hardware-accelerated<br>
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; server.<br>
I believe both of those represent the rough consensus of the RDDP<br>
WG. &nbsp;To the extent that they do not, we're back in the alignment<br>
morass, as those were crucial to settling the alignment issue, so ...<br>
<br>
I am putting the RDDP WG on notice that if this line of discussion<br>
leads to a need to reopen the alignment issue, I may have no choice<br>
but to conclude that the technical issues involved in RDDP over TCP<br>
are of sufficient intractability that I need to recommend to the ADs<br>
that the pending RDDP recharter should delete all TCP work. &nbsp;<br>
<br>
&gt; Most of the applications that will be using the RDMA protocols already<br>
&gt; have existing TCP/IP implementations, and they are not enhanced or<br>
&gt; accelerated by a software implementation of RDMA. &nbsp;If systems only<br>
&gt; have a standard NIC, then the applications should continue to use TCP/IP.<br>
<br>
That needs a lot more discussion, as among other things it excludes<br>
most NFS clients (by count of systems, not count of independently<br>
developed implementations). &nbsp;This reminds me of the early days of iSCSI<br>
work when the &quot;conventional wisdom&quot; was that iSCSI was going to need<br>
accelerated hardware - at the moment software iSCSI implementations<br>
are coming in with acceptable performance, albeit not as good as could<br>
be achieved in hardware.<br>
<br>
It is fine to discuss whether the CRC should be optional, but that<br>
discussion MUST respect the WG rough consensus that *finally* closed<br>
the alignment issue if any progress is going to be made. &nbsp;Consideration<br>
of all-software clients and their requirements is necessitated by the<br>
current state of the RDDP WG, unless people want to reopen the alignment<br>
issue (see above if you want to do that).<br>
<br>
Thanks,<br>
--David<br>
----------------------------------------------------<br>
David L. Black, Senior Technologist<br>
EMC Corporation, 176 South St., Hopkinton, MA &nbsp;01748<br>
+1 (508) 293-7953 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; FAX: +1 (508) 293-7786<br>
black_david@emc.com &nbsp; &nbsp; &nbsp; &nbsp;Mobile: +1 (978) 394-7754<br>
----------------------------------------------------<br>
_______________________________________________<br>
rddp mailing list<br>
rddp@ietf.org<br>
https://www1.ietf.org/mailman/listinfo/rddp<br>
</font>
<br>
<br>
--=_alternative 0010845688256D05_=--
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Fri Apr 11 00:45:46 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA17868
	for <rddp-archive@odin.ietf.org>; Fri, 11 Apr 2003 00:45:46 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3B4pkk13012
	for rddp-archive@odin.ietf.org; Fri, 11 Apr 2003 00:51:46 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3B4pk813009
	for <rddp-web-archive@optimus.ietf.org>; Fri, 11 Apr 2003 00:51:46 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA17853
	for <rddp-web-archive@ietf.org>; Fri, 11 Apr 2003 00:45:15 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 193q9U-0006aL-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 00:28:36 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 193q9U-0006aI-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 00:28:36 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3B4p7812996;
	Fri, 11 Apr 2003 00:51:07 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3B4on812972
	for <rddp@optimus.ietf.org>; Fri, 11 Apr 2003 00:50:49 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA17850
	for <rddp@ietf.org>; Fri, 11 Apr 2003 00:44:18 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 193q8Z-0006aF-00
	for rddp@ietf.org; Fri, 11 Apr 2003 00:27:39 -0400
Received: from palrel12.hp.com ([156.153.255.237])
	by ietf-mx with esmtp (Exim 4.12)
	id 193q8Z-0006aC-00
	for rddp@ietf.org; Fri, 11 Apr 2003 00:27:39 -0400
Received: from esmail.cup.hp.com (esmail.cup.hp.com [15.0.65.164])
	by palrel12.hp.com (Postfix) with ESMTP id 43DE51C015B1
	for <rddp@ietf.org>; Thu, 10 Apr 2003 21:46:51 -0700 (PDT)
Received: from mk731915.cup.hp.com ([15.113.161.2])
	by esmail.cup.hp.com (8.11.1/8.8.6) with ESMTP id h3B4hJl29734
	for <rddp@ietf.org>; Thu, 10 Apr 2003 21:43:19 -0700 (PDT)
Message-Id: <5.1.0.14.2.20030410213912.08daea20@15.13.130.73>
X-Sender: krause@15.13.130.73
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Thu, 10 Apr 2003 21:46:45 -0700
To: <rddp@ietf.org>
From: Michael Krause <krause@cup.hp.com>
Subject: RE: [rddp] Option or mandatory CRC in MPA
In-Reply-To: <NMEALCLOIBCHBDHLCMIJAEGMDDAA.somesh_gupta@silverbacksystem
 s.com>
References: <5.1.0.14.2.20030410133602.01ffe130@15.13.130.73>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>


>
> > Not clear.  Whether one walks memory to do a checksum calculation, a CRC
> > calculation, or a copy operation, the net effect is the processor data
> > cache is made hot with these cache lines.  Given applications
> > typically ask
> > for data once it has arrived, the fact that is in cache mitigates
> > the cache
> > load latency thus the subsequent performance impact.   I do not
> > see this as
> > creating a significant performance impact.  Do you have data to show that
> > there is such an impact?
>
>   I think this brings up a number of interesting points:
>
>1. We should assume checksum is offloaded as fairly all NICs do
>    it now.

It is common for single segment, IPv4 packets.  It is not common for IPv4 
fragments or for IPv6 (IPv6 is complicated by the next header field making 
it more difficult to support checksum off-load as the number of headers, 
etc. is somewhat unpredictable).

>2. There are a certain class of apps that do not touch the data
>    but just shunt it from one place to another. A classic example
>    is any kind of storage server. These are actually the ones
>    that benefit from zero copy the most and may use means other
>    than RDDP to get the benefit (i.e. handle SGLs)

This primarily is true for the outbound path where one encapsulates the 
file system or storage block.  The inbound path usually requires a copy 
operation from the mbuf to the storage block / file system due to the 
deltas in size between the layer 2 packet and the blocks up above.

>3. Obviously there are many other apps that chew on the data.
>    In those cases, there is just a whole range of factors that
>    come into play. The rate at which data can be burst vs the
>    apps consumption of data, the size of the data caches etc
>    etc. In a number of cases, it does not matter how much
>    network acceleration you provide - the processing of the data
>    consumes bulk of the cycles anyway.

Acceleration does play a role especially when it comes to RDMA because the 
data is placed in the right place from the start thus the correct thread / 
processor is always chosen to process the data.  This is very helpful for 
inbound traffic.  Outbound just takes advantage of copy avoidance if there 
is OS bypass or special mappings to allow a kernel network stack to see 
into user-space (not always true or simple to support on many OS offerings).

> > I thought there was general consensus that the focus of RDMA is hardware
> > acceleration and that it must be able to be implemented in
> > software.  There were no requirements put forth that I'm aware of that 
> require
> > software and  hardware to have an equal footing especially as some 
> operations such as a
> > RDMA Read will always outperform a software implementation as no
> > processor interactions are required when implemented in hardware.
>
>   The statement of requirements should be very explicit on this.
>   We can all agree that hardware acceleration will provide higher
>   performance under all cases, but that is not the point.

I believe there was a question of whether the two types of implementations 
were on equal footing - if one performs potentially radically differently 
than the other and has additional functionality that while possible to 
emulate in software is clearly not the same in terms of remote endnode 
interaction / solution cost, then it is not possible to be on equal footing 
and thus the motivation to compromise on functionality may be quite a bit 
less given this delta.

Mike

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Fri Apr 11 10:23:45 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA14449
	for <rddp-archive@odin.ietf.org>; Fri, 11 Apr 2003 10:23:45 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3BETu330262
	for rddp-archive@odin.ietf.org; Fri, 11 Apr 2003 10:29:56 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BETt830259
	for <rddp-web-archive@optimus.ietf.org>; Fri, 11 Apr 2003 10:29:55 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA14408
	for <rddp-web-archive@ietf.org>; Fri, 11 Apr 2003 10:23:14 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 193zAo-0002YJ-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 10:06:34 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 193zAn-0002YG-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 10:06:33 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BETC830153;
	Fri, 11 Apr 2003 10:29:12 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BENk829819
	for <rddp@optimus.ietf.org>; Fri, 11 Apr 2003 10:23:46 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA14164
	for <rddp@ietf.org>; Fri, 11 Apr 2003 10:17:04 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 193z4q-0002Ut-00
	for rddp@ietf.org; Fri, 11 Apr 2003 10:00:24 -0400
Received: from ztxmail05.ztx.compaq.com ([161.114.1.209])
	by ietf-mx with esmtp (Exim 4.12)
	id 193z4p-0002Ul-00
	for rddp@ietf.org; Fri, 11 Apr 2003 10:00:23 -0400
Received: from cceexg12.americas.cpqcorp.net (cceexg12.americas.cpqcorp.net [16.110.250.124])
	by ztxmail05.ztx.compaq.com (Postfix) with ESMTP
	id 6C7538477; Fri, 11 Apr 2003 09:19:07 -0500 (CDT)
Received: from cceexc18.americas.cpqcorp.net ([16.110.250.64]) by cceexg12.americas.cpqcorp.net with Microsoft SMTPSVC(5.0.2195.2966);
	 Fri, 11 Apr 2003 09:12:38 -0500
X-MimeOLE: Produced By Microsoft Exchange V6.0.6375.0
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Subject: RE: [rddp] Option or mandatory CRC in MPA - WG Chair speaks
Date: Fri, 11 Apr 2003 09:12:37 -0500
Message-ID: <4D027986353D1341ADA4F2A4F8A170762A050637@cceexc18.americas.cpqcorp.net>
Thread-Topic: [rddp] Option or mandatory CRC in MPA - WG Chair speaks
Thread-Index: AcL/yaqbce/sYT2CRje0hoyGJbXFHQAZQkcw
From: "Culley, Paul" <Paul.Culley@hp.com>
To: <Black_David@emc.com>, <hufferd@us.ibm.com>, <rddp@ietf.org>
X-OriginalArrivalTime: 11 Apr 2003 14:12:38.0295 (UTC) FILETIME=[68329270:01C30034]
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by www1.ietf.org id h3BENk829820
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 8bit
Content-Transfer-Encoding: 8bit

I agree that the WG consensus was to "The RDDP protocols must allow
implementations over unmodified TCP/IP stacks." (as quoted from your
"Alignment consensus attempt" email.

I was unable to locate any minutes describing the fan-in scenarios that
would lead to a conclusion of "consensus".  That said, I do believe that
such scenarios will exist.

In both cases I am not quite so sure that the group agreed to
"emphasize" operation over Unmodified SW stacks; rather, I for one, only
think these should be enabled.  Perhaps I am reading too much into your
choice of words.

In my mind "enabling" these scenarios, while important, should not mean
that we go to great lengths to optimize or reduce overhead on the
software side at the expense of making the overall protocol less robust,
or adding multiple options that applications will have to sort through
and all concerned will have to test.

I believe that most of the scenarios where clients might be implemented
in software will also be able to afford the relatively small extra
overhead that use of CRCs requires.  These clients will already be
incurring the overhead of copies on receive, so only CPU time (not cache
dirties or extra memory cycles) should be added in reasonably good code.
The transmit side may incur be a bit more overhead, but I note that many
client applications are asymmetrical anyway, they receive more than they
send.

Also, I firmly believe that we cannot completely drop CRCs, that
essentially all customers will be quite nervous about allowing memory
access by DDP headers that have not been checked by something better
than a checksum.  So there will be that overhead anyway.  I think that
checking the remaining user data will be important enough to the
majority of customers that putting the option in for the minority that
don't care that much is just not worth it.

Paul R. Culley
HP Senior Fellow
281-514-5543


-----Original Message-----
From: Black_David@emc.com [mailto:Black_David@emc.com] 
Sent: Thursday, April 10, 2003 8:15 PM
To: hufferd@us.ibm.com; rddp@ietf.org
Subject: RE: [rddp] Option or mandatory CRC in MPA - WG Chair speaks


> I find some of these arguments somewhat strange.  We have all entered 
> into this effort to bring optimized processing with Remote Direct Data

> Placement, yet we continue to talk about, and give serious weight to, 
> the argument concerning an all software implementation.  It seems that

> we might think about a complete software implementation, in passing, 
> but this is NOT the goal of RDMA or this RDDP workgroup (in my 
> opinion).

I need to say some fairly strong words on this.  That statement and the
implication that only hardware implementations need apply are at direct
odds with the discussion leading to WG rough consensus on the ULPDU
alignment with TCP segment issue, and the resulting rough consensus. It
is very clear that the discussion leading to that rough consensus:
- Emphasized the ability to run over existing unmodified TCP stacks.
- Emphasized the importance of fan-in scenarios in which a number
	of software clients communicated with a hardware-accelerated
	server.
I believe both of those represent the rough consensus of the RDDP WG.
To the extent that they do not, we're back in the alignment morass, as
those were crucial to settling the alignment issue, so ...

I am putting the RDDP WG on notice that if this line of discussion leads
to a need to reopen the alignment issue, I may have no choice but to
conclude that the technical issues involved in RDDP over TCP are of
sufficient intractability that I need to recommend to the ADs that the
pending RDDP recharter should delete all TCP work.  

> Most of the applications that will be using the RDMA protocols already

> have existing TCP/IP implementations, and they are not enhanced or 
> accelerated by a software implementation of RDMA.  If systems only 
> have a standard NIC, then the applications should continue to use 
> TCP/IP.

That needs a lot more discussion, as among other things it excludes most
NFS clients (by count of systems, not count of independently developed
implementations).  This reminds me of the early days of iSCSI work when
the "conventional wisdom" was that iSCSI was going to need accelerated
hardware - at the moment software iSCSI implementations are coming in
with acceptable performance, albeit not as good as could be achieved in
hardware.

It is fine to discuss whether the CRC should be optional, but that
discussion MUST respect the WG rough consensus that *finally* closed the
alignment issue if any progress is going to be made.  Consideration of
all-software clients and their requirements is necessitated by the
current state of the RDDP WG, unless people want to reopen the alignment
issue (see above if you want to do that).

Thanks,
--David
----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
black_david@emc.com        Mobile: +1 (978) 394-7754
----------------------------------------------------
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Fri Apr 11 12:54:33 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA21036
	for <rddp-archive@odin.ietf.org>; Fri, 11 Apr 2003 12:54:33 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3BH0mL09945
	for rddp-archive@odin.ietf.org; Fri, 11 Apr 2003 13:00:48 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BH0m809942
	for <rddp-web-archive@optimus.ietf.org>; Fri, 11 Apr 2003 13:00:48 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA21015
	for <rddp-web-archive@ietf.org>; Fri, 11 Apr 2003 12:54:03 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1941pO-0004C5-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 12:56:38 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 1941pO-0004C2-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 12:56:38 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BH08809904;
	Fri, 11 Apr 2003 13:00:08 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BGxK809837
	for <rddp@optimus.ietf.org>; Fri, 11 Apr 2003 12:59:20 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA20976
	for <rddp@ietf.org>; Fri, 11 Apr 2003 12:52:35 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1941ny-0004Be-00
	for rddp@ietf.org; Fri, 11 Apr 2003 12:55:10 -0400
Received: from dmz1.silverbacksystems.com ([65.172.158.82])
	by ietf-mx with esmtp (Exim 4.12)
	id 1941ny-0004B3-00
	for rddp@ietf.org; Fri, 11 Apr 2003 12:55:10 -0400
Received: from ns.silverbacksystems.com (gate-camp-hme0.silverbacksystems.com [65.172.158.93])
	by dmz1.silverbacksystems.com (Postfix on SuSE Linux 7.3 (i386)) with ESMTP id 74528D36E
	for <rddp@ietf.org>; Fri, 11 Apr 2003 09:54:26 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1])
	by ns.silverbacksystems.com (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 50FD412F7F
	for <rddp@ietf.org>; Fri, 11 Apr 2003 09:54:26 -0700 (PDT)
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
	charset="us-ascii"
Date: Fri, 11 Apr 2003 09:54:18 -0700
From: "Somesh Gupta" <somesh_gupta@silverbacksystems.com>
Importance: Normal
In-Reply-To: <5.1.0.14.2.20030410213912.08daea20@15.13.130.73>
Message-ID: <NMEALCLOIBCHBDHLCMIJMEHCDDAA.somesh_gupta@silverbacksystems.com>
MIME-Version: 1.0
Received: from ns.silverbacksystems.com (localhost [127.0.0.1])
	by localhost (AvMailGate-2.0.0.6) id 11082-1331B806;
	Fri, 11 Apr 2003 09:54:05 -0700
Received: from somesh (somesh.corp.silverbacksystems.com [10.0.16.35])
	by ns.silverbacksystems.com (Postfix on SuSE Linux eMail Server 3.0) with SMTP id 84C1C6F75
	for <rddp@ietf.org>; Fri, 11 Apr 2003 09:54:05 -0700 (PDT)
Subject: RE: [rddp] Option or mandatory CRC in MPA
To: <rddp@ietf.org>
X-AntiVirus: OK! AntiVir MailGate Version 2.0.0.6
	 at ns.silverbacksystems.com has not found any known virus in this email.
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700
X-MSMail-Priority: Normal
X-Priority: 3 (Normal)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit



> -----Original Message-----
> From: rddp-admin@ietf.org [mailto:rddp-admin@ietf.org]On Behalf Of
> Michael Krause
> Sent: Thursday, April 10, 2003 9:47 PM
> To: rddp@ietf.org
> Subject: RE: [rddp] Option or mandatory CRC in MPA
>
>
>
> >
> > > Not clear.  Whether one walks memory to do a checksum
> calculation, a CRC
> > > calculation, or a copy operation, the net effect is the processor data
> > > cache is made hot with these cache lines.  Given applications
> > > typically ask
> > > for data once it has arrived, the fact that is in cache mitigates
> > > the cache
> > > load latency thus the subsequent performance impact.   I do not
> > > see this as
> > > creating a significant performance impact.  Do you have data
> to show that
> > > there is such an impact?
> >
> >   I think this brings up a number of interesting points:
> >
> >1. We should assume checksum is offloaded as fairly all NICs do
> >    it now.
>
> It is common for single segment, IPv4 packets.  It is not common for IPv4
> fragments or for IPv6 (IPv6 is complicated by the next header
> field making
> it more difficult to support checksum off-load as the number of headers,
> etc. is somewhat unpredictable).

  The IPV4 single segment continues to be > 99% of the traffic over TCP.
  And many of the NICs do support checksum for IPV6 (the driver may not).
  The host may be required to fix some stuff.

>
> >2. There are a certain class of apps that do not touch the data
> >    but just shunt it from one place to another. A classic example
> >    is any kind of storage server. These are actually the ones
> >    that benefit from zero copy the most and may use means other
> >    than RDDP to get the benefit (i.e. handle SGLs)
>
> This primarily is true for the outbound path where one encapsulates the
> file system or storage block.  The inbound path usually requires a copy
> operation from the mbuf to the storage block / file system due to the
> deltas in size between the layer 2 packet and the blocks up above.

  Since the SCSI/FC controllers do support scatter-gather lists, it
  is a buffer management issue. With TOE and header-data split, those
  that do not want to complicate their buffer management can get the
  benefits without a change to the wire protocol.

>
> >3. Obviously there are many other apps that chew on the data.
> >    In those cases, there is just a whole range of factors that
> >    come into play. The rate at which data can be burst vs the
> >    apps consumption of data, the size of the data caches etc
> >    etc. In a number of cases, it does not matter how much
> >    network acceleration you provide - the processing of the data
> >    consumes bulk of the cycles anyway.
>
> Acceleration does play a role especially when it comes to RDMA
> because the
> data is placed in the right place from the start thus the correct
> thread /
> processor is always chosen to process the data.  This is very helpful for
> inbound traffic.  Outbound just takes advantage of copy avoidance
> if there
> is OS bypass or special mappings to allow a kernel network stack to see
> into user-space (not always true or simple to support on many OS
> offerings).
>
> > > I thought there was general consensus that the focus of RDMA
> is hardware
> > > acceleration and that it must be able to be implemented in
> > > software.  There were no requirements put forth that I'm
> aware of that
> > require
> > > software and  hardware to have an equal footing especially as some
> > operations such as a
> > > RDMA Read will always outperform a software implementation as no
> > > processor interactions are required when implemented in hardware.
> >
> >   The statement of requirements should be very explicit on this.
> >   We can all agree that hardware acceleration will provide higher
> >   performance under all cases, but that is not the point.
>
> I believe there was a question of whether the two types of
> implementations
> were on equal footing - if one performs potentially radically differently
> than the other and has additional functionality that while possible to
> emulate in software is clearly not the same in terms of remote endnode
> interaction / solution cost, then it is not possible to be on
> equal footing
> and thus the motivation to compromise on functionality may be quite a bit
> less given this delta.
>
> Mike
>
> _______________________________________________
> rddp mailing list
> rddp@ietf.org
> https://www1.ietf.org/mailman/listinfo/rddp

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Fri Apr 11 13:04:30 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA21358
	for <rddp-archive@odin.ietf.org>; Fri, 11 Apr 2003 13:04:30 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3BHAjs11226
	for rddp-archive@odin.ietf.org; Fri, 11 Apr 2003 13:10:45 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BHAj811223
	for <rddp-web-archive@optimus.ietf.org>; Fri, 11 Apr 2003 13:10:45 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA21350
	for <rddp-web-archive@ietf.org>; Fri, 11 Apr 2003 13:03:59 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1941z0-0004G9-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 13:06:34 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 1941yz-0004G6-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 13:06:33 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BHA2811197;
	Fri, 11 Apr 2003 13:10:02 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BH9X811170
	for <rddp@optimus.ietf.org>; Fri, 11 Apr 2003 13:09:33 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA21327
	for <rddp@ietf.org>; Fri, 11 Apr 2003 13:02:48 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1941xr-0004G3-00
	for rddp@ietf.org; Fri, 11 Apr 2003 13:05:23 -0400
Received: from thebe.your-site.com ([140.186.45.26])
	by ietf-mx with esmtp (Exim 4.12)
	id 1941xr-0004Fp-00
	for rddp@ietf.org; Fri, 11 Apr 2003 13:05:23 -0400
Received: from 192.168.0.2 (64-144-5-25.client.dsl.net [64.144.5.25])
	by thebe.your-site.com (Postfix) with ESMTP id 6F35B244C65
	for <rddp@ietf.org>; Fri, 11 Apr 2003 13:05:02 -0400 (EDT)
Date: Fri, 11 Apr 2003 12:07:15 -0500
From: Caitlin Bestler <cait@asomi.com>
To: rddp@ietf.org
X-Priority: 3
Message-ID: <r01050300-1024-0B59DE0F6C4011D78DE4003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Subject: [rddp] Optimized Processing not necessarily Hardware Optimized
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

David chose the following paragraph from John Hufferd.

> I find some of these arguments somewhat strange.  We have
> all entered into this effort to bring optimized
> processing with Remote Direct Data Placement, yet we
> continue to talk about, and give serious weight to, the
> argument concerning an all software implementation.  It
> seems that we might think about a complete software
> implementation, in passing, but this is NOT the goal of
> RDMA or this RDDP workgroup (in my opinion).

Read carefully, the statement correctly states that the
purpose of RDDP is optimized processing. It does not state
that the goal is hardware optimized processing, allthough I
think most people read it that way.

This is a very critical distinction.

In my opinion, the cause of RDMA has been hindered by
arguments made on its behalf that assume it will be
implemented in hardware. Indeed, that its implementability
in hardware is a justification for RDMA.

Ultimately, a justification for RDMA that current stacks
result in too many context switches do not justify the
development of a new transport protocol. They justify the
development of new motherboards.

But RDMA is not merely a hardware optimization. It
fundamentally improves the collaboration between. It
allows two hosts to achieve the same results with less
work, no matter how the hosts are organized/built.

With RDMA a host (through whatver combination of hardware,
firmware, kernel drivers, co-processors and software
libraries):

    is able to directly place the payload from received DDP     
    Segments into user memory, even if the DDP segments are 
    received out of order.
    
    has fewer required interactions with ULP software.
    
    is able to send content in the order most convenient to
    the Data Source, not in an arbitrary order that the
    protocol requires the Data Sink to dictate.
    
It is these fundamental optimizations that justify RDMA.
The fact that they are simple powerful algorithms is *why*
they can be implemented in hardware. But their hardware
implementability is the result of their strength, not the
other way around.


I would like to examine the question of "optional" CRCs in a
similar way. What is the fundamental difference in the work
that is required with or without mandatory CRCs? To what
extent does the proposed protocol require work that is
an undue burden for some applications?

As many have already argued, the need for payload validity
checking increases with network speed. While there are a
handlful of applications that do not require protection
against payload corruption (including many that I have
worked on such as streaming media), this is clearly an
exception. 

At the speeds we are envisioning supporting, payload
validity checking is not a specialized requirement of
storage applications being imposed upon everyone else.
The question is more whether the handful of appliations that
do not need it should be able to impose it being variable.

I believe the impact on the applications that can live
with data loss is very slight, in addition to those
applications representing a small portion of total
network usage.

First, the term "optional" CRC coverage is misleading.
CRC coverage for the DDP header is mandatory. RDDP cannot
responsibly place data directly into user buffers based
soley upon TCP validation.

That would require the application to confine the Protection
Domain to only those buffers that could be corrupted. With
only TCP protection, all buffers exposed through the
Protection Domain would be subject to being falsely
overwritten by any flawed TCP segment that escaped
detection.

Working around this would require additional Protection
Domains, and hence additional connections. Clearly this is
unacceptable.

So the "optional" CRC is really a *shortened* CRC. It covers
only the header length, rather than the full length.

This is additional complexity, and requires a mode-setup
message exchange at the start of a connection (although
one-way messages in each direction are adequate, so at least
no additional round trip is mandated).

All receivers, still must be able to calculate CRCs.

The only difference is that some of them may be able to stop
after the DDP header, while most application will want to
cover the entire FULPDU.

What is the benefit in stopping, however? What is the undue
burden being imposed on a "software implementation"?

The basic argument is that calculating the full CRC will
result in *additional* memory touches.

But examine that more closely. Obviously the memory is
touched. It has to be copied to the user buffer.

The reason that this is an "additional" touch is that on
many systems there is already an *existing* hardware
solution to perform that memory copy, but that *existing*
hardware solution does not know how to perform a CRC on an
MPA FULPDU.

Further, that existing hardware *does* know how to validate
a TCP segment. If it didn't the pass to validate the TCP
segment could also validate the MPA FULPDU, so there would
be no "additional" memory touch required at all.

As I stated earlier, if the only problem with TCP/SCTP and
other existing transports is that they required too many
conext switches, then the correct response would be that new
motherboards needed to be revised, not new transports.

Is the only justification for a shortened CRC that many
motherboards have dedicated co-processors ("DMA Engines")
that understand TCP validation but not MPA?

If so, I would argue that we are complicating the protocol
for the benefit of *existing* hardware.

Clearly, calculating the CRCs is possible because it will
have to be done for the duration of the header and for the
vast majority of payload anyway.

The required processing is very slight, so the only real
issue is the "extra" memory touch. As Mike Krause pointed
out earlier, in many cases cache dynamic will eliminate that
overhead anyway. And as just covered, it is only
"additional" because of the limitations of *current*
hardware designs.

The overhead is minor *and* transitional. It provides no
justification for the complexity of variable CRC coverage.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Fri Apr 11 13:17:24 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA21949
	for <rddp-archive@odin.ietf.org>; Fri, 11 Apr 2003 13:17:24 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3BHNeJ12017
	for rddp-archive@odin.ietf.org; Fri, 11 Apr 2003 13:23:40 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BHNd812014
	for <rddp-web-archive@optimus.ietf.org>; Fri, 11 Apr 2003 13:23:39 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA21941
	for <rddp-web-archive@ietf.org>; Fri, 11 Apr 2003 13:16:53 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1942BV-0004OY-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 13:19:29 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 1942BV-0004OV-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 13:19:29 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BHN1811965;
	Fri, 11 Apr 2003 13:23:01 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BHN0811951
	for <rddp@optimus.ietf.org>; Fri, 11 Apr 2003 13:23:00 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA21911
	for <rddp@ietf.org>; Fri, 11 Apr 2003 13:16:14 -0400 (EDT)
From: Black_David@emc.com
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1942As-0004O5-00
	for rddp@ietf.org; Fri, 11 Apr 2003 13:18:50 -0400
Received: from [128.222.32.10] (helo=mxic1.corp.emc.com)
	by ietf-mx with esmtp (Exim 4.12)
	id 1942Ar-0004O1-00
	for rddp@ietf.org; Fri, 11 Apr 2003 13:18:49 -0400
Received: by mxic1.corp.emc.com with Internet Mail Service (5.5.2653.19)
	id <2PQXVLTQ>; Fri, 11 Apr 2003 13:18:19 -0400
Message-ID: <277DD60FB639D511AC0400B0D068B71E0564CB0C@corpmx14.us.dg.com>
To: Paul.Culley@hp.com, rddp@ietf.org
Subject: RE: [rddp] Option or mandatory CRC in MPA - WG Chair speaks
Date: Fri, 11 Apr 2003 13:13:16 -0400
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

> I agree that the WG consensus was to "The RDDP protocols must allow
> implementations over unmodified TCP/IP stacks." (as quoted from your
> "Alignment consensus attempt" email.
> 
> I was unable to locate any minutes describing the fan-in scenarios that
> would lead to a conclusion of "consensus".  That said, I do believe that
> such scenarios will exist.

Good - I hope I don't have to spend the time to extract the relevant
supporting discussions from the WG email list archives.  

> In both cases I am not quite so sure that the group agreed to
> "emphasize" operation over Unmodified SW stacks; rather, I for one, only
> think these should be enabled.  Perhaps I am reading too much into your
> choice of words.

Perhaps - my use of the word "emphasize" was solely with respect to
issues that were important in establishing rough consensus on
the alignment issue.  Unmodified software stacks are important, but
my post was not about any notion of "emphasizing" them to the exclusion
of other implementation structures. 

> In my mind "enabling" these scenarios, while important, should not mean
> that we go to great lengths to optimize or reduce overhead on the
> software side

That's ok - somewhere in the range from "enabling" to "go to great
lengths" (including the endpoints) there should be an engineering
tradeoff that will yield a protocol that is reasonably implementable
in both hardware and software.  This is in stark contrast to John's
original call to ignore considerations for software implementations:

	It seems that we might think about a complete software
	implementation, in passing, but this is NOT the goal of
	RDMA or this RDDP workgroup (in my opinion).

Just in case it's not clear - while John is entitled to his opinion,
the immediately above statement is simply wrong about the IETF RDDP
WG, as it is at odds with prior WG rough consensus.  Attempting to
change the position of the WG to match the quoted statement carries
significant risk of unraveling the rough consensus on the alignment
issue, which may have dramatic consequences for the scope of the WG's
efforts.  That's the clue-by-four - I would advise against inviting
the use of anything larger ...

Paul's email goes on to touch on a couple of other issues that
I'm going to take up under different subject lines.

Thanks,
--David
----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
black_david@emc.com        Mobile: +1 (978) 394-7754
----------------------------------------------------
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Fri Apr 11 13:38:30 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA22472
	for <rddp-archive@odin.ietf.org>; Fri, 11 Apr 2003 13:38:30 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3BHike13742
	for rddp-archive@odin.ietf.org; Fri, 11 Apr 2003 13:44:46 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BHik813739
	for <rddp-web-archive@optimus.ietf.org>; Fri, 11 Apr 2003 13:44:46 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA22441
	for <rddp-web-archive@ietf.org>; Fri, 11 Apr 2003 13:38:00 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1942Vv-0004W6-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 13:40:35 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 1942Vv-0004W3-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 13:40:35 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BHi2813671;
	Fri, 11 Apr 2003 13:44:02 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BHem813546
	for <rddp@optimus.ietf.org>; Fri, 11 Apr 2003 13:40:48 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA22363
	for <rddp@ietf.org>; Fri, 11 Apr 2003 13:34:01 -0400 (EDT)
From: Black_David@emc.com
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1942S5-0004Uw-00
	for rddp@ietf.org; Fri, 11 Apr 2003 13:36:37 -0400
Received: from mxic2.corp.emc.com ([128.221.31.40])
	by ietf-mx with esmtp (Exim 4.12)
	id 1942S4-0004Uf-00
	for rddp@ietf.org; Fri, 11 Apr 2003 13:36:36 -0400
Received: by mxic2.corp.emc.com with Internet Mail Service (5.5.2653.19)
	id <2JMPX2DT>; Fri, 11 Apr 2003 13:36:03 -0400
Message-ID: <277DD60FB639D511AC0400B0D068B71E0564CB0D@corpmx14.us.dg.com>
To: rddp@ietf.org
Date: Fri, 11 Apr 2003 13:31:08 -0400
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: [rddp] Number of CRCs
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

The last paragraph of Paul Culley's recent post touches
on an old issue that we need to return to:

> Also, I firmly believe that we cannot completely drop CRCs, that
> essentially all customers will be quite nervous about allowing memory
> access by DDP headers that have not been checked by something better
> than a checksum.  So there will be that overhead anyway.  I think that
> checking the remaining user data will be important enough to the
> majority of customers that putting the option in for the minority that
> don't care that much is just not worth it.

This is our old friend the "one vs. two CRCs" issue.  My recollection
of prior discussion on it is that this issue was not settled because
it depended on the open "TCP alignment" issue that has now been settled
(as of the San Francisco meeting, and I believe that rough consensus
is holding).

I read Paul's email as favoring a single mandatory CRC covering
both header and data, combined with an insistence that if separate
header and data CRCs are decided upon, the header CRC needs to be
mandatory in order to increase the assurance of proper placement.

I'd invite further discussion, but this group generally doesn't
need that sort of invitation ;-).  In any case, this issue does
need to be discussed and resolved, and some of Caitlin's recent
post is also on this topic.

Thanks,
--David

----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
black_david@emc.com        Mobile: +1 (978) 394-7754
----------------------------------------------------
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Fri Apr 11 13:45:59 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA22673
	for <rddp-archive@odin.ietf.org>; Fri, 11 Apr 2003 13:45:59 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3BHqFY14137
	for rddp-archive@odin.ietf.org; Fri, 11 Apr 2003 13:52:15 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BHqF814134
	for <rddp-web-archive@optimus.ietf.org>; Fri, 11 Apr 2003 13:52:15 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA22654
	for <rddp-web-archive@ietf.org>; Fri, 11 Apr 2003 13:45:29 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1942dA-0004ZQ-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 13:48:04 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 1942dA-0004ZN-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 13:48:04 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BHp3814059;
	Fri, 11 Apr 2003 13:51:03 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BHoT814022
	for <rddp@optimus.ietf.org>; Fri, 11 Apr 2003 13:50:29 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA22614
	for <rddp@ietf.org>; Fri, 11 Apr 2003 13:43:42 -0400 (EDT)
From: Black_David@emc.com
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1942bS-0004YO-00
	for rddp@ietf.org; Fri, 11 Apr 2003 13:46:18 -0400
Received: from [128.222.32.10] (helo=mxic1.corp.emc.com)
	by ietf-mx with esmtp (Exim 4.12)
	id 1942bR-0004YF-00
	for rddp@ietf.org; Fri, 11 Apr 2003 13:46:17 -0400
Received: by mxic1.corp.emc.com with Internet Mail Service (5.5.2653.19)
	id <2PQXVND9>; Fri, 11 Apr 2003 13:45:47 -0400
Message-ID: <277DD60FB639D511AC0400B0D068B71E0564CB0E@corpmx14.us.dg.com>
To: rddp@ietf.org
Date: Fri, 11 Apr 2003 13:40:40 -0400
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: [rddp] Options and Markers
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

Pulling another topic out of Paul Culley's post:

> In my mind "enabling" these scenarios, while important, should not mean
> that we go to great lengths to optimize or reduce overhead on the
> software side at the expense of making the overall protocol less robust,
> or adding multiple options that applications will have to sort through
> and all concerned will have to test.

I have no problem with robustness considerations, but the vehemence
of opposition to options in any form from a significant portion of
this WG has been among the things that have hindered progress.

I think some form of option is going to be hard to avoid for
markers, as there is "running code" out there for a class of
implementations (RDDP + TCP with full buffering offloaded to
a souped up NIC) that do not need markers.  Requiring that this
class of implementation use markers solely to avoid putting an
option in the protocol appears to be very hard to justify.

One alternate possibility would be to make marker usage optional,
but put control of this "option" entirely in the hands of
the receiver - senders MUST implement both markers and not-markers,
receivers MUST implement one OR the other, and each receiver
gets to pick which one is used as the benefits of markers are
primarily on the receiver side.

Comments?

Thanks,
--David

----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
black_david@emc.com        Mobile: +1 (978) 394-7754
----------------------------------------------------
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Fri Apr 11 13:47:38 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA22711
	for <rddp-archive@odin.ietf.org>; Fri, 11 Apr 2003 13:47:37 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3BHrsH14212
	for rddp-archive@odin.ietf.org; Fri, 11 Apr 2003 13:53:54 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BHrs814209
	for <rddp-web-archive@optimus.ietf.org>; Fri, 11 Apr 2003 13:53:54 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA22692
	for <rddp-web-archive@ietf.org>; Fri, 11 Apr 2003 13:47:07 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1942el-0004a3-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 13:49:43 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 1942ek-0004a0-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 13:49:42 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BHr1814187;
	Fri, 11 Apr 2003 13:53:01 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BHqS814160
	for <rddp@optimus.ietf.org>; Fri, 11 Apr 2003 13:52:28 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA22661;
	Fri, 11 Apr 2003 13:45:42 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1942dN-0004Za-00; Fri, 11 Apr 2003 13:48:17 -0400
Received: from e32.co.us.ibm.com ([32.97.110.130])
	by ietf-mx with esmtp (Exim 4.12)
	id 1942dM-0004ZL-00; Fri, 11 Apr 2003 13:48:17 -0400
Received: from westrelay04.boulder.ibm.com (westrelay04.boulder.ibm.com [9.17.193.32])
	by e32.co.us.ibm.com (8.12.9/8.12.2) with ESMTP id h3BHjvve110440;
	Fri, 11 Apr 2003 13:45:57 -0400
Received: from d03nm115.boulder.ibm.com (d03av02.boulder.ibm.com [9.17.193.82])
	by westrelay04.boulder.ibm.com (8.12.8/NCO/VER6.5) with ESMTP id h3BHlZ2C176610;
	Fri, 11 Apr 2003 11:47:35 -0600
To: Black_David@emc.com
Cc: rddp@ietf.org, rddp-admin@ietf.org
MIME-Version: 1.0
Subject: RE: [rddp] Option or mandatory CRC in MPA - WG Chair speaks
X-Mailer: Lotus Notes Release 5.0.11   July 24, 2002
From: John Hufferd <hufferd@us.ibm.com>
Message-ID: <OFFF387AE3.C662D1DB-ON88256D05.0060FD96-88256D05.0061B67D@us.ibm.com>
Date: Fri, 11 Apr 2003 10:43:29 -0700
X-MIMETrack: Serialize by Router on D03NM115/03/M/IBM(Release 6.0 [IBM]|December 16, 2002) at
 04/11/2003 11:43:14,
	Serialize complete at 04/11/2003 11:43:14
Content-Type: multipart/alternative; boundary="=_alternative 0061B57B88256D05_="
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

This is a multipart message in MIME format.
--=_alternative 0061B57B88256D05_=
Content-Type: text/plain; charset="us-ascii"

David,
There was no intent in my note, to attempt to invalidate previous 
agreements.  In fact it was to accept the previous agreement and move on 
-- without attempting to ALSO apply extensive work to optimization the 
software performance issues -- that I was attempting to motivate.   That 
is quite different than dealing with the issues of unmodified TCP Stacks, 
that applied to the alignment issue.

So, to the extent that my words lead you to think I was attempting to 
revolt from the previous agreements, which I support, I regret that.  It 
was NOT my intention.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/System Group, San Jose CA
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Home Office: (408) 997-6136, Cell: (408) 499-9702
Internet Address: hufferd@us.ibm.com




Black_David@emc.com
Sent by: rddp-admin@ietf.org
04/11/2003 10:13 AM

 
        To:     Paul.Culley@hp.com, rddp@ietf.org
        cc: 
        Subject:        RE: [rddp] Option or mandatory CRC in MPA - WG Chair speaks



> I agree that the WG consensus was to "The RDDP protocols must allow
> implementations over unmodified TCP/IP stacks." (as quoted from your
> "Alignment consensus attempt" email.
> 
> I was unable to locate any minutes describing the fan-in scenarios that
> would lead to a conclusion of "consensus".  That said, I do believe that
> such scenarios will exist.

Good - I hope I don't have to spend the time to extract the relevant
supporting discussions from the WG email list archives. 

> In both cases I am not quite so sure that the group agreed to
> "emphasize" operation over Unmodified SW stacks; rather, I for one, only
> think these should be enabled.  Perhaps I am reading too much into your
> choice of words.

Perhaps - my use of the word "emphasize" was solely with respect to
issues that were important in establishing rough consensus on
the alignment issue.  Unmodified software stacks are important, but
my post was not about any notion of "emphasizing" them to the exclusion
of other implementation structures. 

> In my mind "enabling" these scenarios, while important, should not mean
> that we go to great lengths to optimize or reduce overhead on the
> software side

That's ok - somewhere in the range from "enabling" to "go to great
lengths" (including the endpoints) there should be an engineering
tradeoff that will yield a protocol that is reasonably implementable
in both hardware and software.  This is in stark contrast to John's
original call to ignore considerations for software implementations:

                 It seems that we might think about a complete software
                 implementation, in passing, but this is NOT the goal of
                 RDMA or this RDDP workgroup (in my opinion).

Just in case it's not clear - while John is entitled to his opinion,
the immediately above statement is simply wrong about the IETF RDDP
WG, as it is at odds with prior WG rough consensus.  Attempting to
change the position of the WG to match the quoted statement carries
significant risk of unraveling the rough consensus on the alignment
issue, which may have dramatic consequences for the scope of the WG's
efforts.  That's the clue-by-four - I would advise against inviting
the use of anything larger ...

Paul's email goes on to touch on a couple of other issues that
I'm going to take up under different subject lines.

Thanks,
--David
----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
black_david@emc.com        Mobile: +1 (978) 394-7754
----------------------------------------------------
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



--=_alternative 0061B57B88256D05_=
Content-Type: text/html; charset="us-ascii"


<br><font size=2 face="sans-serif">David,</font>
<br><font size=2 face="sans-serif">There was no intent in my note, to attempt to invalidate previous agreements. &nbsp;In fact it was to accept the previous agreement and move on -- without attempting to ALSO apply extensive work to optimization the software performance issues -- that I was attempting to motivate. &nbsp; That is quite different than dealing with the issues of unmodified TCP Stacks, that applied to the alignment issue.</font>
<br>
<br><font size=2 face="sans-serif">So, to the extent that my words lead you to think I was attempting to &nbsp;revolt from the previous agreements, which I support, I regret that. &nbsp;It was NOT my intention.</font>
<br><font size=2 face="sans-serif">.<br>
.<br>
John L. Hufferd<br>
Senior Technical Staff Member (STSM)<br>
IBM/System Group, San Jose CA<br>
Main Office: (408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688<br>
Home Office: (408) 997-6136, Cell: (408) 499-9702<br>
Internet Address: hufferd@us.ibm.com</font>
<br>
<br>
<br>
<table width=100%>
<tr valign=top>
<td>
<td><font size=1 face="sans-serif"><b>Black_David@emc.com</b></font>
<br><font size=1 face="sans-serif">Sent by: rddp-admin@ietf.org</font>
<p><font size=1 face="sans-serif">04/11/2003 10:13 AM</font>
<br>
<td><font size=1 face="Arial">&nbsp; &nbsp; &nbsp; &nbsp; </font>
<br><font size=1 face="sans-serif">&nbsp; &nbsp; &nbsp; &nbsp; To: &nbsp; &nbsp; &nbsp; &nbsp;Paul.Culley@hp.com, rddp@ietf.org</font>
<br><font size=1 face="sans-serif">&nbsp; &nbsp; &nbsp; &nbsp; cc: &nbsp; &nbsp; &nbsp; &nbsp;</font>
<br><font size=1 face="sans-serif">&nbsp; &nbsp; &nbsp; &nbsp; Subject: &nbsp; &nbsp; &nbsp; &nbsp;RE: [rddp] Option or mandatory CRC in MPA - WG Chair speaks</font>
<br></table>
<br>
<br>
<br><font size=2 face="Courier New">&gt; I agree that the WG consensus was to &quot;The RDDP protocols must allow<br>
&gt; implementations over unmodified TCP/IP stacks.&quot; (as quoted from your<br>
&gt; &quot;Alignment consensus attempt&quot; email.<br>
&gt; <br>
&gt; I was unable to locate any minutes describing the fan-in scenarios that<br>
&gt; would lead to a conclusion of &quot;consensus&quot;. &nbsp;That said, I do believe that<br>
&gt; such scenarios will exist.<br>
<br>
Good - I hope I don't have to spend the time to extract the relevant<br>
supporting discussions from the WG email list archives. &nbsp;<br>
<br>
&gt; In both cases I am not quite so sure that the group agreed to<br>
&gt; &quot;emphasize&quot; operation over Unmodified SW stacks; rather, I for one, only<br>
&gt; think these should be enabled. &nbsp;Perhaps I am reading too much into your<br>
&gt; choice of words.<br>
<br>
Perhaps - my use of the word &quot;emphasize&quot; was solely with respect to<br>
issues that were important in establishing rough consensus on<br>
the alignment issue. &nbsp;Unmodified software stacks are important, but<br>
my post was not about any notion of &quot;emphasizing&quot; them to the exclusion<br>
of other implementation structures. <br>
<br>
&gt; In my mind &quot;enabling&quot; these scenarios, while important, should not mean<br>
&gt; that we go to great lengths to optimize or reduce overhead on the<br>
&gt; software side<br>
<br>
That's ok - somewhere in the range from &quot;enabling&quot; to &quot;go to great<br>
lengths&quot; (including the endpoints) there should be an engineering<br>
tradeoff that will yield a protocol that is reasonably implementable<br>
in both hardware and software. &nbsp;This is in stark contrast to John's<br>
original call to ignore considerations for software implementations:<br>
<br>
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; It seems that we might think about a complete software<br>
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; implementation, in passing, but this is NOT the goal of<br>
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; RDMA or this RDDP workgroup (in my opinion).<br>
<br>
Just in case it's not clear - while John is entitled to his opinion,<br>
the immediately above statement is simply wrong about the IETF RDDP<br>
WG, as it is at odds with prior WG rough consensus. &nbsp;Attempting to<br>
change the position of the WG to match the quoted statement carries<br>
significant risk of unraveling the rough consensus on the alignment<br>
issue, which may have dramatic consequences for the scope of the WG's<br>
efforts. &nbsp;That's the clue-by-four - I would advise against inviting<br>
the use of anything larger ...<br>
<br>
Paul's email goes on to touch on a couple of other issues that<br>
I'm going to take up under different subject lines.<br>
<br>
Thanks,<br>
--David<br>
----------------------------------------------------<br>
David L. Black, Senior Technologist<br>
EMC Corporation, 176 South St., Hopkinton, MA &nbsp;01748<br>
+1 (508) 293-7953 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; FAX: +1 (508) 293-7786<br>
black_david@emc.com &nbsp; &nbsp; &nbsp; &nbsp;Mobile: +1 (978) 394-7754<br>
----------------------------------------------------<br>
_______________________________________________<br>
rddp mailing list<br>
rddp@ietf.org<br>
https://www1.ietf.org/mailman/listinfo/rddp<br>
</font>
<br>
<br>
--=_alternative 0061B57B88256D05_=--
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Fri Apr 11 14:01:55 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA23001
	for <rddp-archive@odin.ietf.org>; Fri, 11 Apr 2003 14:01:55 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3BI8B415680
	for rddp-archive@odin.ietf.org; Fri, 11 Apr 2003 14:08:11 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BI8B815677
	for <rddp-web-archive@optimus.ietf.org>; Fri, 11 Apr 2003 14:08:11 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA22987
	for <rddp-web-archive@ietf.org>; Fri, 11 Apr 2003 14:01:24 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1942sa-0004eD-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 14:04:00 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 1942sZ-0004eA-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 14:03:59 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BI7K815211;
	Fri, 11 Apr 2003 14:07:20 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BI6w814770
	for <rddp@optimus.ietf.org>; Fri, 11 Apr 2003 14:06:58 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA22961
	for <rddp@ietf.org>; Fri, 11 Apr 2003 14:00:11 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1942rP-0004dp-00
	for rddp@ietf.org; Fri, 11 Apr 2003 14:02:47 -0400
Received: from fmr04.intel.com ([143.183.121.6] helo=caduceus.sc.intel.com)
	by ietf-mx with esmtp (Exim 4.12)
	id 1942rP-0004dk-00
	for rddp@ietf.org; Fri, 11 Apr 2003 14:02:47 -0400
Received: from talaria.sc.intel.com (talaria.sc.intel.com [10.3.253.5])
	by caduceus.sc.intel.com (8.11.6p2/8.11.6/d: outer.mc,v 1.51 2002/09/23 20:43:23 dmccart Exp $) with ESMTP id h3BI15g27526
	for <rddp@ietf.org>; Fri, 11 Apr 2003 18:01:05 GMT
Received: from fmsmsxvs042.fm.intel.com (fmsmsxvs042.fm.intel.com [132.233.42.128])
	by talaria.sc.intel.com (8.11.6p2/8.11.6/d: inner.mc,v 1.28 2003/01/13 19:44:39 dmccart Exp $) with SMTP id h3BHvNp16179
	for <rddp@ietf.org>; Fri, 11 Apr 2003 17:57:23 GMT
Received: from FMSMSX018.fm.intel.com ([132.233.42.197])
 by fmsmsxvs042.fm.intel.com (NAVGW 2.5.2.11) with SMTP id M2003041111053830661
 for <rddp@ietf.org>; Fri, 11 Apr 2003 11:05:38 -0700
Received: by fmsmsx018.fm.intel.com with Internet Mail Service (5.5.2653.19)
	id <24JDQFAL>; Fri, 11 Apr 2003 11:02:10 -0700
Message-ID: <D9223EB959A5D511A98F00508B68C20C168FD97B@orsmsx108.jf.intel.com>
From: "Larsen, Roy K" <roy.k.larsen@intel.com>
To: rddp@ietf.org
Subject: RE: [rddp] Optimized Processing not necessarily Hardware Optimize
	d
Date: Fri, 11 Apr 2003 11:02:02 -0700
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="ISO-8859-1"
X-MIME-Autoconverted: from quoted-printable to 8bit by www1.ietf.org id h3BI6x814771
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
X-MIME-Autoconverted: from 8bit to quoted-printable by www1.ietf.org id h3BI7K815211
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by www1.ietf.org id h3BI8B815677
Content-Transfer-Encoding: 8bit


> At the speeds we are envisioning supporting, payload
> validity checking is not a specialized requirement of
> storage applications being imposed upon everyone else.
> The question is more whether the handful of appliations that
> do not need it should be able to impose it being variable.
>
> I believe the impact on the applications that can live
> with data loss is very slight, in addition to those
> applications representing a small portion of total
> network usage.

Pardon my naiveté, but should I conclude from this CRC thread that it is
generally accepted within the IETF community that TCP is no longer
considered a reliable protocol and it (as well as all applications that
depend on its reliability) will become increasingly irrelevant as network
speeds increase?  If so, why would we want to spend any time integrating
RDMA with an obsolete protocol?  If the tsvwg wants to keep TCP relevant,
shouldn't they be the group addressing data integrity?

Roy
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Fri Apr 11 14:38:34 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA24151
	for <rddp-archive@odin.ietf.org>; Fri, 11 Apr 2003 14:38:34 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3BIiqf18238
	for rddp-archive@odin.ietf.org; Fri, 11 Apr 2003 14:44:52 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BIiq818235
	for <rddp-web-archive@optimus.ietf.org>; Fri, 11 Apr 2003 14:44:52 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA24138
	for <rddp-web-archive@ietf.org>; Fri, 11 Apr 2003 14:38:04 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1943S4-0004rT-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 14:40:40 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 1943S3-0004rQ-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 14:40:39 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BIi2818208;
	Fri, 11 Apr 2003 14:44:02 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BIhA818166
	for <rddp@optimus.ietf.org>; Fri, 11 Apr 2003 14:43:10 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA24107
	for <rddp@ietf.org>; Fri, 11 Apr 2003 14:36:22 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1943QQ-0004qx-00
	for rddp@ietf.org; Fri, 11 Apr 2003 14:38:58 -0400
Received: from emulex.emulex.com ([138.239.112.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 1943QL-0004qW-00
	for rddp@ietf.org; Fri, 11 Apr 2003 14:38:53 -0400
Received: from xbl.ad.emulex.com (xbl.ma.emulex.com [172.16.12.11])
	by emulex.emulex.com (8.12.9/8.12.8) with ESMTP id h3BIcGLW004988
	for <rddp@ietf.org>; Fri, 11 Apr 2003 11:38:17 -0700 (PDT)
Received: by xbl.ma.emulex.com with Internet Mail Service (5.5.2653.19)
	id <HN1WWRJ5>; Fri, 11 Apr 2003 14:38:16 -0400
Message-ID: <3356669BBE90C448AD4645C843E2BF289B904F@xbl.ma.emulex.com>
From: "Williams, Jim" <Jim.Williams@Emulex.com>
To: rddp@ietf.org
Subject: RE: [rddp] Optimized Processing not necessarily Hardware Optimize
	d
Date: Fri, 11 Apr 2003 14:38:15 -0400
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="ISO-8859-1"
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>



Caitlin Bestler wrote:
> As many have already argued, the need for payload validity
> checking increases with network speed. 

I have seen this stated, but missed the justification
for this.  Are you implicitly assuming that 
higher network speeds mean lower MTBFs?
(In this context a TCP checksum escape is a failure.)
It seems possible that this might be the case, but
not inevitable.  TCP checksum escapes are almost
always NIC or switch bugs, or bit errors on links
not protected by link CRCs.  The latter should
if anything become less common at higher speeds.
The former might or might not become more
common, but I would certainly hope to see less
of that.  Is there any empirical or theoretical
data supporting the assertion that the need
for validity checking increases with network
speed.

> First, the term "optional" CRC coverage is misleading.
> CRC coverage for the DDP header is mandatory.

Why is it more important to protect
the header than the data?  Bad header means putting
the right data in the wrong buffer.  Bad data means
putting the wrong data in the right buffer.  It seems
that both result in the exact same result in that
the application gets a buffer that has the wrong thing
in it.  

Usually silent propagation of bad data is about the
worst thing that can happen, but I am at a loss to 
understand why undetected bad headers are any WORSE
that undetected bad payloads.

> RDDP cannot
> responsibly place data directly into user buffers based
> soley upon TCP validation.

But it can place data whose correctness relies solely on 
TCP validation?  Why is there any difference?

> That would require the application to confine the Protection
> Domain to only those buffers that could be corrupted. With
> only TCP protection, all buffers exposed through the
> Protection Domain would be subject to being falsely
> overwritten by any flawed TCP segment that escaped
> detection.
> 
> Working around this would require additional Protection
> Domains, and hence additional connections. Clearly this is
> unacceptable.

Maybe I am being slow here, but I am not sure I follow.
Assume for sake of argument that there are errors that
escape TCP but no errors that escape CRC.

Without header CRC, all buffers exposed are subject to
corruption.  With header CRC, but no payload CRC, only
those buffers actually written by the remote end are
subject to corruption.

Why is the former worse than the latter, and why are 
we exposing buffers that are not written?  (These
are not rhetorical questions, there may be good answers.)

Anyway, I regarded the primary motivation for not
using CRCs to be that for many networks the TCP
checksum is sufficiently reliable given the characteristics
of the network and its reliability.  In this case,
TCP checksum escapes are totally unacceptable regardless
of whether they are header or payload.  But if they
don't happen, the CRC adds no value.

> This is additional complexity, and requires a mode-setup
> message exchange at the start of a connection (although
> one-way messages in each direction are adequate, so at least
> no additional round trip is mandated).

Weighing the complexity of supporting an
option versus the overhead of computing the CRC
in software when it is not needed seems like the
critical tradeoff to me.  The mode-setup may
be a non-issue if it required anyway for other
reasons.

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Fri Apr 11 15:44:36 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA27240
	for <rddp-archive@odin.ietf.org>; Fri, 11 Apr 2003 15:44:36 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3BJotA22644
	for rddp-archive@odin.ietf.org; Fri, 11 Apr 2003 15:50:55 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BJot822641
	for <rddp-web-archive@optimus.ietf.org>; Fri, 11 Apr 2003 15:50:55 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA27215
	for <rddp-web-archive@ietf.org>; Fri, 11 Apr 2003 15:44:05 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1944Ty-0005CW-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 15:46:42 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 1944Tx-0005CS-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 15:46:41 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BJo9822583;
	Fri, 11 Apr 2003 15:50:09 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BJm4822418
	for <rddp@optimus.ietf.org>; Fri, 11 Apr 2003 15:48:04 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA26879
	for <rddp@ietf.org>; Fri, 11 Apr 2003 15:41:14 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1944RD-00057l-00
	for rddp@ietf.org; Fri, 11 Apr 2003 15:43:51 -0400
Received: from thebe.your-site.com ([140.186.45.26])
	by ietf-mx with esmtp (Exim 4.12)
	id 1944RC-00057a-00
	for rddp@ietf.org; Fri, 11 Apr 2003 15:43:50 -0400
Received: from 192.168.0.2 (64-144-5-25.client.dsl.net [64.144.5.25])
	by thebe.your-site.com (Postfix) with ESMTP
	id 3F2D1245B67; Fri, 11 Apr 2003 15:43:30 -0400 (EDT)
Date: Fri, 11 Apr 2003 14:45:43 -0500
From: Caitlin Bestler <cait@asomi.com>
Subject: RE: [rddp] Optimized Processing not necessarily Hardware Optimize d
To: rddp@ietf.org
Cc: "Williams, Jim" <Jim.Williams@Emulex.com>
X-Priority: 3
In-Reply-To: <3356669BBE90C448AD4645C843E2BF289B904F@xbl.ma.emulex.com>
Message-ID: <r01050300-1024-2EA4F0266C5611D78DE4003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

On 4/11/03, Williams, Jim wrote:

>> First, the term "optional" CRC coverage is misleading.
>> CRC coverage for the DDP header is mandatory.
>
>Why is it more important to protect
>the header than the data?  Bad header means putting
>the right data in the wrong buffer.  Bad data means
>putting the wrong data in the right buffer.  It seems
>that both result in the exact same result in that
>the application gets a buffer that has the wrong thing
>in it.  
>
>Usually silent propagation of bad data is about the
>worst thing that can happen, but I am at a loss to 
>understand why undetected bad headers are any WORSE
>that undetected bad payloads.
>
>> RDDP cannot
>> responsibly place data directly into user buffers based
>> soley upon TCP validation.
>
>But it can place data whose correctness relies solely on 
>TCP validation?  Why is there any difference?


With a non-RDMA interface the only risk is the wrong data in
the correct buffer. Further, the correct buffer is
determined on a per-read basis.

When a read completes, the worst that can happen is that the
data in the buffer is bad. The application can do a quick
reasonableness check, and then proceed. Does it look like
MPEG-4 data? If so, play it. The worst that will happen
is noise. [And depending on the taste of the consumer,
that's the best that can happen too, ;)]

Without header checking *any* address that is open to *any*
STag valid within the Protection Domain is targettable by an
incorrect (and unchecked) DDP header. To be safe,
applications would have to very actively invalidate STags
after every received message.

Without at least CRC-32c protection on the header, a serious
argument could be made that RDMA over TCP posed an
unacceptable risk to system integrity and security.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Fri Apr 11 15:59:43 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA27771
	for <rddp-archive@odin.ietf.org>; Fri, 11 Apr 2003 15:59:43 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3BK62q24440
	for rddp-archive@odin.ietf.org; Fri, 11 Apr 2003 16:06:02 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BK62824437
	for <rddp-web-archive@optimus.ietf.org>; Fri, 11 Apr 2003 16:06:02 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA27726
	for <rddp-web-archive@ietf.org>; Fri, 11 Apr 2003 15:59:12 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1944ia-0005JL-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 16:01:48 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 1944ia-0005JI-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 16:01:48 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BK51823468;
	Fri, 11 Apr 2003 16:05:01 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BJsP822812
	for <rddp@optimus.ietf.org>; Fri, 11 Apr 2003 15:54:25 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA27376
	for <rddp@ietf.org>; Fri, 11 Apr 2003 15:47:36 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1944XM-0005EN-00
	for rddp@ietf.org; Fri, 11 Apr 2003 15:50:12 -0400
Received: from thebe.your-site.com ([140.186.45.26])
	by ietf-mx with esmtp (Exim 4.12)
	id 1944XM-0005E3-00
	for rddp@ietf.org; Fri, 11 Apr 2003 15:50:12 -0400
Received: from 192.168.0.2 (64-144-5-25.client.dsl.net [64.144.5.25])
	by thebe.your-site.com (Postfix) with ESMTP
	id 98B60244D42; Fri, 11 Apr 2003 15:49:51 -0400 (EDT)
Date: Fri, 11 Apr 2003 14:52:04 -0500
From: Caitlin Bestler <cait@asomi.com>
Subject: RE: [rddp] Option or mandatory CRC in MPA
To: Somesh Gupta <somesh_gupta@silverbacksystems.com>, rddp@ietf.org
X-Priority: 3
In-Reply-To: <NMEALCLOIBCHBDHLCMIJMEHCDDAA.somesh_gupta@silverbacksystems.com>
Message-ID: <r01050300-1024-120292AD6C5711D78DE4003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

On 4/11/03, Somesh Gupta wrote:

>> It is common for single segment, IPv4 packets.  It is
>> not common for IPv4 fragments or for IPv6 (IPv6 is
>> complicated by the next header field making it more
>> difficult to support checksum off-load as the number of
>> headers, etc. is somewhat unpredictable).
>
> The IPV4 single segment continues to be > 99% of the
> traffic over TCP. And many of the NICs do support
> checksum for IPV6 (the driver may not). The host may be
> required to fix some stuff.

Making protocol design decisions on the presumption that
future Internet traffic will remain IPv4 is unlikely to be
persuasive with the Area Directors.

The fact that some *current* hardware knows how to handle
IPv4 validation, but not IPv6, merely re-inforces my earlier
point that these are not intrinisic flaws with the protocol.
They are only transitional issues. Unless there is a severe
penalty with the interim solutions, I would be very
reluctant to impose a *permanent* penalty (in the form of
option tracking and negotiation) to solve an *interim*
problem.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Fri Apr 11 16:46:34 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA29322
	for <rddp-archive@odin.ietf.org>; Fri, 11 Apr 2003 16:46:34 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3BKqrr28674
	for rddp-archive@odin.ietf.org; Fri, 11 Apr 2003 16:52:53 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BKqr828671
	for <rddp-web-archive@optimus.ietf.org>; Fri, 11 Apr 2003 16:52:53 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA29305
	for <rddp-web-archive@ietf.org>; Fri, 11 Apr 2003 16:46:03 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1945Ru-0005dY-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 16:48:38 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 1945Rt-0005dV-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 16:48:37 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BKqA828642;
	Fri, 11 Apr 2003 16:52:10 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BKpS828592
	for <rddp@optimus.ietf.org>; Fri, 11 Apr 2003 16:51:28 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA29141
	for <rddp@ietf.org>; Fri, 11 Apr 2003 16:44:39 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1945QX-0005a2-00
	for rddp@ietf.org; Fri, 11 Apr 2003 16:47:13 -0400
Received: from thebe.your-site.com ([140.186.45.26])
	by ietf-mx with esmtp (Exim 4.12)
	id 1945QX-0005Yt-00
	for rddp@ietf.org; Fri, 11 Apr 2003 16:47:13 -0400
Received: from 192.168.0.2 (64-144-5-25.client.dsl.net [64.144.5.25])
	by thebe.your-site.com (Postfix) with ESMTP
	id 30994244E98; Fri, 11 Apr 2003 16:46:53 -0400 (EDT)
Date: Fri, 11 Apr 2003 15:49:06 -0500
From: Caitlin Bestler <cait@asomi.com>
Subject: RE: [rddp] Optimized Processing not necessarily Hardware Optimize d
To: "Larsen, Roy K" <roy.k.larsen@intel.com>
Cc: rddp@ietf.org
X-Priority: 3
In-Reply-To: <D9223EB959A5D511A98F00508B68C20C168FD97B@orsmsx108.jf.intel.com>
Message-ID: <r01050300-1024-0976EAFB6C5F11D78DE4003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=ISO-8859-1
X-Mailer: Mailsmith 1.5.3 (Blindsider)
X-MIME-Autoconverted: from quoted-printable to 8bit by www1.ietf.org id h3BKpS828593
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
X-MIME-Autoconverted: from 8bit to quoted-printable by www1.ietf.org id h3BKqA828642
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by www1.ietf.org id h3BKqr828671
Content-Transfer-Encoding: 8bit

On 4/11/03, Larsen, Roy K wrote:

>Pardon my naiveté, but should I conclude from this CRC
>thread that it is generally accepted within the IETF
>community that TCP is no longer considered a reliable
>protocol and it (as well as all applications that depend
>on its reliability) will become increasingly irrelevant as
>network speeds increase?  If so, why would we want to
>spend any time integrating RDMA with an obsolete protocol?
> If the tsvwg wants to keep TCP relevant, shouldn't they
>be the group addressing data integrity?

That solution would be called SCTP, which has a CRC-32c.

I see no need to define "TCP-2" when it would merely
be a single-streamed single-homed subset of SCTP.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Fri Apr 11 17:14:21 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA00731
	for <rddp-archive@odin.ietf.org>; Fri, 11 Apr 2003 17:14:21 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3BLKfL31622
	for rddp-archive@odin.ietf.org; Fri, 11 Apr 2003 17:20:41 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BLKf831619
	for <rddp-web-archive@optimus.ietf.org>; Fri, 11 Apr 2003 17:20:41 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA00699
	for <rddp-web-archive@ietf.org>; Fri, 11 Apr 2003 17:13:51 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1945sn-0005xV-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 17:16:25 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 1945sn-0005xS-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 17:16:25 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BLK2831588;
	Fri, 11 Apr 2003 17:20:02 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BLHi831451
	for <rddp@optimus.ietf.org>; Fri, 11 Apr 2003 17:17:44 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA00625
	for <rddp@ietf.org>; Fri, 11 Apr 2003 17:10:54 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1945px-0005wv-00
	for rddp@ietf.org; Fri, 11 Apr 2003 17:13:29 -0400
Received: from fmr05.intel.com ([134.134.136.6] helo=hermes.jf.intel.com)
	by ietf-mx with esmtp (Exim 4.12)
	id 1945pw-0005wm-00
	for rddp@ietf.org; Fri, 11 Apr 2003 17:13:29 -0400
Received: from petasus.jf.intel.com (petasus.jf.intel.com [10.7.209.6])
	by hermes.jf.intel.com (8.11.6p2/8.11.6/d: outer.mc,v 1.51 2002/09/23 20:43:23 dmccart Exp $) with ESMTP id h3BLAxr28262
	for <rddp@ietf.org>; Fri, 11 Apr 2003 21:10:59 GMT
Received: from orsmsxvs040.jf.intel.com (orsmsxvs040.jf.intel.com [192.168.65.206])
	by petasus.jf.intel.com (8.11.6p2/8.11.6/d: inner.mc,v 1.28 2003/01/13 19:44:39 dmccart Exp $) with SMTP id h3BL8uO28542
	for <rddp@ietf.org>; Fri, 11 Apr 2003 21:08:56 GMT
Received: from orsmsx26.jf.intel.com ([192.168.65.26])
 by orsmsxvs040.jf.intel.com (NAVGW 2.5.2.11) with SMTP id M2003041114214505948
 ; Fri, 11 Apr 2003 14:21:45 -0700
Received: by orsmsx26.jf.intel.com with Internet Mail Service (5.5.2653.19)
	id <249MQGSW>; Fri, 11 Apr 2003 14:12:57 -0700
Message-ID: <D9223EB959A5D511A98F00508B68C20C168FD97C@orsmsx108.jf.intel.com>
From: "Larsen, Roy K" <roy.k.larsen@intel.com>
To: "'Caitlin Bestler'" <cait@asomi.com>
Cc: rddp@ietf.org
Subject: RE: [rddp] Optimized Processing not necessarily Hardware Optimize
	 d
Date: Fri, 11 Apr 2003 14:12:55 -0700
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
X-MIME-Autoconverted: from quoted-printable to 8bit by www1.ietf.org id h3BLHj831452
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
X-MIME-Autoconverted: from 8bit to quoted-printable by www1.ietf.org id h3BLK2831588
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by www1.ietf.org id h3BLKf831619
Content-Transfer-Encoding: 8bit

That leaves open my question as to why this group would want to spend time
implementing RDMAP on top of a protocol that folks seem to believe no longer
supplies the quality of service its designers intended.  Doesn't that put us
into the surrogate TCP-2 definition space?

Roy

-----Original Message-----
From: Caitlin Bestler [mailto:cait@asomi.com]
Sent: Friday, April 11, 2003 1:49 PM
To: Larsen, Roy K
Cc: rddp@ietf.org
Subject: RE: [rddp] Optimized Processing not necessarily Hardware
Optimize d


On 4/11/03, Larsen, Roy K wrote:

>Pardon my naiveté, but should I conclude from this CRC
>thread that it is generally accepted within the IETF
>community that TCP is no longer considered a reliable
>protocol and it (as well as all applications that depend
>on its reliability) will become increasingly irrelevant as
>network speeds increase?  If so, why would we want to
>spend any time integrating RDMA with an obsolete protocol?
> If the tsvwg wants to keep TCP relevant, shouldn't they
>be the group addressing data integrity?

That solution would be called SCTP, which has a CRC-32c.

I see no need to define "TCP-2" when it would merely
be a single-streamed single-homed subset of SCTP.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Fri Apr 11 17:22:20 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA01183
	for <rddp-archive@odin.ietf.org>; Fri, 11 Apr 2003 17:22:20 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3BLSef32067
	for rddp-archive@odin.ietf.org; Fri, 11 Apr 2003 17:28:40 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BLSe832064
	for <rddp-web-archive@optimus.ietf.org>; Fri, 11 Apr 2003 17:28:40 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA01123
	for <rddp-web-archive@ietf.org>; Fri, 11 Apr 2003 17:21:49 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 19460W-00063P-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 17:24:24 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 19460W-00063M-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 17:24:24 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BLS2831951;
	Fri, 11 Apr 2003 17:28:02 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BLN6831736
	for <rddp@optimus.ietf.org>; Fri, 11 Apr 2003 17:23:06 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA00801
	for <rddp@ietf.org>; Fri, 11 Apr 2003 17:16:15 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1945v8-0005yP-00
	for rddp@ietf.org; Fri, 11 Apr 2003 17:18:50 -0400
Received: from thebe.your-site.com ([140.186.45.26])
	by ietf-mx with esmtp (Exim 4.12)
	id 1945v8-0005yE-00
	for rddp@ietf.org; Fri, 11 Apr 2003 17:18:50 -0400
Received: from 192.168.0.2 (64-144-5-25.client.dsl.net [64.144.5.25])
	by thebe.your-site.com (Postfix) with ESMTP
	id 03213244F09; Fri, 11 Apr 2003 17:18:30 -0400 (EDT)
Date: Fri, 11 Apr 2003 16:20:43 -0500
From: Caitlin Bestler <cait@asomi.com>
Subject: Re: [rddp] Options and Markers
To: Black_David@emc.com, rddp@ietf.org
X-Priority: 3
In-Reply-To: <277DD60FB639D511AC0400B0D068B71E0564CB0E@corpmx14.us.dg.com>
Message-ID: <r01050300-1024-741C01416C6311D78DE4003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

On 4/11/03, Black_David@emc.com wrote:


>
>One alternate possibility would be to make marker usage
>optional, but put control of this "option" entirely in the
>hands of the receiver - senders MUST implement both
>markers and not-markers, receivers MUST implement one OR
>the other, and each receiver gets to pick which one is
>used as the benefits of markers are primarily on the
>receiver side.
>
>Comments?
>

A very solid suggestion.

Omitting markers simplifies packet handling, improves
network utilization slightly, but at the cost of increasing
the receivers buffering for out-of-order segments.

A receiver that knew it had sufficient buffering would have
every reason to request that it not be sent markers. As
pointed out during the prior discussions on this issue, once
you had an external memory device you might as well add a
lot. Low memory RNICs are most likely those that are trying
to work without an external memory device.

There is one gotcha that would have to be worked out.
All of the other options discussed involve a sender
describing how they send. That makes it compatible with
a single initialization FULPDU that proceeds the data
FULPDUs. There is no need for special marking or for an
extra round trip.

With this option, however, the receiver must know *when* its
partner received its configuration. That means either one of
two things:

-- The establishment of "FULPDU" types, analogous to SCTP
   chunk types, so that configuration changes can be
   acknowledged even after data chunks have been sent.
-- Requiring a round-trip so that each side can confirm
   its modes *after* hearing what the other side wants.
   
In either case, this could be optimized for the case where
one side believes it knows what the other side wants.
The "init" chunk would include not only the sender's
marker mode, but what it believes the other side's is.

When an "init" chunk is received that correctly predicts
the receivers marker mode, no further action is required.
The connection can proceed to "data" mode immediately.
Otherwise a "correction" chunk must be sent.

Similiarly, the receiver can predict whether the other
side would have to send a "data" chunk.

When an incorrect prediction is made to an unknown host,
a round trip is still required. Unless there is a convention
for public servers, this could be a problem (an example of a
connection would be providing a mode hint in the URL).

The alternate would require an "escape" FULPDU. Such a frame
could be designed. The best "escape" protocol I can think of
is that control chunks could be placed immediately after a
zero length FULPDU.

Even so, changing modes would be tricky, because the change
would have to be negotiated without any data in flight. The
benefit is that a receiver could take the first data chunk
in the "wrong" mode, and then decide if it was worth going
idle in order to change.


A third option would be to explicitly allow receivers to
mark themselves as being "sender compliant". This might be
the best solution for public servers that deal with a large
number of intermittent clients, especially for short lived
connections (i.e., an RDMA optimized web server).



BTW, I still dislike options. Every option has a permanent
and ongoing cost in QA testing.


Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Fri Apr 11 17:22:23 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA01201
	for <rddp-archive@odin.ietf.org>; Fri, 11 Apr 2003 17:22:23 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3BLShN32087
	for rddp-archive@odin.ietf.org; Fri, 11 Apr 2003 17:28:43 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BLSh832084
	for <rddp-web-archive@optimus.ietf.org>; Fri, 11 Apr 2003 17:28:43 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA01130
	for <rddp-web-archive@ietf.org>; Fri, 11 Apr 2003 17:21:53 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 19460Z-00063b-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 17:24:27 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 19460Z-00063Y-00
	for rddp-web-archive@ietf.org; Fri, 11 Apr 2003 17:24:27 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BLS1831935;
	Fri, 11 Apr 2003 17:28:01 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3BLMX831716
	for <rddp@optimus.ietf.org>; Fri, 11 Apr 2003 17:22:33 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA00792
	for <rddp@ietf.org>; Fri, 11 Apr 2003 17:15:43 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1945uc-0005yB-00
	for rddp@ietf.org; Fri, 11 Apr 2003 17:18:18 -0400
Received: from ztxmail05.ztx.compaq.com ([161.114.1.209])
	by ietf-mx with esmtp (Exim 4.12)
	id 1945ub-0005y8-00
	for rddp@ietf.org; Fri, 11 Apr 2003 17:18:17 -0400
Received: from cceexg12.americas.cpqcorp.net (cceexg12.americas.cpqcorp.net [16.110.250.124])
	by ztxmail05.ztx.compaq.com (Postfix) with ESMTP
	id 2062D9077; Fri, 11 Apr 2003 16:17:47 -0500 (CDT)
Received: from cceexc16.americas.cpqcorp.net ([16.110.250.58]) by cceexg12.americas.cpqcorp.net with Microsoft SMTPSVC(5.0.2195.2966);
	 Fri, 11 Apr 2003 16:17:45 -0500
X-MimeOLE: Produced By Microsoft Exchange V6.0.6375.0
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: RE: [rddp] Options and Markers
Date: Fri, 11 Apr 2003 16:17:44 -0500
Message-ID: <01CA1B1F74A4EC40A9C887F7B58F872403143131@cceexc16.americas.cpqcorp.net>
Thread-Topic: [rddp] Options and Markers
Thread-Index: AcMAU1UKN3Q35Q9CSnGGIa+XfBVKfAAF6ifg
From: "Barron, Dwight" <Dwight.Barron@hp.com>
To: <Black_David@emc.com>, <rddp@ietf.org>
X-OriginalArrivalTime: 11 Apr 2003 21:17:45.0596 (UTC) FILETIME=[CBBBD3C0:01C3006F]
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by www1.ietf.org id h3BLMX831717
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 8bit
Content-Transfer-Encoding: 8bit



> -----Original Message-----
> From: Black_David@emc.com [mailto:Black_David@emc.com]
> Sent: Friday, April 11, 2003 12:41 PM
> To: rddp@ietf.org
> Subject: [rddp] Options and Markers
> 
> 
> Pulling another topic out of Paul Culley's post:
> 
> > In my mind "enabling" these scenarios, while important, 
> should not mean
> > that we go to great lengths to optimize or reduce overhead on the
> > software side at the expense of making the overall protocol 
> less robust,
> > or adding multiple options that applications will have to 
> sort through
> > and all concerned will have to test.
> 
> I have no problem with robustness considerations, but the vehemence
> of opposition to options in any form from a significant portion of
> this WG has been among the things that have hindered progress.

I do not believe it is a "vehemence of opposition to options" per se, but rather a strong desire to see valid engineering justification for such options. I believe we established, per your comments at the last F2F, that receive side copying is not likely to be avoided, regardless of marker interval and we went on to focus on transmit side impact of marker interval options. I recall that there were two likely implementation scenarios on the transmit side, single buffer pointer per transmit/system call and gather lists per transmit/system call. My personal assertion was that environments with single buffer pointers per transmit/system call would quickly resort to copying to coalesce the RDDP header and payload to avoid system call overhead and marker interval wasn't significant in the copy loop (1/512 is 0.2%, taking it 0.02%, while numerically a 10X improvement, is not a valid justification for an option, IMHO). Others asserted that gather lists were sufficiently large as to !
not warrant additional marker intervals. Neither of these scenarios seem to have been challenged at all, much less to an extent to convince me I am being unreasonable and vehement. 

> 
> I think some form of option is going to be hard to avoid for
> markers, as there is "running code" out there for a class of
> implementations (RDDP + TCP with full buffering offloaded to
> a souped up NIC) that do not need markers.  Requiring that this
> class of implementation use markers solely to avoid putting an
> option in the protocol appears to be very hard to justify.

How does this argument not cross the line from what I would consider debate in search of best architectural practice for RDDP to that of defending an existing RDDP implementation of "running code". What RDDP specification was this running code conformant to such that it should impact the design of future RDDP specifications? 

> 
> One alternate possibility would be to make marker usage optional,
> but put control of this "option" entirely in the hands of
> the receiver - senders MUST implement both markers and not-markers,
> receivers MUST implement one OR the other, and each receiver
> gets to pick which one is used as the benefits of markers are
> primarily on the receiver side.
> 
> Comments?
> 
> Thanks,
> --David
> 
> ----------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 176 South St., Hopkinton, MA  01748
> +1 (508) 293-7953             FAX: +1 (508) 293-7786
> black_david@emc.com        Mobile: +1 (978) 394-7754
> ----------------------------------------------------
> _______________________________________________
> rddp mailing list
> rddp@ietf.org
> https://www1.ietf.org/mailman/listinfo/rddp
> 
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Sun Apr 13 15:56:44 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04808
	for <rddp-archive@odin.ietf.org>; Sun, 13 Apr 2003 15:56:44 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3DK42211664
	for rddp-archive@odin.ietf.org; Sun, 13 Apr 2003 16:04:02 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3DK42811661
	for <rddp-web-archive@optimus.ietf.org>; Sun, 13 Apr 2003 16:04:02 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04801
	for <rddp-web-archive@ietf.org>; Sun, 13 Apr 2003 15:56:14 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 194ncl-00061z-00
	for rddp-web-archive@ietf.org; Sun, 13 Apr 2003 15:58:47 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 194ncl-00061v-00
	for rddp-web-archive@ietf.org; Sun, 13 Apr 2003 15:58:47 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3DK3K811641;
	Sun, 13 Apr 2003 16:03:20 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3DK2i811624
	for <rddp@optimus.ietf.org>; Sun, 13 Apr 2003 16:02:44 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04792
	for <rddp@ietf.org>; Sun, 13 Apr 2003 15:54:57 -0400 (EDT)
From: Black_David@emc.com
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 194nbW-00061r-00
	for rddp@ietf.org; Sun, 13 Apr 2003 15:57:30 -0400
Received: from mxic2.corp.emc.com ([128.221.31.40])
	by ietf-mx with esmtp (Exim 4.12)
	id 194nbV-00061n-00
	for rddp@ietf.org; Sun, 13 Apr 2003 15:57:29 -0400
Received: by mxic2.corp.emc.com with Internet Mail Service (5.5.2653.19)
	id <2JMPYPM2>; Sun, 13 Apr 2003 15:56:58 -0400
Message-ID: <277DD60FB639D511AC0400B0D068B71E0564CB13@corpmx14.corp.emc.com>
To: Dwight.Barron@hp.com, rddp@ietf.org
Subject: RE: [rddp] Options and Markers
Date: Sun, 13 Apr 2003 15:51:56 -0400
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

Dwight,

> > I have no problem with robustness considerations, but the vehemence
> > of opposition to options in any form from a significant portion of
> > this WG has been among the things that have hindered progress.
> 
> I do not believe it is a "vehemence of opposition to options" 
> per se, but rather a strong desire to see valid engineering 
> justification for such options. I believe we established, per 
> your comments at the last F2F, that receive side copying is 
> not likely to be avoided, regardless of marker interval and 
> we went on to focus on transmit side impact of marker 
> interval options.

The discussion in San Francisco concerned a receive side full-software
implementation.  It did not cover a "full-offload with full TCP buffering"
class of implementation, as the discussion would up considering only
full-software and "flow-through" RNIC implementations.  There's a third
class that is relevant.

[... material on full-software implementations snipped ... ]

> > I think some form of option is going to be hard to avoid for
> > markers, as there is "running code" out there for a class of
> > implementations (RDDP + TCP with full buffering offloaded to
> > a souped up NIC) that do not need markers.  Requiring that this
> > class of implementation use markers solely to avoid putting an
> > option in the protocol appears to be very hard to justify.
> 
> How does this argument not cross the line from what I would 
> consider debate in search of best architectural practice for 
> RDDP to that of defending an existing RDDP implementation of 
> "running code". What RDDP specification was this running code 
> conformant to such that it should impact the design of future 
> RDDP specifications? 

The assertion that there is a single "best architectural practice"
for RDDP and it is a "flow-through RNIC" with minimal buffering
is part of the problem.  The implementation I referred to above
is based on the VI architecture and runs it over TCP/IP with
all RDMA processing done outboard.  AFAIK, the protocol used
is specific to that implementation; among the reasons for this is
that VI is an interface, not a protocol.

The reason that this discussion doesn't cross the line, is that
the "running code" serves to demonstrate the viability of the
third class of implementations, speaking to your above desire for
"valid engineering justification" for protocol design concerns
raised by that class.  While I can't speak for the implementers,
I think they plan to update the implementation in question to
whatever specifications the RDDP WG adopts.  Given the viability
of that class of implementation (FWIW, it was extensively discussed
prior to formation of the RDDP WG), design decisions which exclude
that class will tend to run afoul of the admonition not to
impose requirements on implementors that are not required for
interoperability (see Section 6 of RFC 2119).

Thanks,
--David
----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
black_david@emc.com        Mobile: +1 (978) 394-7754
----------------------------------------------------
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Sun Apr 13 15:58:22 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04833
	for <rddp-archive@odin.ietf.org>; Sun, 13 Apr 2003 15:58:22 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3DK5dw11726
	for rddp-archive@odin.ietf.org; Sun, 13 Apr 2003 16:05:39 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3DK5d811723
	for <rddp-web-archive@optimus.ietf.org>; Sun, 13 Apr 2003 16:05:39 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04829
	for <rddp-web-archive@ietf.org>; Sun, 13 Apr 2003 15:57:51 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 194neK-00062K-00
	for rddp-web-archive@ietf.org; Sun, 13 Apr 2003 16:00:24 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 194neK-00062H-00
	for rddp-web-archive@ietf.org; Sun, 13 Apr 2003 16:00:24 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3DK51811700;
	Sun, 13 Apr 2003 16:05:01 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3DK45811678
	for <rddp@optimus.ietf.org>; Sun, 13 Apr 2003 16:04:05 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04806
	for <rddp@ietf.org>; Sun, 13 Apr 2003 15:56:17 -0400 (EDT)
From: Black_David@emc.com
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 194ncp-000624-00
	for rddp@ietf.org; Sun, 13 Apr 2003 15:58:51 -0400
Received: from [128.222.32.10] (helo=mxic1.corp.emc.com)
	by ietf-mx with esmtp (Exim 4.12)
	id 194nco-00061u-00
	for rddp@ietf.org; Sun, 13 Apr 2003 15:58:50 -0400
Received: by mxic1.corp.emc.com with Internet Mail Service (5.5.2653.19)
	id <2PQXWS07>; Sun, 13 Apr 2003 15:58:20 -0400
Message-ID: <277DD60FB639D511AC0400B0D068B71E0564CB14@corpmx14.corp.emc.com>
To: cait@asomi.com, rddp@ietf.org
Date: Sun, 13 Apr 2003 15:53:20 -0400
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: [rddp] STag invalidation
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

WG Chair hat off:

> Without header checking *any* address that is open to *any*
> STag valid within the Protection Domain is targettable by an
> incorrect (and unchecked) DDP header. To be safe,
> applications would have to very actively invalidate STags
> after every received message.

An application that cares about data integrity and does not fully
trust its senders (there are lots of examples, as this is likely to
be the case for anything other than a tightly integrated cluster)
is likely to want to invalidate the STag after each received message
in order to avoid attacks based on a remote overwrite
after some application validity check has been made on the received
data.  See Section 4.1 of the DDP and RDMA Concerns draft.

Thanks,
--David
----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
black_david@emc.com        Mobile: +1 (978) 394-7754
----------------------------------------------------


_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Sun Apr 13 17:54:35 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA06343
	for <rddp-archive@odin.ietf.org>; Sun, 13 Apr 2003 17:54:35 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3DM1tk18991
	for rddp-archive@odin.ietf.org; Sun, 13 Apr 2003 18:01:55 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3DM1t818988
	for <rddp-web-archive@optimus.ietf.org>; Sun, 13 Apr 2003 18:01:55 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA06327
	for <rddp-web-archive@ietf.org>; Sun, 13 Apr 2003 17:54:04 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 194pSo-0006Lf-00
	for rddp-web-archive@ietf.org; Sun, 13 Apr 2003 17:56:38 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 194pSo-0006Lc-00
	for rddp-web-archive@ietf.org; Sun, 13 Apr 2003 17:56:38 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3DM1H818934;
	Sun, 13 Apr 2003 18:01:17 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3DM0X818899
	for <rddp@optimus.ietf.org>; Sun, 13 Apr 2003 18:00:33 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA06316
	for <rddp@ietf.org>; Sun, 13 Apr 2003 17:52:42 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 194pRU-0006LM-00
	for rddp@ietf.org; Sun, 13 Apr 2003 17:55:16 -0400
Received: from thebe.your-site.com ([140.186.45.26])
	by ietf-mx with esmtp (Exim 4.12)
	id 194pRU-0006LJ-00
	for rddp@ietf.org; Sun, 13 Apr 2003 17:55:16 -0400
Received: from 192.168.0.212 (208-41-143-203.client.dsl.net [208.41.143.203])
	by thebe.your-site.com (Postfix) with ESMTP
	id 0EA022448BC; Sun, 13 Apr 2003 17:55:01 -0400 (EDT)
Date: Sun, 13 Apr 2003 16:54:57 -0500
From: Caitlin Bestler <cait@asomi.com>
To: Black_David@emc.com
Cc: rddp@ietf.org
X-Priority: 3
In-Reply-To: <277DD60FB639D511AC0400B0D068B71E0564CB14@corpmx14.corp.emc.com>
Message-ID: <r01050300-1024-911DF15D6DFA11D78329000393814604@[192.168.0.212]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=ISO-8859-1
X-Mailer: Mailsmith 1.5.3 (Blindsider)
X-MIME-Autoconverted: from quoted-printable to 8bit by www1.ietf.org id h3DM0X818900
Subject: [rddp] Re: STag invalidation
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
X-MIME-Autoconverted: from 8bit to quoted-printable by www1.ietf.org id h3DM1H818934
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by www1.ietf.org id h3DM1t818988
Content-Transfer-Encoding: 8bit

> > Without header checking *any* address that is open to *any*
> > STag valid within the Protection Domain is targettable by an
> > incorrect (and unchecked) DDP header. To be safe,
> > applications would have to very actively invalidate STags
> > after every received message.
> 
> An application that cares about data integrity and does not fully
> trust its senders (there are lots of examples, as this is likely to
> be the case for anything other than a tightly integrated cluster)
> is likely to want to invalidate the STag after each received message
> in order to avoid attacks based on a remote overwrite
> after some application validity check has been made on the received
> data.  See Section 4.1 of the DDP and RDMA Concerns draft.

If you don't trust a client, it is really only necessary to protect work done
for other clients from meddling. If an untrusted client wants to sabotage
work done on their behalf, well let them.

Creating a Protection Domain for each client is more than enough to ensure
that clients cannot interfere with other clients (and standard application layer
security).

The original point is that without header protection, an error within a
session can invalidate work done for that session -- unless almost
draconian measures of STag invalidation are undertaken. That would
*include* having no more than one operation pending at a time.

 -- 
Caitlin Bestler <asomi@earthlink.net>
Ï`ò�œÐÏ`ò�œÐÏ`��T  q@Ä.�� ýò��\@`��ƒ`p\{��h¡¿ò��(Ä@��H¤ýò�œÐÄ
ïzKïO0xÄ   gggRp
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Mon Apr 14 08:51:33 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA03329
	for <rddp-archive@odin.ietf.org>; Mon, 14 Apr 2003 08:51:33 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3ECxBe18569
	for rddp-archive@odin.ietf.org; Mon, 14 Apr 2003 08:59:11 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3ECxA818566
	for <rddp-web-archive@optimus.ietf.org>; Mon, 14 Apr 2003 08:59:10 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA03320
	for <rddp-web-archive@ietf.org>; Mon, 14 Apr 2003 08:51:02 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1953So-0001sA-00
	for rddp-web-archive@ietf.org; Mon, 14 Apr 2003 08:53:34 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 1953Sn-0001s7-00
	for rddp-web-archive@ietf.org; Mon, 14 Apr 2003 08:53:33 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3ECwI818514;
	Mon, 14 Apr 2003 08:58:19 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3ECvC818466
	for <rddp@optimus.ietf.org>; Mon, 14 Apr 2003 08:57:12 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA03139
	for <rddp@ietf.org>; Mon, 14 Apr 2003 08:49:04 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1953Qu-0001o9-00
	for rddp@ietf.org; Mon, 14 Apr 2003 08:51:36 -0400
Received: from palrel11.hp.com ([156.153.255.246])
	by ietf-mx with esmtp (Exim 4.12)
	id 1953Qt-0001o6-00
	for rddp@ietf.org; Mon, 14 Apr 2003 08:51:35 -0400
Received: from esmail.cup.hp.com (esmail.cup.hp.com [15.0.65.164])
	by palrel11.hp.com (Postfix) with ESMTP id 4DA711C01C95
	for <rddp@ietf.org>; Mon, 14 Apr 2003 05:51:40 -0700 (PDT)
Received: from mk731915.cup.hp.com ([15.113.161.2])
	by esmail.cup.hp.com (8.11.1/8.8.6) with ESMTP id h3ECm6l03864
	for <rddp@ietf.org>; Mon, 14 Apr 2003 05:48:06 -0700 (PDT)
Message-Id: <5.1.0.14.2.20030412234443.0209fe48@15.13.130.73>
X-Sender: krause@15.13.130.73
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Sat, 12 Apr 2003 23:48:30 -0700
To: <rddp@ietf.org>
From: Michael Krause <krause@cup.hp.com>
Subject: RE: [rddp] Option or mandatory CRC in MPA
In-Reply-To: <NMEALCLOIBCHBDHLCMIJMEHCDDAA.somesh_gupta@silverbacksystem
 s.com>
References: <5.1.0.14.2.20030410213912.08daea20@15.13.130.73>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

At 09:54 AM 4/11/2003 -0700, Somesh Gupta wrote:

>   The IPV4 single segment continues to be > 99% of the traffic over TCP.
>   And many of the NICs do support checksum for IPV6 (the driver may not).
>   The host may be required to fix some stuff.

Some NICs support IPv6 checksum off-load if the next header is the 
transport header.  If the next header is something else, then checksum 
off-load is relatively rare.  There is also the lack of support in many 
NICs when things get fragmented.

> > >2. There are a certain class of apps that do not touch the data
> > >    but just shunt it from one place to another. A classic example
> > >    is any kind of storage server. These are actually the ones
> > >    that benefit from zero copy the most and may use means other
> > >    than RDDP to get the benefit (i.e. handle SGLs)
> >
> > This primarily is true for the outbound path where one encapsulates the
> > file system or storage block.  The inbound path usually requires a copy
> > operation from the mbuf to the storage block / file system due to the
> > deltas in size between the layer 2 packet and the blocks up above.
>
>   Since the SCSI/FC controllers do support scatter-gather lists, it
>   is a buffer management issue. With TOE and header-data split, those
>   that do not want to complicate their buffer management can get the
>   benefits without a change to the wire protocol.

I thought we were discussing a software implementation and not a protocol 
off-load in hardware solution?  The performance issues that were raised 
were about the impact to the software implementation thus jumping into a 
hybrid isn't quite an apples-to-apples comparison.

Mike

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Mon Apr 14 11:24:09 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA09144
	for <rddp-archive@odin.ietf.org>; Mon, 14 Apr 2003 11:24:08 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3EFVo630048
	for rddp-archive@odin.ietf.org; Mon, 14 Apr 2003 11:31:50 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3EFVo830045
	for <rddp-web-archive@optimus.ietf.org>; Mon, 14 Apr 2003 11:31:50 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA09136
	for <rddp-web-archive@ietf.org>; Mon, 14 Apr 2003 11:23:38 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1955qU-0002w2-00
	for rddp-web-archive@ietf.org; Mon, 14 Apr 2003 11:26:10 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 1955qU-0002vz-00
	for rddp-web-archive@ietf.org; Mon, 14 Apr 2003 11:26:10 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3EFV9830025;
	Mon, 14 Apr 2003 11:31:09 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3EFUG829967
	for <rddp@optimus.ietf.org>; Mon, 14 Apr 2003 11:30:16 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA09113
	for <rddp@ietf.org>; Mon, 14 Apr 2003 11:22:04 -0400 (EDT)
From: Black_David@emc.com
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1955oy-0002vd-00
	for rddp@ietf.org; Mon, 14 Apr 2003 11:24:36 -0400
Received: from mxic2.corp.emc.com ([128.221.31.40])
	by ietf-mx with esmtp (Exim 4.12)
	id 1955oy-0002vL-00
	for rddp@ietf.org; Mon, 14 Apr 2003 11:24:36 -0400
Received: by mxic2.corp.emc.com with Internet Mail Service (5.5.2653.19)
	id <2JMPZRCX>; Mon, 14 Apr 2003 11:24:11 -0400
Message-ID: <277DD60FB639D511AC0400B0D068B71E0564CB19@corpmx14.corp.emc.com>
To: cait@asomi.com
Cc: rddp@ietf.org
Date: Mon, 14 Apr 2003 11:19:08 -0400
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: [rddp] RE: STag invalidation
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

WG chair hat still off ...

> > > Without header checking *any* address that is open to *any*
> > > STag valid within the Protection Domain is targettable by an
> > > incorrect (and unchecked) DDP header. To be safe,
> > > applications would have to very actively invalidate STags
> > > after every received message.
> > 
> > An application that cares about data integrity and does not fully
> > trust its senders (there are lots of examples, as this is likely to
> > be the case for anything other than a tightly integrated cluster)
> > is likely to want to invalidate the STag after each received message
> > in order to avoid attacks based on a remote overwrite
> > after some application validity check has been made on the received
> > data.  See Section 4.1 of the DDP and RDMA Concerns draft.
> 
> If you don't trust a client, it is really only necessary to protect work
done
> for other clients from meddling. If an untrusted client wants to sabotage
> work done on their behalf, well let them.

Only if the work has no lasting value after it's done.  If I store a
file to a fileserver and 3 months later the fileserver tells me that
it is corrupted due to something that probably happened when the
file was originally stored ... I'm going to go buy a fileserver that
actually works unlike that piece of <bleep>.

> Creating a Protection Domain for each client is more than enough to ensure
> that clients cannot interfere with other clients (and standard application
layer
> security).

Only if "standard application security" includes some serious defenses
against attacks on the application integrity (generally denial of service,
but could be worse).  The scenario of concern is that a data structure
placed by  RDDP is used by application code that doesn't recheck its
assumptions at every if statement just in case the structure was
overwritten.  Again, invalidating the STag may well be in order.

> The original point is that without header protection, an error within a
> session can invalidate work done for that session -- unless almost
> draconian measures of STag invalidation are undertaken. That would
> *include* having no more than one operation pending at a time.

I think that conclusion starts from a decision not to trust the TCP/IP
checksums.  A different starting point could reach a different conclusion,
but I don't have a problem with a mandatory header CRC.

Thanks,
--David
----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
black_david@emc.com        Mobile: +1 (978) 394-7754
----------------------------------------------------
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Mon Apr 14 13:32:09 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA13274
	for <rddp-archive@odin.ietf.org>; Mon, 14 Apr 2003 13:32:09 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3EHdsk08599
	for rddp-archive@odin.ietf.org; Mon, 14 Apr 2003 13:39:54 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3EHds808596
	for <rddp-web-archive@optimus.ietf.org>; Mon, 14 Apr 2003 13:39:54 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA13237
	for <rddp-web-archive@ietf.org>; Mon, 14 Apr 2003 13:31:39 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1957qO-0003ip-00
	for rddp-web-archive@ietf.org; Mon, 14 Apr 2003 13:34:12 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 1957qN-0003il-00
	for rddp-web-archive@ietf.org; Mon, 14 Apr 2003 13:34:11 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3EHdD808513;
	Mon, 14 Apr 2003 13:39:13 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3EHZA807547
	for <rddp@optimus.ietf.org>; Mon, 14 Apr 2003 13:35:10 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA12948
	for <rddp@ietf.org>; Mon, 14 Apr 2003 13:26:55 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1957ln-0003gM-00
	for rddp@ietf.org; Mon, 14 Apr 2003 13:29:27 -0400
Received: from atlrel9.hp.com ([156.153.255.214])
	by ietf-mx with esmtp (Exim 4.12)
	id 1957ln-0003fe-00
	for rddp@ietf.org; Mon, 14 Apr 2003 13:29:27 -0400
Received: from rosemail.rose.hp.com (rosemail.rose.hp.com [15.96.64.26])
	by atlrel9.hp.com (Postfix) with ESMTP
	id 017F91C0178C; Mon, 14 Apr 2003 13:28:09 -0400 (EDT)
Received: from swathi (dhcp-rosefl-bba009.rose.hp.com [15.3.110.9]) by rosemail.rose.hp.com with SMTP (8.7.1/8.7.3 SMKit7.02) id KAA27316; Mon, 14 Apr 2003 10:28:08 -0700 (PDT)
Message-ID: <002e01c302ab$3688bd20$096e030f@rose.hp.com>
From: "Mallikarjun C." <cbm@rose.hp.com>
To: <rddp@ietf.org>
Subject: Re: [rddp] Options and Markers
Date: Mon, 14 Apr 2003 10:28:06 -0700
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2600.0000
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

My take-away from the San Francisco RDDP meeting was that there
was rough consensus for mandating markers, though there was quite a 
bit of discussion around marker interval (I just went back to the minutes
and they seem to confirm this recollection).  I am somewhat surprised 
to see this discussion brought up again.

>design decisions which exclude
> that class will tend to run afoul of the admonition not to
> impose requirements on implementors that are not required for
> interoperability (see Section 6 of RFC 2119).

This WG is trying to solve some architectural problems around 
performance/scalability bottlenecks.  Markers is one of the architectural 
elements designed to solve those bottlenecks and once we accept it's a 
desired element of the architecture, it is of course *required* for 
interoperability to implement and use it.  

So I guess you are questioning the value of the markers as a desired
element of RDDP/TCP architecture.  Again, I thought we're past that 
question.

>The implementation I referred to above
> is based on the VI architecture and runs it over TCP/IP with
> all RDMA processing done outboard.

Let's not assume that a commercially available VI-based adapter running
at relatively low speeds, compared to what this WG is trying to design for, 
defines the "architectural best practices" for RDDP/TCP.  I would be much
more convinced about this VI-based design if there is "running code" for 10G.
--
Mallikarjun

Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions
Hewlett-Packard MS 5668 
Roseville CA 95747
cbm@rose.hp.com

----- Original Message ----- 
From: <Black_David@emc.com>
To: <Dwight.Barron@hp.com>; <rddp@ietf.org>
Sent: Sunday, April 13, 2003 12:51 PM
Subject: RE: [rddp] Options and Markers


> Dwight,
> 
> > > I have no problem with robustness considerations, but the vehemence
> > > of opposition to options in any form from a significant portion of
> > > this WG has been among the things that have hindered progress.
> > 
> > I do not believe it is a "vehemence of opposition to options" 
> > per se, but rather a strong desire to see valid engineering 
> > justification for such options. I believe we established, per 
> > your comments at the last F2F, that receive side copying is 
> > not likely to be avoided, regardless of marker interval and 
> > we went on to focus on transmit side impact of marker 
> > interval options.
> 
> The discussion in San Francisco concerned a receive side full-software
> implementation.  It did not cover a "full-offload with full TCP buffering"
> class of implementation, as the discussion would up considering only
> full-software and "flow-through" RNIC implementations.  There's a third
> class that is relevant.
> 
> [... material on full-software implementations snipped ... ]
> 
> > > I think some form of option is going to be hard to avoid for
> > > markers, as there is "running code" out there for a class of
> > > implementations (RDDP + TCP with full buffering offloaded to
> > > a souped up NIC) that do not need markers.  Requiring that this
> > > class of implementation use markers solely to avoid putting an
> > > option in the protocol appears to be very hard to justify.
> > 
> > How does this argument not cross the line from what I would 
> > consider debate in search of best architectural practice for 
> > RDDP to that of defending an existing RDDP implementation of 
> > "running code". What RDDP specification was this running code 
> > conformant to such that it should impact the design of future 
> > RDDP specifications? 
> 
> The assertion that there is a single "best architectural practice"
> for RDDP and it is a "flow-through RNIC" with minimal buffering
> is part of the problem.  The implementation I referred to above
> is based on the VI architecture and runs it over TCP/IP with
> all RDMA processing done outboard.  AFAIK, the protocol used
> is specific to that implementation; among the reasons for this is
> that VI is an interface, not a protocol.
> 
> The reason that this discussion doesn't cross the line, is that
> the "running code" serves to demonstrate the viability of the
> third class of implementations, speaking to your above desire for
> "valid engineering justification" for protocol design concerns
> raised by that class.  While I can't speak for the implementers,
> I think they plan to update the implementation in question to
> whatever specifications the RDDP WG adopts.  Given the viability
> of that class of implementation (FWIW, it was extensively discussed
> prior to formation of the RDDP WG), design decisions which exclude
> that class will tend to run afoul of the admonition not to
> impose requirements on implementors that are not required for
> interoperability (see Section 6 of RFC 2119).
> 
> Thanks,
> --David
> ----------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 176 South St., Hopkinton, MA  01748
> +1 (508) 293-7953             FAX: +1 (508) 293-7786
> black_david@emc.com        Mobile: +1 (978) 394-7754
> ----------------------------------------------------
> _______________________________________________
> rddp mailing list
> rddp@ietf.org
> https://www1.ietf.org/mailman/listinfo/rddp
> 

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Mon Apr 14 13:32:11 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA13288
	for <rddp-archive@odin.ietf.org>; Mon, 14 Apr 2003 13:32:11 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3EHdtU08615
	for rddp-archive@odin.ietf.org; Mon, 14 Apr 2003 13:39:55 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3EHdt808612
	for <rddp-web-archive@optimus.ietf.org>; Mon, 14 Apr 2003 13:39:55 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA13241
	for <rddp-web-archive@ietf.org>; Mon, 14 Apr 2003 13:31:40 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1957qP-0003iv-00
	for rddp-web-archive@ietf.org; Mon, 14 Apr 2003 13:34:13 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 1957qP-0003is-00
	for rddp-web-archive@ietf.org; Mon, 14 Apr 2003 13:34:13 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3EHdF808555;
	Mon, 14 Apr 2003 13:39:15 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3EHcf808490
	for <rddp@optimus.ietf.org>; Mon, 14 Apr 2003 13:38:41 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA13071;
	Mon, 14 Apr 2003 13:30:25 -0400 (EDT)
From: Black_David@emc.com
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1957pC-0003i5-00; Mon, 14 Apr 2003 13:32:58 -0400
Received: from [128.222.32.10] (helo=mxic1.corp.emc.com)
	by ietf-mx with esmtp (Exim 4.12)
	id 1957pC-0003ho-00; Mon, 14 Apr 2003 13:32:58 -0400
Received: by mxic1.corp.emc.com with Internet Mail Service (5.5.2653.19)
	id <2PQXX7ZX>; Mon, 14 Apr 2003 13:32:19 -0400
Message-ID: <277DD60FB639D511AC0400B0D068B71E0564CB1B@corpmx14.corp.emc.com>
To: minutes@ietf.org
Cc: rddp@ietf.org
Date: Mon, 14 Apr 2003 13:26:48 -0400
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: [rddp] RDDP WG San Francisco Final meeting minutes
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

RDDP WG Meeting Minutes
56th IETF
March 17 & 20 2003
------------------

Thanks to Allyn Romanow for taking notes.

-- TCP Mapping Requirements (David Black)

This is a reminder from the WG chair in combination with the Area Directors
of what is expected of the RDDP WG's TCP mapping work.  

(1) The "proof" requirement that two communicating stacks can't get confused
about the format of communication applies to the entire RDDP protocol stack
running over TCP, not just the issue of whether a modified vs. unmodified
TCP
is being used.  32 bits of zeroes (first marker) to distinguish what is
going
on is not enough, 128 bits randomly generated for the connection is enough
(sufficient, but may be more than needed).  Relying on ULP to do this is
considerably weaker than solving the problem in RDDP.  Negotiation is not
required - a declaration mechanism like the SCTP adaption indication
exchange
is sufficient (no additional round trip is required).

(2) The nfsv4 WG is starting work on application of RDDP to NFSv4.  NFSv4's
current level of data integrity does not require a CRC; if this remains
the case for NFSv4 over RDDP, the RDDP CRC will need to be optional to use.

(3) Fixed interval markers are a potential problem for software
implementations
(see below, as a great deal of time was spent on this issue later).

-- Administrivia

Reminder that the problem statement and architecture drafts are in WG
Last Call.  Please read and comment on the list.

-- SCTP Mapping Draft (Randy Stewart)

See slides (will be posted).  Draft is relatively far along.
Note that SCTP uses (completely) unordered delivery when carrying RDDP
	traffic.  This requires DDP to track its operation ordering
	in order to enforce any delivery order requirements.

-- Applicability Statement Draft (Randy Stewart for Caitlin Bestler)

Plan has changed to a 3 draft approach - (1) TCP mapping, (2) SCTP mapping,
and (3) an applicability statement draft covering both mappings.  Lode Coene
(editor of SCTP's applicability statement, which set a good example) has
agreed to help out with the RDDP applicability statement (many thanks).

Advice from the WG chairs and ADs: A good applicability statement is
guidance on utility, not advertising to promote usage.  An important
target audience is ULP designers who are looking advice on whether
to use RDDP and with which transport.

Both the SCTP mapping draft and applicability statement draft were approved
to be official RDDP WG drafts - next versions will have draft-ietf-rddp-*
names.

-- Allyn Romanow - NICELI Workshop reminder and security update

For NICELI, see:
http://www.acm.org/sigs/sigcomm/sigcomm2003/workshop/niceli/

Security update - folks working on this issue are making good progress
with Catherine Meadows (security advisor).  Initial draft will be posted
for discussion in the next 2-3 weeks.

-- DDP Update -  Hemal Shah
couple of small changes.  

-- RDMAP Update - Paul Culley (for Renato Recio)
a couple of small changes (no presentation slides)

-- TCP mapping
- MPA Update - Paul Culley

Packing of multiple ULPDUS into single TCP segment is now allowed.
MPA is now specified as an "adaptation layer" for RDDP that must
be explicitly enabled by higher-level protocol (i.e., DDP/RDMA or
ULP further up the stack).

- IFT update - Jim Williams

Added adaption indication, based to some extent on the one used
with SCTP.  CRC is optional and negotiable
Markers are negotiable, with some details to be fleshed out.
Speculative placement proposed for use when markers not provided -
	this received a rather negative reaction in the meeting.

-- TCP mapping issues

David Black worked through several major outstanding issues for the WG:

(1) Alignment and TCP Mapping

This is about aligning transmitted ULPDUS (w/RDDP headers) to TCP segment
boundaries.  

David Black summarized from the mailing list discussion and made a proposal
Alignment is not required for interoperability, the RDDP receiver
cannot rely on it, but it is a potentially useful optimization.

Therefore, the proposal is to treat alignment as an implementation
	optimization only, and hence not use any MUST or SHOULD (in the
	RFC 2119 sense) for alignment, but instead use (at most) lower
	case "should" along with a description of the circumstances
	in which alignment is a useful optimization.

This represents rough consensus of the Working Group participants in the
room.

Additional note from WG chair: While nothing is certain, this approach is
	very likely to be acceptable to the tsvwg (Transport Area) WG, where
	past alignment work has generated controversy.

(2) Markers - this generated a *lot* of discussion, mostly focused on marker
interval.

Markers are motivated by a  need to regain synchronization for placement
after a packet drop or reordering without having to waiting for retransmit.
Markers allow finding where the header is in byte stream.
MPA currently requires markers at 512 byte fixed intervals

A potential problem with markers is the overhead for existing stacks
For example, for an 8KB transfer, 1 send operation becomes 16 sends
(Some people suggest fewer than 16 because can use gather technique.)
This overhead is a potential problem for a software implementation.

Markers have been motivated by flow through designs that may be overly
optimized, especially given above rough consensus on alignment.  The
discussion focused on buffering requirements, and whether a link
frame per connection is required - it clearly is in the worst case,
but there are enough shorter packets/frames (e.g., about 500 bytes
on Ethernet) that on average across multiple connections, one can
use less, and hence a marker interval shorter than the max link
frame size (e.g., about 1500 bytes for Ethernet without jumbo frames)
does save implementation resources.

There was at least one comment that Markers should be optional.

There is no rough consensus, but David notes that there is a lot of
support for markers and the 512 byte interval in particular.

(3) NFSV4 and CRCs
The nfsv4 WG charter now includes NFSV4 over RDDP problem statement
and requirements.

NFS does not require stronger data integrity than TCP/IP checksums, so
a mandatory RDDP CRC may be excessive although optional RDDP CRC could
be useful.

The argument in the other direction is that RDDP is new and wants to
provide a better level of service (data integrity), so demanding that
all users of RDDP step up to that level of service is within reason.

After much discussion the conclusion is to defer the issue of whether
to require CRC usage for further input from the nfsv4 WG's requirements
work.

-David concludes:

alignment - there is rough consensus in the room, will recheck on list.
marker interval - there is clear support for a 512B interval
CRC - will defer "MUST use" issue until nfsv4 WG is further along in
	considering requirements for RDDP with NFSv4.

Hope to close remaining issues in this space by summer IETF (Vienna)
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Mon Apr 14 15:27:17 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA19111
	for <rddp-archive@odin.ietf.org>; Mon, 14 Apr 2003 15:27:17 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3EJZ3p16665
	for rddp-archive@odin.ietf.org; Mon, 14 Apr 2003 15:35:03 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3EJZ2816662
	for <rddp-web-archive@optimus.ietf.org>; Mon, 14 Apr 2003 15:35:02 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA19080
	for <rddp-web-archive@ietf.org>; Mon, 14 Apr 2003 15:26:46 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1959dl-0004hH-00
	for rddp-web-archive@ietf.org; Mon, 14 Apr 2003 15:29:18 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 1959dl-0004hD-00
	for rddp-web-archive@ietf.org; Mon, 14 Apr 2003 15:29:17 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3EJYD816611;
	Mon, 14 Apr 2003 15:34:13 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3EJXq816591
	for <rddp@optimus.ietf.org>; Mon, 14 Apr 2003 15:33:52 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA19044
	for <rddp@ietf.org>; Mon, 14 Apr 2003 15:25:36 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1959ce-0004gl-00
	for rddp@ietf.org; Mon, 14 Apr 2003 15:28:08 -0400
Received: from emulex.emulex.com ([138.239.112.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 1959cd-0004gS-00
	for rddp@ietf.org; Mon, 14 Apr 2003 15:28:07 -0400
Received: from xbl.ad.emulex.com (xbl.ma.emulex.com [172.16.12.11])
	by emulex.emulex.com (8.12.9/8.12.8) with ESMTP id h3EJRfLW017781
	for <rddp@ietf.org>; Mon, 14 Apr 2003 12:27:42 -0700 (PDT)
Received: by xbl.ma.emulex.com with Internet Mail Service (5.5.2653.19)
	id <HN1WWVDP>; Mon, 14 Apr 2003 15:27:41 -0400
Message-ID: <3356669BBE90C448AD4645C843E2BF289B9059@xbl.ma.emulex.com>
From: "Williams, Jim" <Jim.Williams@Emulex.com>
To: rddp@ietf.org
Subject: RE: [rddp] Options and Markers
Date: Mon, 14 Apr 2003 15:27:40 -0400
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="ISO-8859-1"
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>



> -----Original Message-----
> From: Mallikarjun C. [mailto:cbm@rose.hp.com]
> Sent: Monday, April 14, 2003 1:28 PM
> To: rddp@ietf.org
> Subject: Re: [rddp] Options and Markers
> 
> 
> My take-away from the San Francisco RDDP meeting was that there
> was rough consensus for mandating markers,

There was certainly not unanimous consensus.  I guess it's 
David's call as to what constitutes rough consensus.

> So I guess you are questioning the value of the markers as a desired
> element of RDDP/TCP architecture.  Again, I thought we're past that 
> question.

I have certainly questioned the value of markers and continue
to question it.  Nothing has transpired that led me to 
believe we were "past that question".

> > The implementation I referred to above
> > is based on the VI architecture and runs it over
> > TCP/IP with all RDMA processing done outboard.
> 
> Let's not assume that a commercially available 
> VI-based adapter running
> at relatively low speeds, compared to what 
> this WG is trying to design for, 
> defines the "architectural best practices" 
> for RDDP/TCP.  I would be much
> more convinced about this VI-based design 
> if there is "running code" for 10G.

I don't presume that this VI-based design
resolves the issue one way or the other,
but it suggests that there is at least one
design point that works better without markers.
I personally don't see any fundamental reason
that this design approach won't scale to 
much higher speeds, but as you point out
this is not convincing until it is actually
done.

Much more to the point, though, is that there
are no working examples (correct me if I am wrong)
of designs that successfully exploit markers.
iSCSI defines optional markers, and iSCSI
implementations that use these to advantage
are conspicuous by their absence.

There are a couple different reasons to add
options to a protocol.  One is that it is 
sometimes beneficial for two different
modes to co-exist.

A second important reason is that during
the initial development of a protocol, it
is often not clear what will and will not
be desirable features long term.  This is
particularly true when there are not yet
any interoperable implementations.  If the
option is successful, it will become a de
facto standard and always be implemented.
If it is not successful, it will be dropped
and forgotten.

Others can comment on the IETF standards
process,  but perhaps it makes sense that
options be defined as the "proposed standard"
level, and then become either mandated or
eliminated as the standard progresses along
the standards track, and experience is gained.

Markers are a feature which is not required
for functional correctness of the protocol,
adds complexity to the protocol, is claimed
(but not demonstrated) to improve the
cost/performance of a subset of the possible
implementations, and hurts the cost/performance
of a different subset of the possible 
implementations.  I would urge that 
significant caution be used when
considering mandating such a feature. 




_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Mon Apr 14 16:47:07 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA21169
	for <rddp-archive@odin.ietf.org>; Mon, 14 Apr 2003 16:47:07 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3EKstT22288
	for rddp-archive@odin.ietf.org; Mon, 14 Apr 2003 16:54:55 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3EKst822285
	for <rddp-web-archive@optimus.ietf.org>; Mon, 14 Apr 2003 16:54:55 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA21159
	for <rddp-web-archive@ietf.org>; Mon, 14 Apr 2003 16:46:37 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195At2-00054U-00
	for rddp-web-archive@ietf.org; Mon, 14 Apr 2003 16:49:09 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 195At2-00054R-00
	for rddp-web-archive@ietf.org; Mon, 14 Apr 2003 16:49:08 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3EKsD822257;
	Mon, 14 Apr 2003 16:54:13 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3EKrg822220
	for <rddp@optimus.ietf.org>; Mon, 14 Apr 2003 16:53:42 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA21103
	for <rddp@ietf.org>; Mon, 14 Apr 2003 16:45:23 -0400 (EDT)
From: Black_David@emc.com
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195Arr-000540-00
	for rddp@ietf.org; Mon, 14 Apr 2003 16:47:55 -0400
Received: from maho3msx2.corp.emc.com ([128.221.11.32])
	by ietf-mx with esmtp (Exim 4.12)
	id 195Arr-00053q-00
	for rddp@ietf.org; Mon, 14 Apr 2003 16:47:55 -0400
Received: by maho3msx2.corp.emc.com with Internet Mail Service (5.5.2653.19)
	id <2PLN3FFH>; Mon, 14 Apr 2003 16:47:31 -0400
Message-ID: <277DD60FB639D511AC0400B0D068B71E0564CB22@corpmx14.corp.emc.com>
To: rddp@ietf.org
Subject: RE: [rddp] Options and Markers
Date: Mon, 14 Apr 2003 16:42:25 -0400
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

> > My take-away from the San Francisco RDDP meeting was that there
> > was rough consensus for mandating markers,
> 
> There was certainly not unanimous consensus.  I guess it's 
> David's call as to what constitutes rough consensus.

It is, and the issue of whether to mandate markers or not was
not part of the discussion in the San Francisco meeting - that is
very much an open issue.

> > So I guess you are questioning the value of the markers as a desired
> > element of RDDP/TCP architecture.  Again, I thought we're past that 
> > question.

We did get past that question - but notice the use of "desired" as
opposed to "mandatory" in the way that Mallikarjun phrased it - things
that aren't "desired" should not be put into a specification, and
markers clearly are "desired".  The conclusion from SF is that markers
will need to be specified with a 512 byte interval.  This discussion
is about "optional to use" and "optional to implement for receivers",
which are different issues.

> > Let's not assume that a commercially available VI-based adapter
> > running at relatively low speeds, compared to what this WG is
> > trying to design for, defines the "architectural best practices"
> > for RDDP/TCP.  I would be much more convinced about this VI-based
> > design if there is "running code" for 10G.

That appears perpetuate the incorrect assumption that high speed
hardware implementations are the only thing worth making design
decisions for.  Please see my previous comments on this topic.

Thanks,
--David
----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
black_david@emc.com        Mobile: +1 (978) 394-7754
----------------------------------------------------
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Tue Apr 15 00:00:54 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA02496
	for <rddp-archive@odin.ietf.org>; Tue, 15 Apr 2003 00:00:54 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3F48p618856
	for rddp-archive@odin.ietf.org; Tue, 15 Apr 2003 00:08:51 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3F48p818853
	for <rddp-web-archive@optimus.ietf.org>; Tue, 15 Apr 2003 00:08:51 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA02493
	for <rddp-web-archive@ietf.org>; Tue, 15 Apr 2003 00:00:23 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195Hep-0007P8-00
	for rddp-web-archive@ietf.org; Tue, 15 Apr 2003 00:02:55 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 195Hep-0007P5-00
	for rddp-web-archive@ietf.org; Tue, 15 Apr 2003 00:02:55 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3F48B818824;
	Tue, 15 Apr 2003 00:08:11 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3F47L818412
	for <rddp@optimus.ietf.org>; Tue, 15 Apr 2003 00:07:22 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA02447
	for <rddp@ietf.org>; Mon, 14 Apr 2003 23:58:54 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195HdO-0007Og-00
	for rddp@ietf.org; Tue, 15 Apr 2003 00:01:26 -0400
Received: from thebe.your-site.com ([140.186.45.26])
	by ietf-mx with esmtp (Exim 4.12)
	id 195HdO-0007Oc-00
	for rddp@ietf.org; Tue, 15 Apr 2003 00:01:26 -0400
Received: from 192.168.0.15 (nmd.ssb.sjc.wayport.net [64.134.12.62])
	by thebe.your-site.com (Postfix) with ESMTP
	id 831712448D5; Tue, 15 Apr 2003 00:01:09 -0400 (EDT)
Date: Mon, 14 Apr 2003 23:01:05 -0500
From: Caitlin Bestler <cait@asomi.com>
Subject: RE: [rddp] Options and Markers
To: Black_David@emc.com
Cc: rddp@ietf.org
X-Priority: 3
In-Reply-To: <277DD60FB639D511AC0400B0D068B71E0564CB22@corpmx14.corp.emc.com>
Message-ID: <r01050300-1024-E1B51BE06EF611D78329000393814604@[192.168.0.15]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

> > > My take-away from the San Francisco RDDP meeting was that there
> > > was rough consensus for mandating markers,
> > 
> > There was certainly not unanimous consensus.  I guess it's 
> > David's call as to what constitutes rough consensus.
> 
> It is, and the issue of whether to mandate markers or not was
> not part of the discussion in the San Francisco meeting - that is
> very much an open issue.
> 
> > > So I guess you are questioning the value of the markers as a desired
> > > element of RDDP/TCP architecture.  Again, I thought we're past that 
> > > question.
> 
> We did get past that question - but notice the use of "desired" as
> opposed to "mandatory" in the way that Mallikarjun phrased it - things
> that aren't "desired" should not be put into a specification, and
> markers clearly are "desired".  The conclusion from SF is that markers
> will need to be specified with a 512 byte interval.  This discussion
> is about "optional to use" and "optional to implement for receivers",
> which are different issues.
> 

This is a very important distinction.

The strongest justification for markers is that they enable low-memory
high-speed RNICs. I believe the case has been made for allowing such RNICs to
mandate markers on connections sent to them.

I also believe that the case has been made that for memory-rich receivers, that
the markers are a nuisance and a slight bandwidth waste.

That leave the real issue being assessing which is the greater imposition:

-- forcing senders to support both marker and no-marker mode.
-- forcing memory-rich receivers to implement markers.

The primary virtue of the latter is that it avoids options. Allowing recievers
to mandate "no markers" is clearly superior for pure-software RDDP stacks
written on top of sockets.
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Tue Apr 15 00:21:44 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA02815
	for <rddp-archive@odin.ietf.org>; Tue, 15 Apr 2003 00:21:44 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3F4TfZ19603
	for rddp-archive@odin.ietf.org; Tue, 15 Apr 2003 00:29:41 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3F4Tf819600
	for <rddp-web-archive@optimus.ietf.org>; Tue, 15 Apr 2003 00:29:41 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA02806
	for <rddp-web-archive@ietf.org>; Tue, 15 Apr 2003 00:21:13 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195Hyz-0007UK-00
	for rddp-web-archive@ietf.org; Tue, 15 Apr 2003 00:23:45 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 195Hyy-0007UH-00
	for rddp-web-archive@ietf.org; Tue, 15 Apr 2003 00:23:44 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3F4T1819570;
	Tue, 15 Apr 2003 00:29:01 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3F4So819549
	for <rddp@optimus.ietf.org>; Tue, 15 Apr 2003 00:28:50 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA02790
	for <rddp@ietf.org>; Tue, 15 Apr 2003 00:20:22 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195HyA-0007Tp-00
	for rddp@ietf.org; Tue, 15 Apr 2003 00:22:54 -0400
Received: from thebe.your-site.com ([140.186.45.26])
	by ietf-mx with esmtp (Exim 4.12)
	id 195Hy9-0007Te-00
	for rddp@ietf.org; Tue, 15 Apr 2003 00:22:53 -0400
Received: from 192.168.0.15 (nmd.ssb.sjc.wayport.net [64.134.12.62])
	by thebe.your-site.com (Postfix) with ESMTP
	id 953C02450CD; Tue, 15 Apr 2003 00:22:40 -0400 (EDT)
Date: Mon, 14 Apr 2003 23:22:36 -0500
From: Caitlin Bestler <cait@asomi.com>
Subject: Re: [rddp] RDDP WG San Francisco Final meeting minutes
To: Black_David@emc.com, rddp@ietf.org
X-Priority: 3
In-Reply-To: <277DD60FB639D511AC0400B0D068B71E0564CB1B@corpmx14.corp.emc.com>
Message-ID: <r01050300-1024-E331FD7C6EF911D78329000393814604@[192.168.0.15]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

> (1) The "proof" requirement that two communicating stacks can't get
> confused about the format of communication applies to the entire RDDP
> protocol stack running over TCP, not just the issue of whether a
> modified vs. unmodified TCP is being used.  32 bits of zeroes (first
> marker) to distinguish what is going on is not enough, 128 bits
> randomly generated for the connection is enough (sufficient, but may
> be more than needed).  Relying on ULP to do this is considerably
> weaker than solving the problem in RDDP.  Negotiation is not required
> - a declaration mechanism like the SCTP adaption indication exchange
> is sufficient (no additional round trip is required).

I am somewhat perplexed by this.

I am unaware of any case being made for this either on rddp or tsvwg.
Further the requirement seems to be unique. It does not apply to TPS,
RPC, SOAP, iSCSI or any of a thousand other TCP ULPs.

The probable reaction at this point is "we've covered this", but I don't
believe that we have. It's been reported as a given. The rationale has
never been presented. What problem is being addressed? What assumptions
are being made about the cost? If this is a general new direction for all new
connection establishment? If so, where was the consensus reached? Has it been
(or will it) discussed on tsvwg? If not there, where?

ULP responsibility for matching peers has been the rule for IP protocols
throughout history. Why the change? And at what cost? Is it even possible
to truly validate with randomly generated data without delaying the
start of real work?

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Tue Apr 15 12:14:44 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA02883
	for <rddp-archive@odin.ietf.org>; Tue, 15 Apr 2003 12:14:43 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3FGMt616388
	for rddp-archive@odin.ietf.org; Tue, 15 Apr 2003 12:22:55 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3FGMt816385
	for <rddp-web-archive@optimus.ietf.org>; Tue, 15 Apr 2003 12:22:55 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA02826
	for <rddp-web-archive@ietf.org>; Tue, 15 Apr 2003 12:14:13 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195T6y-0003Ky-00
	for rddp-web-archive@ietf.org; Tue, 15 Apr 2003 12:16:44 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 195T6y-0003Kv-00
	for rddp-web-archive@ietf.org; Tue, 15 Apr 2003 12:16:44 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3FGM1816331;
	Tue, 15 Apr 2003 12:22:01 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3FGJk816144
	for <rddp@optimus.ietf.org>; Tue, 15 Apr 2003 12:19:46 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA02684
	for <rddp@ietf.org>; Tue, 15 Apr 2003 12:11:04 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195T3v-0003JM-00
	for rddp@ietf.org; Tue, 15 Apr 2003 12:13:35 -0400
Received: from ztxmail03.ztx.compaq.com ([161.114.1.207])
	by ietf-mx with esmtp (Exim 4.12)
	id 195T3u-0003Is-00
	for rddp@ietf.org; Tue, 15 Apr 2003 12:13:34 -0400
Received: from cceexg13.americas.cpqcorp.net (cceexg13.americas.cpqcorp.net [16.110.250.119])
	by ztxmail03.ztx.compaq.com (Postfix) with ESMTP
	id 1E2CC87ED; Tue, 15 Apr 2003 11:12:12 -0500 (CDT)
Received: from cceexc16.americas.cpqcorp.net ([16.110.250.58]) by cceexg13.americas.cpqcorp.net with Microsoft SMTPSVC(5.0.2195.2966);
	 Tue, 15 Apr 2003 11:12:08 -0500
X-MimeOLE: Produced By Microsoft Exchange V6.0.6375.0
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: RE: [rddp] Options and Markers
Date: Tue, 15 Apr 2003 11:12:07 -0500
Message-ID: <01CA1B1F74A4EC40A9C887F7B58F872403143137@cceexc16.americas.cpqcorp.net>
Thread-Topic: [rddp] Options and Markers
Thread-Index: AcMB9tnnQYmh1TQXQPKmLQeplje2iQBba/Lg
From: "Barron, Dwight" <Dwight.Barron@hp.com>
To: <Black_David@emc.com>, <rddp@ietf.org>
X-OriginalArrivalTime: 15 Apr 2003 16:12:08.0369 (UTC) FILETIME=[C38C0A10:01C30369]
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by www1.ietf.org id h3FGJk816145
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 8bit
Content-Transfer-Encoding: 8bit



> -----Original Message-----
> From: Black_David@emc.com [mailto:Black_David@emc.com]
> Sent: Sunday, April 13, 2003 2:52 PM
> To: Barron, Dwight; rddp@ietf.org
> Subject: RE: [rddp] Options and Markers
> 
> 
> Dwight,
> 
> > > I have no problem with robustness considerations, but the 
> vehemence
> > > of opposition to options in any form from a significant portion of
> > > this WG has been among the things that have hindered progress.
> > 
> > I do not believe it is a "vehemence of opposition to options" 
> > per se, but rather a strong desire to see valid engineering 
> > justification for such options. I believe we established, per 
> > your comments at the last F2F, that receive side copying is 
> > not likely to be avoided, regardless of marker interval and 
> > we went on to focus on transmit side impact of marker 
> > interval options.
> 
> The discussion in San Francisco concerned a receive side full-software
> implementation.  It did not cover a "full-offload with full 
> TCP buffering"
> class of implementation, as the discussion would up considering only
> full-software and "flow-through" RNIC implementations.  
> There's a third
> class that is relevant.

I haven't heard anyone assert that this relevant third class implementation does not fall into one of the two previously mentioned catagories. I would suspect, given the embedded nature of such designs, that they have relatively unencumbered access to a NIC with scatter/gather capability. 

> 
> [... material on full-software implementations snipped ... ]
> 
> > > I think some form of option is going to be hard to avoid for
> > > markers, as there is "running code" out there for a class of
> > > implementations (RDDP + TCP with full buffering offloaded to
> > > a souped up NIC) that do not need markers.  Requiring that this
> > > class of implementation use markers solely to avoid putting an
> > > option in the protocol appears to be very hard to justify.
> > 
> > How does this argument not cross the line from what I would 
> > consider debate in search of best architectural practice for 
> > RDDP to that of defending an existing RDDP implementation of 
> > "running code". What RDDP specification was this running code 
> > conformant to such that it should impact the design of future 
> > RDDP specifications? 
> 
> The assertion that there is a single "best architectural practice"
> for RDDP and it is a "flow-through RNIC" with minimal buffering
> is part of the problem.  

I'm sorry, but I disagree. This is the Remote Direct Data Placement WG, not the Remote Store and Forward WG. From the WG Description at https://www1.ietf.org/html.charters/rddp-charter.html ..RDDP capabilities refer to the ability to place data directly from the NIC into application buffers, without intensive CPU usage. This strategy avoids the costs of data copying and enables using IP as a method for high speed buffer to buffer transfer....
I have no quarrel with anyone that wants to use assembly buffers in their design, every design will need some amount of buffering. I do take exception that we mandate an option that essentially diminishes or precludes a flow-through design for those of us that are taking the *Direct* part seriously. 

> The implementation I referred to above
> is based on the VI architecture and runs it over TCP/IP with
> all RDMA processing done outboard.  AFAIK, the protocol used
> is specific to that implementation; among the reasons for this is
> that VI is an interface, not a protocol.
> The reason that this discussion doesn't cross the line, is that
> the "running code" serves to demonstrate the viability of the
> third class of implementations, speaking to your above desire for
> "valid engineering justification" for protocol design concerns
> raised by that class.  While I can't speak for the implementers,
> I think they plan to update the implementation in question to
> whatever specifications the RDDP WG adopts.  Given the viability
> of that class of implementation (FWIW, it was extensively discussed
> prior to formation of the RDDP WG), design decisions which exclude
> that class will tend to run afoul of the admonition not to
> impose requirements on implementors that are not required for
> interoperability (see Section 6 of RFC 2119).

I cannot find said product available for sale today, I have been told it is withdrawn from the market and it is not on the vendor's website (if this comment seems untowardly callous, the same is true for my own company's VI product). That said, I have not heard that the product is incapable of supporting markers. Given the general catagory of a dedicated CPU and souped up NIC capable of being upgraded by firmware, does that product become (more) commercially unviable due to the overhead of inserting markers? As for excluding a class as a whole, I think we are down to quibbling whether an implementation should use an XXX or YYY MHz embedded processor to hit ZZZ% of wire speed. 

> 
> Thanks,
> --David
> ----------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 176 South St., Hopkinton, MA  01748
> +1 (508) 293-7953             FAX: +1 (508) 293-7786
> black_david@emc.com        Mobile: +1 (978) 394-7754
> ----------------------------------------------------
> 
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Tue Apr 15 13:43:58 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA05225
	for <rddp-archive@odin.ietf.org>; Tue, 15 Apr 2003 13:43:58 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3FHqCA22808
	for rddp-archive@odin.ietf.org; Tue, 15 Apr 2003 13:52:12 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3FHqB822805
	for <rddp-web-archive@optimus.ietf.org>; Tue, 15 Apr 2003 13:52:11 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA05202
	for <rddp-web-archive@ietf.org>; Tue, 15 Apr 2003 13:43:27 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195UVL-0003iE-00
	for rddp-web-archive@ietf.org; Tue, 15 Apr 2003 13:45:59 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 195UVK-0003iA-00
	for rddp-web-archive@ietf.org; Tue, 15 Apr 2003 13:45:58 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3FHpD822760;
	Tue, 15 Apr 2003 13:51:13 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3FHlb822619
	for <rddp@optimus.ietf.org>; Tue, 15 Apr 2003 13:47:37 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA05069
	for <rddp@ietf.org>; Tue, 15 Apr 2003 13:38:53 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195UQu-0003gR-00
	for rddp@ietf.org; Tue, 15 Apr 2003 13:41:24 -0400
Received: from dmz1.silverbacksystems.com ([65.172.158.82])
	by ietf-mx with esmtp (Exim 4.12)
	id 195UQt-0003g3-00
	for rddp@ietf.org; Tue, 15 Apr 2003 13:41:24 -0400
Received: from ns.silverbacksystems.com (gate-camp-hme0.silverbacksystems.com [65.172.158.93])
	by dmz1.silverbacksystems.com (Postfix on SuSE Linux 7.3 (i386)) with ESMTP id C7DB6B9D3
	for <rddp@ietf.org>; Tue, 15 Apr 2003 10:40:42 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1])
	by ns.silverbacksystems.com (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 9813C1E1D0
	for <rddp@ietf.org>; Tue, 15 Apr 2003 10:40:42 -0700 (PDT)
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
	charset="iso-8859-1"
Date: Tue, 15 Apr 2003 10:40:21 -0700
From: "Somesh Gupta" <somesh_gupta@silverbacksystems.com>
Importance: Normal
In-Reply-To: <01CA1B1F74A4EC40A9C887F7B58F872403143137@cceexc16.americas.cpqcorp.net>
Message-ID: <NMEALCLOIBCHBDHLCMIJEEINDDAA.somesh_gupta@silverbacksystems.com>
MIME-Version: 1.0
Received: from ns.silverbacksystems.com (localhost [127.0.0.1])
	by localhost (AvMailGate-2.0.0.6) id 24978-432CBD1E;
	Tue, 15 Apr 2003 10:40:08 -0700
Received: from somesh (somesh.corp.silverbacksystems.com [10.0.16.35])
	by ns.silverbacksystems.com (Postfix on SuSE Linux eMail Server 3.0) with SMTP id E4B311E60D
	for <rddp@ietf.org>; Tue, 15 Apr 2003 10:40:07 -0700 (PDT)
Subject: RE: [rddp] Options and Markers
To: <rddp@ietf.org>
X-AntiVirus: OK! AntiVir MailGate Version 2.0.0.6
	 at ns.silverbacksystems.com has not found any known virus in this email.
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700
X-MSMail-Priority: Normal
X-Priority: 3 (Normal)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

Dwight,

My reading of the charter of the workgroup is that
the rddp protocol address the problem of data copying
in the host (from the NIC into application buffers).

The charter does not indicate how the NIC is implemented.

When the Work Group was being chartered, there was a
fairly big debate about whether there should even be
a TCP mapping, since TCP would always require some
buffering or roto-tilling of TCP (the argument is over
how much buffering). Those of us who wanted
a mapping over TCP pushed for it on the ground that
we could achieve the mapping without trying to create a
different version of TCP - and it did not matter from
the perspective of the problem being solved what we did in
the NIC.

Maybe I was being overly optimistic at that time, not
in any way about the engineering challenges, but about
the process. Seems like there is a strong desire to get
the legitimacy provided by using the TCP protocol, but
at the same time by significant modifications to it
(or by creating a new supertransport to overcome the
limitations of TCP).

Regards,
Somesh

> -----Original Message-----
> From: rddp-admin@ietf.org [mailto:rddp-admin@ietf.org]On Behalf Of
> Barron, Dwight
> Sent: Tuesday, April 15, 2003 9:12 AM
> To: Black_David@emc.com; rddp@ietf.org
> Subject: RE: [rddp] Options and Markers
> 
> 
> 
> 
> > -----Original Message-----
> > From: Black_David@emc.com [mailto:Black_David@emc.com]
> > Sent: Sunday, April 13, 2003 2:52 PM
> > To: Barron, Dwight; rddp@ietf.org
> > Subject: RE: [rddp] Options and Markers
> > 
> > 
> > Dwight,
> > 
> > > > I have no problem with robustness considerations, but the 
> > vehemence
> > > > of opposition to options in any form from a significant portion of
> > > > this WG has been among the things that have hindered progress.
> > > 
> > > I do not believe it is a "vehemence of opposition to options" 
> > > per se, but rather a strong desire to see valid engineering 
> > > justification for such options. I believe we established, per 
> > > your comments at the last F2F, that receive side copying is 
> > > not likely to be avoided, regardless of marker interval and 
> > > we went on to focus on transmit side impact of marker 
> > > interval options.
> > 
> > The discussion in San Francisco concerned a receive side full-software
> > implementation.  It did not cover a "full-offload with full 
> > TCP buffering"
> > class of implementation, as the discussion would up considering only
> > full-software and "flow-through" RNIC implementations.  
> > There's a third
> > class that is relevant.
> 
> I haven't heard anyone assert that this relevant third class 
> implementation does not fall into one of the two previously 
> mentioned catagories. I would suspect, given the embedded nature 
> of such designs, that they have relatively unencumbered access to 
> a NIC with scatter/gather capability. 
> 
> > 
> > [... material on full-software implementations snipped ... ]
> > 
> > > > I think some form of option is going to be hard to avoid for
> > > > markers, as there is "running code" out there for a class of
> > > > implementations (RDDP + TCP with full buffering offloaded to
> > > > a souped up NIC) that do not need markers.  Requiring that this
> > > > class of implementation use markers solely to avoid putting an
> > > > option in the protocol appears to be very hard to justify.
> > > 
> > > How does this argument not cross the line from what I would 
> > > consider debate in search of best architectural practice for 
> > > RDDP to that of defending an existing RDDP implementation of 
> > > "running code". What RDDP specification was this running code 
> > > conformant to such that it should impact the design of future 
> > > RDDP specifications? 
> > 
> > The assertion that there is a single "best architectural practice"
> > for RDDP and it is a "flow-through RNIC" with minimal buffering
> > is part of the problem.  
> 
> I'm sorry, but I disagree. This is the Remote Direct Data 
> Placement WG, not the Remote Store and Forward WG. From the WG 
> Description at 
> https://www1.ietf.org/html.charters/rddp-charter.html ..RDDP 
> capabilities refer to the ability to place data directly from the 
> NIC into application buffers, without intensive CPU usage. This 
> strategy avoids the costs of data copying and enables using IP as 
> a method for high speed buffer to buffer transfer....
> I have no quarrel with anyone that wants to use assembly buffers 
> in their design, every design will need some amount of buffering. 
> I do take exception that we mandate an option that essentially 
> diminishes or precludes a flow-through design for those of us 
> that are taking the *Direct* part seriously. 
> 
> > The implementation I referred to above
> > is based on the VI architecture and runs it over TCP/IP with
> > all RDMA processing done outboard.  AFAIK, the protocol used
> > is specific to that implementation; among the reasons for this is
> > that VI is an interface, not a protocol.
> > The reason that this discussion doesn't cross the line, is that
> > the "running code" serves to demonstrate the viability of the
> > third class of implementations, speaking to your above desire for
> > "valid engineering justification" for protocol design concerns
> > raised by that class.  While I can't speak for the implementers,
> > I think they plan to update the implementation in question to
> > whatever specifications the RDDP WG adopts.  Given the viability
> > of that class of implementation (FWIW, it was extensively discussed
> > prior to formation of the RDDP WG), design decisions which exclude
> > that class will tend to run afoul of the admonition not to
> > impose requirements on implementors that are not required for
> > interoperability (see Section 6 of RFC 2119).
> 
> I cannot find said product available for sale today, I have been 
> told it is withdrawn from the market and it is not on the 
> vendor's website (if this comment seems untowardly callous, the 
> same is true for my own company's VI product). That said, I have 
> not heard that the product is incapable of supporting markers. 
> Given the general catagory of a dedicated CPU and souped up NIC 
> capable of being upgraded by firmware, does that product become 
> (more) commercially unviable due to the overhead of inserting 
> markers? As for excluding a class as a whole, I think we are down 
> to quibbling whether an implementation should use an XXX or YYY 
> MHz embedded processor to hit ZZZ% of wire speed. 
> 
> > 
> > Thanks,
> > --David
> > ----------------------------------------------------
> > David L. Black, Senior Technologist
> > EMC Corporation, 176 South St., Hopkinton, MA  01748
> > +1 (508) 293-7953             FAX: +1 (508) 293-7786
> > black_david@emc.com        Mobile: +1 (978) 394-7754
> > ----------------------------------------------------
> > 
> _______________________________________________
> rddp mailing list
> rddp@ietf.org
> https://www1.ietf.org/mailman/listinfo/rddp
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Tue Apr 15 15:38:32 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA09258
	for <rddp-archive@odin.ietf.org>; Tue, 15 Apr 2003 15:38:31 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3FJkk531123
	for rddp-archive@odin.ietf.org; Tue, 15 Apr 2003 15:46:46 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3FJkk831120
	for <rddp-web-archive@optimus.ietf.org>; Tue, 15 Apr 2003 15:46:46 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA09242
	for <rddp-web-archive@ietf.org>; Tue, 15 Apr 2003 15:38:01 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195WIB-0004DS-00
	for rddp-web-archive@ietf.org; Tue, 15 Apr 2003 15:40:31 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 195WIA-0004DP-00
	for rddp-web-archive@ietf.org; Tue, 15 Apr 2003 15:40:30 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3FJju831064;
	Tue, 15 Apr 2003 15:45:56 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3FJh0830963
	for <rddp@optimus.ietf.org>; Tue, 15 Apr 2003 15:43:00 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA09127
	for <rddp@ietf.org>; Tue, 15 Apr 2003 15:34:15 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195WEX-0004Ca-00
	for rddp@ietf.org; Tue, 15 Apr 2003 15:36:45 -0400
Received: from ztxmail03.ztx.compaq.com ([161.114.1.207])
	by ietf-mx with esmtp (Exim 4.12)
	id 195WEW-0004CL-00
	for rddp@ietf.org; Tue, 15 Apr 2003 15:36:44 -0400
Received: from cceexg13.americas.cpqcorp.net (cceexg13.americas.cpqcorp.net [16.110.250.119])
	by ztxmail03.ztx.compaq.com (Postfix) with ESMTP
	id 022808F15; Tue, 15 Apr 2003 14:36:22 -0500 (CDT)
Received: from cceexc16.americas.cpqcorp.net ([16.110.250.58]) by cceexg13.americas.cpqcorp.net with Microsoft SMTPSVC(5.0.2195.2966);
	 Tue, 15 Apr 2003 14:36:17 -0500
X-MimeOLE: Produced By Microsoft Exchange V6.0.6375.0
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: RE: [rddp] Options and Markers
Date: Tue, 15 Apr 2003 14:36:16 -0500
Message-ID: <01CA1B1F74A4EC40A9C887F7B58F87240314313A@cceexc16.americas.cpqcorp.net>
Thread-Topic: [rddp] Options and Markers
Thread-Index: AcMDeD98bx8yIkeLSrCP8TGzuAPcUAAAzO2Q
From: "Barron, Dwight" <Dwight.Barron@hp.com>
To: "Somesh Gupta" <somesh_gupta@silverbacksystems.com>, <rddp@ietf.org>
X-OriginalArrivalTime: 15 Apr 2003 19:36:17.0426 (UTC) FILETIME=[488DCF20:01C30386]
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by www1.ietf.org id h3FJh1830964
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 8bit
Content-Transfer-Encoding: 8bit



> -----Original Message-----
> From: Somesh Gupta [mailto:somesh_gupta@silverbacksystems.com]
> Sent: Tuesday, April 15, 2003 12:40 PM
> To: rddp@ietf.org
> Subject: RE: [rddp] Options and Markers
> 
> 
> Dwight,
> 
> My reading of the charter of the workgroup is that
> the rddp protocol address the problem of data copying
> in the host (from the NIC into application buffers).
> 
> The charter does not indicate how the NIC is implemented.

As I stated in my posting, I have no quarrel over implementations with buffering, nor am I advocating this WG dictate implementation practice. It should not, and cannot. My point was that the WG charter suggests that there is an architectural goal which in turn suggests best engineering practices to reach that goal, which in turn suggests an appropriate weighting when forced to make the inevitable trade offs between a rock and hard place. I believe the absence of markers to be a much more serious impediment to the viability of a flow-through implementation than any additional overhead they impose on the other classes of design. The proponents of markers presented a case for such impact to memory in the elzur paper and quantified the impact as something that statistically scales with on the order of per connection overhead, which is three orders of magnitude for 1000 connections and another order of magnitude for 10Gig as a very conservative design point. I have not seen a si!
milar attempt to quantify the overhead of markers and personally find it very hard to believe they will be more than 10% in either class of software based implementation. 

> 
> When the Work Group was being chartered, there was a
> fairly big debate about whether there should even be
> a TCP mapping, since TCP would always require some
> buffering or roto-tilling of TCP (the argument is over
> how much buffering). Those of us who wanted
> a mapping over TCP pushed for it on the ground that
> we could achieve the mapping without trying to create a
> different version of TCP - and it did not matter from
> the perspective of the problem being solved what we did in
> the NIC.
> 
> Maybe I was being overly optimistic at that time, not
> in any way about the engineering challenges, but about
> the process. Seems like there is a strong desire to get
> the legitimacy provided by using the TCP protocol, but
> at the same time by significant modifications to it
> (or by creating a new supertransport to overcome the
> limitations of TCP).

Does the existence of iSCSI markers and CRC create any more of a new supertransport protocol by overcoming the same limitations of TCP? If any consumer is to be aware of a new transport protocol, it will the ULPs that are specifically defined to use RDDP, in which case I would much rather provide them with the same RDDP behaviors and implementation impact for both SCTP and TCP mappings. 

Regards,
Dwight

> 
> Regards,
> Somesh
> 
> > -----Original Message-----
> > From: rddp-admin@ietf.org [mailto:rddp-admin@ietf.org]On Behalf Of
> > Barron, Dwight
> > Sent: Tuesday, April 15, 2003 9:12 AM
> > To: Black_David@emc.com; rddp@ietf.org
> > Subject: RE: [rddp] Options and Markers
> > 
> > 
> > 
> > 
> > > -----Original Message-----
> > > From: Black_David@emc.com [mailto:Black_David@emc.com]
> > > Sent: Sunday, April 13, 2003 2:52 PM
> > > To: Barron, Dwight; rddp@ietf.org
> > > Subject: RE: [rddp] Options and Markers
> > > 
> > > 
> > > Dwight,
> > > 
> > > > > I have no problem with robustness considerations, but the 
> > > vehemence
> > > > > of opposition to options in any form from a 
> significant portion of
> > > > > this WG has been among the things that have hindered progress.
> > > > 
> > > > I do not believe it is a "vehemence of opposition to options" 
> > > > per se, but rather a strong desire to see valid engineering 
> > > > justification for such options. I believe we established, per 
> > > > your comments at the last F2F, that receive side copying is 
> > > > not likely to be avoided, regardless of marker interval and 
> > > > we went on to focus on transmit side impact of marker 
> > > > interval options.
> > > 
> > > The discussion in San Francisco concerned a receive side 
> full-software
> > > implementation.  It did not cover a "full-offload with full 
> > > TCP buffering"
> > > class of implementation, as the discussion would up 
> considering only
> > > full-software and "flow-through" RNIC implementations.  
> > > There's a third
> > > class that is relevant.
> > 
> > I haven't heard anyone assert that this relevant third class 
> > implementation does not fall into one of the two previously 
> > mentioned catagories. I would suspect, given the embedded nature 
> > of such designs, that they have relatively unencumbered access to 
> > a NIC with scatter/gather capability. 
> > 
> > > 
> > > [... material on full-software implementations snipped ... ]
> > > 
> > > > > I think some form of option is going to be hard to avoid for
> > > > > markers, as there is "running code" out there for a class of
> > > > > implementations (RDDP + TCP with full buffering offloaded to
> > > > > a souped up NIC) that do not need markers.  Requiring 
> that this
> > > > > class of implementation use markers solely to avoid putting an
> > > > > option in the protocol appears to be very hard to justify.
> > > > 
> > > > How does this argument not cross the line from what I would 
> > > > consider debate in search of best architectural practice for 
> > > > RDDP to that of defending an existing RDDP implementation of 
> > > > "running code". What RDDP specification was this running code 
> > > > conformant to such that it should impact the design of future 
> > > > RDDP specifications? 
> > > 
> > > The assertion that there is a single "best architectural practice"
> > > for RDDP and it is a "flow-through RNIC" with minimal buffering
> > > is part of the problem.  
> > 
> > I'm sorry, but I disagree. This is the Remote Direct Data 
> > Placement WG, not the Remote Store and Forward WG. From the WG 
> > Description at 
> > https://www1.ietf.org/html.charters/rddp-charter.html ..RDDP 
> > capabilities refer to the ability to place data directly from the 
> > NIC into application buffers, without intensive CPU usage. This 
> > strategy avoids the costs of data copying and enables using IP as 
> > a method for high speed buffer to buffer transfer....
> > I have no quarrel with anyone that wants to use assembly buffers 
> > in their design, every design will need some amount of buffering. 
> > I do take exception that we mandate an option that essentially 
> > diminishes or precludes a flow-through design for those of us 
> > that are taking the *Direct* part seriously. 
> > 
> > > The implementation I referred to above
> > > is based on the VI architecture and runs it over TCP/IP with
> > > all RDMA processing done outboard.  AFAIK, the protocol used
> > > is specific to that implementation; among the reasons for this is
> > > that VI is an interface, not a protocol.
> > > The reason that this discussion doesn't cross the line, is that
> > > the "running code" serves to demonstrate the viability of the
> > > third class of implementations, speaking to your above desire for
> > > "valid engineering justification" for protocol design concerns
> > > raised by that class.  While I can't speak for the implementers,
> > > I think they plan to update the implementation in question to
> > > whatever specifications the RDDP WG adopts.  Given the viability
> > > of that class of implementation (FWIW, it was extensively 
> discussed
> > > prior to formation of the RDDP WG), design decisions which exclude
> > > that class will tend to run afoul of the admonition not to
> > > impose requirements on implementors that are not required for
> > > interoperability (see Section 6 of RFC 2119).
> > 
> > I cannot find said product available for sale today, I have been 
> > told it is withdrawn from the market and it is not on the 
> > vendor's website (if this comment seems untowardly callous, the 
> > same is true for my own company's VI product). That said, I have 
> > not heard that the product is incapable of supporting markers. 
> > Given the general catagory of a dedicated CPU and souped up NIC 
> > capable of being upgraded by firmware, does that product become 
> > (more) commercially unviable due to the overhead of inserting 
> > markers? As for excluding a class as a whole, I think we are down 
> > to quibbling whether an implementation should use an XXX or YYY 
> > MHz embedded processor to hit ZZZ% of wire speed. 
> > 
> > > 
> > > Thanks,
> > > --David
> > > ----------------------------------------------------
> > > David L. Black, Senior Technologist
> > > EMC Corporation, 176 South St., Hopkinton, MA  01748
> > > +1 (508) 293-7953             FAX: +1 (508) 293-7786
> > > black_david@emc.com        Mobile: +1 (978) 394-7754
> > > ----------------------------------------------------
> > > 
> > _______________________________________________
> > rddp mailing list
> > rddp@ietf.org
> > https://www1.ietf.org/mailman/listinfo/rddp
> _______________________________________________
> rddp mailing list
> rddp@ietf.org
> https://www1.ietf.org/mailman/listinfo/rddp
> 
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Tue Apr 15 15:44:27 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA09405
	for <rddp-archive@odin.ietf.org>; Tue, 15 Apr 2003 15:44:27 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3FJqgC31354
	for rddp-archive@odin.ietf.org; Tue, 15 Apr 2003 15:52:42 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3FJqg831351
	for <rddp-web-archive@optimus.ietf.org>; Tue, 15 Apr 2003 15:52:42 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA09394
	for <rddp-web-archive@ietf.org>; Tue, 15 Apr 2003 15:43:56 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195WNu-0004EL-00
	for rddp-web-archive@ietf.org; Tue, 15 Apr 2003 15:46:26 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 195WNu-0004EI-00
	for rddp-web-archive@ietf.org; Tue, 15 Apr 2003 15:46:26 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3FJmj831197;
	Tue, 15 Apr 2003 15:48:45 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3FJkV831106
	for <rddp@optimus.ietf.org>; Tue, 15 Apr 2003 15:46:31 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA09233
	for <rddp@ietf.org>; Tue, 15 Apr 2003 15:37:45 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195WHv-0004DM-00
	for rddp@ietf.org; Tue, 15 Apr 2003 15:40:15 -0400
Received: from emulex.emulex.com ([138.239.112.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 195WHv-0004DJ-00
	for rddp@ietf.org; Tue, 15 Apr 2003 15:40:15 -0400
Received: from xbl.ad.emulex.com (xbl.ma.emulex.com [172.16.12.11])
	by emulex.emulex.com (8.12.9/8.12.8) with ESMTP id h3FJdqLW014074
	for <rddp@ietf.org>; Tue, 15 Apr 2003 12:39:52 -0700 (PDT)
Received: by xbl.ma.emulex.com with Internet Mail Service (5.5.2653.19)
	id <HN1WWXWL>; Tue, 15 Apr 2003 15:39:52 -0400
Message-ID: <3356669BBE90C448AD4645C843E2BF289B905C@xbl.ma.emulex.com>
From: "Williams, Jim" <Jim.Williams@Emulex.com>
To: rddp@ietf.org
Subject: RE: [rddp] Options and Markers
Date: Tue, 15 Apr 2003 15:39:51 -0400
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="ISO-8859-1"
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

> From: Caitlin Bestler [mailto:cait@asomi.com]
> [...]
> That leave the real issue being assessing which is the 
> greater imposition:
> 
> -- forcing senders to support both marker and no-marker mode.
> -- forcing memory-rich receivers to implement markers.
> 
> The primary virtue of the latter is that it avoids options. 
> Allowing recievers
> to mandate "no markers" is clearly superior for pure-software 
> RDDP stacks
> written on top of sockets.

If you will forgive a bit of oversimplification, I view
the problem as follows.

If markers are optional to implement, then there are two
possible outcomes:

    A:  Markers turn out to be valuable, and all implementations
        will either support them or be non-competitive.

    B:  Markers turn out NOT to be valuable, and will not
        be implemented.

If markers are mandatory to implement (regardless of whether
optional or mandatory to use) then there are two possible
outcomes:

    C:  Markers turn out to be valuable and are always used.

    D:  Markers turn out NOT to be valuable, but must be 
        implemented anyway for compliance.

So A and C are virtually equivalent outcomes, but B is clearly
preferable to D.  D is clearly worse if markers are also 
mandatory to use.  Overall, my preference is to make
markers optional to implement unless or until their 
value is demonstrated through actual implementations.

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Tue Apr 15 19:39:53 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA16699
	for <rddp-archive@odin.ietf.org>; Tue, 15 Apr 2003 19:39:52 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3FNmEc16111
	for rddp-archive@odin.ietf.org; Tue, 15 Apr 2003 19:48:14 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3FNmD816108
	for <rddp-web-archive@optimus.ietf.org>; Tue, 15 Apr 2003 19:48:13 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA16692
	for <rddp-web-archive@ietf.org>; Tue, 15 Apr 2003 19:39:22 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195a3l-0005Pz-00
	for rddp-web-archive@ietf.org; Tue, 15 Apr 2003 19:41:53 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 195a3l-0005Pw-00
	for rddp-web-archive@ietf.org; Tue, 15 Apr 2003 19:41:53 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3FNlM816049;
	Tue, 15 Apr 2003 19:47:22 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3FNk0815974
	for <rddp@optimus.ietf.org>; Tue, 15 Apr 2003 19:46:00 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA16639
	for <rddp@ietf.org>; Tue, 15 Apr 2003 19:37:09 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195a1c-0005Oz-00
	for rddp@ietf.org; Tue, 15 Apr 2003 19:39:40 -0400
Received: from thebe.your-site.com ([140.186.45.26])
	by ietf-mx with esmtp (Exim 4.12)
	id 195a1c-0005Ok-00
	for rddp@ietf.org; Tue, 15 Apr 2003 19:39:40 -0400
Received: from 148.87.1.40 (nic5.oracle.com [148.87.1.40])
	by thebe.your-site.com (Postfix) with ESMTP
	id B3DAA244BC2; Tue, 15 Apr 2003 19:39:30 -0400 (EDT)
Date: Tue, 15 Apr 2003 18:39:25 -0500
From: Caitlin Bestler <cait@asomi.com>
Subject: RE: [rddp] Options and Markers
To: "Williams, Jim" <Jim.Williams@Emulex.com>, rddp@ietf.org
X-Priority: 3
In-Reply-To: <3356669BBE90C448AD4645C843E2BF289B905C@xbl.ma.emulex.com>
Message-ID: <r01050300-1024-7E7CD6226F9B11D79E7E000393814604@[148.87.1.40]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

> > From: Caitlin Bestler [mailto:cait@asomi.com]
> > [...]
> > That leave the real issue being assessing which is the 
> > greater imposition:
> > 
> > -- forcing senders to support both marker and no-marker mode.
> > -- forcing memory-rich receivers to implement markers.
> > 
> > The primary virtue of the latter is that it avoids options. 
> > Allowing recievers
> > to mandate "no markers" is clearly superior for pure-software 
> > RDDP stacks
> > written on top of sockets.
> 
> If you will forgive a bit of oversimplification, I view
> the problem as follows.
> 
> If markers are optional to implement, then there are two
> possible outcomes:
> 
>     A:  Markers turn out to be valuable, and all implementations
>         will either support them or be non-competitive.
> 
>     B:  Markers turn out NOT to be valuable, and will not
>         be implemented.
> 
> If markers are mandatory to implement (regardless of whether
> optional or mandatory to use) then there are two possible
> outcomes:
> 
>     C:  Markers turn out to be valuable and are always used.
> 
>     D:  Markers turn out NOT to be valuable, but must be 
>         implemented anyway for compliance.
> 
> So A and C are virtually equivalent outcomes, but B is clearly
> preferable to D.  D is clearly worse if markers are also 
> mandatory to use.  Overall, my preference is to make
> markers optional to implement unless or until their 
> value is demonstrated through actual implementations.
> 

You are assuming that markers being "valuable" is a universal thing.
The point I was making is that markers are extermely valuable to some
receivers (those with low memory resources) and of no value to those
without that restriction (indeed, the markers becomes a slight nuisance).

I believe there is an overwhelming case that some recievers require markers,
and that failure to provide them will have a severe impact on their
performance. As pointed out, the buffering requirements scale by the
maximum in flight allowance for each connection.

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Tue Apr 15 20:21:38 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA17686
	for <rddp-archive@odin.ietf.org>; Tue, 15 Apr 2003 20:21:38 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3G0U0p18332
	for rddp-archive@odin.ietf.org; Tue, 15 Apr 2003 20:30:00 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3G0U0818329
	for <rddp-web-archive@optimus.ietf.org>; Tue, 15 Apr 2003 20:30:00 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA17662
	for <rddp-web-archive@ietf.org>; Tue, 15 Apr 2003 20:21:07 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195aiA-0005aO-00
	for rddp-web-archive@ietf.org; Tue, 15 Apr 2003 20:23:38 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 195aiA-0005aL-00
	for rddp-web-archive@ietf.org; Tue, 15 Apr 2003 20:23:38 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3G0T8818286;
	Tue, 15 Apr 2003 20:29:08 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3G0Sh818262
	for <rddp@optimus.ietf.org>; Tue, 15 Apr 2003 20:28:43 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA17623
	for <rddp@ietf.org>; Tue, 15 Apr 2003 20:19:50 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195agw-0005Zk-00
	for rddp@ietf.org; Tue, 15 Apr 2003 20:22:22 -0400
Received: from palrel11.hp.com ([156.153.255.246])
	by ietf-mx with esmtp (Exim 4.12)
	id 195agv-0005Zh-00
	for rddp@ietf.org; Tue, 15 Apr 2003 20:22:21 -0400
Received: from wera.hpl.hp.com (wera.hpl.hp.com [15.9.120.80])
	by palrel11.hp.com (Postfix) with ESMTP
	id 9A1A41C00B49; Tue, 15 Apr 2003 17:22:29 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1])
	by wera.hpl.hp.com (8.12.9/8.12.2) with SMTP id h3G0MT8o013995;
	Tue, 15 Apr 2003 17:22:29 -0700 (PDT)
From: Jeffrey Mogul <Jeff.Mogul@hp.com>
Message-Id: <200304160022.h3G0MT8o013995@wera.hpl.hp.com>
X-Authentication-Warning: wera.hpl.hp.com: localhost [127.0.0.1] didn't use HELO protocol
To: "Williams, Jim" <Jim.Williams@emulex.com>
Cc: rddp@ietf.org
Subject: Re: [rddp] Options and Markers 
In-reply-to: Your message of "Tue, 15 Apr 2003 15:39:51 EDT."
             <3356669BBE90C448AD4645C843E2BF289B905C@xbl.ma.emulex.com> 
X-Organization: Hewlett-Packard Labs, Large Scale Systems group
Date: Tue, 15 Apr 2003 17:22:29 -0700
X-Mts: smtp
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

After watching this debate from the cheap seats for a while,
and realizing that I don't have a strong technical preference
either for or against either markers and CRCs, Jim's case
analysis inspired me to comment.

My sense of the debate is that:

    for all X in the set {Markers, CRCs}

	1. Some people want X
	2. Some people don't want X
	3. X has some overhead that might lead to suboptimal performance
	4. X is not hard to implement

	and the choices for the RDDP-WG are:

	(a) Don't put X in the protocol
	(b) Make X mandatory
	(c) Make X fully optional
	(d) Make X mandatory to implement, but optional to use

But rather than to address this as a technical question, I'd like
to address it as a procedural question.

First, our goal is to achieve "genuine consensus" (or to give
up).  Unanimity is not required by IETF procedures.  So I think
we should consider which choices are most likely to lead to
consensus.

   This means that choices (a) and (b) are a problem, because
   they lead to continued debate (and perhaps re-opening the
   debate at future stages of the IETF process.)

Second, my experience from several past IETF standards (Path MTU
Discovery, HTTP/1.1) is that interoperability suffers unless the
specification leaves very little wiggle room.  Unlike Jim, I am
not optimistic that the marketplace will save us from products
that don't support "valuable" features whose implementation is
optional.  (For example, in a corporate network with 50,000 nodes
and hundreds or thousands of routers, it only takes one or two
routers that black-hole PMTUD messages to force nearly everyone
to turn off their end-host PMTUD support.)

   This means that choice (c) is a problem, because if anyone
   does want X, then we'll always be fixing interoperability bugs
   in deployed systems.

Third, the IETF process does allow changes to the protocol design
between the Proposed Standard and Draft Standard stages.  But in
practice, it's very hard to add a feature here, because the IESG
seems to be suspicious of new features without sufficient
testing.  So they may require us to go through the Proposed
Standard stage again (with all the delay that implies) if we try
to add something.

    This means that choice (a) is a problem if some people really
    do want X.

So I lean towards choice (d): Make X mandatory to implement, but
optional to use.  This will make fans of X happy, will add some
implementation overhead for opponents of X, and will annoy people
who don't want any options.

So, are options really that bad?  I don't think so.  First of
all, if X is mandatory to implement but optional to use, and the
receiver gets to choose, then you can make this work without
extra round-trips by having the sender start the connection using
the option, and then let the receiver tell the sender to turn it
off.

Second, my guess is that any solution to the "prove MPA doesn't
get confused with unmodified TCP" requirement will provide us
with a convenient place to pass some options.  (This is a
requirement because the IESG says it is a requirement; it doesn't
matter what my personal technical judgement is, because the IESG
has the last say.  I've been there before.)

Third, I'm really nervous about publishing *any* protocol spec
that doesn't include a version number that the WG can change
relatively easily.  Especially one where people are clearly
tempted to deploy systems based on "Proposed Standard"
specifications.  We had this problem with HTTP/1.1, and by not
making the decision to increment the version number between
Proposed and Draft, we created massive and permanent
interoperability problems.  If you have no version number at all,
you're equally screwed.

At this point, I don't believe MPA carries an explicit version
number, and I think this is a disaster waiting to happen.  If we
fix that, we again are likely to have a place to put options,
too.

-Jeff
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Wed Apr 16 08:40:08 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA27558
	for <rddp-archive@odin.ietf.org>; Wed, 16 Apr 2003 08:40:08 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3GCmjv16838
	for rddp-archive@odin.ietf.org; Wed, 16 Apr 2003 08:48:45 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GCmj816835
	for <rddp-web-archive@optimus.ietf.org>; Wed, 16 Apr 2003 08:48:45 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA27542
	for <rddp-web-archive@ietf.org>; Wed, 16 Apr 2003 08:39:38 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195mEq-0000n3-00
	for rddp-web-archive@ietf.org; Wed, 16 Apr 2003 08:42:08 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 195mEq-0000n0-00
	for rddp-web-archive@ietf.org; Wed, 16 Apr 2003 08:42:08 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GCm6816772;
	Wed, 16 Apr 2003 08:48:06 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GCl1816697
	for <rddp@optimus.ietf.org>; Wed, 16 Apr 2003 08:47:01 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA27485
	for <rddp@ietf.org>; Wed, 16 Apr 2003 08:37:54 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195mDA-0000lq-00
	for rddp@ietf.org; Wed, 16 Apr 2003 08:40:24 -0400
Received: from emulex.emulex.com ([138.239.112.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 195mDA-0000li-00
	for rddp@ietf.org; Wed, 16 Apr 2003 08:40:24 -0400
Received: from xbl.ad.emulex.com (xbl.ma.emulex.com [172.16.12.11])
	by emulex.emulex.com (8.12.9/8.12.8) with ESMTP id h3GCe2LW029869
	for <rddp@ietf.org>; Wed, 16 Apr 2003 05:40:02 -0700 (PDT)
Received: by xbl.ma.emulex.com with Internet Mail Service (5.5.2653.19)
	id <HN1WWY0V>; Wed, 16 Apr 2003 08:40:02 -0400
Message-ID: <3356669BBE90C448AD4645C843E2BF289B9060@xbl.ma.emulex.com>
From: "Williams, Jim" <Jim.Williams@Emulex.com>
To: rddp@ietf.org
Subject: RE: [rddp] Options and Markers 
Date: Wed, 16 Apr 2003 08:39:52 -0400
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="ISO-8859-1"
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>


> Second, my experience from several past IETF standards (Path MTU
> Discovery, HTTP/1.1) is that interoperability suffers unless the
> specification leaves very little wiggle room.  Unlike Jim, I am
> not optimistic that the marketplace will save us from products
> that don't support "valuable" features whose implementation is
> optional.  (For example, in a corporate network with 50,000 nodes
> and hundreds or thousands of routers, it only takes one or two
> routers that black-hole PMTUD messages to force nearly everyone
> to turn off their end-host PMTUD support.)

Jeff,

I think your point has a lot of merit.  I would suggest, though, 
that markers are not QUITE as bad as the PMTU example you give.
Specifically, if markers are optional to implement, and an endpoint
chooses not to implement them, the worst case is that the performance
of connections to THAT endpoint may be slower.  There are no
functional or interoperability problems, an other connections are
not penalized.

 - Jim

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Wed Apr 16 12:01:56 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA06891
	for <rddp-archive@odin.ietf.org>; Wed, 16 Apr 2003 12:01:56 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3GGAaZ01479
	for rddp-archive@odin.ietf.org; Wed, 16 Apr 2003 12:10:36 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GGAa801476
	for <rddp-web-archive@optimus.ietf.org>; Wed, 16 Apr 2003 12:10:36 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA06849
	for <rddp-web-archive@ietf.org>; Wed, 16 Apr 2003 12:01:25 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195pO7-0002OT-00
	for rddp-web-archive@ietf.org; Wed, 16 Apr 2003 12:03:55 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 195pO6-0002OQ-00
	for rddp-web-archive@ietf.org; Wed, 16 Apr 2003 12:03:54 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GG9I801286;
	Wed, 16 Apr 2003 12:09:18 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GG2q832388
	for <rddp@optimus.ietf.org>; Wed, 16 Apr 2003 12:02:52 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA06500
	for <rddp@ietf.org>; Wed, 16 Apr 2003 11:53:42 -0400 (EDT)
From: Black_David@emc.com
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195pGd-0002L4-00
	for rddp@ietf.org; Wed, 16 Apr 2003 11:56:11 -0400
Received: from mxic2.corp.emc.com ([128.221.31.40])
	by ietf-mx with esmtp (Exim 4.12)
	id 195pGd-0002L0-00
	for rddp@ietf.org; Wed, 16 Apr 2003 11:56:11 -0400
Received: by mxic2.corp.emc.com with Internet Mail Service (5.5.2653.19)
	id <JAALMXA0>; Wed, 16 Apr 2003 11:55:43 -0400
Message-ID: <277DD60FB639D511AC0400B0D068B71E0564CB31@corpmx14.corp.emc.com>
To: Dwight.Barron@hp.com, rddp@ietf.org
Subject: RE: [rddp] Options and Markers
Date: Wed, 16 Apr 2003 11:50:32 -0400
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

> > The discussion in San Francisco concerned a receive side full-software
> > implementation.  It did not cover a "full-offload with full TCP
buffering"
> > class of implementation, as the discussion would up considering only
> > full-software and "flow-through" RNIC implementations.  There's a third
> > class that is relevant.
> 
> I haven't heard anyone assert that this relevant third class 
> implementation does not fall into one of the two previously 
> mentioned catagories.

Here's the assertion:

A full-offload full-buffering implementation (1) can achieve copy
avoidance via direct data placement between the NIC and host memory
(2) without using markers.  Item (1) distinguishes this class from
full-software implementations, and item (2) distinguishes it from RNICs.

> > The assertion that there is a single "best architectural practice"
> > for RDDP and it is a "flow-through RNIC" with minimal buffering
> > is part of the problem.  
> 
> I'm sorry, but I disagree. This is the Remote Direct Data 
> Placement WG, not the Remote Store and Forward WG. From the 
> WG Description at http://www.ietf.org/html.charters/rddp-charter.html
>  ..RDDP capabilities refer to the ability to place data directly from
>	the NIC into application buffers, without intensive CPU usage.
>	This strategy avoids the costs of data copying and enables using
>	IP as a method for high speed buffer to buffer transfer....

That text deliberately does not specify what is implemented in the NIC
or how the NIC does its buffering, so a "full-offload full-buffering"
NIC is within the scope of the description, and this was intended when
the charter was written.

> I have no quarrel with anyone that wants to use assembly buffers in
> their design, every design will need some amount of buffering. I do
> take exception that we mandate an option that essentially diminishes
> or precludes a flow-through design

Could we see "valid engineering justification" for the latter statement,
especially the "precludes" assertion?  Keep in mind that the "option"
under discussion allows a receiver who benefits from markers to require
their use.

I snipped material on commercial success of implementations as being
basically irrelevant to the scope issues under discussion.  I would
caution against further raising of this sort of issue, as I suspect
an IESG discussion of the commercial merits of InfiniBand would not
bode well for the future of this WG, and I'd just as soon not
encourage the IESG to go there.

Thanks,
--David
----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
black_david@emc.com        Mobile: +1 (978) 394-7754
----------------------------------------------------
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Wed Apr 16 14:02:22 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA10182
	for <rddp-archive@odin.ietf.org>; Wed, 16 Apr 2003 14:02:22 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3GIB6b10137
	for rddp-archive@odin.ietf.org; Wed, 16 Apr 2003 14:11:06 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GIB5810134
	for <rddp-web-archive@optimus.ietf.org>; Wed, 16 Apr 2003 14:11:05 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA10179
	for <rddp-web-archive@ietf.org>; Wed, 16 Apr 2003 14:01:52 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195rGg-0002vY-00
	for rddp-web-archive@ietf.org; Wed, 16 Apr 2003 14:04:22 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 195rGf-0002vU-00
	for rddp-web-archive@ietf.org; Wed, 16 Apr 2003 14:04:21 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GIAE810105;
	Wed, 16 Apr 2003 14:10:14 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GI9J810069
	for <rddp@optimus.ietf.org>; Wed, 16 Apr 2003 14:09:19 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA10101
	for <rddp@ietf.org>; Wed, 16 Apr 2003 14:00:06 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195rEx-0002vB-00
	for rddp@ietf.org; Wed, 16 Apr 2003 14:02:35 -0400
Received: from ztxmail04.ztx.compaq.com ([161.114.1.208])
	by ietf-mx with esmtp (Exim 4.12)
	id 195rEx-0002v2-00
	for rddp@ietf.org; Wed, 16 Apr 2003 14:02:35 -0400
Received: from cceexg12.americas.cpqcorp.net (cceexg12.americas.cpqcorp.net [16.110.250.124])
	by ztxmail04.ztx.compaq.com (Postfix) with ESMTP
	id 74B0A7F61; Wed, 16 Apr 2003 13:02:12 -0500 (CDT)
Received: from cceexc16.americas.cpqcorp.net ([16.110.250.58]) by cceexg12.americas.cpqcorp.net with Microsoft SMTPSVC(5.0.2195.2966);
	 Wed, 16 Apr 2003 13:02:12 -0500
X-MimeOLE: Produced By Microsoft Exchange V6.0.6375.0
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: RE: [rddp] Options and Markers
Date: Wed, 16 Apr 2003 13:02:11 -0500
Message-ID: <01CA1B1F74A4EC40A9C887F7B58F87240314313C@cceexc16.americas.cpqcorp.net>
Thread-Topic: [rddp] Options and Markers
Thread-Index: AcMEMKj+tgs9G/ecRDGwQ4cflpEYywAB1Dfg
From: "Barron, Dwight" <Dwight.Barron@hp.com>
To: <Black_David@emc.com>, <rddp@ietf.org>
X-OriginalArrivalTime: 16 Apr 2003 18:02:12.0327 (UTC) FILETIME=[4E39CF70:01C30442]
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by www1.ietf.org id h3GI9J810070
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 8bit
Content-Transfer-Encoding: 8bit



> -----Original Message-----
> From: Black_David@emc.com [mailto:Black_David@emc.com]
> Sent: Wednesday, April 16, 2003 10:51 AM
> To: Barron, Dwight; rddp@ietf.org
> Subject: RE: [rddp] Options and Markers
> 
> 
> > > The discussion in San Francisco concerned a receive side 
> full-software
> > > implementation.  It did not cover a "full-offload with full TCP
> buffering"
> > > class of implementation, as the discussion would up 
> considering only
> > > full-software and "flow-through" RNIC implementations.  
> There's a third
> > > class that is relevant.
> > 
> > I haven't heard anyone assert that this relevant third class 
> > implementation does not fall into one of the two previously 
> > mentioned catagories.
> 
> Here's the assertion:
> 
> A full-offload full-buffering implementation (1) can achieve copy
> avoidance via direct data placement between the NIC and host memory
> (2) without using markers.  Item (1) distinguishes this class from
> full-software implementations, and item (2) distinguishes it 
> from RNICs.

My comments on the third class (how about BNIC, for buffered RDDP+TCP?) was (perhaps incorrectly) based on the following previous discussions/data points:
- The performance overhead disadvantage of markers primarily impacts the transmitter (per the San Francisco discussion)
- There are two basic ways to insert markers in a transmit data stream in the two classes I was envisioning: gather lists or copying (e.g. using explicit BNIC or host buffer pool. Just for completeness, RNICs are assumed to use inline insertion, which is a third category assumed beyond the reach of BNICs and hosts)
- At the time, reference was made to software upgrade of an existing product to incorporate RDDP. I will focus solely on the architectural connotation of *software* in that context; underneath the covers of a BNIC are still one of two basic marker insertion mechanisms discussed in the host context, gather lists or copying. 
- Which comes back to my previous question, which one of these methods is marker insertion overloading?

> 
> > > The assertion that there is a single "best architectural practice"
> > > for RDDP and it is a "flow-through RNIC" with minimal buffering
> > > is part of the problem.  
> > 
> > I'm sorry, but I disagree. This is the Remote Direct Data 
> > Placement WG, not the Remote Store and Forward WG. From the 
> > WG Description at 
> http://www.ietf.org/html.charters/rddp-charter.html
> >  ..RDDP capabilities refer to the ability to place data 
> directly from
> >	the NIC into application buffers, without intensive CPU usage.
> >	This strategy avoids the costs of data copying and enables using
> >	IP as a method for high speed buffer to buffer transfer....
> 
> That text deliberately does not specify what is implemented in the NIC
> or how the NIC does its buffering, so a "full-offload full-buffering"
> NIC is within the scope of the description, and this was intended when
> the charter was written.

Again, I did not suggest the charter mandated a specific design. I suggested it points to an architectural goal against which one may evaluate, ahem, their options.

> 
> > I have no quarrel with anyone that wants to use assembly buffers in
> > their design, every design will need some amount of buffering. I do
> > take exception that we mandate an option that essentially diminishes
> > or precludes a flow-through design
> 
> Could we see "valid engineering justification" for the latter 
> statement,
> especially the "precludes" assertion?  Keep in mind that the "option"
> under discussion allows a receiver who benefits from markers 
> to require
> their use.

You are correct, the "preclude" assertion does not apply if we get to the point that markers are mandatory to implement/optional to use at all receivers (are you putting your WG Chair hat on?). However, this means that markers are mandatory to implement at the transmitter also, which seems to be where the argument for the most marker pain has manifested itself for those that are opposed to markers. Once implemented, is there sufficient delta to a) warrant an option and b) operate the connection in an asymmetric mode? I apologize in advance for slipping into "commercial merits diatribe" but options cost real money once the ink is dry on the spec. I like to reserve their use/cost for things that are to the left of the decimal point, rather than the right. 

> 
> I snipped material on commercial success of implementations as being
> basically irrelevant to the scope issues under discussion.  I would
> caution against further raising of this sort of issue, as I suspect
> an IESG discussion of the commercial merits of InfiniBand would not
> bode well for the future of this WG, and I'd just as soon not
> encourage the IESG to go there.

My previous comment was in the context of a software upgrade for an existing product. I think the word on VI and IB has already leaked out, but just in case, I will be happy to constain further comments to architectural lessons learned from those experiences. 

Regards,
Dwight

> 
> Thanks,
> --David
> ----------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 176 South St., Hopkinton, MA  01748
> +1 (508) 293-7953             FAX: +1 (508) 293-7786
> black_david@emc.com        Mobile: +1 (978) 394-7754
> ----------------------------------------------------
> 
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Wed Apr 16 15:33:11 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA13921
	for <rddp-archive@odin.ietf.org>; Wed, 16 Apr 2003 15:33:11 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3GJfuK17117
	for rddp-archive@odin.ietf.org; Wed, 16 Apr 2003 15:41:56 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GJfu817114
	for <rddp-web-archive@optimus.ietf.org>; Wed, 16 Apr 2003 15:41:56 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA13879
	for <rddp-web-archive@ietf.org>; Wed, 16 Apr 2003 15:32:40 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195sgY-0003KT-00
	for rddp-web-archive@ietf.org; Wed, 16 Apr 2003 15:35:10 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 195sgY-0003KQ-00
	for rddp-web-archive@ietf.org; Wed, 16 Apr 2003 15:35:10 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GJf6817050;
	Wed, 16 Apr 2003 15:41:06 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GJef817011
	for <rddp@optimus.ietf.org>; Wed, 16 Apr 2003 15:40:41 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA13824
	for <rddp@ietf.org>; Wed, 16 Apr 2003 15:31:25 -0400 (EDT)
From: Black_David@emc.com
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195sfL-0003JO-00
	for rddp@ietf.org; Wed, 16 Apr 2003 15:33:55 -0400
Received: from mxic2.corp.emc.com ([128.221.31.40])
	by ietf-mx with esmtp (Exim 4.12)
	id 195sfL-0003J5-00
	for rddp@ietf.org; Wed, 16 Apr 2003 15:33:55 -0400
Received: by mxic2.corp.emc.com with Internet Mail Service (5.5.2653.19)
	id <JAALN1WG>; Wed, 16 Apr 2003 15:33:35 -0400
Message-ID: <277DD60FB639D511AC0400B0D068B71E0564CB36@corpmx14.corp.emc.com>
To: Dwight.Barron@hp.com, rddp@ietf.org
Date: Wed, 16 Apr 2003 15:28:24 -0400
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: [rddp] Marker Progress?
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

> > > I do take exception that we mandate an option that essentially
> > > diminishes or precludes a flow-through design
> >
> > Could we see "valid engineering justification" for the latter statement,
> > especially the "precludes" assertion?  Keep in mind that the "option"
> > under discussion allows a receiver who benefits from markers 
> > to require their use.
> 
> You are correct, the "preclude" assertion does not apply if 
> we get to the point that markers are mandatory to 
> implement/optional to use at all receivers (are you putting 
> your WG Chair hat on?).

Not exactly, and not yet.  Here's the proposal that was floated to
start this thread:

	One alternate possibility would be to make marker usage optional,
	but put control of this "option" entirely in the hands of
	the receiver - senders MUST implement both markers and not-markers,
	receivers MUST implement one OR the other, and each receiver
	gets to pick which one is used as the benefits of markers are
	primarily on the receiver side.

See Jeff Mogul's email for a long explanation of why something like
this has a better chance of making progress than the "MUST always
use markers" and "markers are completely OPTIONAL" positions.

> However, this means that markers are 
> mandatory to implement at the transmitter also, which seems 
> to be where the argument for the most marker pain has 
> manifested itself for those that are opposed to markers.

Yes.  This now puts responsibility for the transmitter consequences
on the receiver.  Receiver implementers who believe that there are
no significant transmitter consequences can implement receivers that
always require markers.  Receiver implementers who are concerned about
transmitter consequences can choose to support not using markers.
OTOH, we could go on arguing about this for the next 6 months ...

> Once implemented, is there sufficient delta to a) warrant an 
> option and b) operate the connection in an asymmetric mode? I 
> apologize in advance for slipping into "commercial merits 
> diatribe" but options cost real money once the ink is dry on 
> the spec. I like to reserve their use/cost for things that 
> are to the left of the decimal point, rather than the right. 

This is an unfortunate example of the "vehemence of opposition
to options" that is causing delay.

For asymmetric mode, consider that some all-software "client"
implementations might be more willing to put in the markers to
help out the receiving "server" if the "client" didn't have to
go to the pointless additional trouble of stripping markers from
the data returned by the server.  Consider that the "client"
might be a small portable device like a PDA or mobile phone.

Thanks,
--David 
----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
black_david@emc.com        Mobile: +1 (978) 394-7754
----------------------------------------------------
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Wed Apr 16 15:43:57 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA14217
	for <rddp-archive@odin.ietf.org>; Wed, 16 Apr 2003 15:43:57 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3GJqhv17578
	for rddp-archive@odin.ietf.org; Wed, 16 Apr 2003 15:52:43 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GJqh817575
	for <rddp-web-archive@optimus.ietf.org>; Wed, 16 Apr 2003 15:52:43 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA14212
	for <rddp-web-archive@ietf.org>; Wed, 16 Apr 2003 15:43:26 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195sqz-0003Mr-00
	for rddp-web-archive@ietf.org; Wed, 16 Apr 2003 15:45:57 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 195sqy-0003Mo-00
	for rddp-web-archive@ietf.org; Wed, 16 Apr 2003 15:45:56 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GJq1817528;
	Wed, 16 Apr 2003 15:52:01 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GJkC817283
	for <rddp@optimus.ietf.org>; Wed, 16 Apr 2003 15:46:12 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA14019
	for <rddp@ietf.org>; Wed, 16 Apr 2003 15:36:56 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195skg-0003LU-00
	for rddp@ietf.org; Wed, 16 Apr 2003 15:39:26 -0400
Received: from msgbas1x.cos.agilent.com ([192.25.240.36])
	by ietf-mx with esmtp (Exim 4.12)
	id 195skg-0003LR-00
	for rddp@ietf.org; Wed, 16 Apr 2003 15:39:26 -0400
Received: from relcos2.cos.agilent.com (relcos2.cos.agilent.com [130.29.152.237])
	by msgbas1x.cos.agilent.com (Postfix) with ESMTP id ECBE81745F
	for <rddp@ietf.org>; Wed, 16 Apr 2003 13:39:34 -0600 (MDT)
Received: from acropora.rose.agilent.com (acropora.rose.agilent.com [130.30.179.5])
	by relcos2.cos.agilent.com (Postfix) with ESMTP id 9A70C662
	for <rddp@ietf.org>; Wed, 16 Apr 2003 13:39:34 -0600 (MDT)
Received: (from dbs@localhost)
	by acropora.rose.agilent.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.1.0) id MAA12303
	for rddp@ietf.org; Wed, 16 Apr 2003 12:36:10 -0700 (PDT)
From: Dave Sheehy <dbs@acropora.rose.agilent.com>
Message-Id: <200304161936.MAA12303@acropora.rose.agilent.com>
Subject: RE: [rddp] Options and Markers
To: rddp@ietf.org
Date: Wed, 16 Apr 2003 12:36:10 PDT
X-Mailer: Elm [revision: 212.4]
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>


Jim Williams wrote:
> > From: Caitlin Bestler [mailto:cait@asomi.com]
> > [...]
> > That leave the real issue being assessing which is the 
> > greater imposition:
> > 
> > -- forcing senders to support both marker and no-marker mode.
> > -- forcing memory-rich receivers to implement markers.
> > 
> > The primary virtue of the latter is that it avoids options. 
> > Allowing recievers
> > to mandate "no markers" is clearly superior for pure-software 
> > RDDP stacks
> > written on top of sockets.
> 
> If you will forgive a bit of oversimplification, I view
> the problem as follows.
> 
> If markers are optional to implement, then there are two
> possible outcomes:
> 
>     A:  Markers turn out to be valuable, and all implementations
>         will either support them or be non-competitive.
> 
>     B:  Markers turn out NOT to be valuable, and will not
>         be implemented.

Neither A or B is the most likely outcome. The most likely outcome is 
that markers will not be implemented at all since they are useless if 
optional.  iSCSI is a perfect example of this.

Dave Sheehy


_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Wed Apr 16 15:57:04 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA14866
	for <rddp-archive@odin.ietf.org>; Wed, 16 Apr 2003 15:57:04 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3GK5oN18301
	for rddp-archive@odin.ietf.org; Wed, 16 Apr 2003 16:05:50 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GK5o818298
	for <rddp-web-archive@optimus.ietf.org>; Wed, 16 Apr 2003 16:05:50 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA14848
	for <rddp-web-archive@ietf.org>; Wed, 16 Apr 2003 15:56:33 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195t3g-0003SA-00
	for rddp-web-archive@ietf.org; Wed, 16 Apr 2003 15:59:04 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 195t3f-0003S7-00
	for rddp-web-archive@ietf.org; Wed, 16 Apr 2003 15:59:03 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GK51818259;
	Wed, 16 Apr 2003 16:05:01 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GK4Z818209
	for <rddp@optimus.ietf.org>; Wed, 16 Apr 2003 16:04:35 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA14801
	for <rddp@ietf.org>; Wed, 16 Apr 2003 15:55:18 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195t2T-0003RZ-00
	for rddp@ietf.org; Wed, 16 Apr 2003 15:57:49 -0400
Received: from dmz1.silverbacksystems.com ([65.172.158.82])
	by ietf-mx with esmtp (Exim 4.12)
	id 195t2S-0003RO-00
	for rddp@ietf.org; Wed, 16 Apr 2003 15:57:49 -0400
Received: from ns.silverbacksystems.com (gate-camp-hme0.silverbacksystems.com [65.172.158.93])
	by dmz1.silverbacksystems.com (Postfix on SuSE Linux 7.3 (i386)) with ESMTP id 656C8D414
	for <rddp@ietf.org>; Wed, 16 Apr 2003 12:57:15 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1])
	by ns.silverbacksystems.com (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 49F371EAAB
	for <rddp@ietf.org>; Wed, 16 Apr 2003 12:57:15 -0700 (PDT)
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
	charset="iso-8859-1"
Date: Wed, 16 Apr 2003 12:57:01 -0700
From: "Somesh Gupta" <somesh_gupta@silverbacksystems.com>
Importance: Normal
In-Reply-To: <277DD60FB639D511AC0400B0D068B71E0564CB36@corpmx14.corp.emc.com>
Message-ID: <NMEALCLOIBCHBDHLCMIJOEJDDDAA.somesh_gupta@silverbacksystems.com>
MIME-Version: 1.0
Received: from ns.silverbacksystems.com (localhost [127.0.0.1])
	by localhost (AvMailGate-2.0.0.6) id 13702-341434F6;
	Wed, 16 Apr 2003 12:56:48 -0700
Received: from somesh (somesh.corp.silverbacksystems.com [10.0.16.35])
	by ns.silverbacksystems.com (Postfix on SuSE Linux eMail Server 3.0) with SMTP id 7EB571EE29
	for <rddp@ietf.org>; Wed, 16 Apr 2003 12:56:48 -0700 (PDT)
Subject: RE: [rddp] Marker Progress?
To: <rddp@ietf.org>
X-AntiVirus: OK! AntiVir MailGate Version 2.0.0.6
	 at ns.silverbacksystems.com has not found any known virus in this email.
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700
X-MSMail-Priority: Normal
X-Priority: 3 (Normal)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit



> -----Original Message-----
> From: rddp-admin@ietf.org [mailto:rddp-admin@ietf.org]On Behalf Of
> Black_David@emc.com
> Sent: Wednesday, April 16, 2003 12:28 PM
> To: Dwight.Barron@hp.com; rddp@ietf.org
> Subject: [rddp] Marker Progress?
> 
> 
> > > > I do take exception that we mandate an option that essentially
> > > > diminishes or precludes a flow-through design
> > >
> > > Could we see "valid engineering justification" for the latter 
> statement,
> > > especially the "precludes" assertion?  Keep in mind that the "option"
> > > under discussion allows a receiver who benefits from markers 
> > > to require their use.
> > 
> > You are correct, the "preclude" assertion does not apply if 
> > we get to the point that markers are mandatory to 
> > implement/optional to use at all receivers (are you putting 
> > your WG Chair hat on?).
> 
> Not exactly, and not yet.  Here's the proposal that was floated to
> start this thread:
> 
> 	One alternate possibility would be to make marker usage optional,
> 	but put control of this "option" entirely in the hands of
> 	the receiver - senders MUST implement both markers and not-markers,
> 	receivers MUST implement one OR the other, and each receiver
> 	gets to pick which one is used as the benefits of markers are
> 	primarily on the receiver side.
> 
> See Jeff Mogul's email for a long explanation of why something like
> this has a better chance of making progress than the "MUST always
> use markers" and "markers are completely OPTIONAL" positions.
> 
> > However, this means that markers are 
> > mandatory to implement at the transmitter also, which seems 
> > to be where the argument for the most marker pain has 
> > manifested itself for those that are opposed to markers.
> 
> Yes.  This now puts responsibility for the transmitter consequences
> on the receiver.  Receiver implementers who believe that there are
> no significant transmitter consequences can implement receivers that
> always require markers.  Receiver implementers who are concerned about
> transmitter consequences can choose to support not using markers.
> OTOH, we could go on arguing about this for the next 6 months ...
> 
> > Once implemented, is there sufficient delta to a) warrant an 
> > option and b) operate the connection in an asymmetric mode? I 
> > apologize in advance for slipping into "commercial merits 
> > diatribe" but options cost real money once the ink is dry on 
> > the spec. I like to reserve their use/cost for things that 
> > are to the left of the decimal point, rather than the right. 
> 
> This is an unfortunate example of the "vehemence of opposition
> to options" that is causing delay.
> 
> For asymmetric mode, consider that some all-software "client"
> implementations might be more willing to put in the markers to
> help out the receiving "server" if the "client" didn't have to
> go to the pointless additional trouble of stripping markers from
> the data returned by the server.  Consider that the "client"
> might be a small portable device like a PDA or mobile phone.

  Or the low end server using an embedded processor (with a
  smaller I and D cache than a P-4) and (or without) a 
  partial or TCP offload NIC may not want to see markers.
  People have indicated that they don't believe there will
  be RDMA based storage servers without hardware acceleration,
  but that is probably more a function of protocol design
  than whether someone will do it or not, or whether there
  is a place in the market for that or not.

> 
> Thanks,
> --David 
> ----------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 176 South St., Hopkinton, MA  01748
> +1 (508) 293-7953             FAX: +1 (508) 293-7786
> black_david@emc.com        Mobile: +1 (978) 394-7754
> ----------------------------------------------------
> _______________________________________________
> rddp mailing list
> rddp@ietf.org
> https://www1.ietf.org/mailman/listinfo/rddp
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Wed Apr 16 15:59:53 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA14937
	for <rddp-archive@odin.ietf.org>; Wed, 16 Apr 2003 15:59:53 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3GK8d219309
	for rddp-archive@odin.ietf.org; Wed, 16 Apr 2003 16:08:39 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GK8d819306
	for <rddp-web-archive@optimus.ietf.org>; Wed, 16 Apr 2003 16:08:39 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA14928
	for <rddp-web-archive@ietf.org>; Wed, 16 Apr 2003 15:59:23 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195t6P-0003T4-00
	for rddp-web-archive@ietf.org; Wed, 16 Apr 2003 16:01:53 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 195t6P-0003T1-00
	for rddp-web-archive@ietf.org; Wed, 16 Apr 2003 16:01:53 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GK81819251;
	Wed, 16 Apr 2003 16:08:01 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GK7K818852
	for <rddp@optimus.ietf.org>; Wed, 16 Apr 2003 16:07:20 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA14899
	for <rddp@ietf.org>; Wed, 16 Apr 2003 15:58:03 -0400 (EDT)
From: Black_David@emc.com
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195t58-0003Sg-00
	for rddp@ietf.org; Wed, 16 Apr 2003 16:00:34 -0400
Received: from mxic2.corp.emc.com ([128.221.31.40])
	by ietf-mx with esmtp (Exim 4.12)
	id 195t57-0003Sc-00
	for rddp@ietf.org; Wed, 16 Apr 2003 16:00:33 -0400
Received: by mxic2.corp.emc.com with Internet Mail Service (5.5.2653.19)
	id <JAALNGJG>; Wed, 16 Apr 2003 16:00:10 -0400
Message-ID: <277DD60FB639D511AC0400B0D068B71E0564CB37@corpmx14.corp.emc.com>
To: rddp@ietf.org
Date: Wed, 16 Apr 2003 15:55:01 -0400
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: [rddp] Supertransport?
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

I'm pulling together a couple of related issues in a single response.

Dwight Barron asked:

> Does the existence of iSCSI markers and CRC create any more
> of a new supertransport protocol by overcoming the same
> limitations of TCP?

No - it's not an issue for iSCSI because iSCSI is unambiguously
a ULP and is not trying to provide a general data transfer mechanism
for use by unrelated protocols.  For example, HTTP over iSCSI makes
little to no sense.  In contrast, RDDP is trying to be a transport
"shim" that provides services to unrelated protocols - for example,
HTTP over RDDP does make sense when large amounts of data are involved.

Caitlin Bestler asked:

> > (1) The "proof" requirement that two communicating stacks can't get
> > confused about the format of communication applies to the entire RDDP
> > protocol stack running over TCP, not just the issue of whether a
> > modified vs. unmodified TCP is being used.
> 
> ULP responsibility for matching peers has been the rule for IP protocols
> throughout history.  Why the change?

What's actually changed here is the willingness to consider the
idea of a transport "shim".  ULPs currently unambiguously select
their transport protocol by causing selection of the IP protocol
number (e.g., TCP = 6, UDP = 17, SCTP = 132).  Two peers who are
confused at this level (e.g., TCP vs. SCTP) won't get very far
because IP protocol stacks tend to demux on the IP protocol number.

The idea of using another IP protocol number to unambiguously
indicate RDDP functionality has been overwhelmingly rejected by
the RDDP WG for reasons that we don't need to go back over here.
Among the reasons for the "proof" requirement is that rejection
of use of a different IP protocol number does not absolve the
RDDP WG of its obligation of ensuring that transport level
functionality is unambiguously selected.  ULPs will still need
to do peer matching above RDDP.

Also, as Jeff Mogul wrote:

	(This is a requirement because the IESG says it is a
	requirement; it doesn't matter what my personal technical
	judgement is, because the IESG has the last say.  I've
	been there before.)

But I do hope I've provided a little more insight ...

Thanks,
--David

----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
black_david@emc.com        Mobile: +1 (978) 394-7754
----------------------------------------------------
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Wed Apr 16 17:59:09 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA18314
	for <rddp-archive@odin.ietf.org>; Wed, 16 Apr 2003 17:59:09 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3GM7vp28721
	for rddp-archive@odin.ietf.org; Wed, 16 Apr 2003 18:07:57 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GM7v828718
	for <rddp-web-archive@optimus.ietf.org>; Wed, 16 Apr 2003 18:07:57 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA18295
	for <rddp-web-archive@ietf.org>; Wed, 16 Apr 2003 17:58:39 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195uxo-00040q-00
	for rddp-web-archive@ietf.org; Wed, 16 Apr 2003 18:01:08 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 195uxn-00040k-00
	for rddp-web-archive@ietf.org; Wed, 16 Apr 2003 18:01:07 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GM7C828057;
	Wed, 16 Apr 2003 18:07:12 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GM6S827809
	for <rddp@optimus.ietf.org>; Wed, 16 Apr 2003 18:06:28 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA18265
	for <rddp@ietf.org>; Wed, 16 Apr 2003 17:57:11 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195uwN-00040W-00
	for rddp@ietf.org; Wed, 16 Apr 2003 17:59:40 -0400
Received: from ztxmail04.ztx.compaq.com ([161.114.1.208])
	by ietf-mx with esmtp (Exim 4.12)
	id 195uwN-00040B-00
	for rddp@ietf.org; Wed, 16 Apr 2003 17:59:39 -0400
Received: from cceexg12.americas.cpqcorp.net (cceexg12.americas.cpqcorp.net [16.110.250.124])
	by ztxmail04.ztx.compaq.com (Postfix) with ESMTP
	id F409895D4; Wed, 16 Apr 2003 16:59:18 -0500 (CDT)
Received: from cceexc16.americas.cpqcorp.net ([16.110.250.58]) by cceexg12.americas.cpqcorp.net with Microsoft SMTPSVC(5.0.2195.2966);
	 Wed, 16 Apr 2003 16:59:18 -0500
X-MimeOLE: Produced By Microsoft Exchange V6.0.6375.0
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Date: Wed, 16 Apr 2003 16:59:18 -0500
Message-ID: <01CA1B1F74A4EC40A9C887F7B58F87240314313E@cceexc16.americas.cpqcorp.net>
Thread-Topic: Marker Progress?
Thread-Index: AcMETxfztNFoV2XBTBKkqPnQqFckVAAE5KXA
From: "Barron, Dwight" <Dwight.Barron@hp.com>
To: <Black_David@emc.com>, <rddp@ietf.org>
X-OriginalArrivalTime: 16 Apr 2003 21:59:18.0796 (UTC) FILETIME=[6DDCB4C0:01C30463]
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by www1.ietf.org id h3GM6S827810
Subject: [rddp] RE: Marker Progress?
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 8bit
Content-Transfer-Encoding: 8bit

David, how does this proposal relate to previous discussions about configurable marker intervals? Would the ability for the receiver to disable markers at the transmitter be sufficient to mitigate the need for additional marker interval options? 

Regards,
Dwight

> -----Original Message-----
> From: Black_David@emc.com [mailto:Black_David@emc.com]
> Sent: Wednesday, April 16, 2003 2:28 PM
> To: Barron, Dwight; rddp@ietf.org
> Subject: Marker Progress?
> 
> 
> > > > I do take exception that we mandate an option that essentially
> > > > diminishes or precludes a flow-through design
> > >
> > > Could we see "valid engineering justification" for the 
> latter statement,
> > > especially the "precludes" assertion?  Keep in mind that 
> the "option"
> > > under discussion allows a receiver who benefits from markers 
> > > to require their use.
> > 
> > You are correct, the "preclude" assertion does not apply if 
> > we get to the point that markers are mandatory to 
> > implement/optional to use at all receivers (are you putting 
> > your WG Chair hat on?).
> 
> Not exactly, and not yet.  Here's the proposal that was floated to
> start this thread:
> 
> 	One alternate possibility would be to make marker usage 
> optional,
> 	but put control of this "option" entirely in the hands of
> 	the receiver - senders MUST implement both markers and 
> not-markers,
> 	receivers MUST implement one OR the other, and each receiver
> 	gets to pick which one is used as the benefits of markers are
> 	primarily on the receiver side.
> 
> See Jeff Mogul's email for a long explanation of why something like
> this has a better chance of making progress than the "MUST always
> use markers" and "markers are completely OPTIONAL" positions.
> 
> > However, this means that markers are 
> > mandatory to implement at the transmitter also, which seems 
> > to be where the argument for the most marker pain has 
> > manifested itself for those that are opposed to markers.
> 
> Yes.  This now puts responsibility for the transmitter consequences
> on the receiver.  Receiver implementers who believe that there are
> no significant transmitter consequences can implement receivers that
> always require markers.  Receiver implementers who are concerned about
> transmitter consequences can choose to support not using markers.
> OTOH, we could go on arguing about this for the next 6 months ...
> 
> > Once implemented, is there sufficient delta to a) warrant an 
> > option and b) operate the connection in an asymmetric mode? I 
> > apologize in advance for slipping into "commercial merits 
> > diatribe" but options cost real money once the ink is dry on 
> > the spec. I like to reserve their use/cost for things that 
> > are to the left of the decimal point, rather than the right. 
> 
> This is an unfortunate example of the "vehemence of opposition
> to options" that is causing delay.
> 
> For asymmetric mode, consider that some all-software "client"
> implementations might be more willing to put in the markers to
> help out the receiving "server" if the "client" didn't have to
> go to the pointless additional trouble of stripping markers from
> the data returned by the server.  Consider that the "client"
> might be a small portable device like a PDA or mobile phone.
> 
> Thanks,
> --David 
> ----------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 176 South St., Hopkinton, MA  01748
> +1 (508) 293-7953             FAX: +1 (508) 293-7786
> black_david@emc.com        Mobile: +1 (978) 394-7754
> ----------------------------------------------------
> 
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Wed Apr 16 18:35:53 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA20165
	for <rddp-archive@odin.ietf.org>; Wed, 16 Apr 2003 18:35:53 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3GMigu30904
	for rddp-archive@odin.ietf.org; Wed, 16 Apr 2003 18:44:42 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GMif830901
	for <rddp-web-archive@optimus.ietf.org>; Wed, 16 Apr 2003 18:44:41 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA20137
	for <rddp-web-archive@ietf.org>; Wed, 16 Apr 2003 18:35:23 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195vXM-00045s-00
	for rddp-web-archive@ietf.org; Wed, 16 Apr 2003 18:37:52 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 195vXL-00045p-00
	for rddp-web-archive@ietf.org; Wed, 16 Apr 2003 18:37:51 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GMi2830840;
	Wed, 16 Apr 2003 18:44:02 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3GMhj830811
	for <rddp@optimus.ietf.org>; Wed, 16 Apr 2003 18:43:45 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA20084
	for <rddp@ietf.org>; Wed, 16 Apr 2003 18:34:26 -0400 (EDT)
From: Black_David@emc.com
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195vWR-00045J-00
	for rddp@ietf.org; Wed, 16 Apr 2003 18:36:55 -0400
Received: from maho3msx2.corp.emc.com ([128.221.11.32])
	by ietf-mx with esmtp (Exim 4.12)
	id 195vWR-00045G-00
	for rddp@ietf.org; Wed, 16 Apr 2003 18:36:55 -0400
Received: by maho3msx2.corp.emc.com with Internet Mail Service (5.5.2653.19)
	id <JAAJGZGK>; Wed, 16 Apr 2003 18:36:35 -0400
Message-ID: <277DD60FB639D511AC0400B0D068B71E0564CB3C@corpmx14.corp.emc.com>
To: Dwight.Barron@hp.com, rddp@ietf.org
Date: Wed, 16 Apr 2003 18:31:25 -0400
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: [rddp] RE: Marker Progress?
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

> David, how does this proposal relate to previous discussions 
> about configurable marker intervals? Would the ability for 
> the receiver to disable markers at the transmitter be 
> sufficient to mitigate the need for additional marker 
> interval options? 

That appears to be likely, but let me go ahead and ask --

	Does anyone want to see support for additional marker
	interval sizes (other than the 512 bytes that was
	deemed important in San Francisco)?

If so, speak up quickly, please.

Thanks,
--David
----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
black_david@emc.com        Mobile: +1 (978) 394-7754
----------------------------------------------------
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Wed Apr 16 20:13:02 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA22038
	for <rddp-archive@odin.ietf.org>; Wed, 16 Apr 2003 20:13:02 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3H0Lrd04354
	for rddp-archive@odin.ietf.org; Wed, 16 Apr 2003 20:21:53 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3H0Lq804351
	for <rddp-web-archive@optimus.ietf.org>; Wed, 16 Apr 2003 20:21:52 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA22035
	for <rddp-web-archive@ietf.org>; Wed, 16 Apr 2003 20:12:31 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195x3N-0004R7-00
	for rddp-web-archive@ietf.org; Wed, 16 Apr 2003 20:15:01 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 195x3M-0004R4-00
	for rddp-web-archive@ietf.org; Wed, 16 Apr 2003 20:15:00 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3H0LC804333;
	Wed, 16 Apr 2003 20:21:12 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3H0Kq804310
	for <rddp@optimus.ietf.org>; Wed, 16 Apr 2003 20:20:52 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA22010
	for <rddp@ietf.org>; Wed, 16 Apr 2003 20:11:31 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 195x2P-0004Qd-00
	for rddp@ietf.org; Wed, 16 Apr 2003 20:14:01 -0400
Received: from e31.co.us.ibm.com ([32.97.110.129])
	by ietf-mx with esmtp (Exim 4.12)
	id 195x2O-0004QU-00
	for rddp@ietf.org; Wed, 16 Apr 2003 20:14:00 -0400
Received: from westrelay04.boulder.ibm.com (westrelay04.boulder.ibm.com [9.17.193.32])
	by e31.co.us.ibm.com (8.12.9/8.12.2) with ESMTP id h3H0C4Qs168520;
	Wed, 16 Apr 2003 20:12:04 -0400
Received: from d03nm803.boulder.ibm.com (d03av02.boulder.ibm.com [9.17.193.82])
	by westrelay04.boulder.ibm.com (8.12.8/NCO/VER6.5) with ESMTP id h3H0DUYt048460;
	Wed, 16 Apr 2003 18:13:30 -0600
Subject: Re: [rddp] RE: Marker Progress?
To: Black_David@emc.com
Cc: Dwight.Barron@hp.com, rddp@ietf.org
X-Mailer: Lotus Notes Release 5.0.7  March 21, 2001
Message-ID: <OF416AA4C1.595B2977-ON86256D0B.00002F13@us.ibm.com>
From: Renato Recio <recio@us.ibm.com>
Date: Wed, 16 Apr 2003 19:13:27 -0500
X-MIMETrack: Serialize by Router on D03NM803/03/M/IBM(Release 6.0 [IBM]|March 31, 2003) at
 04/16/2003 18:13:30
MIME-Version: 1.0
Content-type: text/plain; charset=US-ASCII
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>





What is the strong value of having variable marker interval sizes?  In my
view, without a really, really good reason, and considering the increase in
complexity (receiver, additional negotiation/mgt/debug field) associated
with variable marker sizes,  I would not want to see support for additional
marker interval sizes.

Thanks,

Renato J Recio
IBM Distinguished Engineer
Member IBM Academy of Technology
IBM I/O Architecture and Technology
Tel 512-838-3685, T/L 678-3685



                                                                                                                        
                      Black_David@emc.c                                                                                 
                      om                       To:       Dwight.Barron@hp.com, rddp@ietf.org                            
                      Sent by:                 cc:                                                                      
                      rddp-admin@ietf.o        Subject:  [rddp] RE: Marker Progress?                                    
                      rg                                                                                                
                                                                                                                        
                                                                                                                        
                      04/16/2003 05:31                                                                                  
                      PM                                                                                                
                                                                                                                        
                                                                                                                        




> David, how does this proposal relate to previous discussions
> about configurable marker intervals? Would the ability for
> the receiver to disable markers at the transmitter be
> sufficient to mitigate the need for additional marker
> interval options?

That appears to be likely, but let me go ahead and ask --

             Does anyone want to see support for additional marker
             interval sizes (other than the 512 bytes that was
             deemed important in San Francisco)?

If so, speak up quickly, please.

Thanks,
--David
----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
black_david@emc.com        Mobile: +1 (978) 394-7754
----------------------------------------------------
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr 17 12:09:04 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA26835
	for <rddp-archive@odin.ietf.org>; Thu, 17 Apr 2003 12:09:04 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3HGIFi16113
	for rddp-archive@odin.ietf.org; Thu, 17 Apr 2003 12:18:15 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HGIF816110
	for <rddp-web-archive@optimus.ietf.org>; Thu, 17 Apr 2003 12:18:15 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA26792
	for <rddp-web-archive@ietf.org>; Thu, 17 Apr 2003 12:08:33 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196ByX-0000yQ-00
	for rddp-web-archive@ietf.org; Thu, 17 Apr 2003 12:11:01 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 196ByX-0000yN-00
	for rddp-web-archive@ietf.org; Thu, 17 Apr 2003 12:11:01 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HGHI816076;
	Thu, 17 Apr 2003 12:17:18 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HGGk816022
	for <rddp@optimus.ietf.org>; Thu, 17 Apr 2003 12:16:46 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA26746
	for <rddp@ietf.org>; Thu, 17 Apr 2003 12:07:04 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196Bx6-0000xh-00
	for rddp@ietf.org; Thu, 17 Apr 2003 12:09:32 -0400
Received: from thebe.your-site.com ([140.186.45.26])
	by ietf-mx with esmtp (Exim 4.12)
	id 196Bx6-0000xZ-00
	for rddp@ietf.org; Thu, 17 Apr 2003 12:09:32 -0400
Received: from 192.168.0.2 (64-144-5-25.client.dsl.net [64.144.5.25])
	by thebe.your-site.com (Postfix) with ESMTP id A3069245157
	for <rddp@ietf.org>; Thu, 17 Apr 2003 12:09:26 -0400 (EDT)
Date: Thu, 17 Apr 2003 11:11:41 -0500
From: Caitlin Bestler <cait@asomi.com>
Subject: Re: [rddp] RE: Marker Progress?
To: rddp@ietf.org
X-Priority: 3
Message-ID: <r01050300-1025-472094B970EF11D7B1DC003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

On 4/16/03, Renato Recio wrote:

>
>
>
>
>What is the strong value of having variable marker
>interval sizes?  In my view, without a really, really good
>reason, and considering the increase in complexity
>(receiver, additional negotiation/mgt/debug field)
>associated with variable marker sizes,  I would not want
>to see support for additional marker interval sizes.
>

I agree.

As I understood it, the intent of variable marker sizes was
to minimize their overhead while still ensuring one marker
per TCP segment.

But for all the reasons previously discussed on why there is
no such thing as an "aligned connection", merely one that
has been aligned so far, a variable marker size cannot be
tailored to the PMTU. PMTU is a measured characteristic
of a connection, not a configured attribute.

Worse, the entire simplicity of markers requires that they 
be unchanged over the lifespan of the connection. A protocol
that allowed the interval to change to X at stream offset Y
is possible, but certain to be complex. For one thing, it
would require MPA to define in-stream "control chunks".

The benefit of a greater marker interval is at best margin.
The complexity it would introduce is considerable,
especially for testing and monitoring.

I believe the real debate is between mandatory fixed
interval markers, and David's suggestion for having the
receiver mandate whether it expected markers or not.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr 17 12:32:38 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA27568
	for <rddp-archive@odin.ietf.org>; Thu, 17 Apr 2003 12:32:38 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3HGfm118103
	for rddp-archive@odin.ietf.org; Thu, 17 Apr 2003 12:41:48 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HGfm818100
	for <rddp-web-archive@optimus.ietf.org>; Thu, 17 Apr 2003 12:41:48 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA27561
	for <rddp-web-archive@ietf.org>; Thu, 17 Apr 2003 12:32:08 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196CLK-00017h-00
	for rddp-web-archive@ietf.org; Thu, 17 Apr 2003 12:34:34 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 196CLJ-00017e-00
	for rddp-web-archive@ietf.org; Thu, 17 Apr 2003 12:34:33 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HGf2818072;
	Thu, 17 Apr 2003 12:41:02 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HGeH818013
	for <rddp@optimus.ietf.org>; Thu, 17 Apr 2003 12:40:17 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA27516
	for <rddp@ietf.org>; Thu, 17 Apr 2003 12:30:37 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196CJr-00017J-00
	for rddp@ietf.org; Thu, 17 Apr 2003 12:33:03 -0400
Received: from mx01.netapp.com ([198.95.226.53])
	by ietf-mx with esmtp (Exim 4.12)
	id 196CJq-00017G-00
	for rddp@ietf.org; Thu, 17 Apr 2003 12:33:02 -0400
Received: from frejya.corp.netapp.com (frejya [10.10.20.91])
	by mx01.netapp.com (8.12.9/8.12.9/NTAP-1.4) with ESMTP id h3HGWeFB001550
	for <rddp@ietf.org>; Thu, 17 Apr 2003 09:32:40 -0700 (PDT)
Received: from TMT.netapp.com (tmt [10.97.1.32])
	by frejya.corp.netapp.com (8.12.9/8.12.9/NTAP-1.4) with ESMTP id h3HGWdAc003453
	for <rddp@ietf.org>; Thu, 17 Apr 2003 09:32:39 -0700 (PDT)
Message-Id: <5.2.0.9.2.20030417122810.00bb6d70@silver.nane.netapp.com>
Date: Thu, 17 Apr 2003 12:31:25 -0400
To: rddp@ietf.org
From: "Talpey, Thomas" <Thomas.Talpey@netapp.com>
Subject: Re: [rddp] RE: Marker Progress?
In-Reply-To: <277DD60FB639D511AC0400B0D068B71E0564CB3C@corpmx14.corp.emc
 .com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

I believe the three interesting values are:
	none (infinity)
	512
	8192
("None" for software receivers, 512 for full generality,
and 8K for local networks with jumbo frames.)

Therefore your question might be rephrased, do we need an
optional 8K marker interval?

I say it's not compelling enough to be worthwhile.

Tom.

At 06:31 PM 4/16/2003, Black_David@emc.com wrote:
>> David, how does this proposal relate to previous discussions 
>> about configurable marker intervals? Would the ability for 
>> the receiver to disable markers at the transmitter be 
>> sufficient to mitigate the need for additional marker 
>> interval options? 
>
>That appears to be likely, but let me go ahead and ask --
>
>	Does anyone want to see support for additional marker
>	interval sizes (other than the 512 bytes that was
>	deemed important in San Francisco)?
>
>If so, speak up quickly, please.
>
>Thanks,
>--David
>----------------------------------------------------
>David L. Black, Senior Technologist
>EMC Corporation, 176 South St., Hopkinton, MA  01748
>+1 (508) 293-7953             FAX: +1 (508) 293-7786
>black_david@emc.com        Mobile: +1 (978) 394-7754
>----------------------------------------------------
>_______________________________________________
>rddp mailing list
>rddp@ietf.org
>https://www1.ietf.org/mailman/listinfo/rddp

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr 17 12:33:31 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA27632
	for <rddp-archive@odin.ietf.org>; Thu, 17 Apr 2003 12:33:31 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3HGgfr18189
	for rddp-archive@odin.ietf.org; Thu, 17 Apr 2003 12:42:41 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HGgf818186
	for <rddp-web-archive@optimus.ietf.org>; Thu, 17 Apr 2003 12:42:41 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA27610
	for <rddp-web-archive@ietf.org>; Thu, 17 Apr 2003 12:33:01 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196CMB-00018G-00
	for rddp-web-archive@ietf.org; Thu, 17 Apr 2003 12:35:27 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 196CMA-00018D-00
	for rddp-web-archive@ietf.org; Thu, 17 Apr 2003 12:35:26 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HGg0818128;
	Thu, 17 Apr 2003 12:42:00 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HGfl818095
	for <rddp@optimus.ietf.org>; Thu, 17 Apr 2003 12:41:47 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA27557
	for <rddp@ietf.org>; Thu, 17 Apr 2003 12:32:06 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196CLI-00017b-00
	for rddp@ietf.org; Thu, 17 Apr 2003 12:34:32 -0400
Received: from thebe.your-site.com ([140.186.45.26])
	by ietf-mx with esmtp (Exim 4.12)
	id 196CLI-00017S-00
	for rddp@ietf.org; Thu, 17 Apr 2003 12:34:32 -0400
Received: from 192.168.0.2 (64-144-5-25.client.dsl.net [64.144.5.25])
	by thebe.your-site.com (Postfix) with ESMTP
	id C8875244E00; Thu, 17 Apr 2003 12:34:26 -0400 (EDT)
Date: Thu, 17 Apr 2003 11:36:42 -0500
From: Caitlin Bestler <cait@asomi.com>
Subject: Re: [rddp] Options and Markers 
To: Jeffrey Mogul <Jeff.Mogul@hp.com>
Cc: "Williams, Jim" <Jim.Williams@emulex.com>, rddp@ietf.org
X-Priority: 3
In-Reply-To: <200304160022.h3G0MT8o013995@wera.hpl.hp.com>
Message-ID: <r01050300-1025-C5A4F86F70F211D7B1DC003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

On 4/15/03, Jeffrey Mogul wrote:

>So, are options really that bad?  I don't think so.  First
>of all, if X is mandatory to implement but optional to
>use, and the receiver gets to choose, then you can make
>this work without extra round-trips by having the sender
>start the connection using the option, and then let the
>receiver tell the sender to turn it off.
>
Starting markers in the middle of the data would be very
complex. I agree that this can be made to work without 
an extra round-trip, but there is still a requirement 
that the markers be present (or absent) from the very
beginning. This does make the first segment special,
but that is a given (if we accept that some form of
protocol marker has been mandated by the powers-that-be).
 
>Second, my guess is that any solution to the "prove MPA
>doesn't get confused with unmodified TCP" requirement will
>provide us with a convenient place to pass some options. 
>(This is a requirement because the IESG says it is a
>requirement; it doesn't matter what my personal technical
>judgement is, because the IESG has the last say.  I've
>been there before.)

The two are definitely related.

Personally, I do not find the "same ULP different de facto
protocol but using the same number" argument compelling.
There is no guard seperating HTTP over TLS-SSL/TCP from
HTTP over TCP. But I will concede that there is at least
some merit to the proposal, and it can be argued that
TLS *should* have been required to do something similiar
but that it is too late.

For the record, though, I still believe this requirement
should be stated to tsvwg and debated there.

Assuming that it is something that RDDP will end up with,
however, it then makes sense to communicate other mode
information at the same time.

I believe optional markers is clearly a case where there
is benefit to using this packet that we have to pay for
anyway (i.e., "free") to set a connection specific option.

I believe the debate has established the following points:

--- There is an implementation strategy ("cut through"
    or "low memory") where omitting markers would
    substantially increase the amount of buffering
    required for an integrated RDDP/MPA/TCP stack.
    
    Not allowing such receivers to mandate markers in
    streams sent to them would have a serious impact
    on their performance.
    
--- There are also implementation strategies based
    upon cleaner layering where the TCP layer has
    sufficient buffering to still provide "single
    touch" placement by simply deferring placement
    until the gap TCP segments arrive.
    
    Having to deal with markers in such environments
    provides no benefits, and is at least a minor
    performance penalty. I believe the remaing dispute
    is whether this is a *severe* penalty.
    
--- It is possible to specify an opening "mode establishing"
    packet which will ensure that each side sends markers
    to match the expectation/demand of the receiver without
    requiring an additional round-trip before real data
    can be exchanged.
    
I remain convinced that the cost of markers to the "software
only" / "buffer rich" / "layered" solutions does not qualify
as "severe" and that it would not justify the creation of a
mode packet.

But, if a mode packet is going to exist *anyway*, why not
enable this simplification for receivers that have valid
reasons to avoid the use of markers? The extra byte required
to set this option up is surely justified. There is the
extra QA cost for senders, which must support both modes,
but I for one am convinced that the case for allowing some
receivers to work without markers has been made
sufficiently.


Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr 17 12:46:45 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA28037
	for <rddp-archive@odin.ietf.org>; Thu, 17 Apr 2003 12:46:45 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3HGtuZ20640
	for rddp-archive@odin.ietf.org; Thu, 17 Apr 2003 12:55:56 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HGtt820637
	for <rddp-web-archive@optimus.ietf.org>; Thu, 17 Apr 2003 12:55:55 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA28014
	for <rddp-web-archive@ietf.org>; Thu, 17 Apr 2003 12:46:15 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196CYz-0001Eh-00
	for rddp-web-archive@ietf.org; Thu, 17 Apr 2003 12:48:41 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 196CYy-0001Ee-00
	for rddp-web-archive@ietf.org; Thu, 17 Apr 2003 12:48:40 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HGq1820473;
	Thu, 17 Apr 2003 12:52:01 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HGpr820447
	for <rddp@optimus.ietf.org>; Thu, 17 Apr 2003 12:51:53 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA27960
	for <rddp@ietf.org>; Thu, 17 Apr 2003 12:42:13 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196CV5-0001DV-00
	for rddp@ietf.org; Thu, 17 Apr 2003 12:44:39 -0400
Received: from e31.co.us.ibm.com ([32.97.110.129])
	by ietf-mx with esmtp (Exim 4.12)
	id 196CV4-0001DI-00
	for rddp@ietf.org; Thu, 17 Apr 2003 12:44:38 -0400
Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.17.195.11])
	by e31.co.us.ibm.com (8.12.9/8.12.2) with ESMTP id h3HGgqQs102642;
	Thu, 17 Apr 2003 12:42:52 -0400
Received: from d03nm803.boulder.ibm.com (d03av02.boulder.ibm.com [9.17.193.82])
	by westrelay02.boulder.ibm.com (8.12.8/NCO/VER6.5) with ESMTP id h3HGiKpt297254;
	Thu, 17 Apr 2003 10:44:20 -0600
Subject: Re: [rddp] RE: Marker Progress?
To: Caitlin Bestler <cait@asomi.com>
Cc: rddp@ietf.org
X-Mailer: Lotus Notes Release 5.0.7  March 21, 2001
Message-ID: <OF26831CA6.00091872-ON86256D0B.005A7077@us.ibm.com>
From: Renato Recio <recio@us.ibm.com>
Date: Thu, 17 Apr 2003 11:44:19 -0500
X-MIMETrack: Serialize by Router on D03NM803/03/M/IBM(Release 6.0 [IBM]|March 31, 2003) at
 04/17/2003 10:44:19
MIME-Version: 1.0
Content-type: text/plain; charset=US-ASCII
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>





Folks,

   So, it seems the real issue is whether normatively defined that:
A)  RDMA/DDP over TCP requires MPA in the middle, or
B)  RDMA/DDP over TCP may use MPA or may not.

If that is the case, then it seems to me we could either decide now, or let
the market decide later.  In my view, the market will go with (A), because
of the benefits it provides.  If we go with (B), we would add complexity to
the (initialization) protocols and a slight increase in implementation
complexity, which would, in my view, not be used much (if at all) in the
market.

So, what's the gain in going with (B)? If its to allow a NIC to be fully
buffered and hence it would not need markers for placement?  It seems to me
that type of NIC would need a lot of buffer capacity to support the number
of connections demanded by future Servers, Appliances, Storage, etc...   It
seems to me the cost of that additional capacity, even considering $/MB
take-down rates, is not going to be competitive when compared to a
flow-through design that doesn't require the additional capacity.

Thanks,

Renato J Recio
IBM Distinguished Engineer
Member IBM Academy of Technology
IBM I/O Architecture and Technology
Tel 512-838-3685, T/L 678-3685



                                                                                                                        
                      Caitlin Bestler                                                                                   
                      <cait@asomi.com>         To:       rddp@ietf.org                                                  
                      Sent by:                 cc:                                                                      
                      rddp-admin@ietf.o        Subject:  Re: [rddp] RE: Marker Progress?                                
                      rg                                                                                                
                                                                                                                        
                                                                                                                        
                      04/17/2003 11:11                                                                                  
                      AM                                                                                                
                                                                                                                        
                                                                                                                        




On 4/16/03, Renato Recio wrote:

>
>
>
>
>What is the strong value of having variable marker
>interval sizes?  In my view, without a really, really good
>reason, and considering the increase in complexity
>(receiver, additional negotiation/mgt/debug field)
>associated with variable marker sizes,  I would not want
>to see support for additional marker interval sizes.
>

I agree.

As I understood it, the intent of variable marker sizes was
to minimize their overhead while still ensuring one marker
per TCP segment.

But for all the reasons previously discussed on why there is
no such thing as an "aligned connection", merely one that
has been aligned so far, a variable marker size cannot be
tailored to the PMTU. PMTU is a measured characteristic
of a connection, not a configured attribute.

Worse, the entire simplicity of markers requires that they
be unchanged over the lifespan of the connection. A protocol
that allowed the interval to change to X at stream offset Y
is possible, but certain to be complex. For one thing, it
would require MPA to define in-stream "control chunks".

The benefit of a greater marker interval is at best margin.
The complexity it would introduce is considerable,
especially for testing and monitoring.

I believe the real debate is between mandatory fixed
interval markers, and David's suggestion for having the
receiver mandate whether it expected markers or not.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr 17 12:48:54 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA28130
	for <rddp-archive@odin.ietf.org>; Thu, 17 Apr 2003 12:48:54 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3HGw4u20773
	for rddp-archive@odin.ietf.org; Thu, 17 Apr 2003 12:58:04 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HGw4820770
	for <rddp-web-archive@optimus.ietf.org>; Thu, 17 Apr 2003 12:58:04 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA28109
	for <rddp-web-archive@ietf.org>; Thu, 17 Apr 2003 12:48:23 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196Cb3-0001Fn-00
	for rddp-web-archive@ietf.org; Thu, 17 Apr 2003 12:50:49 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 196Cb3-0001Fk-00
	for rddp-web-archive@ietf.org; Thu, 17 Apr 2003 12:50:49 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HGs1820542;
	Thu, 17 Apr 2003 12:54:01 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HGrX820519
	for <rddp@optimus.ietf.org>; Thu, 17 Apr 2003 12:53:33 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA27981
	for <rddp@ietf.org>; Thu, 17 Apr 2003 12:43:52 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196CWg-0001E0-00
	for rddp@ietf.org; Thu, 17 Apr 2003 12:46:18 -0400
Received: from thebe.your-site.com ([140.186.45.26])
	by ietf-mx with esmtp (Exim 4.12)
	id 196CWg-0001Dl-00
	for rddp@ietf.org; Thu, 17 Apr 2003 12:46:18 -0400
Received: from 192.168.0.2 (64-144-5-25.client.dsl.net [64.144.5.25])
	by thebe.your-site.com (Postfix) with ESMTP
	id 178FC245A9D; Thu, 17 Apr 2003 12:46:13 -0400 (EDT)
Date: Thu, 17 Apr 2003 11:48:28 -0500
From: Caitlin Bestler <cait@asomi.com>
Subject: RE: [rddp] Options and Markers
To: "Barron, Dwight" <Dwight.Barron@hp.com>
Cc: Black_David@emc.com, rddp@ietf.org
X-Priority: 3
In-Reply-To: <01CA1B1F74A4EC40A9C887F7B58F87240314313C@cceexc16.americas.cpqcorp.net>
Message-ID: <r01050300-1025-6A86E88570F411D7B1DC003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

On 4/16/03, Barron, Dwight wrote:

>You are correct, the "preclude" assertion does not apply
>if we get to the point that markers are mandatory to
>implement/optional to use at all receivers (are you
>putting your WG Chair hat on?). However, this means that
>markers are mandatory to implement at the transmitter
>also, which seems to be where the argument for the most
>marker pain has manifested itself for those that are
>opposed to markers. Once implemented, is there sufficient
>delta to a) warrant an option and b) operate the
>connection in an asymmetric mode? I apologize in advance
>for slipping into "commercial merits diatribe" but options
>cost real money once the ink is dry on the spec. I like to
>reserve their use/cost for things that are to the left of
>the decimal point, rather than the right.

If we agree that a sender MUST have the capability to send
with markers, then the only question is whether there is a
cost in disabling that capability.

Given that the same stack will need to send unmodified TCP
traffic it obviously needs that capability anyway.

Other than QA testing, I don't see the cost to a sender
of making this receiver specifiable. QA testing *is* a
serious cost, but the benefit to some receivers, IMHO,
clearly justifies it.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr 17 13:43:06 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA29866
	for <rddp-archive@odin.ietf.org>; Thu, 17 Apr 2003 13:43:05 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3HHqHX25770
	for rddp-archive@odin.ietf.org; Thu, 17 Apr 2003 13:52:17 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HHqH825767
	for <rddp-web-archive@optimus.ietf.org>; Thu, 17 Apr 2003 13:52:17 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA29845
	for <rddp-web-archive@ietf.org>; Thu, 17 Apr 2003 13:42:35 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196DRV-0001b1-00
	for rddp-web-archive@ietf.org; Thu, 17 Apr 2003 13:45:01 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 196DRV-0001ay-00
	for rddp-web-archive@ietf.org; Thu, 17 Apr 2003 13:45:01 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HHm7825608;
	Thu, 17 Apr 2003 13:48:07 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HHlL825561
	for <rddp@optimus.ietf.org>; Thu, 17 Apr 2003 13:47:21 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA29717
	for <rddp@ietf.org>; Thu, 17 Apr 2003 13:37:39 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196DMj-0001ZE-00
	for rddp@ietf.org; Thu, 17 Apr 2003 13:40:05 -0400
Received: from thebe.your-site.com ([140.186.45.26])
	by ietf-mx with esmtp (Exim 4.12)
	id 196DMj-0001Yx-00
	for rddp@ietf.org; Thu, 17 Apr 2003 13:40:05 -0400
Received: from 192.168.0.2 (64-144-5-25.client.dsl.net [64.144.5.25])
	by thebe.your-site.com (Postfix) with ESMTP id 997DC2449E8
	for <rddp@ietf.org>; Thu, 17 Apr 2003 13:39:59 -0400 (EDT)
Date: Thu, 17 Apr 2003 12:42:15 -0500
From: Caitlin Bestler <cait@asomi.com>
To: rddp@ietf.org
X-Priority: 3
Message-ID: <r01050300-1025-EDC4968370FB11D7B1DC003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Subject: [rddp] Header CRC mandatory?
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

Have we reached a consensus that CRC-32 coverage of at least
the header is a requirement?

Without CRC coverage over the header an application CANNOT
apply its own protection to data until AFTER it invalidates
all STags that can reference the memory.

This is far in excess of the use of "one shot" STags. The
existence of another STag that has the option of referencing
the same memory exposes it to an error risk -- even when
there is a solid basis for trusting that the peer is not
a risk for malicious tampering.

In prior discussions on STags, we rejected mandating the use
of STag invalidation (requiring all STags to be "one shot").
There were numerous application profiles recognized where
flexible buffering was of value. Without header CRC
protection, all such applications would be subject to
increased risk from undetected TCP errors.

Further, this would be an *increased* risk.

With pure TCP, only the buffered offered for a specific read
is ever at risk of receiving bad data. Once delivered, it is
no longer at risk.  RDMA over TCP without header CRC
protection would place all advertised buffers at risk
from undetected TCP errors for the duration of the
advertisement.

It is my understanding of our charter that we CANNOT
increase the risk to applications over the use of non-RDMA
enabled TCP or SCTP.

I do not believe that a variable-coverage CRC is of
sufficient value to justify the increased complexity
for specifying the variation, and of testing it. But such
a decision would be valid under our charter. I do not
believe we have an option to propose direct memory placemnt
based upon unprotected headers.



Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr 17 13:55:39 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA00309
	for <rddp-archive@odin.ietf.org>; Thu, 17 Apr 2003 13:55:39 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3HI4p326508
	for rddp-archive@odin.ietf.org; Thu, 17 Apr 2003 14:04:51 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HI4p826505
	for <rddp-web-archive@optimus.ietf.org>; Thu, 17 Apr 2003 14:04:51 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA00285
	for <rddp-web-archive@ietf.org>; Thu, 17 Apr 2003 13:55:08 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196Dde-0001fn-00
	for rddp-web-archive@ietf.org; Thu, 17 Apr 2003 13:57:34 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 196Dde-0001fk-00
	for rddp-web-archive@ietf.org; Thu, 17 Apr 2003 13:57:34 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HI42826448;
	Thu, 17 Apr 2003 14:04:02 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HI3u826410
	for <rddp@optimus.ietf.org>; Thu, 17 Apr 2003 14:03:56 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA00226
	for <rddp@ietf.org>; Thu, 17 Apr 2003 13:54:14 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196Dcm-0001fB-00
	for rddp@ietf.org; Thu, 17 Apr 2003 13:56:40 -0400
Received: from thebe.your-site.com ([140.186.45.26])
	by ietf-mx with esmtp (Exim 4.12)
	id 196Dcm-0001ej-00
	for rddp@ietf.org; Thu, 17 Apr 2003 13:56:40 -0400
Received: from 192.168.0.2 (64-144-5-25.client.dsl.net [64.144.5.25])
	by thebe.your-site.com (Postfix) with ESMTP
	id 7972E24517B; Thu, 17 Apr 2003 13:56:34 -0400 (EDT)
Date: Thu, 17 Apr 2003 12:58:50 -0500
From: Caitlin Bestler <cait@asomi.com>
Subject: Re: [rddp] RE: Marker Progress?
To: Renato Recio <recio@us.ibm.com>
Cc: rddp@ietf.org
X-Priority: 3
In-Reply-To: <OF26831CA6.00091872-ON86256D0B.005A7077@us.ibm.com>
Message-ID: <r01050300-1025-3ECB764A70FE11D7B1DC003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

On 4/17/03, Renato Recio wrote:

>
>
>
>
>Folks,
>
> So, it seems the real issue is whether normatively
> defined that:
>A)  RDMA/DDP over TCP requires MPA in the middle, or
>B)  RDMA/DDP over TCP may use MPA or may not.
>

I believe that MPA is required no matter what. See my
immediate prior posting on Header CRCs being mandatory.
So the real question is whether Markers are optional where
the option is dictated by the receiver. I believe there is a
strong consensus that receivers MUST have the option to
mandate that streams sent to them have markers.


>If that is the case, then it seems to me we could either
>decide now, or let the market decide later.  In my view,
>the market will go with (A), because of the benefits it
>provides.  If we go with (B), we would add complexity to
>the (initialization) protocols and a slight increase in
>implementation complexity, which would, in my view, not be
>used much (if at all) in the market.
>
To be of value to the recievers, the standard really needs
to mandate that senders MUST comply with receiver
requirements. So, I do not see how market forces will have
an impact here. Even if 99% of the market goes with markers,
the 1% that decided otherwise would be entitled to have
senders comply with their receive requirements.

It is a valid question as to whether the existence of the
option is justified. That is, how much of a benefit is
no-marker reception to buffer-rich receivers, and are
buffer-rich receivers a valid design likely to be
implemented by at least a signifigant portion of the market?


>So, what's the gain in going with (B)? If its to allow a
>NIC to be fully buffered and hence it would not need
>markers for placement?  It seems to me that type of NIC
>would need a lot of buffer capacity to support the number
>of connections demanded by future Servers, Appliances,
>Storage, etc...   It seems to me the cost of that
>additional capacity, even considering $/MB take-down
>rates, is not going to be competitive when compared to a
>flow-through design that doesn't require the additional
>capacity.
>
If I'm desiging and/or buying an RNIC, I agree. I'd prefer a
low to zero buffer RNIC with minimal functionality. But then
I've gotten used to the idea that my preferences are not
binding upon the market. Given that AmigaDOS is my favorite
OS to date, we can go further and conclude that my
preferences are not even a valid predictor of the market.

However, the alternate design is a valid approach. It
particularly offers benefits when optimizing for a mix
of TCP and RDDP/TCP connections. A buffer rich NIC offers
optimizations for *all* TCP connections.

I have never bought the argument that markers imposed a
severe cost on "software" implementations. I remain opposed
to creating a mode marker *for this purpose*. But there is
a benefit to eliminating markers over these connections.

So let me flip the question. Other than QA testing, what
is the cost of omitting markers when the receiver will
not benefit from them?

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr 17 14:58:56 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA02359
	for <rddp-archive@odin.ietf.org>; Thu, 17 Apr 2003 14:58:56 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3HJ8Al32022
	for rddp-archive@odin.ietf.org; Thu, 17 Apr 2003 15:08:10 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HJ8A832019
	for <rddp-web-archive@optimus.ietf.org>; Thu, 17 Apr 2003 15:08:10 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA02325
	for <rddp-web-archive@ietf.org>; Thu, 17 Apr 2003 14:58:25 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196Ecu-0001z3-00
	for rddp-web-archive@ietf.org; Thu, 17 Apr 2003 15:00:52 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 196Ecu-0001yz-00
	for rddp-web-archive@ietf.org; Thu, 17 Apr 2003 15:00:52 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HJ7A831265;
	Thu, 17 Apr 2003 15:07:10 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HJ6E830912
	for <rddp@optimus.ietf.org>; Thu, 17 Apr 2003 15:06:14 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA02190
	for <rddp@ietf.org>; Thu, 17 Apr 2003 14:56:30 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196Eb3-0001wi-00
	for rddp@ietf.org; Thu, 17 Apr 2003 14:58:57 -0400
Received: from magic-mail.adaptec.com ([208.236.45.100] helo=magic.adaptec.com)
	by ietf-mx with esmtp (Exim 4.12)
	id 196Eb2-0001wC-00
	for rddp@ietf.org; Thu, 17 Apr 2003 14:58:56 -0400
Received: from redfish.adaptec.com (redfish.adaptec.com [162.62.50.11])
	by magic.adaptec.com (8.11.6/8.11.6) with ESMTP id h3HIuRZ26487
	for <rddp@ietf.org>; Thu, 17 Apr 2003 11:56:27 -0700
Received: from aimexc07.adaptec.com (aimexc07.adaptec.com [162.62.62.47])
	by redfish.adaptec.com (8.8.8p2+Sun/8.8.8) with ESMTP id LAA28720
	for <rddp@ietf.org>; Thu, 17 Apr 2003 11:58:37 -0700 (PDT)
Received: by aimexc07.adaptec.com with Internet Mail Service (5.5.2653.19)
	id <H3YSDCQJ>; Thu, 17 Apr 2003 11:58:37 -0700
Message-ID: <E156A23F1885D4119ED800B0D0498A9F04309F9D@aimexc07.adaptec.com>
From: "Carrier, John" <John_Carrier@adaptec.com>
To: rddp@ietf.org
Subject: RE: [rddp] Header CRC mandatory?
Date: Thu, 17 Apr 2003 11:58:36 -0700
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

I agree that the CRC should be mandatory, but believe that we need to
protect the entire payload not just the header.  RDMAP & DDP were originally
designed with the assumption of a CRC32 digest of all data received.  SCTP
already provides this level of protection and the CRC was added to MPA to
add the digest to TCP.  As a result, RDMAP & DDP over SCTP or over MPA/TCP
can assume the same level of data intergrity.  This is not a ULP option, but
a requirement of the RDMAP & DDP protocols.

--jc

> -----Original Message-----
> From: Caitlin Bestler [mailto:cait@asomi.com]
> Sent: Thursday, April 17, 2003 10:42 AM
> To: rddp@ietf.org
> Subject: [rddp] Header CRC mandatory?
> 
> 
> Have we reached a consensus that CRC-32 coverage of at least
> the header is a requirement?
> 
> Without CRC coverage over the header an application CANNOT
> apply its own protection to data until AFTER it invalidates
> all STags that can reference the memory.
> 
> This is far in excess of the use of "one shot" STags. The
> existence of another STag that has the option of referencing
> the same memory exposes it to an error risk -- even when
> there is a solid basis for trusting that the peer is not
> a risk for malicious tampering.
> 
> In prior discussions on STags, we rejected mandating the use
> of STag invalidation (requiring all STags to be "one shot").
> There were numerous application profiles recognized where
> flexible buffering was of value. Without header CRC
> protection, all such applications would be subject to
> increased risk from undetected TCP errors.
> 
> Further, this would be an *increased* risk.
> 
> With pure TCP, only the buffered offered for a specific read
> is ever at risk of receiving bad data. Once delivered, it is
> no longer at risk.  RDMA over TCP without header CRC
> protection would place all advertised buffers at risk
> from undetected TCP errors for the duration of the
> advertisement.
> 
> It is my understanding of our charter that we CANNOT
> increase the risk to applications over the use of non-RDMA
> enabled TCP or SCTP.
> 
> I do not believe that a variable-coverage CRC is of
> sufficient value to justify the increased complexity
> for specifying the variation, and of testing it. But such
> a decision would be valid under our charter. I do not
> believe we have an option to propose direct memory placemnt
> based upon unprotected headers.
> 
> 
> 
> Caitlin Bestler
> http://asomi.com/CaitlinBestler/
> _______________________________________________
> rddp mailing list
> rddp@ietf.org
> https://www1.ietf.org/mailman/listinfo/rddp
> 
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr 17 15:35:36 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04679
	for <rddp-archive@odin.ietf.org>; Thu, 17 Apr 2003 15:35:36 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3HJipj02057
	for rddp-archive@odin.ietf.org; Thu, 17 Apr 2003 15:44:51 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HJip802054
	for <rddp-web-archive@optimus.ietf.org>; Thu, 17 Apr 2003 15:44:51 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04675
	for <rddp-web-archive@ietf.org>; Thu, 17 Apr 2003 15:35:06 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196FCP-00028Q-00
	for rddp-web-archive@ietf.org; Thu, 17 Apr 2003 15:37:33 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 196FCO-00028N-00
	for rddp-web-archive@ietf.org; Thu, 17 Apr 2003 15:37:32 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HJi3802002;
	Thu, 17 Apr 2003 15:44:03 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HJhh801978
	for <rddp@optimus.ietf.org>; Thu, 17 Apr 2003 15:43:43 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04626
	for <rddp@ietf.org>; Thu, 17 Apr 2003 15:33:58 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196FBJ-00027u-00
	for rddp@ietf.org; Thu, 17 Apr 2003 15:36:25 -0400
Received: from palrel10.hp.com ([156.153.255.245])
	by ietf-mx with esmtp (Exim 4.12)
	id 196FBI-00027p-00
	for rddp@ietf.org; Thu, 17 Apr 2003 15:36:24 -0400
Received: from esmail.cup.hp.com (esmail.cup.hp.com [15.0.65.164])
	by palrel10.hp.com (Postfix) with ESMTP
	id D08391C0289E; Thu, 17 Apr 2003 12:36:37 -0700 (PDT)
Received: from mk731915.cup.hp.com ([15.113.161.2])
	by esmail.cup.hp.com (8.11.1/8.8.6) with ESMTP id h3HJX0l10010;
	Thu, 17 Apr 2003 12:33:00 -0700 (PDT)
Message-Id: <5.1.0.14.2.20030417122938.0373aab8@15.13.130.73>
X-Sender: krause@15.13.130.73
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Thu, 17 Apr 2003 12:36:32 -0700
To: Caitlin Bestler <cait@asomi.com>
From: Michael Krause <krause@cup.hp.com>
Subject: RE: [rddp] Options and Markers
Cc: Black_David@emc.com, rddp@ietf.org
In-Reply-To: <r01050300-1025-6A86E88570F411D7B1DC003065D48EE0@[192.168.0
 .2]>
References: <01CA1B1F74A4EC40A9C887F7B58F87240314313C@cceexc16.americas.cpqcorp.net>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

At 11:48 AM 4/17/2003 -0500, Caitlin Bestler wrote:
>On 4/16/03, Barron, Dwight wrote:
>
> >You are correct, the "preclude" assertion does not apply
> >if we get to the point that markers are mandatory to
> >implement/optional to use at all receivers (are you
> >putting your WG Chair hat on?). However, this means that
> >markers are mandatory to implement at the transmitter
> >also, which seems to be where the argument for the most
> >marker pain has manifested itself for those that are
> >opposed to markers. Once implemented, is there sufficient
> >delta to a) warrant an option and b) operate the
> >connection in an asymmetric mode? I apologize in advance
> >for slipping into "commercial merits diatribe" but options
> >cost real money once the ink is dry on the spec. I like to
> >reserve their use/cost for things that are to the left of
> >the decimal point, rather than the right.
>
>If we agree that a sender MUST have the capability to send
>with markers, then the only question is whether there is a
>cost in disabling that capability.
>
>Given that the same stack will need to send unmodified TCP
>traffic it obviously needs that capability anyway.

There may be two stacks: a host-based software stack for non-RDMA traffic 
and a hardware-based RDMA stack.  There is no requirement that the 
hardware-based solution support raw access to the transport layers for 
non-RDMA access just as there is no requirement that the software-stack 
support RDMA - this is all in the implementation space and outside the 
scope of the RDDP charter.

>Other than QA testing, I don't see the cost to a sender of making this 
>receiver specifiable. QA testing *is* a serious cost, but the benefit to 
>some receivers, IMHO, clearly justifies it.

QA often becomes the dominant cost in delivery technology - it isn't just a 
one-time thing but is on-going and applies to multiple permutations thus 
options have a multiplying effect.  There is also the cost issue on the 
transmitter to support both options - its design may be such that removing 
markers increases complexity (I believe this was noted by several people 
prior to the recent IETF event) and thus cost.  Development, QA, etc. costs 
all impact the market adoption rate, TTM for solutions, etc.  Each option 
also comes at an opportunity cost as gates, engineers, etc. spent on the 
option are not free and delay or prevent other technologies from being 
incorporated into design points (numerous examples of this in the marketplace).

Mike

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr 17 15:36:41 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04724
	for <rddp-archive@odin.ietf.org>; Thu, 17 Apr 2003 15:36:41 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3HJjtq02165
	for rddp-archive@odin.ietf.org; Thu, 17 Apr 2003 15:45:55 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HJjt802162
	for <rddp-web-archive@optimus.ietf.org>; Thu, 17 Apr 2003 15:45:55 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04708
	for <rddp-web-archive@ietf.org>; Thu, 17 Apr 2003 15:36:10 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196FDR-00028f-00
	for rddp-web-archive@ietf.org; Thu, 17 Apr 2003 15:38:37 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 196FDR-00028c-00
	for rddp-web-archive@ietf.org; Thu, 17 Apr 2003 15:38:37 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HJj1802092;
	Thu, 17 Apr 2003 15:45:01 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HJib802044
	for <rddp@optimus.ietf.org>; Thu, 17 Apr 2003 15:44:37 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA04667
	for <rddp@ietf.org>; Thu, 17 Apr 2003 15:34:52 -0400 (EDT)
From: Black_David@emc.com
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196FCB-00028E-00
	for rddp@ietf.org; Thu, 17 Apr 2003 15:37:19 -0400
Received: from [128.222.32.10] (helo=mxic1.corp.emc.com)
	by ietf-mx with esmtp (Exim 4.12)
	id 196FCA-000289-00
	for rddp@ietf.org; Thu, 17 Apr 2003 15:37:18 -0400
Received: by mxic1.corp.emc.com with Internet Mail Service (5.5.2653.19)
	id <JAAPFFY1>; Thu, 17 Apr 2003 15:36:57 -0400
Message-ID: <277DD60FB639D511AC0400B0D068B71E0564CB45@corpmx14.corp.emc.com>
To: rddp@ietf.org
Date: Thu, 17 Apr 2003 15:31:38 -0400
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: [rddp] FW: Adaptec IPR Notice
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

FYI - this may also be relevant to some RDDP work.  --David

p.s. Timing of this message with respect to John Carrier's
	is pure coincidence; please don't read anything into it.

-----Original Message-----
From: Sperry, Todd [mailto:Todd_Sperry@adaptec.com]
Sent: Thursday, April 17, 2003 1:30 PM
To: ips@ece.cmu.edu
Cc: 'Elizabeth G. Rodriguez'; 'Black_David@emc.com'; Sperry, Todd
Subject: IPS: Adaptec IPR Notice



Adaptec has made a patent application that may relate to the
work underway in the IPS working group, specifically to the
work derived from draft-ietf-ips-iscsi-20.txt.  This patent application 
describes a method and process for calculating a CRC on data 
received out of order without having to store and assemble the data.

If any patents are issued to Adaptec with claims that are necessary 
for practicing this standard, any party will be able to obtain the right 
to implement, use and distribute the technology or works when implementing,
using or distributing technology based upon the specific specification
under openly specified, reasonable, non-discriminatory terms.

Requests may be sent to:

	Adaptec Legal Department
	691 South Milpitas Boulevard
	Milpitas, Ca. 95035

	Phone:  (408) 957-1718
	Email:   legal@adaptec.com
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr 17 16:47:29 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA07168
	for <rddp-archive@odin.ietf.org>; Thu, 17 Apr 2003 16:47:29 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3HKuji07121
	for rddp-archive@odin.ietf.org; Thu, 17 Apr 2003 16:56:45 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HKuj807118
	for <rddp-web-archive@optimus.ietf.org>; Thu, 17 Apr 2003 16:56:45 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA07119
	for <rddp-web-archive@ietf.org>; Thu, 17 Apr 2003 16:46:58 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196GJx-0002Wr-00
	for rddp-web-archive@ietf.org; Thu, 17 Apr 2003 16:49:25 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 196GJx-0002Wo-00
	for rddp-web-archive@ietf.org; Thu, 17 Apr 2003 16:49:25 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HKu4807079;
	Thu, 17 Apr 2003 16:56:04 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3HKtI807026
	for <rddp@optimus.ietf.org>; Thu, 17 Apr 2003 16:55:18 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA07076
	for <rddp@ietf.org>; Thu, 17 Apr 2003 16:45:31 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196GIY-0002WK-00
	for rddp@ietf.org; Thu, 17 Apr 2003 16:47:58 -0400
Received: from thebe.your-site.com ([140.186.45.26])
	by ietf-mx with esmtp (Exim 4.12)
	id 196GIY-0002WB-00
	for rddp@ietf.org; Thu, 17 Apr 2003 16:47:58 -0400
Received: from 192.168.0.2 (64-144-5-25.client.dsl.net [64.144.5.25])
	by thebe.your-site.com (Postfix) with ESMTP
	id 35344245584; Thu, 17 Apr 2003 16:47:55 -0400 (EDT)
Date: Thu, 17 Apr 2003 15:50:09 -0500
From: Caitlin Bestler <cait@asomi.com>
Subject: Re: [rddp] RE: Marker Progress?
To: Michael Krause <krause@cup.hp.com>, rddp@ietf.org
X-Priority: 3
In-Reply-To: <5.1.0.14.2.20030417122202.036b5e60@15.13.130.73>
Message-ID: <r01050300-1025-2D9FEEB2711611D7B1DC003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

On 4/17/03, Michael Krause wrote:

>>It is a valid question as to whether the existence of the
>>option is justified. That is, how much of a benefit is
>>no-marker reception to buffer-rich receivers, and are
>>buffer-rich receivers a valid design likely to be
>>implemented by at least a signifigant portion of the
market?
>
>They may be valid for a 100 Mbps or perhaps even a Gbps
>link (but with GbE prices falling so quickly why would
>anyone tolerate the price differential) but anything
>faster and the need for dual-port, high-speed SRAM, etc.
>far outweighs any benefits.

Agreed.

Which is why I am firmly in the camp that says that
high-speed low-buffer RNICs MUST be able to mandate
that they will receive markers.

But I believe RDMA has a far wider applicability than
machines that justify 10 GbE ports. Even if the 10 GbE
port were free, a 10 GbE link may not be available.

A Gbit NIC may more than cover all of the traffic a given
host may ever see. That does not mean that it, or the
servers it connects to, would not benefit from RDMA.

It is a fair point that the overhead of doing markers in
software needs to be considered at the single Gbit range.
Identifying TCP connections alone would seem to make a pure
software solution at 10 Gbits unfeasible, with or without
markers. So if no consensus can be reached on
receiver-driven optional markers, I would fully expect that
software-driven receivers will still be feasible and will
still be deployed.

>>Other than QA testing, I don't see the cost to a sender
>>of making this receiver specifiable. QA testing *is* a
>>serious cost, but the benefit to some receivers, IMHO,
>>clearly justifies it.
>
>QA often becomes the dominant cost in delivery technology
>- it isn't just a one-time thing but is on-going and
>applies to multiple permutations thus options have a
>multiplying effect.  There is also the cost issue on the
>transmitter to support both options - its design may be
>such that removing markers increases complexity (I believe
>this was noted by several people prior to the recent IETF
>event) and thus cost.  Development, QA, etc. costs all
>impact the market adoption rate, TTM for solutions, etc. 
>Each option also comes at an opportunity cost as gates,
>engineers, etc. spent on the option are not free and delay
>or prevent other technologies from being incorporated into
>design points (numerous examples of this in the
>marketplace).

This is the crucial point of comparison. In the long run,
QA and compliance testing will dwarf development costs.
I reamin opposed to variable CRC coverage precisely because
the benefits of header-only coverage seem too slight to
cover the long term testing requirement.

If the consensus is that TOEs will not be part of the long
term solution mix and/or that the impact on TOE/software
stacks for markers is minimal then the QA cost is not
justified.

However, you have to be careful about what you define the
"long term" to be.

In my opinion, during the lifespan of RDDP over MPA/TCP,
there is likely to be a signifigant portion of traffic
that is non-RDMA over TCP through the GigE (or 10 GigE)
ports. If anything, the inertia against changing
applications will take longer to overcome than the inertia
to shift to RDDP over SCTP.

Over that timeframe I strongly suspect that low buffer
RNICs will probably be the dominant solution, but I doubt
that they will be the only one.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Fri Apr 18 03:37:27 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA05227
	for <rddp-archive@odin.ietf.org>; Fri, 18 Apr 2003 03:37:27 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3I7kuE27159
	for rddp-archive@odin.ietf.org; Fri, 18 Apr 2003 03:46:56 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3I7kt827156
	for <rddp-web-archive@optimus.ietf.org>; Fri, 18 Apr 2003 03:46:55 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA05197
	for <rddp-web-archive@ietf.org>; Fri, 18 Apr 2003 03:36:56 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196QSw-0005lu-00
	for rddp-web-archive@ietf.org; Fri, 18 Apr 2003 03:39:22 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 196QSv-0005lr-00
	for rddp-web-archive@ietf.org; Fri, 18 Apr 2003 03:39:21 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3I7kD827131;
	Fri, 18 Apr 2003 03:46:13 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3I7iZ827070
	for <rddp@optimus.ietf.org>; Fri, 18 Apr 2003 03:44:35 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA05130
	for <rddp@ietf.org>; Fri, 18 Apr 2003 03:34:36 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196QQg-0005kt-00
	for rddp@ietf.org; Fri, 18 Apr 2003 03:37:02 -0400
Received: from d12lmsgate-5.de.ibm.com ([194.196.100.238])
	by ietf-mx with esmtp (Exim 4.12)
	id 196QQf-0005km-00
	for rddp@ietf.org; Fri, 18 Apr 2003 03:37:01 -0400
Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23])
	by d12lmsgate-5.de.ibm.com (8.12.9/8.12.8) with ESMTP id h3I7ajju057714
	for <rddp@ietf.org>; Fri, 18 Apr 2003 09:36:45 +0200
Received: from d10ml001.telaviv.ibm.com (d10ml001.telaviv.ibm.com [9.148.216.55])
	by d12relay02.de.ibm.com (8.12.8/NCO/VER6.5) with ESMTP id h3I7aeHa178152
	for <rddp@ietf.org>; Fri, 18 Apr 2003 09:36:44 +0200
To: rddp@ietf.org
MIME-Version: 1.0
X-Mailer: Lotus Notes Release 6.0.1 February 07, 2003
From: "Julian Satran" <Julian_Satran@il.ibm.com>
Message-ID: <OF97F70062.78AF9ABD-ONC2256D0C.0028A345-C2256D0C.0029CFC0@telaviv.ibm.com>
Date: Fri, 18 Apr 2003 10:36:40 +0300
X-MIMETrack: Serialize by Router on D10ML001/10/M/IBM(Release 5.0.9a |January 7, 2002) at
 18/04/2003 10:36:44,
	Serialize complete at 18/04/2003 10:36:44
Content-Type: multipart/alternative; boundary="=_alternative 0029126AC2256D0C_="
Subject: [rddp] Fw: IPS:  Adaptec IPR Notice
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

This is a multipart message in MIME format.
--=_alternative 0029126AC2256D0C_=
Content-Type: text/plain; charset="US-ASCII"

David Black, senior technologist at EMC Corp. (NYSE: EMC - message board) 
who co-chairs the IETF IP Storage working group, says the Adaptec claims 
appear to not be applicable to all implementations.
He says they "may be applicable to people who are building seriously 
hot-rodded hardware." He adds that Adaptec's patent application could also 
potentially affect implementations of standards being developed by the 
IETF's Remote Direct Data Placement (RDDP) working group. RDDP refers to 
the ability to place data directly from a network adapter into application 
buffers, without intensive CPU usage.



Considering the attached you may want to look at the details of our 
earlier presenation and judge what (if any) patent apply.
The technique we presented is to be considered public domain knowledge,

Julo
----- Forwarded by Julian Satran/Haifa/IBM on 18/04/03 10:23 -----

Julian Satran/Haifa/IBM
18/04/03 09:39

To
"Sperry, Todd" <Todd_Sperry@adaptec.com>
cc
Dafna Sheinwald, Ilan Shimony/Haifa/IBM@IBMIL, HRL IP 
Department/Haifa/IBM@IBMIL
Subject
Re: IPS:  Adaptec IPR Notice





For what it is worth - a team from IBM has documented such a technique in 
an open presentation to the RDMA consortium (publisdhed also on the forum) 
several months ago and a paper was forwarded to a conference and a 
journal. This is publicly available knowledge.  IBM has made not patent 
claims on it and I will make it soon available on my site.

The presentation to the RDMA forum is still a working paper of the 
consortium. The extend to which the two techniques are similar or not is 
for you to judge.


Julo



"Sperry, Todd" <Todd_Sperry@adaptec.com> 
Sent by: owner-ips@ece.cmu.edu
17/04/03 20:30

To
ips@ece.cmu.edu
cc
"'Elizabeth G. Rodriguez'" <Elizabeth.G.Rodriguez@123mail.net>, 
"'Black_David@emc.com'" <Black_David@emc.com>, "Sperry, Todd" 
<Todd_Sperry@adaptec.com>
Subject
IPS:  Adaptec IPR Notice







Adaptec has made a patent application that may relate to the
work underway in the IPS working group, specifically to the
work derived from draft-ietf-ips-iscsi-20.txt.  This patent application 
describes a method and process for calculating a CRC on data 
received out of order without having to store and assemble the data.

If any patents are issued to Adaptec with claims that are necessary 
for practicing this standard, any party will be able to obtain the right 
to implement, use and distribute the technology or works when 
implementing,
using or distributing technology based upon the specific specification
under openly specified, reasonable, non-discriminatory terms.

Requests may be sent to:

                 Adaptec Legal Department
                 691 South Milpitas Boulevard
                 Milpitas, Ca. 95035

                 Phone:  (408) 957-1718
                 Email:   legal@adaptec.com



--=_alternative 0029126AC2256D0C_=
Content-Type: text/html; charset="US-ASCII"


<table>
<tr valign=top>
<td><font size=2><br>
</font>
<table width=100%>
<tr>
<td width=100%><font size=2><br>
</font>
<table>
<tr valign=top>
<td><font size=2><br>
</font>
<table>
<tr>
<td><font size=1 face="Arial">David Black, senior technologist at </font><a href=http://www.emc.com/ target=new><font size=1 color=blue face="Arial"><u>EMC
Corp.</u></font></a><font size=1 face="Arial"> (NYSE: EMC - </font><a href="http://www.byteandswitch.com/boards/thread_view.asp?thread_topic=19&amp;thread_key=EMC&amp;thread_title=EMC"><font size=1 color=blue face="Arial"><u>message
board</u></font></a><font size=1 face="Arial">) who co-chairs the IETF IP
Storage working group, says the Adaptec claims appear to <i>not</i> be
applicable to all implementations.</font>
<p><font size=1 face="Arial">He says they &quot;may be applicable to people
who are building seriously hot-rodded hardware.&quot; He adds that Adaptec's
patent application could also potentially affect implementations of standards
being developed by the IETF's Remote Direct Data Placement (RDDP) working
group. RDDP refers to the ability to place data directly from a network
adapter into application buffers, without intensive CPU usage.</font></table>
<br></table>
<br></table>
<br></table>
<br><font size=2 face="sans-serif">Considering the attached you may want
to look at the details of our earlier presenation and judge what (if any)
patent apply.</font>
<br><font size=2 face="sans-serif">The technique we presented is to be
considered public domain knowledge,</font>
<br>
<br><font size=2 face="sans-serif">Julo</font>
<br><font size=1 color=#800080 face="sans-serif">----- Forwarded by Julian
Satran/Haifa/IBM on 18/04/03 10:23 -----</font>
<br>
<table width=100%>
<tr valign=top>
<td width=40%><font size=1 face="sans-serif"><b>Julian Satran/Haifa/IBM</b></font>
<p><font size=1 face="sans-serif">18/04/03 09:39</font>
<td width=59%>
<table width=100%>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">To</font></div>
<td valign=top><font size=1 face="sans-serif">&quot;Sperry, Todd&quot;
&lt;Todd_Sperry@adaptec.com&gt;</font>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">cc</font></div>
<td valign=top><font size=1 face="sans-serif">Dafna Sheinwald, Ilan Shimony/Haifa/IBM@IBMIL,
HRL IP Department/Haifa/IBM@IBMIL</font>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">Subject</font></div>
<td valign=top><font size=1 face="sans-serif">Re: IPS: &nbsp;Adaptec IPR
Notice</font><a href=Notes:///C225670D0041573F/170CE2954CD1A249C2256B1E00635F3E/30AD679FF933D660C2256D0B0060571E>Link</a></table>
<br>
<table>
<tr valign=top>
<td>
<td></table>
<br></table>
<br>
<br><font size=2 face="sans-serif">For what it is worth - a team from IBM
has documented such a technique in an open presentation to the RDMA consortium
(publisdhed also on the forum) several months ago and a paper was forwarded
to a conference and a journal. This is publicly available knowledge. &nbsp;IBM
has made not patent claims on it and I will make it soon available on my
site.</font>
<br>
<br><font size=2 face="sans-serif">The presentation to the RDMA forum is
still a working paper of the consortium. The extend to which the two techniques
are similar or not is for you to judge.</font>
<br>
<br>
<br><font size=2 face="sans-serif">Julo</font>
<br>
<br>
<br>
<table width=100%>
<tr valign=top>
<td width=40%><font size=1 face="sans-serif"><b>&quot;Sperry, Todd&quot;
&lt;Todd_Sperry@adaptec.com&gt;</b> </font>
<br><font size=1 face="sans-serif">Sent by: owner-ips@ece.cmu.edu</font>
<p><font size=1 face="sans-serif">17/04/03 20:30</font>
<td width=59%>
<table width=100%>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">To</font></div>
<td valign=top><font size=1 face="sans-serif">ips@ece.cmu.edu</font>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">cc</font></div>
<td valign=top><font size=1 face="sans-serif">&quot;'Elizabeth G. Rodriguez'&quot;
&lt;Elizabeth.G.Rodriguez@123mail.net&gt;, &quot;'Black_David@emc.com'&quot;
&lt;Black_David@emc.com&gt;, &quot;Sperry, Todd&quot; &lt;Todd_Sperry@adaptec.com&gt;</font>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">Subject</font></div>
<td valign=top><font size=1 face="sans-serif">IPS: &nbsp;Adaptec IPR Notice</font></table>
<br>
<table>
<tr valign=top>
<td>
<td></table>
<br></table>
<br>
<br>
<br><font size=2><tt><br>
Adaptec has made a patent application that may relate to the<br>
work underway in the IPS working group, specifically to the<br>
work derived from draft-ietf-ips-iscsi-20.txt. &nbsp;This patent application
<br>
describes a method and process for calculating a CRC on data <br>
received out of order without having to store and assemble the data.<br>
<br>
If any patents are issued to Adaptec with claims that are necessary <br>
for practicing this standard, any party will be able to obtain the right
<br>
to implement, use and distribute the technology or works when implementing,<br>
using or distributing technology based upon the specific specification<br>
under openly specified, reasonable, non-discriminatory terms.<br>
<br>
Requests may be sent to:<br>
<br>
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
Adaptec Legal Department<br>
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
691 South Milpitas Boulevard<br>
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
Milpitas, Ca. 95035<br>
<br>
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
Phone: &nbsp;(408) 957-1718<br>
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
Email: &nbsp; legal@adaptec.com<br>
<br>
</tt></font>
<br>
--=_alternative 0029126AC2256D0C_=--
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Fri Apr 18 12:16:20 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA18498
	for <rddp-archive@odin.ietf.org>; Fri, 18 Apr 2003 12:16:20 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3IGQ1327575
	for rddp-archive@odin.ietf.org; Fri, 18 Apr 2003 12:26:01 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3IGQ0827572
	for <rddp-web-archive@optimus.ietf.org>; Fri, 18 Apr 2003 12:26:00 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA18483
	for <rddp-web-archive@ietf.org>; Fri, 18 Apr 2003 12:15:49 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196YZ6-0000OD-00
	for rddp-web-archive@ietf.org; Fri, 18 Apr 2003 12:18:16 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 196YZ5-0000OA-00
	for rddp-web-archive@ietf.org; Fri, 18 Apr 2003 12:18:15 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3IGPD827550;
	Fri, 18 Apr 2003 12:25:13 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3IGO6827503
	for <rddp@optimus.ietf.org>; Fri, 18 Apr 2003 12:24:06 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA18457
	for <rddp@ietf.org>; Fri, 18 Apr 2003 12:13:55 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196YXF-0000Nk-00
	for rddp@ietf.org; Fri, 18 Apr 2003 12:16:21 -0400
Received: from thebe.your-site.com ([140.186.45.26])
	by ietf-mx with esmtp (Exim 4.12)
	id 196YXE-0000Ng-00
	for rddp@ietf.org; Fri, 18 Apr 2003 12:16:21 -0400
Received: from 192.168.0.2 (64-144-5-25.client.dsl.net [64.144.5.25])
	by thebe.your-site.com (Postfix) with ESMTP id 8ED9C244A5F
	for <rddp@ietf.org>; Fri, 18 Apr 2003 12:16:15 -0400 (EDT)
Date: Fri, 18 Apr 2003 11:18:32 -0500
From: Caitlin Bestler <cait@asomi.com>
Subject: Re: [rddp] Fw: IPS:  Adaptec IPR Notice
To: rddp@ietf.org
X-Priority: 3
In-Reply-To: <OF97F70062.78AF9ABD-ONC2256D0C.0028A345-C2256D0C.0029CFC0@telaviv.ibm.com>
Message-ID: <r01050300-1025-6623EDBA71B911D794AC003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit


>
>Adaptec has made a patent application that may relate to
>the work underway in the IPS working group, specifically
>to the work derived from draft-ietf-ips-iscsi-20.txt. 
>This patent application describes a method and process for
>calculating a CRC on data received out of order without
>having to store and assemble the data.
>
I'm curious as to what is being claimed as novel here. The
fact that XOR operations are commutative has been known for
some time, and that for a CRC upon any fixed sized block
that each bit in the final CRC is an XOR of a subset of
the source bits.

But more to the point. Where is this applicable to rddp?
I can only think of one case where data arrives out of order
that is subject to a CRC, and that would be when an MPA
FULPDU has been delivered both out-of-order and unaligned.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Fri Apr 18 20:02:02 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA01829
	for <rddp-archive@odin.ietf.org>; Fri, 18 Apr 2003 20:02:02 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3J0Bos24588
	for rddp-archive@odin.ietf.org; Fri, 18 Apr 2003 20:11:50 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3J0Bo824585
	for <rddp-web-archive@optimus.ietf.org>; Fri, 18 Apr 2003 20:11:50 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA01810
	for <rddp-web-archive@ietf.org>; Fri, 18 Apr 2003 20:01:31 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196fpk-0002Bu-00
	for rddp-web-archive@ietf.org; Fri, 18 Apr 2003 20:03:56 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 196fpk-0002Bq-00
	for rddp-web-archive@ietf.org; Fri, 18 Apr 2003 20:03:56 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3J0BB824568;
	Fri, 18 Apr 2003 20:11:11 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3J0Ap824549
	for <rddp@optimus.ietf.org>; Fri, 18 Apr 2003 20:10:51 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA01659
	for <rddp@ietf.org>; Fri, 18 Apr 2003 20:00:33 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 196fon-00029e-00
	for rddp@ietf.org; Fri, 18 Apr 2003 20:02:57 -0400
Received: from palrel11.hp.com ([156.153.255.246])
	by ietf-mx with esmtp (Exim 4.12)
	id 196fon-00029a-00
	for rddp@ietf.org; Fri, 18 Apr 2003 20:02:57 -0400
Received: from wera.hpl.hp.com (wera.hpl.hp.com [15.9.120.80])
	by palrel11.hp.com (Postfix) with ESMTP
	id B61841C00D8A; Fri, 18 Apr 2003 17:03:12 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1])
	by wera.hpl.hp.com (8.12.9/8.12.2) with SMTP id h3J03CZL019355;
	Fri, 18 Apr 2003 17:03:12 -0700 (PDT)
From: Jeffrey Mogul <Jeff.Mogul@hp.com>
Message-Id: <200304190003.h3J03CZL019355@wera.hpl.hp.com>
X-Authentication-Warning: wera.hpl.hp.com: localhost [127.0.0.1] didn't use HELO protocol
To: Caitlin Bestler <cait@asomi.com>
Cc: rddp@ietf.org
Subject: Re: [rddp] Options and Markers 
In-reply-to: Your message of "Thu, 17 Apr 2003 11:36:42 CDT."
             <r01050300-1025-C5A4F86F70F211D7B1DC003065D48EE0@[192.168.0.2]> 
X-Organization: Hewlett-Packard Labs, Large Scale Systems group
Date: Fri, 18 Apr 2003 17:03:12 -0700
X-Mts: smtp
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

Caitlin Bestler wrote:

    On 4/15/03, Jeffrey Mogul wrote:
    >So, are options really that bad?  I don't think so.  First
    >of all, if X is mandatory to implement but optional to
    >use, and the receiver gets to choose, then you can make
    >this work without extra round-trips by having the sender
    >start the connection using the option, and then let the
    >receiver tell the sender to turn it off.
    
    Starting markers in the middle of the data would be very
    complex. I agree that this can be made to work without 
    an extra round-trip, but there is still a requirement 
    that the markers be present (or absent) from the very
    beginning. This does make the first segment special,
    but that is a given (if we accept that some form of
    protocol marker has been mandated by the powers-that-be).

Note that, in the paragraph you quoted, I didn't propose *starting*
markers in the middle of the data.  Ending them in the middle
seems a lot simpler.  For example, one could have a marker value
that means "this is the last marker on this connection."  Doesn't
the MPA spec include some RESERVED bits in the market format?

Note: I don't really understand all 35 pages of the MPA spec,
so maybe I'm confused.

At any rate, my goal is not to re-engineer the specs, it's to
see if we can find some sort of acceptable compromise that
will bring working-group consensus closer.

-Jeff
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Sun Apr 20 01:48:32 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id BAA18022
	for <rddp-archive@odin.ietf.org>; Sun, 20 Apr 2003 01:48:31 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3K5wwW30418
	for rddp-archive@odin.ietf.org; Sun, 20 Apr 2003 01:58:58 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3K5ww830415
	for <rddp-web-archive@optimus.ietf.org>; Sun, 20 Apr 2003 01:58:58 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id BAA18009
	for <rddp-web-archive@ietf.org>; Sun, 20 Apr 2003 01:48:01 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1977ib-0001ID-00
	for rddp-web-archive@ietf.org; Sun, 20 Apr 2003 01:50:26 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 1977ib-0001I9-00
	for rddp-web-archive@ietf.org; Sun, 20 Apr 2003 01:50:25 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3K5wH830382;
	Sun, 20 Apr 2003 01:58:17 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3K5qw830208
	for <rddp@optimus.ietf.org>; Sun, 20 Apr 2003 01:52:58 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id BAA17807
	for <rddp@ietf.org>; Sun, 20 Apr 2003 01:42:02 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 1977co-0001G4-00
	for rddp@ietf.org; Sun, 20 Apr 2003 01:44:26 -0400
Received: from thebe.your-site.com ([140.186.45.26])
	by ietf-mx with esmtp (Exim 4.12)
	id 1977co-0001G0-00
	for rddp@ietf.org; Sun, 20 Apr 2003 01:44:26 -0400
Received: from 192.168.0.2 (64-144-5-25.client.dsl.net [64.144.5.25])
	by thebe.your-site.com (Postfix) with ESMTP
	id 2CB07244981; Sun, 20 Apr 2003 01:44:27 -0400 (EDT)
Date: Sun, 20 Apr 2003 00:46:48 -0500
From: Caitlin Bestler <cait@asomi.com>
Subject: Re: [rddp] Options and Markers 
To: Jeffrey Mogul <Jeff.Mogul@hp.com>
Cc: rddp@ietf.org
X-Priority: 3
In-Reply-To: <200304190003.h3J03CZL019355@wera.hpl.hp.com>
Message-ID: <r01050300-1025-7A63242872F311D78A85003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

On 4/18/03, Jeffrey Mogul wrote:

>Caitlin Bestler wrote:
>
>    On 4/15/03, Jeffrey Mogul wrote:
>    >So, are options really that bad?  I don't think so.
First
>    >of all, if X is mandatory to implement but optional to
>    >use, and the receiver gets to choose, then you can
make
>    >this work without extra round-trips by having the
sender
>    >start the connection using the option, and then let
the
>    >receiver tell the sender to turn it off.
>    
>   Starting markers in the middle of the data would be very
>   complex. I agree that this can be made to work without > 
>   an extra round-trip, but there is still a requirement 
>   that the markers be present (or absent) from the very
>   beginning. This does make the first segment special, but
>   that is a given (if we accept that some form of protocol
>   marker has been mandated by the powers-that-be).
>
>Note that, in the paragraph you quoted, I didn't propose
>*starting* markers in the middle of the data.  Ending them
>in the middle seems a lot simpler.  For example, one could
>have a marker value that means "this is the last marker on
>this connection."  Doesn't the MPA spec include some
>RESERVED bits in the market format?
>

Changing the marker mode, rather on or off, requires a
round-trip.

The problem is that any change implies that there is a
point in the stream where the change occurs. There also
has to be some point in the stream at or before that where
the intent to make the change is given.

You have ot guarantee that the notice has been received,
and processed, before any segments in the new mode arrive.
The only two methods that I can think of require an explicit
ack or giving notice further in advance than the
connection's window allows (hence the notice and first data
in the new mode cannot both be in flight at the same time).

The latter only works if the notice is handled by MPA
itself. If it is a ULP message, then a ULP ack is needed.
Google some of the earlier discussions on *entering* RDDP
mode, its pretty much the same problem.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Sun Apr 20 02:16:13 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA29567
	for <rddp-archive@odin.ietf.org>; Sun, 20 Apr 2003 02:16:13 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3K6QeA10469
	for rddp-archive@odin.ietf.org; Sun, 20 Apr 2003 02:26:40 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3K6Qe810466
	for <rddp-web-archive@optimus.ietf.org>; Sun, 20 Apr 2003 02:26:40 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA29516
	for <rddp-web-archive@ietf.org>; Sun, 20 Apr 2003 02:15:42 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 19789P-0001MW-00
	for rddp-web-archive@ietf.org; Sun, 20 Apr 2003 02:18:07 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 19789O-0001MT-00
	for rddp-web-archive@ietf.org; Sun, 20 Apr 2003 02:18:06 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3K6Q1810445;
	Sun, 20 Apr 2003 02:26:01 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3K6PA810407
	for <rddp@optimus.ietf.org>; Sun, 20 Apr 2003 02:25:10 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id CAA29389;
	Sun, 20 Apr 2003 02:14:12 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 19787x-0001MP-00; Sun, 20 Apr 2003 02:16:37 -0400
Received: from d12lmsgate-2.de.ibm.com ([194.196.100.235])
	by ietf-mx with esmtp (Exim 4.12)
	id 19787w-0001MM-00; Sun, 20 Apr 2003 02:16:36 -0400
Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23])
	by d12lmsgate-2.de.ibm.com (8.12.9/8.12.8) with ESMTP id h3K6GKUm066362;
	Sun, 20 Apr 2003 08:16:20 +0200
Received: from d10ml001.telaviv.ibm.com (d10ml001.telaviv.ibm.com [9.148.216.55])
	by d12relay02.de.ibm.com (8.12.8/NCO/VER6.5) with ESMTP id h3K6GJli281502;
	Sun, 20 Apr 2003 08:16:20 +0200
In-Reply-To: <r01050300-1025-7A63242872F311D78A85003065D48EE0@[192.168.0.2]>
Subject: Re: [rddp] Options and Markers
To: Jeffrey Mogul <Jeff.Mogul@hp.com>
Cc: Caitlin Bestler <cait@asomi.com>, rddp@ietf.org, rddp-admin@ietf.org
X-Mailer: Lotus Notes Release 6.0 September 26, 2002
Message-ID: <OFEC028BAE.779AEE01-ONC2256D0E.00223E87-C2256D0E.002273AE@telaviv.ibm.com>
From: "Vadim Makhervaks" <VADIK@il.ibm.com>
Date: Sun, 20 Apr 2003 09:16:17 +0300
X-MIMETrack: Serialize by Router on D10ML001/10/M/IBM(Release 5.0.9a |January 7, 2002) at
 20/04/2003 09:16:19
MIME-Version: 1.0
Content-type: text/plain; charset=us-ascii
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

Absolutely agree. To change the mode one need to stop the traffic, make
sure that the change-mode-notification has been received and accepted, and
only then proceed with the data transmission. Otherwise, think about
out-of-order received segments ...

Vadim


|---------+---------------------------->
|         |           Caitlin Bestler  |
|         |           <cait@asomi.com> |
|         |           Sent by:         |
|         |           rddp-admin@ietf.o|
|         |           rg               |
|         |                            |
|         |                            |
|         |           20/04/03 08:46 AM|
|---------+---------------------------->
  >-------------------------------------------------------------------------------------------------------------------------------|
  |                                                                                                                               |
  |       To:       Jeffrey Mogul <Jeff.Mogul@hp.com>                                                                             |
  |       cc:       rddp@ietf.org                                                                                                 |
  |       Subject:  Re: [rddp] Options and Markers                                                                                |
  >-------------------------------------------------------------------------------------------------------------------------------|




On 4/18/03, Jeffrey Mogul wrote:

>Caitlin Bestler wrote:
>
>    On 4/15/03, Jeffrey Mogul wrote:
>    >So, are options really that bad?  I don't think so.
First
>    >of all, if X is mandatory to implement but optional to
>    >use, and the receiver gets to choose, then you can
make
>    >this work without extra round-trips by having the
sender
>    >start the connection using the option, and then let
the
>    >receiver tell the sender to turn it off.
>
>   Starting markers in the middle of the data would be very
>   complex. I agree that this can be made to work without >
>   an extra round-trip, but there is still a requirement
>   that the markers be present (or absent) from the very
>   beginning. This does make the first segment special, but
>   that is a given (if we accept that some form of protocol
>   marker has been mandated by the powers-that-be).
>
>Note that, in the paragraph you quoted, I didn't propose
>*starting* markers in the middle of the data.  Ending them
>in the middle seems a lot simpler.  For example, one could
>have a marker value that means "this is the last marker on
>this connection."  Doesn't the MPA spec include some
>RESERVED bits in the market format?
>

Changing the marker mode, rather on or off, requires a
round-trip.

The problem is that any change implies that there is a
point in the stream where the change occurs. There also
has to be some point in the stream at or before that where
the intent to make the change is given.

You have ot guarantee that the notice has been received,
and processed, before any segments in the new mode arrive.
The only two methods that I can think of require an explicit
ack or giving notice further in advance than the
connection's window allows (hence the notice and first data
in the new mode cannot both be in flight at the same time).

The latter only works if the notice is handled by MPA
itself. If it is a ULP message, then a ULP ack is needed.
Google some of the earlier discussions on *entering* RDDP
mode, its pretty much the same problem.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Sun Apr 20 23:17:02 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA22181
	for <rddp-archive@odin.ietf.org>; Sun, 20 Apr 2003 23:17:02 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3L3Rsq16079
	for rddp-archive@odin.ietf.org; Sun, 20 Apr 2003 23:27:54 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3L3Rs816076
	for <rddp-web-archive@optimus.ietf.org>; Sun, 20 Apr 2003 23:27:54 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA22166
	for <rddp-web-archive@ietf.org>; Sun, 20 Apr 2003 23:16:32 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 197RpW-0005Y4-00
	for rddp-web-archive@ietf.org; Sun, 20 Apr 2003 23:18:54 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 197RpW-0005Y1-00
	for rddp-web-archive@ietf.org; Sun, 20 Apr 2003 23:18:54 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3L3RE816035;
	Sun, 20 Apr 2003 23:27:14 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3L3QI816006
	for <rddp@optimus.ietf.org>; Sun, 20 Apr 2003 23:26:18 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA22120
	for <rddp@ietf.org>; Sun, 20 Apr 2003 23:14:56 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 197Rnz-0005XZ-00
	for rddp@ietf.org; Sun, 20 Apr 2003 23:17:19 -0400
Received: from dmz1.silverbacksystems.com ([65.172.158.82])
	by ietf-mx with esmtp (Exim 4.12)
	id 197Rny-0005XO-00
	for rddp@ietf.org; Sun, 20 Apr 2003 23:17:19 -0400
Received: from ns.silverbacksystems.com (gate-camp-hme0.silverbacksystems.com [65.172.158.93])
	by dmz1.silverbacksystems.com (Postfix on SuSE Linux 7.3 (i386)) with ESMTP id B967CD36E
	for <rddp@ietf.org>; Sun, 20 Apr 2003 20:16:50 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1])
	by ns.silverbacksystems.com (Postfix on SuSE Linux eMail Server 3.0) with ESMTP id 8C51C1D464
	for <rddp@ietf.org>; Sun, 20 Apr 2003 20:16:50 -0700 (PDT)
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
	charset="us-ascii"
Date: Sun, 20 Apr 2003 20:16:13 -0700
From: "Somesh Gupta" <somesh_gupta@silverbacksystems.com>
Importance: Normal
In-Reply-To: <r01050300-1025-7A63242872F311D78A85003065D48EE0@[192.168.0.2]>
Message-ID: <NMEALCLOIBCHBDHLCMIJMEKJDDAA.somesh_gupta@silverbacksystems.com>
MIME-Version: 1.0
Received: from ns.silverbacksystems.com (localhost [127.0.0.1])
	by localhost (AvMailGate-2.0.0.6) id 13020-6DFA5FB9;
	Sun, 20 Apr 2003 20:16:00 -0700
Received: from somesh (somesh.corp.silverbacksystems.com [10.0.16.35])
	by ns.silverbacksystems.com (Postfix on SuSE Linux eMail Server 3.0) with SMTP id 297B518F37
	for <rddp@ietf.org>; Sun, 20 Apr 2003 20:16:00 -0700 (PDT)
Subject: RE: [rddp] Options and Markers 
To: "Rddp" <rddp@ietf.org>
X-AntiVirus: OK! AntiVir MailGate Version 2.0.0.6
	 at ns.silverbacksystems.com has not found any known virus in this email.
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700
X-MSMail-Priority: Normal
X-Priority: 3 (Normal)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

The RDDP protocol suite will require a reasonable number of
exchanges between the two systems before data can be transferred
in a manner that makes use of the RDDP protocol.

1. Authentication of end points is for all practical purposes,
a very strong requirement.
2. Exchange of buffer information is needed for direct data
placement
3. Storage connections are long lived in any case.
4. IPC connections - looking at SDP as an example - have a 
 number of exchanges for setup.

Given this, one approach should be to see how the cost of
MPA parameter exchange can be minimized instead of using
that as a reason for not having options.

I understand testing as a concern but let me give an
example for the iSCSI protocol.

The immediate/unsolicited data option in iSCSI is an optional
feature. Making it mandatory overlooks the fact that most
existing targets do not support that capability. Not
having it in the protocol overlooks the fact that it removes
write latency. Having it as an option allows for faster
deployment of iSCSI, while in the future allowing for this
as a performance improvement. So overall we had the following
choices:

1. Immediate data on or off
2. Unsolicited data on or off
3. The amount of immediate data
4. The amount of unsolicited data

Of course it is one of the examples of option creep. But
we have gotten over it and people are ready to deploy
products.

In RDMA we are talking about a handful of options on features
where I feel we have reasonable difference of opinion
on various aspects of those features.

Somesh
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Mon Apr 21 10:56:01 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA18718
	for <rddp-archive@odin.ietf.org>; Mon, 21 Apr 2003 10:56:00 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3LF76Z03451
	for rddp-archive@odin.ietf.org; Mon, 21 Apr 2003 11:07:06 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3LF76803448
	for <rddp-web-archive@optimus.ietf.org>; Mon, 21 Apr 2003 11:07:06 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA18714
	for <rddp-web-archive@ietf.org>; Mon, 21 Apr 2003 10:55:30 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 197cjv-0000ll-00
	for rddp-web-archive@ietf.org; Mon, 21 Apr 2003 10:57:51 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 197cjv-0000lh-00
	for rddp-web-archive@ietf.org; Mon, 21 Apr 2003 10:57:51 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3LF6J803365;
	Mon, 21 Apr 2003 11:06:20 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3LF5T803316
	for <rddp@optimus.ietf.org>; Mon, 21 Apr 2003 11:05:29 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA18695
	for <rddp@ietf.org>; Mon, 21 Apr 2003 10:53:53 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 197ciN-0000lS-00
	for rddp@ietf.org; Mon, 21 Apr 2003 10:56:15 -0400
Received: from thebe.your-site.com ([140.186.45.26])
	by ietf-mx with esmtp (Exim 4.12)
	id 197ciM-0000lO-00
	for rddp@ietf.org; Mon, 21 Apr 2003 10:56:14 -0400
Received: from 192.168.0.2 (64-144-5-25.client.dsl.net [64.144.5.25])
	by thebe.your-site.com (Postfix) with ESMTP
	id 4C46F244FD9; Mon, 21 Apr 2003 10:56:18 -0400 (EDT)
Date: Mon, 21 Apr 2003 09:58:41 -0500
From: Caitlin Bestler <cait@asomi.com>
Subject: RE: [rddp] Options and Markers 
To: Somesh Gupta <somesh_gupta@silverbacksystems.com>, rddp@ietf.org
X-Priority: 3
In-Reply-To: <NMEALCLOIBCHBDHLCMIJMEKJDDAA.somesh_gupta@silverbacksystems.com>
Message-ID: <r01050300-1025-BDAEA5C6740911D7BE47003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

On 4/20/03, Somesh Gupta wrote:

>The RDDP protocol suite will require a reasonable number
>of exchanges between the two systems before data can be
>transferred in a manner that makes use of the RDDP
>protocol.
>
RDDP is not an application layer protocol, it rides just
above the traditional transport layer and essentially
functions as a transport layer.

As with any transport, it should avoid making presumptions
about what the ULP will do or not do.

Any distinctions should relate to the inherent nature of
RDDP, not to presumptions about its use. For example, RDDP
might not handle a given interchange well. That is not a
problem if the given interchange does not meet the
characteristics that RDDP was intended to optimize.

Ultimately, the defining characteristic of ULPs that benefit
from RDDP is that there is a relatively large amount of data
that is exchanged for each ULP signifigant event.

That characteristic does not support your assertions.

In particular, the assertion offered would not apply to
a ULP that used RDMA to allow anonymous clients to fetch
files or other named assets (such as HTTP over RDDP).

>1. Authentication of end points is for all practical
>purposes, a very strong requirement.

Authentication is seldom required for fetching content.
A single request could be sent which identified the resource
requested and supplied the advertised buffer where it was to
be deposited. The RDMA transfers and a single reply could
complete the session.

>2. Exchange of buffer information is needed for direct data
>placement

DAFS is already an example of a ULP that includes the buffer
advertisement in the request. No additional exchanges are
required. DAFS is an authenticated file system. It chooses
to set up sessions to authenticate clients, but that is part
of that ULP, it is not intrinsic to its use of RDMA.

>3. Storage connections are long lived in any case.

Classic storage is only one use of RDMA.

HTTP connections are highly optimizable using RDMA and are
only as long lived as required to fetch a single page.

>4. IPC connections - looking at SDP as an example - have a 
> number of exchanges for setup.
>

Again, that is a single ULP. In fact it is one where the two
SDP peers are managing a complex shared buffering resource.


There is obviously a minimum duration of a connection to be
worth designating RDMA in the first place. There simply
isn't a signifigant penalty in being forced to serialize
data transfer unless you are transferring at least a certain
amount of data. Nobody worries about reordering buffers for
a single 16 KB file.



 
>Given this, one approach should be to see how the cost of
>MPA parameter exchange can be minimized instead of using
>that as a reason for not having options.
>

Given that MPA supports bundling, including a MPA parameter
block at the start of MPA-mode is definitely acceptable.

Valid objections remain as to the QA cost of each option,
but the network impact of one-way MPA paramter notices that
can be bundled with actual data is clearly acceptable.

MPA parameters that require acknowledgement, however, add a
round trip delay. There are a large class of transfers that
could have benefitted from RDMA transfer, but would not when
an extra round trip was added to the start of the exchange.

With high speed connections, a *lot* of data could have been
transferred in the time it takes for an extra round trip.
And yes, the first round trip won't be very fast anyway. But
we've also delayed attempting to send large amounts of data
and expanding the transmit window. Ultimately, a round-trip
delay is a round-trip delay.








Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Mon Apr 21 11:05:53 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA19109
	for <rddp-archive@odin.ietf.org>; Mon, 21 Apr 2003 11:05:53 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3LFGxn04651
	for rddp-archive@odin.ietf.org; Mon, 21 Apr 2003 11:16:59 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3LFGx804648
	for <rddp-web-archive@optimus.ietf.org>; Mon, 21 Apr 2003 11:16:59 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA19105
	for <rddp-web-archive@ietf.org>; Mon, 21 Apr 2003 11:05:23 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 197ctV-0000sU-00
	for rddp-web-archive@ietf.org; Mon, 21 Apr 2003 11:07:45 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 197ctU-0000sQ-00
	for rddp-web-archive@ietf.org; Mon, 21 Apr 2003 11:07:44 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3LFGC804614;
	Mon, 21 Apr 2003 11:16:12 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3LFFs804568
	for <rddp@optimus.ietf.org>; Mon, 21 Apr 2003 11:15:54 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA19079
	for <rddp@ietf.org>; Mon, 21 Apr 2003 11:04:18 -0400 (EDT)
Received: from localhost ([127.0.0.1] helo=ietf-mx)
	by ietf-mx with esmtp (Exim 4.12)
	id 197csS-0000rs-00
	for rddp@ietf.org; Mon, 21 Apr 2003 11:06:40 -0400
Received: from palrel11.hp.com ([156.153.255.246])
	by ietf-mx with esmtp (Exim 4.12)
	id 197csR-0000rC-00
	for rddp@ietf.org; Mon, 21 Apr 2003 11:06:39 -0400
Received: from esmail.cup.hp.com (esmail.cup.hp.com [15.0.65.164])
	by palrel11.hp.com (Postfix) with ESMTP id 4B07D1C01E1C
	for <rddp@ietf.org>; Mon, 21 Apr 2003 08:06:56 -0700 (PDT)
Received: from mk731915.cup.hp.com ([15.113.161.2])
	by esmail.cup.hp.com (8.11.1/8.8.6) with ESMTP id h3LF3Fl04771
	for <rddp@ietf.org>; Mon, 21 Apr 2003 08:03:15 -0700 (PDT)
Message-Id: <5.1.0.14.2.20030421065857.036ce7a0@15.13.130.73>
X-Sender: krause@15.13.130.73
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Mon, 21 Apr 2003 07:07:41 -0700
To: <rddp@ietf.org>
From: Michael Krause <krause@cup.hp.com>
Subject: RE: [rddp] Options and Markers 
In-Reply-To: <NMEALCLOIBCHBDHLCMIJMEKJDDAA.somesh_gupta@silverbacksystem
 s.com>
References: <r01050300-1025-7A63242872F311D78A85003065D48EE0@[192.168.0.2]>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

At 08:16 PM 4/20/2003 -0700, Somesh Gupta wrote:
>The RDDP protocol suite will require a reasonable number of
>exchanges between the two systems before data can be transferred
>in a manner that makes use of the RDDP protocol.
>
>1. Authentication of end points is for all practical purposes,
>a very strong requirement.

There is no additional requirement beyond what is required for normal 
connection / stream establishment - anything else is a ULP / application 
issue outside the scope of the protocol.

>2. Exchange of buffer information is needed for direct data
>placement

Ideally, a ULP mapping will simply map the existing messages exchange on 
top of the protocol thus there is no increase in the number of messages 
exchanged.

>3. Storage connections are long lived in any case.

The protocol is not focused on any one market segment - there is nothing to 
preclude using RDMA for shorter lived connections.

>4. IPC connections - looking at SDP as an example - have a
>  number of exchanges for setup.

There is a hello / ack message exchange similar to an iSCSI login thus 
there is an additional round trip at most.

>Given this, one approach should be to see how the cost of
>MPA parameter exchange can be minimized instead of using
>that as a reason for not having options.

Keeping as many options out of the "data mover" layer of an implementation 
is a good thing irrespective of whether the ULP above it are smart in their 
execution and thus minimization of options / number of messages exchanged / 
etc.

>I understand testing as a concern but let me give an example for the iSCSI 
>protocol.
>
>The immediate/unsolicited data option in iSCSI is an optional feature. 
>Making it mandatory overlooks the fact that most existing targets do not 
>support that capability. Not having it in the protocol overlooks the fact 
>that it removes write latency. Having it as an option allows for faster 
>deployment of iSCSI, while in the future allowing for this as a 
>performance improvement. So overall we had the following choices:
>
>1. Immediate data on or off
>2. Unsolicited data on or off
>3. The amount of immediate data
>4. The amount of unsolicited data
>
>Of course it is one of the examples of option creep. But we have gotten 
>over it and people are ready to deploy products.

While an interesting, option creep has and will always be a bad thing to 
simply accept and move onward.  I've been involved in other technologies 
where too many options have been nearly the death of the technology itself 
- it wasn't until we all agreed on what the options were thus making them 
no longer options did people actually get the technology implemented in any 
credible form.

>In RDMA we are talking about a handful of options on features where I feel 
>we have reasonable difference of opinion on various aspects of those features.

My preference is to not have options aside from resource depths wherever 
possible.  Keeping options out of the wire protocols is a good thing for 
all.   To date, I have only seen a couple of people claim the marker option 
is important while many others claim otherwise.  Are there additional 
people who believe one way or another that should be heard?

Mike

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Mon Apr 21 18:26:46 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA05973
	for <rddp-archive@odin.ietf.org>; Mon, 21 Apr 2003 18:26:45 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3LMc0L02364
	for rddp-archive@odin.ietf.org; Mon, 21 Apr 2003 18:38:00 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3LMc0802361
	for <rddp-web-archive@optimus.ietf.org>; Mon, 21 Apr 2003 18:38:00 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA05967
	for <rddp-web-archive@ietf.org>; Mon, 21 Apr 2003 18:26:15 -0400 (EDT)
Received: from ietf-mx ([132.151.6.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 197jm8-0003yR-00
	for rddp-web-archive@ietf.org; Mon, 21 Apr 2003 18:28:36 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 197jm8-0003yN-00
	for rddp-web-archive@ietf.org; Mon, 21 Apr 2003 18:28:36 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3LMbG801933;
	Mon, 21 Apr 2003 18:37:16 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3LMUn801271
	for <rddp@optimus.ietf.org>; Mon, 21 Apr 2003 18:30:49 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA05904
	for <rddp@ietf.org>; Mon, 21 Apr 2003 18:19:04 -0400 (EDT)
From: Black_David@emc.com
Received: from ietf-mx ([132.151.6.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 197jfC-0003xU-00
	for rddp@ietf.org; Mon, 21 Apr 2003 18:21:26 -0400
Received: from mxic2.corp.emc.com ([128.221.31.40])
	by ietf-mx with esmtp (Exim 4.12)
	id 197jfB-0003xP-00
	for rddp@ietf.org; Mon, 21 Apr 2003 18:21:25 -0400
Received: by mxic2.corp.emc.com with Internet Mail Service (5.5.2653.19)
	id <JL712AL0>; Mon, 21 Apr 2003 18:21:12 -0400
Message-ID: <277DD60FB639D511AC0400B0D068B71E0564CB61@corpmx14.corp.emc.com>
To: rddp@ietf.org
Subject: RE: [rddp] WG Last Call: Problem Statement and Architecture
Date: Mon, 21 Apr 2003 18:15:46 -0400
Importance: high
X-Priority: 1
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

The WG Last Call on these drafts is well and truly concluded -
I should have sent this email out several weeks ago, but I've
been spending too much time at high altitude in pressurized
aluminum tubes.  The only comment I have seen on these drafts
is Caitlin Bestler's on whether the application vs. the system
is responsible for supplying all untagged buffers - in essence
all received must be authorized by the ULP, but system buffers
may be used provided that the local implementation allows this
an the ULP chooses to take advantage of this feature.

Having seen no other comments, these drafts have passed WG Last
Call, so the editor (Tom Talpey) should make this change plus
any editorial changes that he deems necessary - at that point it
will fall to me (WG Chair) to check the drafts for formatting nits
and the like prior to submission to the ADs/IESG.

Thanks,
--David
----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
black_david@emc.com        Mobile: +1 (978) 394-7754
----------------------------------------------------

> -----Original Message-----
> From: Black_David@emc.com [mailto:Black_David@emc.com]
> Sent: Wednesday, March 05, 2003 10:24 AM
> To: rddp@ietf.org
> Subject: [rddp] WG Last Call: Problem Statement and Architecture
> Importance: High
> 
> 
> This message announces an informal Working Group
> Last Call on the RDDP problem statement and architecture
> drafts that are intended to become informational RFCs:
> 
> - RDMA over IP Problem Statement,
> 	draft-ietf-rddp-problem-statement-01.txt
> - The Architecture of Direct Data Placement (DDP) And 
> 	Remote Direct Memory Access (RDMA) On Internet
> 	Protocols, draft-ietf-rddp-arch-01.txt
> 
> Due to the timing of the IETF meetings in San Francisco,
> this WG Last Call will run slightly longer than typical -
> it will end at 9am Eastern Time on Monday, March 31st,
> giving everyone a full week after the IETF meetings
> to review and comment on these drafts.
> 
> Editorial comments should be sent directly to the
> draft editor: Thomas Talpey [Thomas.Talpey@netapp.com].
> Technical comments and requests for changes must be
> sent to the RDDP mailing list [rddp@ietf.org] for
> discussion.
> 
> Thanks,
> --David [RDDP WG chair]
> 
> ----------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 176 South St., Hopkinton, MA  01748
> +1 (508) 293-7953             FAX: +1 (508) 293-7786
> black_david@emc.com        Mobile: +1 (978) 394-7754
> ----------------------------------------------------
> _______________________________________________
> rddp mailing list
> rddp@ietf.org
> https://www1.ietf.org/mailman/listinfo/rddp
> 
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr 24 12:37:28 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA22923
	for <rddp-archive@odin.ietf.org>; Thu, 24 Apr 2003 12:37:28 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3OGeAY24586
	for rddp-archive@odin.ietf.org; Thu, 24 Apr 2003 12:40:10 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3OGeA824583
	for <rddp-web-archive@optimus.ietf.org>; Thu, 24 Apr 2003 12:40:10 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA22900
	for <rddp-web-archive@ietf.org>; Thu, 24 Apr 2003 12:36:57 -0400 (EDT)
Received: from ietf-mx ([132.151.6.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 198jkj-0007BI-00
	for rddp-web-archive@ietf.org; Thu, 24 Apr 2003 12:39:17 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 198jki-0007BE-00
	for rddp-web-archive@ietf.org; Thu, 24 Apr 2003 12:39:16 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3OGdK824536;
	Thu, 24 Apr 2003 12:39:20 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3OGcW824473
	for <rddp@optimus.ietf.org>; Thu, 24 Apr 2003 12:38:32 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA22804
	for <rddp@ietf.org>; Thu, 24 Apr 2003 12:35:19 -0400 (EDT)
Received: from ietf-mx ([132.151.6.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 198jj8-00079y-00
	for rddp@ietf.org; Thu, 24 Apr 2003 12:37:38 -0400
Received: from emulex.emulex.com ([138.239.112.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 198jj8-00079g-00
	for rddp@ietf.org; Thu, 24 Apr 2003 12:37:38 -0400
Received: from xbl.ad.emulex.com (xbl.ma.emulex.com [172.16.12.11])
	by emulex.emulex.com (8.12.9/8.12.8) with ESMTP id h3OGbYLW005176
	for <rddp@ietf.org>; Thu, 24 Apr 2003 09:37:35 -0700 (PDT)
Received: by xbl.ma.emulex.com with Internet Mail Service (5.5.2653.19)
	id <HN1WXJBN>; Thu, 24 Apr 2003 12:37:34 -0400
Message-ID: <3356669BBE90C448AD4645C843E2BF289B9062@xbl.ma.emulex.com>
From: "Williams, Jim" <Jim.Williams@Emulex.com>
To: rddp@ietf.org
Subject: RE: [rddp] Header CRC mandatory?
Date: Thu, 24 Apr 2003 12:37:31 -0400
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="ISO-8859-1"
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

The argument for making CRC optional
is that there will be a sizeable subset of implementations
that have a software implementation on one end of the 
connection, AND the underlying network is sufficiently
reliable that the combination of TCP checksum and
link CRC provide adequate coverage.  In this case
NEITHER the header or data require CRC.

The argument against optional CRCs is that there does
NOT exist a sizeable subset of implementations with the
above properties.  In that case BOTH the header
and payload should be protected by CRC.

I believe you are suggesting that there is a third
possibility which is a ULP that can tolerate payload
errors but not header errors, possibly because the
ULP applies its own CRC to the payload.  This seems
possible, but less compelling to me because it seems
less likely.  But even if true, it doesn't invalidate
either of the above two arguments.

> -----Original Message-----
> From: Caitlin Bestler [mailto:cait@asomi.com]
> Sent: Thursday, April 17, 2003 1:42 PM
> To: rddp@ietf.org
> Subject: [rddp] Header CRC mandatory?
> 
> 
> Have we reached a consensus that CRC-32 coverage of at least
> the header is a requirement?
> 
> Without CRC coverage over the header an application CANNOT
> apply its own protection to data until AFTER it invalidates
> all STags that can reference the memory.
> 
> This is far in excess of the use of "one shot" STags. The
> existence of another STag that has the option of referencing
> the same memory exposes it to an error risk -- even when
> there is a solid basis for trusting that the peer is not
> a risk for malicious tampering.
> 
> In prior discussions on STags, we rejected mandating the use
> of STag invalidation (requiring all STags to be "one shot").
> There were numerous application profiles recognized where
> flexible buffering was of value. Without header CRC
> protection, all such applications would be subject to
> increased risk from undetected TCP errors.
> 
> Further, this would be an *increased* risk.
> 
> With pure TCP, only the buffered offered for a specific read
> is ever at risk of receiving bad data. Once delivered, it is
> no longer at risk.  RDMA over TCP without header CRC
> protection would place all advertised buffers at risk
> from undetected TCP errors for the duration of the
> advertisement.
> 
> It is my understanding of our charter that we CANNOT
> increase the risk to applications over the use of non-RDMA
> enabled TCP or SCTP.
> 
> I do not believe that a variable-coverage CRC is of
> sufficient value to justify the increased complexity
> for specifying the variation, and of testing it. But such
> a decision would be valid under our charter. I do not
> believe we have an option to propose direct memory placemnt
> based upon unprotected headers.
> 
> 
> 
> Caitlin Bestler
> http://asomi.com/CaitlinBestler/
> _______________________________________________
> rddp mailing list
> rddp@ietf.org
> https://www1.ietf.org/mailman/listinfo/rddp
> 
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr 24 13:11:36 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA23809
	for <rddp-archive@odin.ietf.org>; Thu, 24 Apr 2003 13:11:36 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3OHEHN26827
	for rddp-archive@odin.ietf.org; Thu, 24 Apr 2003 13:14:17 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3OHEH826824
	for <rddp-web-archive@optimus.ietf.org>; Thu, 24 Apr 2003 13:14:17 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA23790
	for <rddp-web-archive@ietf.org>; Thu, 24 Apr 2003 13:11:05 -0400 (EDT)
Received: from ietf-mx ([132.151.6.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 198kHi-0007RD-00
	for rddp-web-archive@ietf.org; Thu, 24 Apr 2003 13:13:22 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 198kHi-0007RA-00
	for rddp-web-archive@ietf.org; Thu, 24 Apr 2003 13:13:22 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3OHDa826718;
	Thu, 24 Apr 2003 13:13:36 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3OH8U826481
	for <rddp@optimus.ietf.org>; Thu, 24 Apr 2003 13:08:30 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA23554
	for <rddp@ietf.org>; Thu, 24 Apr 2003 13:05:19 -0400 (EDT)
Received: from ietf-mx ([132.151.6.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 198kC8-0007Ng-00
	for rddp@ietf.org; Thu, 24 Apr 2003 13:07:36 -0400
Received: from palrel13.hp.com ([156.153.255.238])
	by ietf-mx with esmtp (Exim 4.12)
	id 198kC8-0007Nd-00
	for rddp@ietf.org; Thu, 24 Apr 2003 13:07:36 -0400
Received: from esmail.cup.hp.com (esmail.cup.hp.com [15.0.65.164])
	by palrel13.hp.com (Postfix) with ESMTP id 5AB681C00C2C
	for <rddp@ietf.org>; Thu, 24 Apr 2003 10:08:03 -0700 (PDT)
Received: from mk731915.cup.hp.com ([15.113.161.2])
	by esmail.cup.hp.com (8.11.1/8.8.6) with ESMTP id h3OH4El27432
	for <rddp@ietf.org>; Thu, 24 Apr 2003 10:04:14 -0700 (PDT)
Message-Id: <5.1.0.14.2.20030424095146.047d7b88@15.13.130.73>
X-Sender: krause@15.13.130.73
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Thu, 24 Apr 2003 10:07:39 -0700
To: rddp@ietf.org
From: Michael Krause <krause@cup.hp.com>
Subject: RE: [rddp] Header CRC mandatory?
In-Reply-To: <3356669BBE90C448AD4645C843E2BF289B9062@xbl.ma.emulex.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

At 12:37 PM 4/24/2003 -0400, Williams, Jim wrote:
>The argument for making CRC optional
>is that there will be a sizeable subset of implementations
>that have a software implementation on one end of the
>connection, AND the underlying network is sufficiently
>reliable that the combination of TCP checksum and
>link CRC provide adequate coverage.  In this case
>NEITHER the header or data require CRC.

Prefer to think of this a bit differently:

- For RDMA to have an impact on performance / functionality offered, at 
least one side of the connection / stream needs to be a hardware 
implementation of the RDDP protocol suite.

         - Note: I have not seen anyone advocate the use of RDMA technology 
in the past 9 years I've been developing this type of technology where both 
sides are software only implementations - it does not make a lot of sense 
to me what value could be derived from using RDMA over a regular 
TCP.  Also, I have not seen anyone describe such a usage configuration and 
subsequent requirements in any detail in this forum but perhaps I missed an 
e-mail that did so.

- A software implementation may be focused primarily on pushing data to the 
hardware implementation so that it can be placed (Write or Send) or sourced 
(Read).  Such implementations exist today, e.g. DAFS that use this type of 
operational model.

- A hardware implementation may be efficiently perform any type of 
operation defined by the protocol drafts and there has been reasonable 
consensus that the headers must be protected to avoid silent data 
corruption.  Based on implementation experience and the CRC analysis as 
bandwidth increases, there is also reasonable consensus that the data 
should also be protected with the existing drafts proposing using a single 
CRC for both (see previous e-mails for that justification).

 From my perspective, software costs are really about packaging the data 
into a format that hardware on the other side of the connection / stream 
can execute in an optimal, single-method manner while providing the 
requisite data integrity.  The costs to software have been about the need 
to include markers and CRC.  My contention has been that such costs can be 
mitigated with good design / implementation such that the end-to-end 
solution is optimal from a customer perspective.  I have not seen data that 
demonstrates that such designs / implementations are not possible or 
overtly burdensome to implement.

Mike


>The argument against optional CRCs is that there does
>NOT exist a sizeable subset of implementations with the
>above properties.  In that case BOTH the header
>and payload should be protected by CRC.
>
>I believe you are suggesting that there is a third
>possibility which is a ULP that can tolerate payload
>errors but not header errors, possibly because the
>ULP applies its own CRC to the payload.  This seems
>possible, but less compelling to me because it seems
>less likely.  But even if true, it doesn't invalidate
>either of the above two arguments.
>
> > -----Original Message-----
> > From: Caitlin Bestler [mailto:cait@asomi.com]
> > Sent: Thursday, April 17, 2003 1:42 PM
> > To: rddp@ietf.org
> > Subject: [rddp] Header CRC mandatory?
> >
> >
> > Have we reached a consensus that CRC-32 coverage of at least
> > the header is a requirement?
> >
> > Without CRC coverage over the header an application CANNOT
> > apply its own protection to data until AFTER it invalidates
> > all STags that can reference the memory.
> >
> > This is far in excess of the use of "one shot" STags. The
> > existence of another STag that has the option of referencing
> > the same memory exposes it to an error risk -- even when
> > there is a solid basis for trusting that the peer is not
> > a risk for malicious tampering.
> >
> > In prior discussions on STags, we rejected mandating the use
> > of STag invalidation (requiring all STags to be "one shot").
> > There were numerous application profiles recognized where
> > flexible buffering was of value. Without header CRC
> > protection, all such applications would be subject to
> > increased risk from undetected TCP errors.
> >
> > Further, this would be an *increased* risk.
> >
> > With pure TCP, only the buffered offered for a specific read
> > is ever at risk of receiving bad data. Once delivered, it is
> > no longer at risk.  RDMA over TCP without header CRC
> > protection would place all advertised buffers at risk
> > from undetected TCP errors for the duration of the
> > advertisement.
> >
> > It is my understanding of our charter that we CANNOT
> > increase the risk to applications over the use of non-RDMA
> > enabled TCP or SCTP.
> >
> > I do not believe that a variable-coverage CRC is of
> > sufficient value to justify the increased complexity
> > for specifying the variation, and of testing it. But such
> > a decision would be valid under our charter. I do not
> > believe we have an option to propose direct memory placemnt
> > based upon unprotected headers.
> >
> >
> >
> > Caitlin Bestler
> > http://asomi.com/CaitlinBestler/
> > _______________________________________________
> > rddp mailing list
> > rddp@ietf.org
> > https://www1.ietf.org/mailman/listinfo/rddp
> >
>_______________________________________________
>rddp mailing list
>rddp@ietf.org
>https://www1.ietf.org/mailman/listinfo/rddp

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr 24 14:18:18 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA25922
	for <rddp-archive@odin.ietf.org>; Thu, 24 Apr 2003 14:18:18 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3OIL1U31713
	for rddp-archive@odin.ietf.org; Thu, 24 Apr 2003 14:21:01 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3OIL1831710
	for <rddp-web-archive@optimus.ietf.org>; Thu, 24 Apr 2003 14:21:01 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA25913
	for <rddp-web-archive@ietf.org>; Thu, 24 Apr 2003 14:17:47 -0400 (EDT)
Received: from ietf-mx ([132.151.6.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 198lKH-00005K-00
	for rddp-web-archive@ietf.org; Thu, 24 Apr 2003 14:20:05 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 198lKH-00005H-00
	for rddp-web-archive@ietf.org; Thu, 24 Apr 2003 14:20:05 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3OIKE831679;
	Thu, 24 Apr 2003 14:20:14 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3OIJ2831610
	for <rddp@optimus.ietf.org>; Thu, 24 Apr 2003 14:19:02 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA25796
	for <rddp@ietf.org>; Thu, 24 Apr 2003 14:15:49 -0400 (EDT)
Received: from ietf-mx ([132.151.6.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 198lIM-00002h-00
	for rddp@ietf.org; Thu, 24 Apr 2003 14:18:06 -0400
Received: from magic-mail.adaptec.com ([208.236.45.100] helo=magic.adaptec.com)
	by ietf-mx with esmtp (Exim 4.12)
	id 198lIM-00002N-00
	for rddp@ietf.org; Thu, 24 Apr 2003 14:18:06 -0400
Received: from redfish.adaptec.com (redfish.adaptec.com [162.62.50.11])
	by magic.adaptec.com (8.11.6/8.11.6) with ESMTP id h3OIFPZ14954
	for <rddp@ietf.org>; Thu, 24 Apr 2003 11:15:25 -0700
Received: from aimexc07.adaptec.com (aimexc07.adaptec.com [162.62.62.47])
	by redfish.adaptec.com (8.8.8p2+Sun/8.8.8) with ESMTP id LAA04611
	for <rddp@ietf.org>; Thu, 24 Apr 2003 11:18:00 -0700 (PDT)
Received: by aimexc07.adaptec.com with Internet Mail Service (5.5.2653.19)
	id <J1DDLSD8>; Thu, 24 Apr 2003 11:18:00 -0700
Message-ID: <E156A23F1885D4119ED800B0D0498A9F0430A33B@aimexc07.adaptec.com>
From: "Carrier, John" <John_Carrier@adaptec.com>
To: rddp@ietf.org
Subject: RE: [rddp] Header CRC mandatory?
Date: Thu, 24 Apr 2003 11:17:53 -0700
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

 
>
> The argument against optional CRCs is that there does
> NOT exist a sizeable subset of implementations with the
> above properties.  In that case BOTH the header
> and payload should be protected by CRC.
>


yes!  

As I've mentioned before we need both header and data CRC.  To my mind there
is no compelling reason to make a digest optional.  Regardless of ULP
requirements and regardless of how MPA is implemented, RDMAP/DDP assume at
least CRC-level of data protection.  It's important that a ULP using these
protocols over TCP has the same expectation of data robustness as when using
these protocols over SCTP.

--jc 
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr 24 20:02:21 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA10624
	for <rddp-archive@odin.ietf.org>; Thu, 24 Apr 2003 20:02:21 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3P05AG23480
	for rddp-archive@odin.ietf.org; Thu, 24 Apr 2003 20:05:10 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3P05A823477
	for <rddp-web-archive@optimus.ietf.org>; Thu, 24 Apr 2003 20:05:10 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA10621
	for <rddp-web-archive@ietf.org>; Thu, 24 Apr 2003 20:01:50 -0400 (EDT)
Received: from ietf-mx ([132.151.6.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 198qhD-00037k-00
	for rddp-web-archive@ietf.org; Thu, 24 Apr 2003 20:04:07 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 198qhD-00037h-00
	for rddp-web-archive@ietf.org; Thu, 24 Apr 2003 20:04:07 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3P04J823439;
	Thu, 24 Apr 2003 20:04:19 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3P03X823406
	for <rddp@optimus.ietf.org>; Thu, 24 Apr 2003 20:03:33 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA10602
	for <rddp@ietf.org>; Thu, 24 Apr 2003 20:00:13 -0400 (EDT)
Received: from ietf-mx ([132.151.6.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 198qfe-00037D-00
	for rddp@ietf.org; Thu, 24 Apr 2003 20:02:30 -0400
Received: from thebe.your-site.com ([140.186.45.26])
	by ietf-mx with esmtp (Exim 4.12)
	id 198qfe-000374-00
	for rddp@ietf.org; Thu, 24 Apr 2003 20:02:30 -0400
Received: from 192.168.0.2 (64-144-5-25.client.dsl.net [64.144.5.25])
	by thebe.your-site.com (Postfix) with ESMTP
	id 105DE2452DB; Thu, 24 Apr 2003 20:02:37 -0400 (EDT)
Date: Thu, 24 Apr 2003 19:05:07 -0500
From: Caitlin Bestler <cait@asomi.com>
Subject: RE: [rddp] Header CRC mandatory?
To: rddp@ietf.org
Cc: "Williams, Jim" <Jim.Williams@Emulex.com>
X-Priority: 3
In-Reply-To: <3356669BBE90C448AD4645C843E2BF289B9062@xbl.ma.emulex.com>
Message-ID: <r01050300-1025-9349E5A576B111D79AEC003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

On 4/24/03, Williams, Jim wrote:

>The argument for making CRC optional
>is that there will be a sizeable subset of implementations
>that have a software implementation on one end of the 
>connection, AND the underlying network is sufficiently
>reliable that the combination of TCP checksum and
>link CRC provide adequate coverage.  In this case
>NEITHER the header or data require CRC.
>
Which presupposes that software calculation of a CRC-32
is difficult and unduly time-consuming. I disagree.
There is no inherent need for additional memory touches
to calculate the CRC-32 for the full payload -- that is
a characteristic of the algorithm. Implementation in
software and/or hardware is irrelevant. The fact that
*current* DMA hardware only performs a subset of this
function is not really a justification for an option.
As Mike already pointed out, current hardware also tends to
be deficient in support of IPv6.

>The argument against optional CRCs is that there does
>NOT exist a sizeable subset of implementations with the
>above properties.  In that case BOTH the header
>and payload should be protected by CRC.
>
>I believe you are suggesting that there is a third
>possibility which is a ULP that can tolerate payload
>errors but not header errors, possibly because the
>ULP applies its own CRC to the payload.  This seems
>possible, but less compelling to me because it seems
>less likely.  But even if true, it doesn't invalidate
>either of the above two arguments.

There are indeed applications that can tolerate payload
errors, especially when the transfer is for final use
and there is no risk of accumulating errors. Whether making
payload CRC optional is of sufficient benefit to them is
another question.

With plain TCP, an application can decide on its own that
it need no further protection of the payload. But with plain
TCP it can inspect the delivered payload *once*.

With RDDP, the payload is at risk until there is NO STag
for the Protection Domain that is valid. Further, because
the header is not validated, the ULP cannot rely upon
automatic STag invalidation. It must do so itself after
receiving a completion notification. Only *then* could it
determine if the received payload was usable.

That amount of caution in an interface is, in my opinion,
unnatural. I find it hard to believe that an application
would truly choose to take those risks with informed
consent. More likely they would expose themselves to these
risks unknowingly, not realizing that without a header CRC
that RDDP/MPA/TCP creates new risks that did not exist
for plain TCP.

RDMA has had to fight against assumptions that direct access
to user memory inherently implied weakened security and
reduced reliability. We don't need to carve out an exception
in order to actually make it true.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Thu Apr 24 20:11:52 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA10815
	for <rddp-archive@odin.ietf.org>; Thu, 24 Apr 2003 20:11:52 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3P0Egm24770
	for rddp-archive@odin.ietf.org; Thu, 24 Apr 2003 20:14:42 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3P0Ef824767
	for <rddp-web-archive@optimus.ietf.org>; Thu, 24 Apr 2003 20:14:42 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA10809
	for <rddp-web-archive@ietf.org>; Thu, 24 Apr 2003 20:11:21 -0400 (EDT)
Received: from ietf-mx ([132.151.6.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 198qqR-0003CZ-00
	for rddp-web-archive@ietf.org; Thu, 24 Apr 2003 20:13:39 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 198qqQ-0003CW-00
	for rddp-web-archive@ietf.org; Thu, 24 Apr 2003 20:13:38 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3P0E3824703;
	Thu, 24 Apr 2003 20:14:03 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3P0DK824584
	for <rddp@optimus.ietf.org>; Thu, 24 Apr 2003 20:13:20 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id UAA10735
	for <rddp@ietf.org>; Thu, 24 Apr 2003 20:10:00 -0400 (EDT)
Received: from ietf-mx ([132.151.6.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 198qp7-0003AC-00
	for rddp@ietf.org; Thu, 24 Apr 2003 20:12:17 -0400
Received: from thebe.your-site.com ([140.186.45.26])
	by ietf-mx with esmtp (Exim 4.12)
	id 198qp7-00039w-00
	for rddp@ietf.org; Thu, 24 Apr 2003 20:12:17 -0400
Received: from 192.168.0.2 (64-144-5-25.client.dsl.net [64.144.5.25])
	by thebe.your-site.com (Postfix) with ESMTP
	id 97DEC245258; Thu, 24 Apr 2003 20:12:29 -0400 (EDT)
Date: Thu, 24 Apr 2003 19:15:00 -0500
From: Caitlin Bestler <cait@asomi.com>
To: Michael Krause <krause@cup.hp.com>
Cc: rddp@ietf.org
X-Priority: 3
In-Reply-To: <5.1.0.14.2.20030424095146.047d7b88@15.13.130.73>
Message-ID: <r01050300-1025-F478C72D76B211D79AEC003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Subject: [rddp] Software-only optimization: was RE Header CRC mandatory?
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

On 4/24/03, Michael Krause wrote:

>Prefer to think of this a bit differently:
>
>- For RDMA to have an impact on performance /
>functionality offered, at least one side of the connection
>/ stream needs to be a hardware implementation of the RDDP
>protocol suite.
>

Not to disagree with your comment on CRCs, but I strongly
believe that RDMA defines a fundamentally improved
interaction between the parties involved (source ULP,
source transport, destination transport and destination
ULP).

That fundamental improvement enables hardware
implementation. But it is a fundamental improvement even if
implemented solely in software.


Whether implemented in hardware or software:

    RDDP still enables placement directly from the MPA Frame 
    / SCTP Data Chunk to user memory without any 
    intermediate buffering.
    
    It still minimizes the number of interactions with the
    ULP, and hence the number of required context switches.
    
    It still frees the data source to send data in the
    order most convenient to it, not an arbitrary order
    matching the data sinks pre-conceptions.
    
    It still enables the data sink to fetch data on
    demand using RDMA Reads.
    
That said, ultimately any simple algorithm with wide
applicability will be implemented better in hardware than in
software. RDMA's strengths enable it to be implemented in
hardware, but if its only strength were truly that it could
be implemented in hardware then it would not be justified.
Hardware should adjust to the needs of the network, not the
other way around. But the network should be designed to
minimize the amount of work each node must do -- and that
work is minimized no matter how it is dividied between the
main processor, co-processors, motherboards, ASICs, FPGAs or
Network Interface Cards.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Fri Apr 25 12:46:49 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA14432
	for <rddp-archive@odin.ietf.org>; Fri, 25 Apr 2003 12:46:49 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3PGo1P09633
	for rddp-archive@odin.ietf.org; Fri, 25 Apr 2003 12:50:01 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3PGo0809629
	for <rddp-web-archive@optimus.ietf.org>; Fri, 25 Apr 2003 12:50:00 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA14386
	for <rddp-web-archive@ietf.org>; Fri, 25 Apr 2003 12:46:18 -0400 (EDT)
Received: from ietf-mx ([132.151.6.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 1996NI-000217-00
	for rddp-web-archive@ietf.org; Fri, 25 Apr 2003 12:48:36 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 1996NI-000213-00
	for rddp-web-archive@ietf.org; Fri, 25 Apr 2003 12:48:36 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3PGnG809526;
	Fri, 25 Apr 2003 12:49:16 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3PGmP809426
	for <rddp@optimus.ietf.org>; Fri, 25 Apr 2003 12:48:25 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA14289
	for <rddp@ietf.org>; Fri, 25 Apr 2003 12:44:43 -0400 (EDT)
From: Black_David@emc.com
Received: from ietf-mx ([132.151.6.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 1996Ll-0001zo-00
	for rddp@ietf.org; Fri, 25 Apr 2003 12:47:01 -0400
Received: from [128.222.32.10] (helo=mxic1.corp.emc.com)
	by ietf-mx with esmtp (Exim 4.12)
	id 1996Ll-0001z4-00
	for rddp@ietf.org; Fri, 25 Apr 2003 12:47:01 -0400
Received: by mxic1.corp.emc.com with Internet Mail Service (5.5.2653.19)
	id <JN85LDA1>; Fri, 25 Apr 2003 12:46:55 -0400
Message-ID: <277DD60FB639D511AC0400B0D068B71E0564CB8D@corpmx14.corp.emc.com>
To: rddp@ietf.org
Date: Fri, 25 Apr 2003 12:41:03 -0400
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: [rddp] Marker Consensus Proposal
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

WG Chair hat firmly in place ...

Looking back on the discussion of markers, I'm going to attempt
to propose a way forward and hope the we can get WG rough consensus
for it.  The core of the proposal is what I originally posted a
couple of weeks ago:

	Make marker usage OPTIONAL,
	but put control of this option entirely in the hands of
	the receiver - senders MUST implement both markers and not-markers,
	receivers MUST implement one OR the other, and each receiver
	gets to pick which one is used as the benefits of markers are
	primarily on the receiver side.

With the following additions:
- When markers are used, the marker interval MUST be 512 bytes.
- This proposed rough consensus assumes that we will be forced
	to do some sort of setup announcement as part of the
	"proof" requirement, which will provide a place to
	transmit options (and a version number).  Details of
	exactly how this works so that each receiver gets to
	pick what is sent to it are TBD.

This is not going to be the "perfect" answer for everyone, but
it's clear to me that WG rough consensus will not be obtained
for the positions at the end of the spectrum, namely:

- 512 byte markers MUST be used at all times
- Markers are completely optional

Further discussion/advocacy of those positions is probably
not productive.

I believe the above proposal is acceptable for the important
classes of implementations, and is technically sound.

Further comment to the list, please.

Thanks,
--David

----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
black_david@emc.com        Mobile: +1 (978) 394-7754
----------------------------------------------------
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Fri Apr 25 13:11:46 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA15313
	for <rddp-archive@odin.ietf.org>; Fri, 25 Apr 2003 13:11:46 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3PHEwj12061
	for rddp-archive@odin.ietf.org; Fri, 25 Apr 2003 13:14:58 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3PHEw812057
	for <rddp-web-archive@optimus.ietf.org>; Fri, 25 Apr 2003 13:14:58 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA15291
	for <rddp-web-archive@ietf.org>; Fri, 25 Apr 2003 13:11:15 -0400 (EDT)
Received: from ietf-mx ([132.151.6.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 1996lR-0002F0-00
	for rddp-web-archive@ietf.org; Fri, 25 Apr 2003 13:13:33 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 1996lR-0002Ex-00
	for rddp-web-archive@ietf.org; Fri, 25 Apr 2003 13:13:33 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3PHE3811994;
	Fri, 25 Apr 2003 13:14:03 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3PHDr811975
	for <rddp@optimus.ietf.org>; Fri, 25 Apr 2003 13:13:53 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA15257
	for <rddp@ietf.org>; Fri, 25 Apr 2003 13:10:10 -0400 (EDT)
Received: from ietf-mx ([132.151.6.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 1996kO-0002E9-00
	for rddp@ietf.org; Fri, 25 Apr 2003 13:12:29 -0400
Received: from magic-mail.adaptec.com ([208.236.45.100] helo=magic.adaptec.com)
	by ietf-mx with esmtp (Exim 4.12)
	id 1996kO-0002Dy-00
	for rddp@ietf.org; Fri, 25 Apr 2003 13:12:28 -0400
Received: from redfish.adaptec.com (redfish.adaptec.com [162.62.50.11])
	by magic.adaptec.com (8.11.6/8.11.6) with ESMTP id h3PH9kZ18983;
	Fri, 25 Apr 2003 10:09:46 -0700
Received: from aimexc07.adaptec.com (aimexc07.adaptec.com [162.62.62.47])
	by redfish.adaptec.com (8.8.8p2+Sun/8.8.8) with ESMTP id KAA15097;
	Fri, 25 Apr 2003 10:12:25 -0700 (PDT)
Received: by aimexc07.adaptec.com with Internet Mail Service (5.5.2653.19)
	id <J1DDLZRP>; Fri, 25 Apr 2003 10:12:25 -0700
Message-ID: <E156A23F1885D4119ED800B0D0498A9F0430A39A@aimexc07.adaptec.com>
From: "Carrier, John" <John_Carrier@adaptec.com>
To: Black_David@emc.com, rddp@ietf.org
Subject: RE: [rddp] Marker Consensus Proposal
Date: Fri, 25 Apr 2003 10:12:22 -0700
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>

Thanks, David.  Good summary and good compromise.

--jc

> -----Original Message-----
> From: Black_David@emc.com [mailto:Black_David@emc.com]
> Sent: Friday, April 25, 2003 9:41 AM
> To: rddp@ietf.org
> Subject: [rddp] Marker Consensus Proposal
> 
> 
> WG Chair hat firmly in place ...
> 
> Looking back on the discussion of markers, I'm going to attempt
> to propose a way forward and hope the we can get WG rough consensus
> for it.  The core of the proposal is what I originally posted a
> couple of weeks ago:
> 
> 	Make marker usage OPTIONAL,
> 	but put control of this option entirely in the hands of
> 	the receiver - senders MUST implement both markers and 
> not-markers,
> 	receivers MUST implement one OR the other, and each receiver
> 	gets to pick which one is used as the benefits of markers are
> 	primarily on the receiver side.
> 
> With the following additions:
> - When markers are used, the marker interval MUST be 512 bytes.
> - This proposed rough consensus assumes that we will be forced
> 	to do some sort of setup announcement as part of the
> 	"proof" requirement, which will provide a place to
> 	transmit options (and a version number).  Details of
> 	exactly how this works so that each receiver gets to
> 	pick what is sent to it are TBD.
> 
> This is not going to be the "perfect" answer for everyone, but
> it's clear to me that WG rough consensus will not be obtained
> for the positions at the end of the spectrum, namely:
> 
> - 512 byte markers MUST be used at all times
> - Markers are completely optional
> 
> Further discussion/advocacy of those positions is probably
> not productive.
> 
> I believe the above proposal is acceptable for the important
> classes of implementations, and is technically sound.
> 
> Further comment to the list, please.
> 
> Thanks,
> --David
> 
> ----------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 176 South St., Hopkinton, MA  01748
> +1 (508) 293-7953             FAX: +1 (508) 293-7786
> black_david@emc.com        Mobile: +1 (978) 394-7754
> ----------------------------------------------------
> _______________________________________________
> rddp mailing list
> rddp@ietf.org
> https://www1.ietf.org/mailman/listinfo/rddp
> 
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Fri Apr 25 13:12:35 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA15348
	for <rddp-archive@odin.ietf.org>; Fri, 25 Apr 2003 13:12:35 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3PHFlp12137
	for rddp-archive@odin.ietf.org; Fri, 25 Apr 2003 13:15:47 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3PHFl812132
	for <rddp-web-archive@optimus.ietf.org>; Fri, 25 Apr 2003 13:15:47 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA15331
	for <rddp-web-archive@ietf.org>; Fri, 25 Apr 2003 13:12:04 -0400 (EDT)
Received: from ietf-mx ([132.151.6.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 1996mE-0002FN-00
	for rddp-web-archive@ietf.org; Fri, 25 Apr 2003 13:14:22 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 1996mE-0002FK-00
	for rddp-web-archive@ietf.org; Fri, 25 Apr 2003 13:14:22 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3PHF2812092;
	Fri, 25 Apr 2003 13:15:02 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3PHE0811980
	for <rddp@optimus.ietf.org>; Fri, 25 Apr 2003 13:14:00 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA15261
	for <rddp@ietf.org>; Fri, 25 Apr 2003 13:10:18 -0400 (EDT)
Received: from ietf-mx ([132.151.6.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 1996kW-0002EG-00
	for rddp@ietf.org; Fri, 25 Apr 2003 13:12:36 -0400
Received: from thebe.your-site.com ([140.186.45.26])
	by ietf-mx with esmtp (Exim 4.12)
	id 1996kW-0002E5-00
	for rddp@ietf.org; Fri, 25 Apr 2003 13:12:36 -0400
Received: from 192.168.0.2 (64-144-5-25.client.dsl.net [64.144.5.25])
	by thebe.your-site.com (Postfix) with ESMTP id ADBAE24501A
	for <rddp@ietf.org>; Fri, 25 Apr 2003 13:12:51 -0400 (EDT)
Date: Fri, 25 Apr 2003 12:15:22 -0500
From: Caitlin Bestler <cait@asomi.com>
Subject: Re: [rddp] Marker Consensus Proposal
To: rddp@ietf.org
X-Priority: 3
In-Reply-To: <277DD60FB639D511AC0400B0D068B71E0564CB8D@corpmx14.corp.emc.com>
Message-ID: <r01050300-1025-7FBE3FA5774111D78CE4003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

On 4/25/03, Black_David@emc.com wrote:

>WG Chair hat firmly in place ...
>
>Looking back on the discussion of markers, I'm going to
>attempt to propose a way forward and hope the we can get
>WG rough consensus for it.  The core of the proposal is
>what I originally posted a couple of weeks ago:
>
I concur that this is the best option that a consensus can
form around. Given that the mode-marker will have to exist
anyway (residual grumbles from those like myself who
disagree aside) there is little reason not to accomodate
receivers who will not benefit from the use of markers.

Clearly, generating markers is more work (and network
traffic) than not generating them. And that simply does not
make sense if the receiver indicates it will not use them.



Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From mailnull@www1.ietf.org  Fri Apr 25 13:20:31 2003
Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA15621
	for <rddp-archive@odin.ietf.org>; Fri, 25 Apr 2003 13:20:30 -0400 (EDT)
Received: (from mailnull@localhost)
	by www1.ietf.org (8.11.6/8.11.6) id h3PHNhB12595
	for rddp-archive@odin.ietf.org; Fri, 25 Apr 2003 13:23:43 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3PHNh812592
	for <rddp-web-archive@optimus.ietf.org>; Fri, 25 Apr 2003 13:23:43 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA15596
	for <rddp-web-archive@ietf.org>; Fri, 25 Apr 2003 13:20:00 -0400 (EDT)
Received: from ietf-mx ([132.151.6.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 1996tu-0002IK-00
	for rddp-web-archive@ietf.org; Fri, 25 Apr 2003 13:22:18 -0400
Received: from ietf.org ([132.151.1.19] helo=www1.ietf.org)
	by ietf-mx with esmtp (Exim 4.12)
	id 1996tu-0002IH-00
	for rddp-web-archive@ietf.org; Fri, 25 Apr 2003 13:22:18 -0400
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3PHN3812561;
	Fri, 25 Apr 2003 13:23:03 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176])
	by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h3PHMA812530
	for <rddp@optimus.ietf.org>; Fri, 25 Apr 2003 13:22:10 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA15566
	for <rddp@ietf.org>; Fri, 25 Apr 2003 13:18:28 -0400 (EDT)
Received: from ietf-mx ([132.151.6.1])
	by ietf-mx with esmtp (Exim 4.12)
	id 1996sQ-0002Hi-00
	for rddp@ietf.org; Fri, 25 Apr 2003 13:20:46 -0400
Received: from thebe.your-site.com ([140.186.45.26])
	by ietf-mx with esmtp (Exim 4.12)
	id 1996sP-0002Ha-00
	for rddp@ietf.org; Fri, 25 Apr 2003 13:20:45 -0400
Received: from 192.168.0.2 (64-144-5-25.client.dsl.net [64.144.5.25])
	by thebe.your-site.com (Postfix) with ESMTP id E67FF24481E
	for <rddp@ietf.org>; Fri, 25 Apr 2003 13:20:55 -0400 (EDT)
Date: Fri, 25 Apr 2003 12:23:27 -0500
From: Caitlin Bestler <cait@asomi.com>
To: rddp@ietf.org
X-Priority: 3
In-Reply-To: <277DD60FB639D511AC0400B0D068B71E0564CB8D@corpmx14.corp.emc.com>
Message-ID: <r01050300-1025-A062217D774211D78CE4003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Subject: [rddp] Marker option - dual mode receivers
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-BeenThere: rddp@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=unsubscribe>
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
List-Post: <mailto:rddp@ietf.org>
List-Help: <mailto:rddp-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/rddp>,
	<mailto:rddp-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit

On 4/25/03, Black_David@emc.com wrote:

>- This proposed rough consensus assumes that we will be
>forced to do some sort of setup announcement as part of
>the "proof" requirement, which will provide a place to
>transmit options (and a version number).  Details of
>exactly how this works so that each receiver gets to pick
>what is sent to it are TBD.

One refinement I would like to see is to allow public
servers that are likely to have many intermittent clients
(such as an RDMA enabled web server) to be ambidexterous.

Specifically, I would propose that the mode information
contain the following flags:

    My preferred reception mode: marker/no-marker.
    My presumption on your mode: marker/no-marker.
    I will match your presumption: true/false.
    
If a peer's presumption is incompatible with the receiver
some action is required to reliably shift mode. This is TBD,
one simple solution is to simply abort the connection and
try again. But that is a seperate discussion.

The above proposal allows two things:

-- When a sender has a priori knowledge of a receivers 
   preferred mode, it can simply start sending. It does
   not have to wait for the peer's mode packet to start.
   
-- It allows a receiver the option of accomodating a
   a peer that made the wrong presumption.
   
Between the two it should be possible to keep the number of
"wrong mode" startup attempts quite minimal. The more
minimal the more acceptable simply closing the connection
and trying again becomes.


Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp