From daemon@ns.ietf.org  Thu Jul 18 01:40:08 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id BAA12629
	for <rddp-archive@odin.ietf.org>; Thu, 18 Jul 2002 01:40:08 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id BAA07515
	for rddp-archive@odin.ietf.org; Thu, 18 Jul 2002 01:41:06 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id BAA07498;
	Thu, 18 Jul 2002 01:40:47 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id BAA07468
	for <rddp@ns.ietf.org>; Thu, 18 Jul 2002 01:40:46 -0400 (EDT)
Received: from portal.hamachi.org ([140.239.227.17])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id BAA12622
	for <rddp@ietf.org>; Thu, 18 Jul 2002 01:39:47 -0400 (EDT)
Received: from syn.hamachi.org (localhost [127.0.0.1])
	by portal.hamachi.org (Postfix) with ESMTP id 33F5D17926
	for <rddp@ietf.org>; Thu, 18 Jul 2002 01:38:01 -0400 (EDT)
Received: from syn.hamachi.org (sommerfeld@localhost)
	by syn.hamachi.org (8.11.6/8.8.8) with ESMTP id g6I5dE500908
	for <rddp@ietf.org>; Thu, 18 Jul 2002 14:39:14 +0900 (JST)
Message-Id: <200207180539.g6I5dE500908@syn.hamachi.org>
From: Bill Sommerfeld <sommerfeld@orchard.arlington.ma.us>
To: rddp@ietf.org
Reply-To: sommerfeld@orchard.arlington.ma.us
Date: Thu, 18 Jul 2002 14:39:14 +0900
Subject: [rddp] comments on draft-black-rdma-concerns-00.txt
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: Remote Data Direct Placement Working Group <rddp.ietf.org>
X-BeenThere: rddp@ietf.org

[aka sommerfeld@sun.com, sending from a different address for now..]

    DDP and RDMA specifications MUST contain mechanisms to prevent
    overwrites from impairing system integrity and to isolate the
    effect of overwrites so that interference among otherwise
    unrelated applications is prevented.

One possible interpretation of this text is that it is not concerned
about application integrity (i.e., applications may crash due to
overwrites but systems remain up).  Some discussion at the RDDP
meeting in Yokohama indicates that this was not the intent of the
words "system integrity"; some clarification is therefore called for..

In particular, my main concern here is that there may be a new class
of overwrite attacks for applications which check a received message
for integrity, then trundle on assuming that it will remain intact and
well formed.  If a malicious peer does a carefully-timed overwrite, it
may then trigger some sort of stack/buffer overrun and take over the
application.

To this end, I believe RDDP/RDMA mechanisms will need to track buffer
ownership very carefully, in particular prohibiting remote writes to a
buffer once an application has been notified that it has been
filled..

				- Bill

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Thu Jul 18 13:48:13 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA09006
	for <rddp-archive@odin.ietf.org>; Thu, 18 Jul 2002 13:48:13 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id NAA02074
	for rddp-archive@odin.ietf.org; Thu, 18 Jul 2002 13:49:10 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id NAA02052;
	Thu, 18 Jul 2002 13:48:36 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id NAA02023
	for <rddp@optimus.ietf.org>; Thu, 18 Jul 2002 13:48:35 -0400 (EDT)
Received: from rp.asomi.net (rp.asomi.net [216.55.142.101])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA08998
	for <rddp@ietf.org>; Thu, 18 Jul 2002 13:47:36 -0400 (EDT)
Received: from 192.168.0.2 (g4.asomi.net [192.168.0.2])
	by rp.asomi.net (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id g6IFOba09083
	for <rddp@ietf.org>; Thu, 18 Jul 2002 10:24:37 -0500
Date: Thu, 18 Jul 2002 12:48:32 -0500
From: Caitlin Bestler <caitlinb@rp.asomi.net>
Subject: Re: [rddp] comments on draft-black-rdma-concerns-00.txt
To: rddp@ietf.org
X-Priority: 3
In-Reply-To: <200207180539.g6I5dE500908@syn.hamachi.org>
Message-ID: <r01050300-1015-93C1589C9A7611D68D9A003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org
Content-Transfer-Encoding: 7bit

On 7/18/02, Bill Sommerfeld wrote:

>[aka sommerfeld@sun.com, sending from a different address
for now..]
>
>   DDP and RDMA specifications MUST contain mechanisms to
>   prevent overwrites from impairing system integrity and 
>   to isolate the effect of overwrites so that interference
>   among otherwise unrelated applications is prevented.
>
>One possible interpretation of this text is that it is not
>concerned about application integrity (i.e., applications
>may crash due to overwrites but systems remain up).  Some
>discussion at the RDDP meeting in Yokohama indicates that
>this was not the intent of the words "system integrity";
>some clarification is therefore called for..
>
>In particular, my main concern here is that there may be a
>new class of overwrite attacks for applications which
>check a received message for integrity, then trundle on
>assuming that it will remain intact and well formed.  If a
>malicious peer does a carefully-timed overwrite, it may
>then trigger some sort of stack/buffer overrun and take
>over the application.
>
>To this end, I believe RDDP/RDMA mechanisms will need to
>track buffer ownership very carefully, in particular
>prohibiting remote writes to a buffer once an application
>has been notified that it has been filled..
>


This is very difficult to deal with unless you standardize
the local interface. There are numerous reasons to avoid
doing so. I'll just toss off two of them:

-- Any 'standard' local interface is going to be heavily
   influenced by the Unix/Windows process model: virtual
   memory, multiple processes, etc.
   
   These assumptions may not be appropriate for embedded
   devices. This is recognized in InfiniBand. Verbs are
   defined for HCAs (and largely incorporate the classic
   Unix/Windows assumptions) but not for TCAs.
   
-- There are major benefits to defining a local interface
   that is compatible with InfiniBand.
   
In the spirit of minimalism, I would propose the following:

-- RDDP may only access ULP buffers to the extent that it
   has been enabled to do so by local interactions with
   the ULP.
   
-- Peers MAY NOT make use of STags except as explicitly 
   provided by the ULP.
   
-- A peer MAY NOT advertise a buffer that it has not enabled
   through its local interface.
   
If you follow these rules you do not need to know what
degree of enforcement your peer has. That is a local matter.
If a given node wishes to use cheaper hardware that exposes
entire pages, because more RAM is cheaper than more
real-time checking, that is a valid local optimization.
It does not affect interoperability, and hence should not
be part of a wire protocol.

Applications can judge whether their local interface
provides adequate protection for their memory, and select
their interface provide based on their needs.



In any event, the benefit of "automatically" expiring an
STag is very slight.

Many applications, especially high performance ones, do not
inform the NIC of each bufer advertisement anyway. Instead
they create a single large STag and advertise buffers within
via ULP to ULP messaging. They would never use such a
feature.

On the contrary side, there are many protocols where
creating an STag for each transaction is a natural solution.
DAFS is an example of a ULP where a buffer advertisement
authorizes the peer to use the supplied buffer only until it
responds to the request.

The question is whether there is a benefit in teaching the
RDDP layers (and presumbly the hardware itself) to
automatically shut down such an STag immediately, rather
than waiting for the ULP to do so itself.

It obviously cannot be done until the ULP notification
conditions are met anyway (generally the Send following the
RDDP puts). There is already an "attack window" concurrent
with the intended  delivery. There is no way that anyone
will design hardware to ensure that each byte in an
advertised buffer is filled at most once.

So the benefit of a one-shot STag is that it reduces the
window of vunerability from the time period when RDDP puts
are going on through the anonymous put, to the same period
plus the time it takes the ULP to respond and turn off the
availability itself.

A local interface could easily authorize an STag to expire
at the next completion on the connection. As a wire protocol
benefit we are talking about the very short period from the
last RDDP put and the anonymous put. Given the total length
of the transaction this is a very narrow time window. I
don't see the justification for complicating the handling of
RDDP puts. It certainly can be done (a "final" bit in the
RDDP put header), but I think keeping tagged RDDP puts as
clean of notification/completion logic as possible is a good
idea.





Caitlin Bestler
Asomi Network Technoogies

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Thu Jul 18 14:37:02 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA10034
	for <rddp-archive@odin.ietf.org>; Thu, 18 Jul 2002 14:37:02 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA04843
	for rddp-archive@odin.ietf.org; Thu, 18 Jul 2002 14:38:00 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA04830;
	Thu, 18 Jul 2002 14:37:46 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA04797
	for <rddp@optimus.ietf.org>; Thu, 18 Jul 2002 14:37:45 -0400 (EDT)
Received: from patan.sun.com (patan.Sun.COM [192.18.98.43])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA10021
	for <rddp@ietf.org>; Thu, 18 Jul 2002 14:36:46 -0400 (EDT)
Received: from jurassic.eng.sun.com ([129.146.17.55])
	by patan.sun.com (8.9.3+Sun/8.9.3) with ESMTP id MAA22446
	for <rddp@ietf.org>; Thu, 18 Jul 2002 12:37:44 -0600 (MDT)
Received: from eng.sun.com (terra.Eng.Sun.COM [129.146.86.99])
	by jurassic.eng.sun.com (8.12.4+Sun/8.12.4) with ESMTP id g6IIbiuO486412
	for <rddp@ietf.org>; Thu, 18 Jul 2002 11:37:44 -0700 (PDT)
Message-ID: <3D370AF6.948D5A76@eng.sun.com>
Date: Thu, 18 Jul 2002 11:37:42 -0700
From: Brent Callaghan <brent@eng.sun.com>
Organization: Sun Microsystems
X-Mailer: Mozilla 4.78 [en] (X11; U; SunOS 5.9 sun4u)
X-Accept-Language: en
MIME-Version: 1.0
To: rddp@ietf.org
Subject: Re: [rddp] comments on draft-black-rdma-concerns-00.txt
References: <200207180539.g6I5dE500908@syn.hamachi.org>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org
Content-Transfer-Encoding: 7bit

Bill Sommerfeld wrote:
> <snip>
> In particular, my main concern here is that there may be a new class
> of overwrite attacks for applications which check a received message
> for integrity, then trundle on assuming that it will remain intact and
> well formed.  If a malicious peer does a carefully-timed overwrite, it
> may then trigger some sort of stack/buffer overrun and take over the
> application.
> 
> To this end, I believe RDDP/RDMA mechanisms will need to track buffer
> ownership very carefully, in particular prohibiting remote writes to a
> buffer once an application has been notified that it has been
> filled..

But the RDDP/RDMA layer doesn't know when the buffer is filled.
The ULP may utilize several RDMA ops to fill a buffer.

Wouldn't it be better for the ULP just to deregister the buffer
before it checks its integrity ?

	Brent

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Thu Jul 18 19:03:26 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA14793
	for <rddp-archive@odin.ietf.org>; Thu, 18 Jul 2002 19:03:26 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id TAA17836
	for rddp-archive@odin.ietf.org; Thu, 18 Jul 2002 19:04:22 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA17824;
	Thu, 18 Jul 2002 19:04:02 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA17791
	for <rddp@optimus.ietf.org>; Thu, 18 Jul 2002 19:04:00 -0400 (EDT)
Received: from rp.asomi.net (rp.asomi.net [216.55.142.101])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA14784
	for <rddp@ietf.org>; Thu, 18 Jul 2002 19:03:03 -0400 (EDT)
Received: from 192.168.0.2 (g4.asomi.net [192.168.0.2])
	by rp.asomi.net (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id g6IKe3a09960
	for <rddp@ietf.org>; Thu, 18 Jul 2002 15:40:04 -0500
Date: Thu, 18 Jul 2002 18:03:59 -0500
From: Caitlin Bestler <caitlinb@rp.asomi.net>
Subject: Re: [rddp] comments on draft-black-rdma-concerns-00.txt
To: rddp@ietf.org
X-Priority: 3
In-Reply-To: <3D370AF6.948D5A76@eng.sun.com>
Message-ID: <r01050300-1015-A4B685409AA211D68D9A003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org
Content-Transfer-Encoding: 7bit

On 7/18/02, Brent Callaghan wrote:

>Bill Sommerfeld wrote:
>> <snip>
>> In particular, my main concern here is that there may be
>> a new class of overwrite attacks for applications which
>> check a received message for integrity, then trundle on
>> assuming that it will remain intact and well formed.  If
>> a malicious peer does a carefully-timed overwrite, it
>> may then trigger some sort of stack/buffer overrun and
>> take over the application.
>> 
>> To this end, I believe RDDP/RDMA mechanisms will need to
>> track buffer ownership very carefully, in particular
>> prohibiting remote writes to a buffer once an
>> application has been notified that it has been filled..
>
>But the RDDP/RDMA layer doesn't know when the buffer is
>filled. The ULP may utilize several RDMA ops to fill a
>buffer.
>
>Wouldn't it be better for the ULP just to deregister the
>buffer before it checks its integrity ?
>
>   Brent
>

If such a feature were implemented, the Data Source ULP
would flag the RDDP put as 'final', meaning that this
connection had no further use of the supplied STag.

Theoretically this allows an ever so slightly earlier
cleanup of the STag, but it does so by making the RDDP layer
aware of STag maintenance. I believe it is cleaner to leave
those issues to the RDMA layer, or preferably the ULP.

Keep in mind that this 'final' flag cannot be simple and
accomplish the goal of enhanching security. In particular it
would be contrary to proper security for the Data Sink to be
forced to reveal to the Data Source whether the STag
supplied was in use on any other connection. Therefore the
'final' flag can do no more than release *this* connection's
claim on the STag. The hardware implementation would have to
countdown the uses before releasing the STag.

All of which favors the broader point that its better to
keep the RDDP layer as simple as possible and leave
application specific logic to the ULP. However, the feature
can be implemented with the ULP retaining full control over
when the STag is released.



_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Sun Jul 21 14:29:06 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA08650
	for <rddp-archive@odin.ietf.org>; Sun, 21 Jul 2002 14:29:06 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA18897
	for rddp-archive@odin.ietf.org; Sun, 21 Jul 2002 14:30:05 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAB18791;
	Sun, 21 Jul 2002 14:29:37 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA18711
	for <rddp@optimus.ietf.org>; Sun, 21 Jul 2002 14:29:34 -0400 (EDT)
Received: from palrel13.hp.com (palrel13.hp.com [156.153.255.238])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA08644
	for <rddp@ietf.org>; Sun, 21 Jul 2002 14:28:34 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel13.hp.com (Postfix) with ESMTP id 766A940038B
	for <rddp@ietf.org>; Sun, 21 Jul 2002 11:29:01 -0700 (PDT)
Received: from mk731915.cup.hp.com ([15.113.161.2])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id LAA05529
	for <rddp@ietf.org>; Sun, 21 Jul 2002 11:29:00 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020721074840.02d1e008@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Sun, 21 Jul 2002 07:56:52 -0700
To: rddp@ietf.org
From: Michael Krause <krause@cup.hp.com>
Subject: Re: [rddp] comments on draft-black-rdma-concerns-00.txt
In-Reply-To: <r01050300-1015-A4B685409AA211D68D9A003065D48EE0@[192.168.0
 .2]>
References: <3D370AF6.948D5A76@eng.sun.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org

At 06:03 PM 7/18/2002 -0500, Caitlin Bestler wrote:
>On 7/18/02, Brent Callaghan wrote:
>
> >Bill Sommerfeld wrote:
> >> <snip>
> >> In particular, my main concern here is that there may be
> >> a new class of overwrite attacks for applications which
> >> check a received message for integrity, then trundle on
> >> assuming that it will remain intact and well formed.  If
> >> a malicious peer does a carefully-timed overwrite, it
> >> may then trigger some sort of stack/buffer overrun and
> >> take over the application.

If one is concerned about malicious attacks, then security should be 
provided such as IPSec which is not a RDMA protocol issue.

> >>
> >> To this end, I believe RDDP/RDMA mechanisms will need to
> >> track buffer ownership very carefully, in particular
> >> prohibiting remote writes to a buffer once an
> >> application has been notified that it has been filled..
> >
> >But the RDDP/RDMA layer doesn't know when the buffer is
> >filled. The ULP may utilize several RDMA ops to fill a
> >buffer.
> >
> >Wouldn't it be better for the ULP just to deregister the
> >buffer before it checks its integrity ?

Absolutely not.  Bind / unbind of a window is not necessarily cheap though 
optimizations can be made.  Anything dealing with allocation / 
de-allocation windows or registration / deregistration of regions would be 
prohibitive.


In general, one should only receive a message once from the transport - see 
the requirements document.  The transport prevents duplicates from 
overwriting the memory.  For non-duplicates, there is no method at the RDMA 
/ DDP layers to distinguish good / bad packets other than what is provided 
by the transport and lower layers.  Hence, if there is a malicious attack 
concern, the lower layer implementation should provide a security solution 
such as IPSec.

My general comment is the document does not highlight new information not 
already stated within the current requirements document.  Perhaps there are 
areas that should be tightened within the requirements document if there is 
any ambiguity which might have been the motivation for this new document 
but I'd prefer to keep the number of documents to a minimum and focus on 
getting them correct from the start if possible.

Mike


_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Mon Jul 22 11:58:03 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA08636
	for <rddp-archive@odin.ietf.org>; Mon, 22 Jul 2002 11:58:02 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id LAA28503
	for rddp-archive@odin.ietf.org; Mon, 22 Jul 2002 11:59:03 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id LAA28273;
	Mon, 22 Jul 2002 11:57:27 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id LAA28226
	for <rddp@optimus.ietf.org>; Mon, 22 Jul 2002 11:57:24 -0400 (EDT)
Received: from mailhub.lss.emc.com (xdch.emc.com [168.159.1.79])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA08589
	for <rddp@ietf.org>; Mon, 22 Jul 2002 11:56:19 -0400 (EDT)
From: Black_David@emc.com
Received: from emc.com (emcmail.lss.emc.com [168.159.48.78])
	by mailhub.lss.emc.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id g6MFukX18433;
	Mon, 22 Jul 2002 11:56:46 -0400 (EDT)
Received: from mxic1.isus.emc.com ([168.159.129.100])
	by emc.com (8.10.1/8.10.1) with ESMTP id g6MFuj724623;
	Mon, 22 Jul 2002 11:56:45 -0400 (EDT)
Received: by MXIC1 with Internet Mail Service (5.5.2653.19)
	id <PJ7SFZFS>; Mon, 22 Jul 2002 11:54:36 -0400
Message-ID: <277DD60FB639D511AC0400B0D068B71E0564C0C4@CORPMX14>
To: krause@cup.hp.com, rddp@ietf.org
Subject: RE: [rddp] comments on draft-black-rdma-concerns-00.txt
Date: Mon, 22 Jul 2002 11:54:34 -0400
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
X-Perlmx-Spam: Gauge=, Probability=0%, Report="EMC0_FROM, EMC1_BODY, NO_REAL_NAME"
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org

Mike,

> > >> In particular, my main concern here is that there may be
> > >> a new class of overwrite attacks for applications which
> > >> check a received message for integrity, then trundle on
> > >> assuming that it will remain intact and well formed.  If
> > >> a malicious peer does a carefully-timed overwrite, it
> > >> may then trigger some sort of stack/buffer overrun and
> > >> take over the application.
> 
> If one is concerned about malicious attacks, then security should be 
> provided such as IPSec which is not a RDMA protocol issue.

That appears to assume that malicious attacks would never come
from peers that are authorized to use the RDDP-enhanced service,
and that's not a good general assumption to make.  For example,
fileservers usually don't completely trust clients that access
files -- having misbehavior by a single client take down the entire
fileserver is usually regarded as undesirable.

> > >> To this end, I believe RDDP/RDMA mechanisms will need to
> > >> track buffer ownership very carefully, in particular
> > >> prohibiting remote writes to a buffer once an
> > >> application has been notified that it has been filled..
> > >
> > >But the RDDP/RDMA layer doesn't know when the buffer is
> > >filled. The ULP may utilize several RDMA ops to fill a
> > >buffer.
> > >
> > >Wouldn't it be better for the ULP just to deregister the
> > >buffer before it checks its integrity ?
> 
> Absolutely not.  Bind / unbind of a window is not necessarily 
> cheap though 
> optimizations can be made.  Anything dealing with allocation / 
> de-allocation windows or registration / deregistration of 
> regions would be prohibitive.

It may be useful to separate the control and data cases.  The
overwrite risk to application integrity applies to control
information, where the application does something different
depending on what it receives (e.g., if DDP were used for some
sort of command data structure).  This overwrite risk probably
does not apply to data that is being passed through (e.g., if a
client overwrites an iSCSI write buffer while the server is
writing it out, the junk that results on the disk is the client's
problem and doesn't affect the server's operational integrity -
the client will get its junk back when it asks for it).

This suggests that the need to prevent this sort of overwrite
depends on the situation, so it may be reasonable to get
the application involved in both the prevention and deciding
when to use it -- for example, "the ULP deregisters the buffer
before it checks its integrity" when buffer integrity matters,
or in situations where it matters, the ULP instructs the stack
to deregister the buffer when the writes to it are complete.

> In general, one should only receive a message once from the transport -
see 
> the requirements document.  The transport prevents duplicates from 
> overwriting the memory.  For non-duplicates, there is no method at the
RDMA 
> / DDP layers to distinguish good / bad packets other than what is provided

> by the transport and lower layers.

I think this is an exercise in strawman demolition.  The concern is
that applications not go down in the face of incorrect input - if
the "bad packets" come first, they have to be dealt with.

> Hence, if there is a malicious attack 
> concern, the lower layer implementation should provide a security solution

> such as IPSec.

I agree that there are some threats that require IPsec or the like to
counter, but one cannot rely on IPsec to ensure that no attacks are
ever attempted through the transport.

> My general comment is the document does not highlight new information not 
> already stated within the current requirements document.  Perhaps there
are 
> areas that should be tightened within the requirements document if there
is 
> any ambiguity which might have been the motivation for this new document 
> but I'd prefer to keep the number of documents to a minimum and focus on 
> getting them correct from the start if possible.

The draft is "input" to the WG - it's not intended to be published as an
RFC.  The text it contains may be useful for things such as the security
considerations sections that will be needed for the main protocol drafts.

Thanks,
--David
---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA  01748
+1 (508) 249-6449            FAX: +1 (508) 497-8018
black_david@emc.com       Mobile: +1 (978) 394-7754
---------------------------------------------------

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Mon Jul 22 14:03:48 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA22953
	for <rddp-archive@odin.ietf.org>; Mon, 22 Jul 2002 14:03:48 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id OAA24019
	for rddp-archive@odin.ietf.org; Mon, 22 Jul 2002 14:04:50 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA23720;
	Mon, 22 Jul 2002 14:04:21 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id OAA23649
	for <rddp@optimus.ietf.org>; Mon, 22 Jul 2002 14:04:16 -0400 (EDT)
Received: from rp.asomi.net (rp.asomi.net [216.55.142.101])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id OAA22873
	for <rddp@ietf.org>; Mon, 22 Jul 2002 14:03:12 -0400 (EDT)
Received: from 192.168.0.2 (g4.asomi.net [192.168.0.2])
	by rp.asomi.net (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id g6MFdAa23551
	for <rddp@ietf.org>; Mon, 22 Jul 2002 10:39:11 -0500
Date: Mon, 22 Jul 2002 13:02:54 -0500
From: Caitlin Bestler <caitlinb@rp.asomi.net>
Subject: RE: [rddp] comments on draft-black-rdma-concerns-00.txt
To: rddp@ietf.org
X-Priority: 3
In-Reply-To: <277DD60FB639D511AC0400B0D068B71E0564C0C4@CORPMX14>
Message-ID: <r01050300-1015-3F0EB9E29D9D11D6BC86003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org
Content-Transfer-Encoding: 7bit

On 7/22/02, Black_David@emc.com wrote:


>
>That appears to assume that malicious attacks would never
>come from peers that are authorized to use the
>RDDP-enhanced service, and that's not a good general
>assumption to make.  For example, fileservers usually
>don't completely trust clients that access files -- having
>misbehavior by a single client take down the entire
>fileserver is usually regarded as undesirable.
>


The DAFS protocol specifically avoid having the file server
expose its own memory resources.

The reasons for this go beyond preventing deliberate
attacks. An advertised buffer is essentially a dedicated
buffer credit. Granting credits to a very large number of
clients who may be using your services on a sporadic basis
is not a good strategy.

This is one of the best justifications for RDMA Reads, BTW.
 



I would agree that the source of the attack is a critical
factor in analyzing the risk.

An outside attacker would either have to forge *multiple*
packets that were part of the same connection or *replace* a
tagged put packet.

If an 'extra' ddp put is forged, the ack counts on the
connection will be off, and there will be no completion
before the connection is torn down.

Yes, the data would have been put. But the contents of
exposed buffers should be considered undefined until the
completion is received.


An insider, could of course simply perform excess DDP puts
before sending the reply to an anonymous buffer. To prevent
this a local interface would be required that fully
understood the ULP buffer rules.

This *could* be done for any given set of ULP rules. But
realistically, existing NIC/memory interfaces were designed
for InfiniBand. I would not anticipate any great vendor
enthusiasm for adding new features to the NIC/memory
interface.

This is especially true because this is *not* an
interoperabiilty issue. As I pointed out previously:

1) A compliant ULP uses ONLY buffers that have been properly
advertised for its use.

2) Those buffers MUST have been enabled for access by the
remote peer.

3) How much more of the peers memory is also exposed is not
relevant to a compliant ULP, because it will not test those
memory locations.


_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@ns.ietf.org  Tue Jul 23 15:41:04 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA01418
	for <rddp-archive@odin.ietf.org>; Tue, 23 Jul 2002 15:41:04 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id PAA12665
	for rddp-archive@odin.ietf.org; Tue, 23 Jul 2002 15:42:05 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA12593;
	Tue, 23 Jul 2002 15:40:56 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA12558
	for <rddp@ns.ietf.org>; Tue, 23 Jul 2002 15:40:55 -0400 (EDT)
Received: from mailhub.lss.emc.com (xdch.emc.com [168.159.1.79])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA01302
	for <rddp@ietf.org>; Tue, 23 Jul 2002 15:39:53 -0400 (EDT)
From: Black_David@emc.com
Received: from emc.com (emcmail.lss.emc.com [168.159.48.78])
	by mailhub.lss.emc.com (Switch-2.2.0/Switch-2.2.0) with ESMTP id g6NJeJb29184
	for <rddp@ietf.org>; Tue, 23 Jul 2002 15:40:20 -0400 (EDT)
Received: from mxic1.isus.emc.com ([168.159.129.100])
	by emc.com (8.10.1/8.10.1) with ESMTP id g6NJeJ711783
	for <rddp@ietf.org>; Tue, 23 Jul 2002 15:40:19 -0400 (EDT)
Received: by MXIC1 with Internet Mail Service (5.5.2653.19)
	id <PJ7SH930>; Tue, 23 Jul 2002 15:38:09 -0400
Message-ID: <277DD60FB639D511AC0400B0D068B71E0564C0D2@CORPMX14>
To: rddp@ietf.org
Date: Tue, 23 Jul 2002 15:38:04 -0400
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: [rddp] rddp Yokohama DRAFT minutes
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org

Thanks to Mike Smith for taking notes.  Here are
the draft minutes.  Please comment either privately
or to the list - the final minutes will be sent to
the secretariat around the end of this week.

Thanks,
--David

IETF Remote Direct Data Placement (rddp) WG - DRAFT
Meeting minutes - July 18, 2002, Yokohama, Japan
-------------------------------------------------

Chair: David L. Black (black_david@emc.com)

-- Agenda Bashing and Administrivia

rddp is now an IETF Working Group, formed based on the ROI
(RDMA Over the Internet) BOFs in Salt Lake City and Minneapolis.

The mailing list is up and running at rddp@ietf.org.  To subscribe,
send email to rddp-request@ietf.org.  Use of the rdma list at Yahoo
Groups for IETF work has ceased.

-- Charter and Goal/Milestone review and discussion

The official WG charter is available at:

	http://www.ietf.org/html.charters/rddp-charter.html

The four work items on the charter encompass the entire scope of
the WG's activities.  TCP framing was deliberately omitted from that
list - TCP framing work belongs in the Transport Area (tsvwg)
WG. The rddp WG is not permitted to standardize changes to transport
protocols - it may recommend (minor) SCTP specification changes to
the tsvwg WG, but TCP specification changes are not permitted.
The rddp WG has a rechartering scheduled for March 2003 whose
outcome (e.g., whether the WG will be allowed to continue) will
be based to a large extent on the progress made between now and then.

The rddp WG is expected to work on mapping its functionality to both
SCTP and TCP - see the charter.  Work to apply RDDP mechanisms to
protocols other than SCTP and TCP is outside the current charter -
those interested in such work, including generic IP-layer mechanisms
applicable to all protocols that use IP, should submit Internet-Drafts
as a basis for further discussion.  The planned WG draft mapping RDDP
to TCP is currently intended to become an Informational RFC rather than
a standards track RFC - any decision to change this is best done
with a draft in hand, so consideration will be deferred to the rddp
WG rechartering currently scheduled for March 2003.  On the
subject of compatibility with proprietary interconnects/protocols
that already support RDDP-like functionality (often called RDMA), the
guidance from the Area Director is that the WG should not go out of
its way to be incompatible, but likewise, heroic efforts motivated
solely by such compatibility are also inadvisable.

The RDMA Consortium (www.rdmaconsortium.org) was brought up.  This
industry consortium was organized to work on providing functionality
over TCP, and has a goal of transferring its work to the IETF and putting
itself out of business.  The RDMA consortium was formed at a time where
there was some confusion over whether and to what extent the IETF
would permit RDDP work over TCP - that confusion has now been resolved
by the rddp WG charter (such work is permitted, but SCTP comes first).
The WG chair will work with the RDMA consortium towards their goal
of moving their work into the IETF.  It should be understood that
"the fix is not in" - while the RDMA Consortium is very likely to submit
drafts describing its work, that does not preclude others from writing
drafts on the same or similar topics for consideration by the WG.

-- RDMA Concerns Draft (draft-black-rdma-concerns-00.txt)

This draft was based on concerns arising from the Minneapolis BOF
and evolved to its current form via discussion at an end2end Research
Group meeting.  This draft is input to the rddp WG, and is not expected
to become an RFC.

The topic of potential Intellectual Property Rights issues was raised.
The only official way to notify the IETF of such issues is to send an
IPR notice to the IETF - mailing list discussions are not "official
notice".  If anyone knows of an IPR claim for which they believe a notice
should have been sent to the IETF but was not, a private email should
be sent to the WG chair as an initial step.

An issue was raised about the absence of "application integrity" from the
RDMA concerns draft - in addition to "system integrity" concerns, the
overwrites made possible by RDDP functionality open up potential attacks on
applications via overwriting data between the time that an application
checks the data and the time that the application makes use of the data.
This concern will be included in the -01 version of the RDMA concerns draft.

-- Problem Statement and Requirements drafts
	(draft-romanow-rdma-over-ip-problem-statement-00.txt)
	(draft-talpey-rdma-over-ip-requirements-00.txt)

-01 versions of both drafts with minor revisions should hit the
Internet-Draft servers shortly after the Yokohama meeting week.

The rddp WG is tasked to produce a problem statement/architecture
draft along with specifications of an RDDP protocol and an RDDP control
protocol (for RDMA functionality) and mappings of those protocols to
SCTP and TCP .  The problem statement draft will be expanded with
architecture text to become the first of these drafts (one possible
source is draft-bailey-roi-ddp-rdma-arch-00.txt).  Randy Stewart has
a draft (draft-stewart-sctp-roi-00.txt) that could serve as the basis
for the mapping to SCTP.

There was a discussion about the relationship between RDDP operation sizes
(as invoked by protocols above RDDP) and transport segment sizes (based on
network Path MTU).  These sizes ought to be independent, which entails
segmentation and reassembly in the RDDP layer with the goal of providing
an RDDP header in each transport segment; SCTP's ability to operate without
SCTP fragmentation helps, but there are some unavoidable situations in the
face of Path MTU changes where transport segments may have to be sent
without an RDDP header in each segment. 

-- Other Business and/or Discussion

The division of work for applying RDDP to various protocols will probably
be similar to the way that the IETF handles MIBs - the rddp WG will do
the basic work to define the RDDP mechanisms, and then other WGs will
handle application/mapping to their respective protocols.  For the
specific case of SRP (SCSI RDMA Protocol), the IP Storage (ips) WG is
one possible place to do that application/mapping work even though
the protocol is specified by T10.

For interface specifications (e.g., application interface to RDDP
implemented in hardware), drafts are welcome (including the envisioned
division of functionality between hardware and software), but the IETF
has not generally been enthusiastic about specifying APIs and the like,
and does not use API specifications to drive protocol definitions. 
The RDMA Consortium is currently working on verbs (abstract API definition),
and it may be appropriate to allow that work to continue there rather
than transferring it to the IETF.  

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@ns.ietf.org  Tue Jul 23 16:16:11 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA02829
	for <rddp-archive@odin.ietf.org>; Tue, 23 Jul 2002 16:16:11 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id QAA15207
	for rddp-archive@odin.ietf.org; Tue, 23 Jul 2002 16:17:12 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA15166;
	Tue, 23 Jul 2002 16:16:16 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id QAA15133
	for <rddp@ns.ietf.org>; Tue, 23 Jul 2002 16:16:14 -0400 (EDT)
Received: from rp.asomi.net (rp.asomi.net [216.55.142.101])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA02781
	for <rddp@ietf.org>; Tue, 23 Jul 2002 16:15:11 -0400 (EDT)
Received: from 192.168.0.2 (g4.asomi.net [192.168.0.2])
	by rp.asomi.net (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id g6NHqUa28336
	for <rddp@ietf.org>; Tue, 23 Jul 2002 12:52:30 -0500
Date: Tue, 23 Jul 2002 15:16:11 -0500
From: Caitlin Bestler <caitlinb@rp.asomi.net>
To: rddp@ietf.org
X-Priority: 3
Message-ID: <r01050300-1015-07F698B69E7911D6BC86003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Subject: [rddp] SCTP rwnd
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org
Content-Transfer-Encoding: 7bit

Assuming that an SCTP association is used for a single RDDP
connection, how should the receive window credits be
calculated?

In a purely layered implementation, the credits represent
the SCTP layer's buffering capacity. SCTP data chunks are
delivered to the SCTP's "ULP", RDDP. When it completes them
the count is returned to the available SCTP buffer capacity.

That works. And you *could* implement that way.

But the goal is allow the "ULP" to be implementable in
hardware. That means that the time it takes to empty the
SCTP layer buffer to an RDDP target buffer is totally
predictable, and several factors shorter than the round trip
time of the connection.

Rather than churning all of these calculations as though the
buffering at the two layers were totally distinct, would it
be acceptable to advertise the receive window credits based
upon the assumption that all received data chunks will have
a valid RDDP target buffer, and that the  transport layer
buffering was only required to the extent necessary to
validate an SCTP packet?

If RDDP's ULP is sending messages that will not have valid
receiving buffers the connection will be torn down anyway.
Is there any need to faithfully exchange SCTP layer buffer
credits? Could the receiver simply set the credits high,
knowing that if it receives a data chunk for which there is
no valid RDDP layer target that it will simply drop the
packet and break the connection?

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Fri Jul 26 09:31:04 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA19155
	for <rddp-archive@odin.ietf.org>; Fri, 26 Jul 2002 09:31:04 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id JAA22665
	for rddp-archive@odin.ietf.org; Fri, 26 Jul 2002 09:32:09 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id JAA22603;
	Fri, 26 Jul 2002 09:31:32 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id JAA22571
	for <rddp@optimus.ietf.org>; Fri, 26 Jul 2002 09:31:31 -0400 (EDT)
Received: from palrel10.hp.com (palrel10.hp.com [156.153.255.245])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA19132
	for <rddp@ietf.org>; Fri, 26 Jul 2002 09:30:25 -0400 (EDT)
Received: from psalpha0.cup.hp.com (psalpha0.cup.hp.com [15.13.189.230])
	by palrel10.hp.com (Postfix) with ESMTP id 917B6C00DFF
	for <rddp@ietf.org>; Fri, 26 Jul 2002 06:30:55 -0700 (PDT)
Received: from mk731915.cup.hp.com ([15.113.161.2])
	by psalpha0.cup.hp.com (8.9.3/8.8.6) with ESMTP id GAA16481
	for <rddp@ietf.org>; Fri, 26 Jul 2002 06:30:54 -0700 (PDT)
Message-Id: <5.1.0.14.2.20020724075028.00a873e0@esalpha2.cup.hp.com>
X-Sender: krause@esalpha2.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.1
Date: Wed, 24 Jul 2002 08:30:57 -0700
To: rddp@ietf.org
From: Michael Krause <krause@cup.hp.com>
Subject: RE: [rddp] comments on draft-black-rdma-concerns-00.txt
In-Reply-To: <277DD60FB639D511AC0400B0D068B71E0564C0C4@CORPMX14>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org

At 11:54 AM 7/22/2002 -0400, Black_David@emc.com wrote:
>Mike,
>
> > > >> In particular, my main concern here is that there may be
> > > >> a new class of overwrite attacks for applications which
> > > >> check a received message for integrity, then trundle on
> > > >> assuming that it will remain intact and well formed.  If
> > > >> a malicious peer does a carefully-timed overwrite, it
> > > >> may then trigger some sort of stack/buffer overrun and
> > > >> take over the application.
> >
> > If one is concerned about malicious attacks, then security should be
> > provided such as IPSec which is not a RDMA protocol issue.
>
>That appears to assume that malicious attacks would never come
>from peers that are authorized to use the RDDP-enhanced service,
>and that's not a good general assumption to make.  For example,
>fileservers usually don't completely trust clients that access
>files -- having misbehavior by a single client take down the entire
>fileserver is usually regarded as undesirable.

Each client should only be given access to a window at a given time thus 
one cannot take down anything other than what has been exposed to that 
client.  File servers should only expose windows and not the entire memory 
region.  Thus if the communication is over a single stream and targeting a 
given window within the region, it does not seem that RDMA creates any new 
security issues that are at least new in this respect.


> > > >> To this end, I believe RDDP/RDMA mechanisms will need to
> > > >> track buffer ownership very carefully, in particular
> > > >> prohibiting remote writes to a buffer once an
> > > >> application has been notified that it has been filled..
> > > >
> > > >But the RDDP/RDMA layer doesn't know when the buffer is
> > > >filled. The ULP may utilize several RDMA ops to fill a
> > > >buffer.
> > > >
> > > >Wouldn't it be better for the ULP just to deregister the
> > > >buffer before it checks its integrity ?
> >
> > Absolutely not.  Bind / unbind of a window is not necessarily
> > cheap though
> > optimizations can be made.  Anything dealing with allocation /
> > de-allocation windows or registration / deregistration of
> > regions would be prohibitive.
>
>It may be useful to separate the control and data cases.  The
>overwrite risk to application integrity applies to control
>information, where the application does something different
>depending on what it receives (e.g., if DDP were used for some
>sort of command data structure).  This overwrite risk probably
>does not apply to data that is being passed through (e.g., if a
>client overwrites an iSCSI write buffer while the server is
>writing it out, the junk that results on the disk is the client's
>problem and doesn't affect the server's operational integrity -
>the client will get its junk back when it asks for it).
>
>This suggests that the need to prevent this sort of overwrite
>depends on the situation, so it may be reasonable to get
>the application involved in both the prevention and deciding
>when to use it -- for example, "the ULP deregisters the buffer
>before it checks its integrity" when buffer integrity matters,
>or in situations where it matters, the ULP instructs the stack
>to deregister the buffer when the writes to it are complete.

Sounds like an application / ULP issue and not a RDMA issue.


> > In general, one should only receive a message once from the transport -
>see
> > the requirements document.  The transport prevents duplicates from
> > overwriting the memory.  For non-duplicates, there is no method at the
>RDMA
> > / DDP layers to distinguish good / bad packets other than what is provided
>
> > by the transport and lower layers.
>
>I think this is an exercise in strawman demolition.  The concern is
>that applications not go down in the face of incorrect input - if
>the "bad packets" come first, they have to be dealt with.

Agreed - an application / ULP issue in this regard.


> > Hence, if there is a malicious attack
> > concern, the lower layer implementation should provide a security solution
>
> > such as IPSec.
>
>I agree that there are some threats that require IPsec or the like to
>counter, but one cannot rely on IPsec to ensure that no attacks are
>ever attempted through the transport.

Not suggesting IPSec solves all problems but if security is an issue, it 
seems a security layer should deal with it as nothing mentioned within the 
document suggests that RDMA is somehow creating security issues or causing 
non-application / ULP control data corruption.

Mike


_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Sat Jul 27 06:04:04 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA29597
	for <rddp-archive@odin.ietf.org>; Sat, 27 Jul 2002 06:04:04 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id GAA00591
	for rddp-archive@odin.ietf.org; Sat, 27 Jul 2002 06:05:09 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id GAA00580;
	Sat, 27 Jul 2002 06:04:47 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id GAA00547
	for <rddp@optimus.ietf.org>; Sat, 27 Jul 2002 06:04:46 -0400 (EDT)
Received: from portal.hamachi.org (portal.hamachi.org [140.239.227.17])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id GAA29591
	for <rddp@ietf.org>; Sat, 27 Jul 2002 06:03:39 -0400 (EDT)
Received: from syn.hamachi.org (localhost [127.0.0.1])
	by portal.hamachi.org (Postfix) with ESMTP
	id 6155417927; Sat, 27 Jul 2002 06:02:58 -0400 (EDT)
Received: from syn.hamachi.org (sommerfeld@localhost)
	by syn.hamachi.org (8.11.6/8.8.8) with ESMTP id g6RA3v906033;
	Sat, 27 Jul 2002 06:03:57 -0400 (EDT)
Message-Id: <200207271003.g6RA3v906033@syn.hamachi.org>
From: Bill Sommerfeld <sommerfeld@orchard.arlington.ma.us>
To: Michael Krause <krause@cup.hp.com>
Cc: rddp@ietf.org
Subject: Re: [rddp] comments on draft-black-rdma-concerns-00.txt 
In-Reply-To: Message from Michael Krause <krause@cup.hp.com> 
   of "Sun, 21 Jul 2002 07:56:52 PDT." <5.1.0.14.2.20020721074840.02d1e008@esalpha2.cup.hp.com> 
Reply-To: sommerfeld@orchard.arlington.ma.us
Date: Sat, 27 Jul 2002 06:03:57 -0400
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org

> If one is concerned about malicious attacks, then security should be 
> provided such as IPSec which is not a RDMA protocol issue.

This represents a fundamental misunderstanding.  IPsec provides a
mechanism to allow for confidentiality and integrity.  It is not
"magic pixie dust" which protects applications against buffer
overruns, etc.,

The mere fact that a peer may be authenticated and even authorized to
talk with an application does not mean that the peer is authorized to
hijack the application.

> > >Wouldn't it be better for the ULP just to deregister the
> > >buffer before it checks its integrity ?
> 
> Absolutely not.  Bind / unbind of a window is not necessarily cheap though 
> optimizations can be made.  Anything dealing with allocation / 
> de-allocation windows or registration / deregistration of regions would be 
> prohibitive.

The WG charter says:

   The working group will ensure that the resulting technology will be
   secure and will not enable new attacks on systems supporting RDDP.

An overwrite race condition sounds to me like a "new attack".

						- Bill

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Sat Jul 27 12:25:03 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA04346
	for <rddp-archive@odin.ietf.org>; Sat, 27 Jul 2002 12:25:03 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id MAA13032
	for rddp-archive@odin.ietf.org; Sat, 27 Jul 2002 12:26:08 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id MAA12875;
	Sat, 27 Jul 2002 12:19:30 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id MAA12845
	for <rddp@optimus.ietf.org>; Sat, 27 Jul 2002 12:19:29 -0400 (EDT)
Received: from rp.asomi.net (rp.asomi.net [216.55.142.101])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA04233
	for <rddp@ietf.org>; Sat, 27 Jul 2002 12:18:22 -0400 (EDT)
Received: from 192.168.0.2 (g4.asomi.net [192.168.0.2])
	by rp.asomi.net (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id g6RDtva10710;
	Sat, 27 Jul 2002 08:55:57 -0500
Date: Sat, 27 Jul 2002 11:19:25 -0500
From: Caitlin Bestler <caitlinb@rp.asomi.net>
Subject: Re: [rddp] comments on draft-black-rdma-concerns-00.txt 
To: rddp@ietf.org
cc: Michael Krause <krause@cup.hp.com>, sommerfeld@orchard.arlington.ma.us
X-Priority: 3
In-Reply-To: <200207271003.g6RA3v906033@syn.hamachi.org>
Message-ID: <r01050300-1015-9E31F374A17C11D69D27003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org
Content-Transfer-Encoding: 7bit

On 7/27/02, Bill Sommerfeld wrote:


>The WG charter says:
>
> The working group will ensure that the resulting
> technology will be secure and will not enable new attacks
> on systems supporting RDDP.
>
>An overwrite race condition sounds to me like a "new
>attack".

This is a local interface issue, not a protocol issue.

Fundamentally, the Data Source does not need, nor should
it have, the complete semantics of the STag that is part
of the buffer advertisement it received.

The only knowledge it requires is the extent of permission
given to it by the ULP. DAFS is probably typical of the
rules most ULPs will establish:

-- the client sends a request to an anonymous buffer that
   contains a buffer advertisement and a request for some
   work to be done using that buffer.
   
-- the server performs that work.

-- the server sends a reply back to client, also to an 
   anonymous buffer,  thereby terminating its rights to
   the advertised buffer.

The ULP Data Sink / Client MUST ensure that it has used
local interfaces to enable RDDP layer memory access with
sufficient rights to match the advertised buffers.

There is no *protocol* justification for mandating that all
Data Sinks MUST protect the rest of their memory. The
protocol MUST NOT *prevent* them from doing so.

Nothing in any of the iWARP/ROI/RDDP proposals has ever done
anything to prevent more precise local protection of memory.





Even if such local  mechanisms are available, it  would be
inappropriate to make  them mandatory. There are common
practices with RDMA Writes which are incompatible with
deregistering the STag automatically:

-- Enabling a single large region specifically  for the 
    client, and then advertising buffers within that STag.
    This avoids having to create and distribute STags for
    each interaction. This model can be very attractive
    when the peers have long lasting connections  and
    many interactions, such as for an RDBMS.
-- Enabling a large buffer that is filled on an ongoing
    basis, but with each delivery being a notified event.
    This can be useful for delivery a series of variable
    sized data, especially for streaming media.
    
So ultimately, disabling of an STag is both a ULP and local
interface question. The question is whether to mandate a
specific, and potentially expensive behavior to close a
"vunerability window" between the time the message is
received that would have automatically disabled an STag and
when the ULP would have responded to the notification and
disabled the STag itself.

But keep in mind that the same buffers were vunerable to
unauthorized overwrites from the very same sources during
the entire period *prior* to the completing message.

Now there may be some applications where additional
complexity on the NIC is justifiable if it reduces
vunerabilty  in *ANY* way. They would probably be better
served by sticking with non-RDDP messages with
authenticating signatures to every update, but conceivably
they could decide that automatic STag release was a valuable
feature for them. However, mandating that *all* Data Sinks
MUST exercise the *same* data protection policies is simply
not called for.
 

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Sat Jul 27 17:06:43 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA07461
	for <rddp-archive@odin.ietf.org>; Sat, 27 Jul 2002 17:06:43 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id RAA23390
	for rddp-archive@odin.ietf.org; Sat, 27 Jul 2002 17:07:49 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id RAA23377;
	Sat, 27 Jul 2002 17:07:25 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id RAA23344
	for <rddp@optimus.ietf.org>; Sat, 27 Jul 2002 17:07:23 -0400 (EDT)
Received: from maho3msx2.isus.emc.com (maho3msx2.isus.emc.com [128.221.11.32])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA07457
	for <rddp@ietf.org>; Sat, 27 Jul 2002 17:06:17 -0400 (EDT)
From: Black_David@emc.com
Received: by MAHO3MSX2 with Internet Mail Service (5.5.2653.19)
	id <PNM08018>; Sat, 27 Jul 2002 17:06:47 -0400
Message-ID: <277DD60FB639D511AC0400B0D068B71E0564C10E@CORPMX14>
To: caitlinb@rp.asomi.net, rddp@ietf.org
Subject: RE: [rddp] comments on draft-black-rdma-concerns-00.txt 
Date: Sat, 27 Jul 2002 17:06:44 -0400
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org

A few comments:

> This is a local interface issue, not a protocol issue.

That doesn't rule it out of scope - if properties of the local
interface are important to the secure use of the protocol, those
properties need to be specified.  It's not necessary to specify
the local interface itself - functional requirements/constraints
on it are sufficient to get the job done.  One example can be found
in the IPsec architecture RFC - see Section 4.4 (and subsections)
of RFC 2401.

> The ULP Data Sink / Client MUST ensure that it has used
> local interfaces to enable RDDP layer memory access with
> sufficient rights to match the advertised buffers.
> 
> There is no *protocol* justification for mandating that all
> Data Sinks MUST protect the rest of their memory. The
> protocol MUST NOT *prevent* them from doing so.

The actual situation may be somewhere in between:
- It's probably appropriate to require (MUST implement) suitable
	data protection mechanisms even if their use is OPTIONAL.
	As I noted earlier, if the RDDP mechanisms are being used
	only to transfer data and not control, this overwrite
	attack can't cause application integrity problems and
	hence protecting against it may not be necessary.
- The "Security Considerations" section of the protocol
	specification(s) will need to discuss this overwrite
	possibility, when/how it poses a risk, and what to do
	about it.  At a minimum this will result in instructions
	that *if* a particular application's use of RDDP is
	vulnerable to this sort of overwrite, then suitable
	preventative measures MUST be employed, a useful
	warning for those figuring out how to use RDDP with
	other protocols.

Thanks,
--David

---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA  01748
+1 (508) 249-6449            FAX: +1 (508) 497-8018
black_david@emc.com       Mobile: +1 (978) 394-7754
---------------------------------------------------


> -----Original Message-----
> From: Caitlin Bestler [mailto:caitlinb@rp.asomi.net]
> Sent: Saturday, July 27, 2002 12:19 PM
> To: rddp@ietf.org
> Cc: Michael Krause; sommerfeld@orchard.arlington.ma.us
> Subject: Re: [rddp] comments on draft-black-rdma-concerns-00.txt 
> 
> 
> On 7/27/02, Bill Sommerfeld wrote:
> 
> 
> >The WG charter says:
> >
> > The working group will ensure that the resulting
> > technology will be secure and will not enable new attacks
> > on systems supporting RDDP.
> >
> >An overwrite race condition sounds to me like a "new attack".
> 
> This is a local interface issue, not a protocol issue.
> 
> Fundamentally, the Data Source does not need, nor should
> it have, the complete semantics of the STag that is part
> of the buffer advertisement it received.
> 
> The only knowledge it requires is the extent of permission
> given to it by the ULP. DAFS is probably typical of the
> rules most ULPs will establish:
> 
> -- the client sends a request to an anonymous buffer that
>    contains a buffer advertisement and a request for some
>    work to be done using that buffer.
>    
> -- the server performs that work.
> 
> -- the server sends a reply back to client, also to an 
>    anonymous buffer,  thereby terminating its rights to
>    the advertised buffer.
> 
> The ULP Data Sink / Client MUST ensure that it has used
> local interfaces to enable RDDP layer memory access with
> sufficient rights to match the advertised buffers.
> 
> There is no *protocol* justification for mandating that all
> Data Sinks MUST protect the rest of their memory. The
> protocol MUST NOT *prevent* them from doing so.
> 
> Nothing in any of the iWARP/ROI/RDDP proposals has ever done
> anything to prevent more precise local protection of memory.
> 
> 
> 
> 
> 
> Even if such local  mechanisms are available, it  would be
> inappropriate to make  them mandatory. There are common
> practices with RDMA Writes which are incompatible with
> deregistering the STag automatically:
> 
> -- Enabling a single large region specifically  for the 
>     client, and then advertising buffers within that STag.
>     This avoids having to create and distribute STags for
>     each interaction. This model can be very attractive
>     when the peers have long lasting connections  and
>     many interactions, such as for an RDBMS.
> -- Enabling a large buffer that is filled on an ongoing
>     basis, but with each delivery being a notified event.
>     This can be useful for delivery a series of variable
>     sized data, especially for streaming media.
>     
> So ultimately, disabling of an STag is both a ULP and local
> interface question. The question is whether to mandate a
> specific, and potentially expensive behavior to close a
> "vunerability window" between the time the message is
> received that would have automatically disabled an STag and
> when the ULP would have responded to the notification and
> disabled the STag itself.
> 
> But keep in mind that the same buffers were vunerable to
> unauthorized overwrites from the very same sources during
> the entire period *prior* to the completing message.
> 
> Now there may be some applications where additional
> complexity on the NIC is justifiable if it reduces
> vunerabilty  in *ANY* way. They would probably be better
> served by sticking with non-RDDP messages with
> authenticating signatures to every update, but conceivably
> they could decide that automatic STag release was a valuable
> feature for them. However, mandating that *all* Data Sinks
> MUST exercise the *same* data protection policies is simply
> not called for.
>  
> 
> _______________________________________________
> rddp mailing list
> rddp@ietf.org
> https://www1.ietf.org/mailman/listinfo/rddp
> 

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Sun Jul 28 12:36:46 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA29236
	for <rddp-archive@odin.ietf.org>; Sun, 28 Jul 2002 12:36:46 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id MAA08325
	for rddp-archive@odin.ietf.org; Sun, 28 Jul 2002 12:37:52 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id MAA08247;
	Sun, 28 Jul 2002 12:34:01 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id MAA08212
	for <rddp@optimus.ietf.org>; Sun, 28 Jul 2002 12:33:49 -0400 (EDT)
Received: from portal.hamachi.org (portal.hamachi.org [140.239.227.17])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA29193
	for <rddp@ietf.org>; Sun, 28 Jul 2002 12:32:39 -0400 (EDT)
Received: from syn.hamachi.org (localhost [127.0.0.1])
	by portal.hamachi.org (Postfix) with ESMTP
	id 7114417927; Sun, 28 Jul 2002 12:32:00 -0400 (EDT)
Received: from syn.hamachi.org (sommerfeld@localhost)
	by syn.hamachi.org (8.11.6/8.8.8) with ESMTP id g6SGXbD08445;
	Sun, 28 Jul 2002 12:33:37 -0400 (EDT)
Message-Id: <200207281633.g6SGXbD08445@syn.hamachi.org>
From: Bill Sommerfeld <sommerfeld@orchard.arlington.ma.us>
To: Black_David@emc.com
Cc: caitlinb@rp.asomi.net, rddp@ietf.org
Subject: Re: [rddp] comments on draft-black-rdma-concerns-00.txt 
In-Reply-To: Message from Black_David@emc.com 
   of "Sat, 27 Jul 2002 17:06:44 EDT." <277DD60FB639D511AC0400B0D068B71E0564C10E@CORPMX14> 
Reply-To: sommerfeld@orchard.arlington.ma.us
Date: Sun, 28 Jul 2002 12:33:36 -0400
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org

> 	As I noted earlier, if the RDDP mechanisms are being used
> 	only to transfer data and not control, this overwrite
> 	attack can't cause application integrity problems and
> 	hence protecting against it may not be necessary.

In thinking some more about this, the "data"/"control" distinction
doesn't quite capture the risks involved with overwrites.  It's more
of a a "read once" vs "read multiple" case -- overwrites which come in
between multiple reads may cause unexpected application behavior.

So, one could conceivably write a parser for a protocol which loads
each byte at most once -- in which case, overwrites are irrelevant.

Consider anther hypothetical system where a checksum is computed by
the receiving application and stored as part of metadata.

An app is notified of rddp'ed data, computes a checksum over the data,
and then schedules the data to be written to stable store (presumably
with another DMA operation).  Thus, despite being "bulk data", each
byte is read twice -- once by whatever computes the checksum, and
again by the disk controller writing it out to stable store.

So, in parallel with this (during or after the checksum computation),
another rddp/rdma message comes in and overwrites one of the pages
being written, causing the image in stable store to not match the
checksum...

Depending on the scope of the checksum (this may be a checksum over a
batch of transactions being written to a journal), this may
effectively deny future access to more than just the data which the
peer had write access to...

Naturally, one sure-fire way to avoid overwrite complications is just
copy the data to a buffer which is out of reach of the rddp/rdma
system, but I thought the whole point of this exercise is to avoid the
cost of data copies..

Besides the Security Considerations section, I think there may also be
fodder for an applicability statements -- the cost of buffer
registration/deregistration needed to make a system or application
safe against unexpected remote overwrites may rival or exceed the cost
of the data copies avoided by use of the rddp/rdma mechanism, at which
point the added complexity makes it a net lose.

					- Bill

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Sun Jul 28 17:47:29 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA03382
	for <rddp-archive@odin.ietf.org>; Sun, 28 Jul 2002 17:47:29 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id RAA19457
	for rddp-archive@odin.ietf.org; Sun, 28 Jul 2002 17:48:35 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id RAA19441;
	Sun, 28 Jul 2002 17:48:04 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id RAA19408
	for <rddp@optimus.ietf.org>; Sun, 28 Jul 2002 17:48:02 -0400 (EDT)
Received: from rp.asomi.net (rp.asomi.net [216.55.142.101])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA03378
	for <rddp@ietf.org>; Sun, 28 Jul 2002 17:46:54 -0400 (EDT)
Received: from 192.168.0.2 (g4.asomi.net [192.168.0.2])
	by rp.asomi.net (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id g6SJIaZ01712;
	Sun, 28 Jul 2002 14:18:36 -0500
Date: Sun, 28 Jul 2002 16:47:59 -0500
From: Caitlin Bestler <caitlinb@rp.asomi.net>
Subject: Re: [rddp] comments on draft-black-rdma-concerns-00.txt 
To: rddp@ietf.org
cc: Black_David@emc.com, sommerfeld@orchard.arlington.ma.us
X-Priority: 3
In-Reply-To: <200207281633.g6SGXbD08445@syn.hamachi.org>
Message-ID: <r01050300-1015-AEF05CBCA27311D6BAFF003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org
Content-Transfer-Encoding: 7bit

On 7/28/02, Bill Sommerfeld wrote:

>>  As I noted earlier, if the RDDP mechanisms are being
used
>>  only to transfer data and not control, this overwrite
>>  attack can't cause application integrity problems and
>>  hence protecting against it may not be necessary.
>
>In thinking some more about this, the "data"/"control"
>distinction doesn't quite capture the risks involved with
>overwrites.  It's more of a a "read once" vs "read
>multiple" case -- overwrites which come in between
>multiple reads may cause unexpected application behavior.
>
>So, one could conceivably write a parser for a protocol
>which loads each byte at most once -- in which case,
>overwrites are irrelevant.
>
>Consider anther hypothetical system where a checksum is
>computed by the receiving application and stored as part
>of metadata.
>
>An app is notified of rddp'ed data, computes a checksum
>over the data, and then schedules the data to be written
>to stable store (presumably with another DMA operation). 
>Thus, despite being "bulk data", each byte is read twice
>-- once by whatever computes the checksum, and again by
>the disk controller writing it out to stable store.
>
>So, in parallel with this (during or after the checksum
>computation), another rddp/rdma message comes in and
>overwrites one of the pages being written, causing the
>image in stable store to not match the checksum...
>
>Depending on the scope of the checksum (this may be a
>checksum over a batch of transactions being written to a
>journal), this may effectively deny future access to more
>than just the data which the peer had write access to...
>
>Naturally, one sure-fire way to avoid overwrite
>complications is just copy the data to a buffer which is
>out of reach of the rddp/rdma system, but I thought the
>whole point of this exercise is to avoid the cost of data
>copies..
>
>Besides the Security Considerations section, I think there
>may also be fodder for an applicability statements -- the
>cost of buffer registration/deregistration needed to make
>a system or application safe against unexpected remote
>overwrites may rival or exceed the cost of the data copies
>avoided by use of the rddp/rdma mechanism, at which point
>the added complexity makes it a net lose.
>


This sort of scenario is indeed the best justification for
automatic de-registration of the STag. Whether or not an
actual checksum is calculated, the notification that a reply
has been received should indicate transactional closure.

The interface I would favor would be one where the Data Sink
ULP indicated that it wanted an STag deregistered when
notification was made on receive-completion.

When the ULP gets that notification it would be ideal that
it knew that there would be no more modifications of the
advertised buffer, so that it could begin checksum
validation and processing.

I will point out that even without such a local interface it
could simply deregister the STag itself, and when *that*
completes it would have achieved the exact same set of
guaranteed behavior. This is especially true if the local
interface provides for synchronous invalidation of an STag.

It should be noted that any such semantics are not
compatible with InfiniBand R-Key semantics. I wouldn't argue
that STags matching IB R-Keys is a requirement, but I would
favor having a justification for each difference.

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Mon Jul 29 22:07:48 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA22698
	for <rddp-archive@odin.ietf.org>; Mon, 29 Jul 2002 22:07:48 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id WAA13341
	for rddp-archive@odin.ietf.org; Mon, 29 Jul 2002 22:08:56 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id WAA13276;
	Mon, 29 Jul 2002 22:07:59 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id WAA13247
	for <rddp@optimus.ietf.org>; Mon, 29 Jul 2002 22:07:57 -0400 (EDT)
Received: from nwkea-mail-1.sun.com (nwkea-mail-1.sun.com [192.18.42.13])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA22687
	for <rddp@ietf.org>; Mon, 29 Jul 2002 22:06:48 -0400 (EDT)
Received: from jurassic.eng.sun.com ([129.146.17.55])
	by nwkea-mail-1.sun.com (8.9.3+Sun/8.9.3) with ESMTP id TAA16486;
	Mon, 29 Jul 2002 19:07:26 -0700 (PDT)
Received: from taipei (taipei.Eng.Sun.COM [129.146.85.178])
	by jurassic.eng.sun.com (8.12.4+Sun/8.12.4) with SMTP id g6U27QXl732425;
	Mon, 29 Jul 2002 19:07:26 -0700 (PDT)
Message-Id: <200207300207.g6U27QXl732425@jurassic.eng.sun.com>
Date: Mon, 29 Jul 2002 19:07:18 -0700 (PDT)
From: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Reply-To: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Subject: Re: [rddp] comments on draft-black-rdma-concerns-00.txt 
To: caitlinb@rp.asomi.net
Cc: rddp@ietf.org
MIME-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
Content-MD5: ozp1i2JWlatwLngSGjyyyA==
X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.4.6_06 SunOS 5.8 sun4u sparc 
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org

>On 7/27/02, Bill Sommerfeld wrote:
>
>
>>The WG charter says:
>>
>> The working group will ensure that the resulting
>> technology will be secure and will not enable new attacks
>> on systems supporting RDDP.
>>
>>An overwrite race condition sounds to me like a "new
>>attack".
>
>This is a local interface issue, not a protocol issue.
>
>Fundamentally, the Data Source does not need, nor should
>it have, the complete semantics of the STag that is part
>of the buffer advertisement it received.

Yes and no. On one extreme the minimal semantic of the STag
the data source needs to know is simply its validity wrt the
given RDMA transaction where the tag is exposed. THe rest is
all left to the implementation. E.g. in an environment where
malicious attacks are possible, implementations must
invalidate/deregister STag at the end of every RDMA transaction.
In other environment and/or application paradigm where attacks
are not an issue, implementations may choose to defer tag
invalidation or reuse/remap tags in order to attain higher
performance. These different levels of protection can be
negotiated upfront.

But the above more liberal approach has the drawback of
putting too much risk and burden on the users of RDDP to "do
the right thing". A better approach might be to define
explicitly the scope of each STag and convey that as well
as the STag to the other side. The scope can be a single
RDMA transaction, or higher (such as a SCTP/TCP session).
Implementations must invalide a STag when the transport
state machine runs beyond the scope of the STag. This
approach may further complicate the RDMA protocol, but seems
to provide much more flexibility to ULPs.

Jerry

>
>The only knowledge it requires is the extent of permission
>given to it by the ULP. DAFS is probably typical of the
>rules most ULPs will establish:
>
>-- the client sends a request to an anonymous buffer that
>   contains a buffer advertisement and a request for some
>   work to be done using that buffer.
>   
>-- the server performs that work.
>
>-- the server sends a reply back to client, also to an 
>   anonymous buffer,  thereby terminating its rights to
>   the advertised buffer.
>
>The ULP Data Sink / Client MUST ensure that it has used
>local interfaces to enable RDDP layer memory access with
>sufficient rights to match the advertised buffers.
>
>There is no *protocol* justification for mandating that all
>Data Sinks MUST protect the rest of their memory. The
>protocol MUST NOT *prevent* them from doing so.
>
>Nothing in any of the iWARP/ROI/RDDP proposals has ever done
>anything to prevent more precise local protection of memory.
>
>
>
>
>
>Even if such local  mechanisms are available, it  would be
>inappropriate to make  them mandatory. There are common
>practices with RDMA Writes which are incompatible with
>deregistering the STag automatically:
>
>-- Enabling a single large region specifically  for the 
>    client, and then advertising buffers within that STag.
>    This avoids having to create and distribute STags for
>    each interaction. This model can be very attractive
>    when the peers have long lasting connections  and
>    many interactions, such as for an RDBMS.
>-- Enabling a large buffer that is filled on an ongoing
>    basis, but with each delivery being a notified event.
>    This can be useful for delivery a series of variable
>    sized data, especially for streaming media.
>    
>So ultimately, disabling of an STag is both a ULP and local
>interface question. The question is whether to mandate a
>specific, and potentially expensive behavior to close a
>"vunerability window" between the time the message is
>received that would have automatically disabled an STag and
>when the ULP would have responded to the notification and
>disabled the STag itself.
>
>But keep in mind that the same buffers were vunerable to
>unauthorized overwrites from the very same sources during
>the entire period *prior* to the completing message.
>
>Now there may be some applications where additional
>complexity on the NIC is justifiable if it reduces
>vunerabilty  in *ANY* way. They would probably be better
>served by sticking with non-RDDP messages with
>authenticating signatures to every update, but conceivably
>they could decide that automatic STag release was a valuable
>feature for them. However, mandating that *all* Data Sinks
>MUST exercise the *same* data protection policies is simply
>not called for.
> 
>
>_______________________________________________
>rddp mailing list
>rddp@ietf.org
>https://www1.ietf.org/mailman/listinfo/rddp


_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Mon Jul 29 22:37:43 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA23540
	for <rddp-archive@odin.ietf.org>; Mon, 29 Jul 2002 22:37:43 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id WAA14654
	for rddp-archive@odin.ietf.org; Mon, 29 Jul 2002 22:38:51 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id WAA14643;
	Mon, 29 Jul 2002 22:38:29 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id WAA14596
	for <rddp@optimus.ietf.org>; Mon, 29 Jul 2002 22:38:27 -0400 (EDT)
Received: from rp.asomi.net (rp.asomi.net [216.55.142.101])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id WAA23514
	for <rddp@ietf.org>; Mon, 29 Jul 2002 22:37:18 -0400 (EDT)
Received: from 192.168.0.2 (g4.asomi.net [192.168.0.2])
	by rp.asomi.net (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id g6U094Z06090;
	Mon, 29 Jul 2002 19:09:04 -0500
Date: Mon, 29 Jul 2002 21:38:22 -0500
From: Caitlin Bestler <caitlinb@rp.asomi.net>
Subject: Re: [rddp] comments on draft-black-rdma-concerns-00.txt 
To: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
cc: rddp@ietf.org
X-Priority: 3
In-Reply-To: <200207300207.g6U27QXl732425@jurassic.eng.sun.com>
Message-ID: <r01050300-1015-6A61FCC9A36511D6BFAA003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org
Content-Transfer-Encoding: 7bit

On 7/29/02, H.K. Jerry Chu wrote:

>>On 7/27/02, Bill Sommerfeld wrote:
>>
>>
>>>The WG charter says:
>>>
>>> The working group will ensure that the resulting
>>> technology will be secure and will not enable new
attacks
>>> on systems supporting RDDP.
>>>
>>>An overwrite race condition sounds to me like a "new
>>>attack".
>>
>>This is a local interface issue, not a protocol issue.
>>
>>Fundamentally, the Data Source does not need, nor should
>>it have, the complete semantics of the STag that is part
>>of the buffer advertisement it received.
>
>Yes and no. On one extreme the minimal semantic of the
>STag the data source needs to know is simply its validity
>wrt the given RDMA transaction where the tag is exposed.
>THe rest is all left to the implementation. E.g. in an
>environment where malicious attacks are possible,
>implementations must invalidate/deregister STag at the end
>of every RDMA transaction. In other environment and/or
>application paradigm where attacks are not an issue,
>implementations may choose to defer tag invalidation or
>reuse/remap tags in order to attain higher performance.
>These different levels of protection can be negotiated
>upfront.
>
>But the above more liberal approach has the drawback of
>putting too much risk and burden on the users of RDDP to
>"do the right thing". A better approach might be to define
>explicitly the scope of each STag and convey that as well
>as the STag to the other side. The scope can be a single
>RDMA transaction, or higher (such as a SCTP/TCP session).
>Implementations must invalide a STag when the transport
>state machine runs beyond the scope of the STag. This
>approach may further complicate the RDMA protocol, but
>seems to provide much more flexibility to ULPs.
>

The scope of each STag is explicitly defined, and it is
explicitly conveyed to the other side. But that can be done
by the ULP without complicating the RDDP layer.

Further, by keeping it a pure ULP issue, we preserve the
right of the Data Sink to tell the Data Source that the STag
is  only valid for a transaction, but then to be lazy about
deregistering it.

How can it enhance security to require the Data Sink to
publish the exact scope of its exposure? Suppose the Data
Sink's NIC has a limited number of available STags, and
therefore it must use a single STag for multiple buffer
advertisements? Of what benefit is it to inform the Data
Source?

IMHO, this is akin to posting "Speed Limit 55", and then
adding a sign stating "No officers available after 6:00 PM,
please be extra vigilant in monitoring your own speed."



Whether the same STag is used for other buffers, or even on
different connections, is not relevant to the Data Source.
The Data Source must be informed of what *it* is being given
permission to do with the STag -- but that is a ULP issue.
DAFS, for example, delivers buffer advertisements in
requests that are only valid until the reply is sent.

All of this can be handled strictly with local interfaces.
We can suggest local interfaces, but definitely should not
mandate them. InfiniBand, for example, only defines verbs
for those Channel Adapters that choose to classify
themselves as Host Channel Adapters. There is no attempt to
specificy the local interface for specialized executatiaon
environments such as embedded devices (they are Target
Channel Adapters, or TCAs). Any processing environment that
finds the resource models for HCAs can simply declare itself
to have a TCA instead. Any RDDP local interface needs a
similar escape clause.


_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Mon Jul 29 23:10:56 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA24159
	for <rddp-archive@odin.ietf.org>; Mon, 29 Jul 2002 23:10:56 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id XAA15908
	for rddp-archive@odin.ietf.org; Mon, 29 Jul 2002 23:12:02 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id XAA15891;
	Mon, 29 Jul 2002 23:11:24 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id XAA15856
	for <rddp@optimus.ietf.org>; Mon, 29 Jul 2002 23:11:22 -0400 (EDT)
Received: from patan.sun.com (patan.Sun.COM [192.18.98.43])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA24153
	for <rddp@ietf.org>; Mon, 29 Jul 2002 23:10:14 -0400 (EDT)
Received: from jurassic.eng.sun.com ([129.146.17.55])
	by patan.sun.com (8.9.3+Sun/8.9.3) with ESMTP id VAA19889;
	Mon, 29 Jul 2002 21:11:20 -0600 (MDT)
Received: from taipei (taipei.Eng.Sun.COM [129.146.85.178])
	by jurassic.eng.sun.com (8.12.4+Sun/8.12.4) with SMTP id g6U3BJXl737698;
	Mon, 29 Jul 2002 20:11:20 -0700 (PDT)
Message-Id: <200207300311.g6U3BJXl737698@jurassic.eng.sun.com>
Date: Mon, 29 Jul 2002 20:11:11 -0700 (PDT)
From: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Reply-To: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Subject: Re: [rddp] comments on draft-black-rdma-concerns-00.txt 
To: caitlinb@rp.asomi.net
Cc: rddp@ietf.org
MIME-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
Content-MD5: ipNPRNd/CpLb3UuaMI5ruw==
X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.4.6_06 SunOS 5.8 sun4u sparc 
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org

>>>>The WG charter says:
>>>>
>>>> The working group will ensure that the resulting
>>>> technology will be secure and will not enable new
>attacks
>>>> on systems supporting RDDP.
>>>>
>>>>An overwrite race condition sounds to me like a "new
>>>>attack".
>>>
>>>This is a local interface issue, not a protocol issue.
>>>
>>>Fundamentally, the Data Source does not need, nor should
>>>it have, the complete semantics of the STag that is part
>>>of the buffer advertisement it received.
>>
>>Yes and no. On one extreme the minimal semantic of the
>>STag the data source needs to know is simply its validity
>>wrt the given RDMA transaction where the tag is exposed.
>>THe rest is all left to the implementation. E.g. in an
>>environment where malicious attacks are possible,
>>implementations must invalidate/deregister STag at the end
>>of every RDMA transaction. In other environment and/or
>>application paradigm where attacks are not an issue,
>>implementations may choose to defer tag invalidation or
>>reuse/remap tags in order to attain higher performance.
>>These different levels of protection can be negotiated
>>upfront.
>>
>>But the above more liberal approach has the drawback of
>>putting too much risk and burden on the users of RDDP to
>>"do the right thing". A better approach might be to define
>>explicitly the scope of each STag and convey that as well
>>as the STag to the other side. The scope can be a single
>>RDMA transaction, or higher (such as a SCTP/TCP session).
>>Implementations must invalide a STag when the transport
>>state machine runs beyond the scope of the STag. This
>>approach may further complicate the RDMA protocol, but
>>seems to provide much more flexibility to ULPs.
>>
>
>The scope of each STag is explicitly defined, and it is
>explicitly conveyed to the other side. But that can be done
>by the ULP without complicating the RDDP layer.

If the scope of STag is defined by a ULP, how would a RNIC know
when to invalidate (i.e. deregister) a STag without understanding
the ULP? (Not having followed discussions in iWARP I may have
missed something here.)

>
>Further, by keeping it a pure ULP issue, we preserve the
>right of the Data Sink to tell the Data Source that the STag
>is  only valid for a transaction, but then to be lazy about
>deregistering it.
>
>How can it enhance security to require the Data Sink to
>publish the exact scope of its exposure? Suppose the Data
>Sink's NIC has a limited number of available STags, and
>therefore it must use a single STag for multiple buffer
>advertisements? Of what benefit is it to inform the Data
>Source?

No benefit. Defining the scope explicitly is more for the local
implementation to comply with. E.g. the local implementation
MUST enforce the EXACT scope.

I guess my (and a number of others') concern is to minimize
security exposure, which is stated as one of the goals in the
WG charter. How that goal can be achieved is up for debate.

>
>IMHO, this is akin to posting "Speed Limit 55", and then
>adding a sign stating "No officers available after 6:00 PM,
>please be extra vigilant in monitoring your own speed."
>
>
>
>Whether the same STag is used for other buffers, or even on
>different connections, is not relevant to the Data Source.
>The Data Source must be informed of what *it* is being given
>permission to do with the STag -- but that is a ULP issue.
>DAFS, for example, delivers buffer advertisements in
>requests that are only valid until the reply is sent.
>
>All of this can be handled strictly with local interfaces.
>We can suggest local interfaces, but definitely should not
>mandate them. InfiniBand, for example, only defines verbs
>for those Channel Adapters that choose to classify
>themselves as Host Channel Adapters. There is no attempt to
>specificy the local interface for specialized executatiaon
>environments such as embedded devices (they are Target
>Channel Adapters, or TCAs). Any processing environment that
>finds the resource models for HCAs can simply declare itself
>to have a TCA instead. Any RDDP local interface needs a
>similar escape clause.

Ok.

Jerry

>


_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Tue Jul 30 00:03:16 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA24892
	for <rddp-archive@odin.ietf.org>; Tue, 30 Jul 2002 00:03:16 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id AAA18062
	for rddp-archive@odin.ietf.org; Tue, 30 Jul 2002 00:04:23 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id AAA18044;
	Tue, 30 Jul 2002 00:04:06 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id AAA18013
	for <rddp@optimus.ietf.org>; Tue, 30 Jul 2002 00:04:04 -0400 (EDT)
Received: from rp.asomi.net (rp.asomi.net [216.55.142.101])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id AAA24888
	for <rddp@ietf.org>; Tue, 30 Jul 2002 00:02:56 -0400 (EDT)
Received: from 192.168.0.2 (g4.asomi.net [192.168.0.2])
	by rp.asomi.net (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id g6U1YeZ06320;
	Mon, 29 Jul 2002 20:34:40 -0500
Date: Mon, 29 Jul 2002 23:03:58 -0500
From: Caitlin Bestler <caitlinb@rp.asomi.net>
Subject: Re: [rddp] comments on draft-black-rdma-concerns-00.txt 
To: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
cc: rddp@ietf.org
X-Priority: 3
In-Reply-To: <200207300311.g6U3BJXl737698@jurassic.eng.sun.com>
Message-ID: <r01050300-1015-5FC6358EA37111D694A5003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org
Content-Transfer-Encoding: 7bit

On 7/29/02, H.K. Jerry Chu wrote:


>>
>>The scope of each STag is explicitly defined, and it is
>>explicitly conveyed to the other side. But that can be
>>done by the ULP without complicating the RDDP layer.
>
>If the scope of STag is defined by a ULP, how would a RNIC
>know when to invalidate (i.e. deregister) a STag without
>understanding the ULP? (Not having followed discussions in
>iWARP I may have missed something here.)
>

Exactly the point.

The Data Sink ULP is responsible for invalidating the STag.

It MAY/SHOULD have a local interface to allow it  do to so
without having to turn around and place a new asynchronous
request back to the NIC.

But the RDDP layer CANNOT know when to invalidate the STag
unless told by the ULP.

This is especially true on the Data Source end, where the
RDDP layer does not even know what STags have been delivered
to the ULP. Only he Data Sink RDDP layer knows about the
STag,  it is a meaningless opaque tag to the Data Source
RDDP layer that is dealt with on a per-RDDP-put basis.


The Data Source ULP *can* know when it is done using an
STag, but there is no need for it to convey this information
any time earlier than its sending the reply to an untagged
buffer, After all, as far as the Data Sink ULP is concerned,
the advertised buffer is indeterminant until it receives
notification of the reply message.



The concern here is not truly security, but completeness.
The buffer is exposed for a long period of time. The period
that an automatic release would close is statistically
insignificant.

The valid concern is that notification is supposed to mean
that prior RDDP puts have completed. To maximize the meaning
of that, it would be theoretically desirable to know that
the buffer could be closed at the same time as the
notification.

The notification says all writes have been placed, automatic
STag release would add *and no others*. (Assuming of course
that the STag was the only reference to the target memory).

That goal can be accomplished strictly with a local
interface. The problem going beyound a MAY/SHOULD
description of the local interface is that it is next to
impossible to describe an STag's meaning *locally* without
making fundamental presumptions about the local hosts memory
addressing.



_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Tue Jul 30 03:20:36 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA07020
	for <rddp-archive@odin.ietf.org>; Tue, 30 Jul 2002 03:20:36 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id DAA04278
	for rddp-archive@odin.ietf.org; Tue, 30 Jul 2002 03:21:43 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id DAA04249;
	Tue, 30 Jul 2002 03:20:36 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id DAA04217
	for <rddp@optimus.ietf.org>; Tue, 30 Jul 2002 03:20:34 -0400 (EDT)
Received: from portal.hamachi.org (portal.hamachi.org [140.239.227.17])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA07005
	for <rddp@ietf.org>; Tue, 30 Jul 2002 03:19:27 -0400 (EDT)
Received: from syn.hamachi.org (localhost [127.0.0.1])
	by portal.hamachi.org (Postfix) with ESMTP
	id 8543F17926; Tue, 30 Jul 2002 03:17:43 -0400 (EDT)
Received: from syn.hamachi.org (sommerfeld@localhost)
	by syn.hamachi.org (8.11.6/8.8.8) with ESMTP id g6U7H5x11271;
	Tue, 30 Jul 2002 03:17:05 -0400 (EDT)
Message-Id: <200207300717.g6U7H5x11271@syn.hamachi.org>
From: Bill Sommerfeld <sommerfeld@orchard.arlington.ma.us>
To: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Cc: caitlinb@rp.asomi.net, rddp@ietf.org
Subject: Re: [rddp] comments on draft-black-rdma-concerns-00.txt 
In-Reply-To: Message from "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com> 
   of "Mon, 29 Jul 2002 19:07:18 PDT." <200207300207.g6U27QXl732425@jurassic.eng.sun.com> 
Reply-To: sommerfeld@orchard.arlington.ma.us
Date: Tue, 30 Jul 2002 03:17:04 -0400
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org

> In other environment and/or application paradigm where attacks
> are not an issue, implementations may choose to defer tag
> invalidation or reuse/remap tags in order to attain higher
> performance. These different levels of protection can be
> negotiated upfront.

So, in thinking further about this, maliciousness is not actually
required for overwrites to be problematic.

Next hypothetical:

Remember that IP networks may duplicate packets.  

1) application receives packet using DDP
2) application gets notified that data is available.
3) application believes it is in a safe and trusting environment, so
RDDP mapping remains valid (neither explicit invalidation nor implicit
invalidate-on-rx takes place)
4) application modifies received data in place
5) duplicate packet arrives, overwriting buffer.
6) application does some processing which depends on data not
being overwritten after (4)

						- Bill

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Tue Jul 30 09:59:52 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA18602
	for <rddp-archive@odin.ietf.org>; Tue, 30 Jul 2002 09:59:52 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id KAA27338
	for rddp-archive@odin.ietf.org; Tue, 30 Jul 2002 10:00:59 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id KAA27063;
	Tue, 30 Jul 2002 10:00:20 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id KAA27030
	for <rddp@optimus.ietf.org>; Tue, 30 Jul 2002 10:00:18 -0400 (EDT)
Received: from ztxmail05.ztx.compaq.com (ztxmail05.ztx.compaq.com [161.114.1.209])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA18562
	for <rddp@ietf.org>; Tue, 30 Jul 2002 09:59:11 -0400 (EDT)
Received: from cceexg13.americas.cpqcorp.net (cceexg13.americas.cpqcorp.net [16.110.250.119])
	by ztxmail05.ztx.compaq.com (Postfix) with ESMTP
	id 113DE3C5D; Tue, 30 Jul 2002 08:59:49 -0500 (CDT)
Received: from cceexc18.americas.cpqcorp.net ([16.110.250.64]) by cceexg13.americas.cpqcorp.net with Microsoft SMTPSVC(5.0.2195.2966);
	 Tue, 30 Jul 2002 08:59:48 -0500
x-mimeole: Produced By Microsoft Exchange V6.0.5762.3
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: RE: [rddp] comments on draft-black-rdma-concerns-00.txt 
Date: Tue, 30 Jul 2002 08:59:47 -0500
Message-ID: <4D027986353D1341ADA4F2A4F8A1707623FE05D3@cceexc18.americas.cpqcorp.net>
Thread-Topic: [rddp] comments on draft-black-rdma-concerns-00.txt 
Thread-Index: AcI3maaUap/OU84TTsiQ3ocex6dCTwALStFQ
From: "Culley, Paul" <Paul.Culley@hp.com>
To: <sommerfeld@orchard.arlington.ma.us>,
        "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Cc: <caitlinb@rp.asomi.net>, <rddp@ietf.org>
X-OriginalArrivalTime: 30 Jul 2002 13:59:48.0780 (UTC) FILETIME=[5E31A6C0:01C237D1]
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by optimus.ietf.org id KAA27031
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org
Content-Transfer-Encoding: 8bit

It seems that the case you bring up should not be allowed in a good implementation.  Once the data is "delivered" to the application (step 2, allowing step 4 to take place), the implementation should not allow duplicate packets to have any effect.  I believe that reliable transports like TCP and SCTP have this semantic, it is very unclear to me that an unreliable transport should be used by an application that wants to treat the data as you suggest.

Paul Culley
HP Sr. Fellow
281-514-5543


-----Original Message-----
From: Bill Sommerfeld [mailto:sommerfeld@orchard.arlington.ma.us]
Sent: Tuesday, July 30, 2002 2:17 AM
To: H.K. Jerry Chu
Cc: caitlinb@rp.asomi.net; rddp@ietf.org
Subject: Re: [rddp] comments on draft-black-rdma-concerns-00.txt 


> In other environment and/or application paradigm where attacks
> are not an issue, implementations may choose to defer tag
> invalidation or reuse/remap tags in order to attain higher
> performance. These different levels of protection can be
> negotiated upfront.

So, in thinking further about this, maliciousness is not actually
required for overwrites to be problematic.

Next hypothetical:

Remember that IP networks may duplicate packets.  

1) application receives packet using DDP
2) application gets notified that data is available.
3) application believes it is in a safe and trusting environment, so
RDDP mapping remains valid (neither explicit invalidation nor implicit
invalidate-on-rx takes place)
4) application modifies received data in place
5) duplicate packet arrives, overwriting buffer.
6) application does some processing which depends on data not
being overwritten after (4)

						- Bill

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Tue Jul 30 10:22:09 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA19915
	for <rddp-archive@odin.ietf.org>; Tue, 30 Jul 2002 10:22:09 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id KAA28876
	for rddp-archive@odin.ietf.org; Tue, 30 Jul 2002 10:23:16 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id KAA28816;
	Tue, 30 Jul 2002 10:22:22 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id KAA28781
	for <rddp@optimus.ietf.org>; Tue, 30 Jul 2002 10:22:20 -0400 (EDT)
Received: from rp.asomi.net (rp.asomi.net [216.55.142.101])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA19869
	for <rddp@ietf.org>; Tue, 30 Jul 2002 10:21:12 -0400 (EDT)
Received: from 192.168.0.2 (g4.asomi.net [192.168.0.2])
	by rp.asomi.net (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id g6UBqtZ08101;
	Tue, 30 Jul 2002 06:52:55 -0500
Date: Tue, 30 Jul 2002 09:22:11 -0500
From: Caitlin Bestler <caitlinb@rp.asomi.net>
Subject: RE: [rddp] comments on draft-black-rdma-concerns-00.txt 
To: "Culley, Paul" <Paul.Culley@hp.com>
cc: sommerfeld@orchard.arlington.ma.us,
        "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>, rddp@ietf.org
X-Priority: 3
In-Reply-To: <4D027986353D1341ADA4F2A4F8A1707623FE05D3@cceexc18.americas.cpqcorp.net>
Message-ID: <r01050300-1015-BCD90864A3C711D6AA90003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org
Content-Transfer-Encoding: 7bit

On 7/30/02, Culley, Paul wrote:

>It seems that the case you bring up should not be allowed
>in a good implementation. Once the data is "delivered" to
>the application (step 2, allowing step 4 to take place),
>the implementation should not allow duplicate packets to
>have any effect.  I believe that reliable transports like
>TCP and SCTP have this semantic, it is very unclear to me
>that an unreliable transport should be used by an
>application that wants to treat the data as you suggest.
>
>Paul Culley
>HP Sr. Fellow
>281-514-5543
>

I would upgrade that from "good implementation" to "valid
implementation". IMHO, RDDP is only valid over two delivery
semantics: exactly-once deliver (reliable), and at-most-once
delivery (unreliable).

I believe prior disucssions have clearly established that
exactly-once delivery is a requirement of the underlying
tranwsport.

The semantics of unreliable delivery have been debated,
especially between Steph and myself. I favor a
"transactional exchange" model where a sequence of RDMA
Writes and the terminating SEND are reported to the Data
Sink ULP as a success only if the entire set is received.
Steph has advocated more of a "hands off" approach where the
transport does as little as possible in unreliable mode, and
expects the ULP to sort everything out (presumably on the
theory that it was the ULP was the troublemaker that
insisted on using an Unreliable mode in the first place). 

In either case, without at-most-once delivery semantics
there is no way to provide meaningful ordering of tagged
puts and notifiable untagged puts. Inherent to the entire
RDMA paradigm is that completion of an anonymous receive
implies reception of prior RDMA writes.

IMHO, if each RDMA write received must be reported to the
ULP then you are no longer truly RDMA. The difference for
"unreliable" RDDP is that exchange N can succeed even if
exchange N-1 failed.





Caitlin Bestler
http://www.asomi.com/CaitlinBestler.html


_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Tue Jul 30 10:48:42 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA21703
	for <rddp-archive@odin.ietf.org>; Tue, 30 Jul 2002 10:48:42 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id KAA00993
	for rddp-archive@odin.ietf.org; Tue, 30 Jul 2002 10:49:49 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id KAA00950;
	Tue, 30 Jul 2002 10:49:18 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id KAA00917
	for <rddp@optimus.ietf.org>; Tue, 30 Jul 2002 10:49:16 -0400 (EDT)
Received: from zcamail04.zca.compaq.com (zcamail04.zca.compaq.com [161.114.32.104])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA21683
	for <rddp@ietf.org>; Tue, 30 Jul 2002 10:48:08 -0400 (EDT)
Received: from cacexg11.americas.cpqcorp.net (cacexg11.americas.cpqcorp.net [16.105.250.94])
	by zcamail04.zca.compaq.com (Postfix) with ESMTP
	id 1A3DD1ABF; Tue, 30 Jul 2002 07:48:46 -0700 (PDT)
Received: from txnexc01.americas.cpqcorp.net ([16.74.7.244]) by cacexg11.americas.cpqcorp.net with Microsoft SMTPSVC(5.0.2195.2966);
	 Tue, 30 Jul 2002 07:48:45 -0700
X-MimeOLE: Produced By Microsoft Exchange V6.0.5762.3
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: RE: [rddp] comments on draft-black-rdma-concerns-00.txt 
Date: Tue, 30 Jul 2002 09:48:44 -0500
Message-ID: <717252EC3E37AE4392E2614EA24E9F2B02AC4F3B@txnexc01.americas.cpqcorp.net>
Thread-Topic: [rddp] comments on draft-black-rdma-concerns-00.txt 
Thread-Index: AcI3maaUap/OU84TTsiQ3ocex6dCTwALStFQAAQSa3A=
From: "Teisberg, Robert" <robert.teisberg@hp.com>
To: "Culley, Paul" <Paul.Culley@hp.com>, <sommerfeld@orchard.arlington.ma.us>,
        "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Cc: <caitlinb@rp.asomi.net>, <rddp@ietf.org>
X-OriginalArrivalTime: 30 Jul 2002 14:48:45.0751 (UTC) FILETIME=[34C3DC70:01C237D8]
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by optimus.ietf.org id KAA00918
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org
Content-Transfer-Encoding: 8bit

Exactly.  Looking at it from the transport's perspective, by the time
the data is delivered, the transport has advanced its ack pointer.  The
hypothetical duplicate packet is outside the receive window and will
be discarded by the transport.  (If you are remembering my spirited
discussion with Doug Otis regarding out-of-order placement, please
also remember that I advocated an "as-if" rule requiring RDDP to make
out-of-order placement undetectable by a properly written ULP.  Placement
of an out-of-window packet would violate this.)

Unreliable transports create an entirely different set of issues, as
Caitlin points out.

> -----Original Message-----
> From: Culley, Paul 
> Sent: Tuesday, July 30, 2002 9:00 AM
> To: sommerfeld@orchard.arlington.ma.us; H.K. Jerry Chu
> Cc: caitlinb@rp.asomi.net; rddp@ietf.org
> Subject: RE: [rddp] comments on draft-black-rdma-concerns-00.txt 
> 
> 
> It seems that the case you bring up should not be allowed in 
> a good implementation.  Once the data is "delivered" to the 
> application (step 2, allowing step 4 to take place), the 
> implementation should not allow duplicate packets to have any 
> effect.  I believe that reliable transports like TCP and SCTP 
> have this semantic, it is very unclear to me that an 
> unreliable transport should be used by an application that 
> wants to treat the data as you suggest.
> 
> Paul Culley
> HP Sr. Fellow
> 281-514-5543
> 
> 
> -----Original Message-----
> From: Bill Sommerfeld [mailto:sommerfeld@orchard.arlington.ma.us]
> Sent: Tuesday, July 30, 2002 2:17 AM
> To: H.K. Jerry Chu
> Cc: caitlinb@rp.asomi.net; rddp@ietf.org
> Subject: Re: [rddp] comments on draft-black-rdma-concerns-00.txt 
> 
> 
> > In other environment and/or application paradigm where attacks
> > are not an issue, implementations may choose to defer tag
> > invalidation or reuse/remap tags in order to attain higher
> > performance. These different levels of protection can be
> > negotiated upfront.
> 
> So, in thinking further about this, maliciousness is not actually
> required for overwrites to be problematic.
> 
> Next hypothetical:
> 
> Remember that IP networks may duplicate packets.  
> 
> 1) application receives packet using DDP
> 2) application gets notified that data is available.
> 3) application believes it is in a safe and trusting environment, so
> RDDP mapping remains valid (neither explicit invalidation nor implicit
> invalidate-on-rx takes place)
> 4) application modifies received data in place
> 5) duplicate packet arrives, overwriting buffer.
> 6) application does some processing which depends on data not
> being overwritten after (4)
> 
> 						- Bill
> 
> _______________________________________________
> rddp mailing list
> rddp@ietf.org
> https://www1.ietf.org/mailman/listinfo/rddp
> 
> _______________________________________________
> rddp mailing list
> rddp@ietf.org
> https://www1.ietf.org/mailman/listinfo/rddp
> 

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Tue Jul 30 11:09:18 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA22746
	for <rddp-archive@odin.ietf.org>; Tue, 30 Jul 2002 11:09:18 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id LAA02345
	for rddp-archive@odin.ietf.org; Tue, 30 Jul 2002 11:10:26 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id LAA02240;
	Tue, 30 Jul 2002 11:08:09 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id LAA02205
	for <rddp@optimus.ietf.org>; Tue, 30 Jul 2002 11:08:07 -0400 (EDT)
Received: from rp.asomi.net (rp.asomi.net [216.55.142.101])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA22630
	for <rddp@ietf.org>; Tue, 30 Jul 2002 11:06:58 -0400 (EDT)
Received: from 192.168.0.2 (g4.asomi.net [192.168.0.2])
	by rp.asomi.net (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id g6UCchZ08233;
	Tue, 30 Jul 2002 07:38:43 -0500
Date: Tue, 30 Jul 2002 10:08:00 -0500
From: Caitlin Bestler <caitlinb@rp.asomi.net>
Subject: Re: [rddp] comments on draft-black-rdma-concerns-00.txt 
To: sommerfeld@orchard.arlington.ma.us
cc: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>, rddp@ietf.org
X-Priority: 3
In-Reply-To: <200207300717.g6U7H5x11271@syn.hamachi.org>
Message-ID: <r01050300-1015-233EB099A3CE11D6AA90003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org
Content-Transfer-Encoding: 7bit

On 7/30/02, Bill Sommerfeld wrote:

>> In other environment and/or application paradigm where
>> attacks are not an issue, implementations may choose to
>> defer tag invalidation or reuse/remap tags in order to
>> attain higher performance. These different levels of
>> protection can be negotiated upfront.
>
>So, in thinking further about this, maliciousness is not
>actually required for overwrites to be problematic.
>
>Next hypothetical:
>
>Remember that IP networks may duplicate packets.
>
>1) application receives packet using DDP 2) application
>gets notified that data is available. 3) application
>believes it is in a safe and trusting environment, so RDDP
>mapping remains valid (neither explicit invalidation nor
>implicit invalidate-on-rx takes place) 4) application
>modifies received data in place 5) duplicate packet
>arrives, overwriting buffer. 6) application does some
>processing which depends on data not being overwritten
>after (4)
>

Paul Culley's reply already dealt with the immediate
solution to this problem.

A more applicable problem, which ironically demonstrates the
limitations to automatic STag deregistration, deals more
with the classic problem of stale pointers.

The intended sequence is as follows...


SEND-request A with buffer STag-A,Address-A,Len-A
SEND-request B with buffer STag-B,Address-B,Len-B

        RDMA-Write to portion of STag-A
        RDMA-Write to portion of STag-A
        SEND-reply A
        RDMA-Write to portion of STag-B
        RDMA-Write to portion of STag-B
        RDMA-Write to portion of STag-B
        SEND-reply B
        
        
Only the respondent code has a bug (gasp), and accidentally
uses the old remote pointer in processing the first portion
of the second command. This would be as follows...
        

        RDMA-Write to portion of STag-A
        RDMA-Write to portion of STag-A
        SEND-reply A
        RDMA-Write to portion of STag-A <<--- BUG!!!!
        RDMA-Write to portion of STag-B
        RDMA-Write to portion of STag-B
        SEND-reply B
        
        
At first glance this seems like the poster-boy example
justifying automatic tag release on receive. After all, the
erroneous RDMA Write will almost certainly be received more
quickly than the ULP will be able to deregister STag A.

But in fact it shows that automatic deregistration is
futile. Consider the following sequence:


        RDMA-Write to portion of STag-A
        RDMA-Write to portion of STag-A
        << lost message >> SEND-reply A
        RDMA-Write to portion of STag-A <<--- BUG!!!!
        RDMA-Write to portion of STag-B
        RDMA-Write to portion of STag-B
        SEND-reply B    
        << retransmit>> SEND-reply A
        
There is no way to determine that the erroneous RDMA-Write
to STag-A is invalid. The missing message could have been an
RMDA Write, rather than a SEND.


The bottom line is that upon notification of the SEND, the
only thing the ULP knows is that the priorr RDMA Writes have
completed. It cannot know that there have been no
*additional* writes.

[Aside:  To prevent that you must add a "at-once-once" rule
for each advertised buffer (each byte within the advertised
buffer will be delivered at most once). I developed one
in-chassis fabric that used exactly such a rule, and there
are major advantages to it. However it requires that disk
drives complete their never have sent a complete packet with
content that has not completedits media-level CRC check.
Internaly we thought this was a valid trade-off. The one
time I mentioned this approach at an iWARP meeting every
iSCSI vendor in ear-reach disagreed. In any event the rule
would require special hand-shaking to disable it for
circular buffers and publishing of statistical data.]

Sticking with RDMA Writes as they have been implemented to
date, the Data Sink ULP must deregister the STag itself
after it receives notification of the reply. Only THEN does
it know that the buffer is stable, whereupon it can validate
it via whatever ULP specific means deemed appropriate.

It is NICE if the STag deregistration can be automatically
applied at the earliest possible moment. But it does not
close any security holes. The buffers are exposed until the
trailing SEND *and* the deregisration have completed.
Consolidating those two completions is a *local*
optimization, it has no interoperabilty implications.

It is A REALLY GOOD IDEA to use STags exactly once, and it
is also A REALLY GOOD IDEA to deregister them as prompty as
possible. It is also A REALLY GOOD IDEA to have the fastest
processor possible and a few spare GB of DDRAM for your host
processor. Redundant power supplies are nice too.

The point is that how many STags are available, how quickly
they must be reused and how efficiently they can be reused
are all economic tradeoffs. The market should be allowed to
determine those answers, not the IETF.






Caitlin Bestler
http://www.asomi.com/CaitlinBestler.html

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Tue Jul 30 21:28:38 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA15241
	for <rddp-archive@odin.ietf.org>; Tue, 30 Jul 2002 21:28:38 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id VAA09464
	for rddp-archive@odin.ietf.org; Tue, 30 Jul 2002 21:29:46 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id VAA09378;
	Tue, 30 Jul 2002 21:27:49 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id VAA09343
	for <rddp@optimus.ietf.org>; Tue, 30 Jul 2002 21:27:47 -0400 (EDT)
Received: from patan.sun.com (patan.Sun.COM [192.18.98.43])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id VAA15221
	for <rddp@ietf.org>; Tue, 30 Jul 2002 21:26:38 -0400 (EDT)
Received: from jurassic.eng.sun.com ([129.146.17.55])
	by patan.sun.com (8.9.3+Sun/8.9.3) with ESMTP id TAA27897;
	Tue, 30 Jul 2002 19:27:46 -0600 (MDT)
Received: from taipei (taipei.Eng.Sun.COM [129.146.85.178])
	by jurassic.eng.sun.com (8.12.4+Sun/8.12.4) with SMTP id g6V1RjXl963385;
	Tue, 30 Jul 2002 18:27:46 -0700 (PDT)
Message-Id: <200207310127.g6V1RjXl963385@jurassic.eng.sun.com>
Date: Tue, 30 Jul 2002 18:27:37 -0700 (PDT)
From: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Reply-To: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Subject: Re: [rddp] comments on draft-black-rdma-concerns-00.txt 
To: caitlinb@rp.asomi.net
Cc: rddp@ietf.org
MIME-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
Content-MD5: 6/tgXZLoj5tKymiLjjWfKQ==
X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.4.6_06 SunOS 5.8 sun4u sparc 
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org

>It is NICE if the STag deregistration can be automatically
>applied at the earliest possible moment. But it does not
>close any security holes. The buffers are exposed until the
>trailing SEND *and* the deregisration have completed.
>Consolidating those two completions is a *local*
>optimization, it has no interoperabilty implications.
>
>It is A REALLY GOOD IDEA to use STags exactly once, and it
>is also A REALLY GOOD IDEA to deregister them as prompty as
>possible. It is also A REALLY GOOD IDEA to have the fastest
>processor possible and a few spare GB of DDRAM for your host
>processor. Redundant power supplies are nice too.
>
>The point is that how many STags are available, how quickly
>they must be reused and how efficiently they can be reused
>are all economic tradeoffs. The market should be allowed to
>determine those answers, not the IETF.

It seems that the last defense here is a good selection of STags
to export. 32 bit STag seems too small for security purpose. In
the least one should analyze the implication on security and
recommend some algorithm for selecting STags that is better against
spoofing. E.g. a simple tag index into a page table will not be
random at all and be trivial to spoof. (But a randomized STag may
be hard to implement on NICs efficiently.) This is similar to
hijacking a TCP connection except that the latter still seems much
hard. One must guess the correct seq# and port # at the right time
and also manage to shut out the other end of the TCP connection...
But here once a valid STag is guessed no other protection is there.
On a busy system the working set of transient network buffers could
be in many megabyte range making the probability of guessing a valid
32 bit STag >> 0. Even with IPsec, a malicious user on an
authenticated system or a buggy application may stomp on some
random place in remote memory causing data corruption that will be
impossible to debug.

Rgds,

Jerry

>
>
>
>
>
>
>Caitlin Bestler
>http://www.asomi.com/CaitlinBestler.html


_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Wed Jul 31 09:30:55 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA25997
	for <rddp-archive@odin.ietf.org>; Wed, 31 Jul 2002 09:30:55 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id JAA09344
	for rddp-archive@odin.ietf.org; Wed, 31 Jul 2002 09:32:02 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id JAA09277;
	Wed, 31 Jul 2002 09:31:34 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id JAA09242
	for <rddp@optimus.ietf.org>; Wed, 31 Jul 2002 09:31:31 -0400 (EDT)
Received: from ztxmail04.ztx.compaq.com (ztxmail04.ztx.compaq.com [161.114.1.208])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA25979
	for <rddp@ietf.org>; Wed, 31 Jul 2002 09:30:23 -0400 (EDT)
Received: from cceexg12.americas.cpqcorp.net (cceexg12.americas.cpqcorp.net [16.110.250.124])
	by ztxmail04.ztx.compaq.com (Postfix) with ESMTP
	id C41332474; Wed, 31 Jul 2002 08:31:01 -0500 (CDT)
Received: from cceexc18.americas.cpqcorp.net ([16.110.250.64]) by cceexg12.americas.cpqcorp.net with Microsoft SMTPSVC(5.0.2195.2966);
	 Wed, 31 Jul 2002 08:31:01 -0500
x-mimeole: Produced By Microsoft Exchange V6.0.5762.3
content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: RE: [rddp] comments on draft-black-rdma-concerns-00.txt 
Date: Wed, 31 Jul 2002 08:31:01 -0500
Message-ID: <4D027986353D1341ADA4F2A4F8A1707623FE061B@cceexc18.americas.cpqcorp.net>
Thread-Topic: [rddp] comments on draft-black-rdma-concerns-00.txt 
Thread-Index: AcI4MX2akzX+UgwHQn+eDJ03npnRUwAYCtxg
From: "Culley, Paul" <Paul.Culley@hp.com>
To: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>, <caitlinb@rp.asomi.net>
Cc: <rddp@ietf.org>
X-OriginalArrivalTime: 31 Jul 2002 13:31:01.0525 (UTC) FILETIME=[83153450:01C23896]
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by optimus.ietf.org id JAA09243
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org
Content-Transfer-Encoding: 8bit

I suspect that many folks are expecting STags to have a NIC wide scope, that is, any connection on the NIC can use any STag.  That definitely opens up guessing attacks.  While likely implementations MAY do that, a much better choice will be to limit the STag scope to an individual connection.

If connection scope is chosen, then an attacker must both "hijack the TCP connection" and guess the correct STag.  This is where careful use of IPSEC can help; it can make it nearly impossible to hijack the connection.

For an example of how the STag scope might be made variable, see VIA Ptags, or InfiniBand Protection Domains.  Note that I don't believe that these mechanisms belong in a wire level specification; but it might be instructive to provide a pointer to, or include a "verbs" level description of these in the security sections of the RDDP document.

Paul Culley
HP Sr. Fellow
281-514-5543


-----Original Message-----
From: H.K. Jerry Chu [mailto:Jerry.Chu@eng.sun.com]
Sent: Tuesday, July 30, 2002 8:28 PM
To: caitlinb@rp.asomi.net
Cc: rddp@ietf.org
Subject: Re: [rddp] comments on draft-black-rdma-concerns-00.txt 


>It is NICE if the STag deregistration can be automatically
>applied at the earliest possible moment. But it does not
>close any security holes. The buffers are exposed until the
>trailing SEND *and* the deregisration have completed.
>Consolidating those two completions is a *local*
>optimization, it has no interoperabilty implications.
>
>It is A REALLY GOOD IDEA to use STags exactly once, and it
>is also A REALLY GOOD IDEA to deregister them as prompty as
>possible. It is also A REALLY GOOD IDEA to have the fastest
>processor possible and a few spare GB of DDRAM for your host
>processor. Redundant power supplies are nice too.
>
>The point is that how many STags are available, how quickly
>they must be reused and how efficiently they can be reused
>are all economic tradeoffs. The market should be allowed to
>determine those answers, not the IETF.

It seems that the last defense here is a good selection of STags
to export. 32 bit STag seems too small for security purpose. In
the least one should analyze the implication on security and
recommend some algorithm for selecting STags that is better against
spoofing. E.g. a simple tag index into a page table will not be
random at all and be trivial to spoof. (But a randomized STag may
be hard to implement on NICs efficiently.) This is similar to
hijacking a TCP connection except that the latter still seems much
hard. One must guess the correct seq# and port # at the right time
and also manage to shut out the other end of the TCP connection...
But here once a valid STag is guessed no other protection is there.
On a busy system the working set of transient network buffers could
be in many megabyte range making the probability of guessing a valid
32 bit STag >> 0. Even with IPsec, a malicious user on an
authenticated system or a buggy application may stomp on some
random place in remote memory causing data corruption that will be
impossible to debug.

Rgds,

Jerry

>
>
>
>
>
>
>Caitlin Bestler
>http://www.asomi.com/CaitlinBestler.html


_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Wed Jul 31 11:41:04 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA03000
	for <rddp-archive@odin.ietf.org>; Wed, 31 Jul 2002 11:41:04 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id LAA16656
	for rddp-archive@odin.ietf.org; Wed, 31 Jul 2002 11:42:12 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id LAA16626;
	Wed, 31 Jul 2002 11:41:42 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id LAA16593
	for <rddp@optimus.ietf.org>; Wed, 31 Jul 2002 11:41:40 -0400 (EDT)
Received: from rp.asomi.net (rp.asomi.net [216.55.142.101])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id LAA02896
	for <rddp@ietf.org>; Wed, 31 Jul 2002 11:40:30 -0400 (EDT)
Received: from 192.168.0.2 (g4.asomi.net [192.168.0.2])
	by rp.asomi.net (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id g6VDCIZ12288;
	Wed, 31 Jul 2002 08:12:19 -0500
Date: Wed, 31 Jul 2002 10:41:31 -0500
From: Caitlin Bestler <caitlinb@rp.asomi.net>
Subject: RE: [rddp] comments on draft-black-rdma-concerns-00.txt 
To: "Culley, Paul" <Paul.Culley@hp.com>
cc: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>, rddp@ietf.org
X-Priority: 3
In-Reply-To: <4D027986353D1341ADA4F2A4F8A1707623FE061B@cceexc18.americas.cpqcorp.net>
Message-ID: <r01050300-1015-FC8F748FA49B11D69E1C003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org
Content-Transfer-Encoding: 7bit

On 7/31/02, Culley, Paul wrote:

>I suspect that many folks are expecting STags to have a
>NIC wide scope, that is, any connection on the NIC can use
>any STag.  That definitely opens up guessing attacks.
>While likely implementations MAY do that, a much better
>choice will be to limit the STag scope to an individual
>connection.
>

The local interface SHOULD allow the ULP to control which
connections an STag will be valid on.

With my application developer hat on, I would like to make
that a MUST. As a NIC developer I would strongly resist
anything more complex than InfiniBand Protection Domains.


This needs to be combined with:

    Any use of an STag on a connection other than the one
    on which it was delivered MUST be explicitly authorized
    by the Upper Layer Protocol.
    
In other words, you cannot assume the STag is valid anywhere
other than the obvious connection except as agreed upon in
advance in a ULP specific fashion.

An STag being valid on exactly one connection is an
excellent default. However, in terms of requirements, it is
important to recognize that there are some major exceptions:

- An STag may be delivered on a primary connection and then
used on an auxiliary connection. DAFS supplies buffer
advertisements on the primary connection to be used on the
RDMA Read channel. While it is possible to plan the STag
reference for the connection it will be used upon, it would
be very error prone to do so.

-- An client connected to each member of a server cluster
may deliver a request with an STag to one member of the
server cluster, which could then pass the work to the
cluster member most capable of fulfillnig the request.


>
>For an example of how the STag scope might be made
>variable, see VIA Ptags, or InfiniBand Protection Domains.
> Note that I don't believe that these mechanisms belong in
>a wire level specification; but it might be instructive to
>provide a pointer to, or include a "verbs" level
>description of these in the security sections of the RDDP
>document.
>

Protection Domains are a very flexible tool, which is why
the DAT Collaborative basically adopted that model for its
model of transport independent RDMA.

A Protection Domain can be NIC wide, one connection or
anythign in-between.


One thing missing from the InfiniBand model is the ability
to create an STag that is valid across multiple NICs. This
can be easily addressed by adding the following:

    The local interface SHOULD allow the ULP to request
    a specific STag value.
    
The ULP would still have to map each NIC to the same memory,
but that is strictly a local interface.










Caitlin Bestler
http://www.asomi.com/CaitlinBestler.html

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Wed Jul 31 12:17:45 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA04982
	for <rddp-archive@odin.ietf.org>; Wed, 31 Jul 2002 12:17:45 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id MAA19281
	for rddp-archive@odin.ietf.org; Wed, 31 Jul 2002 12:18:52 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id MAA19263;
	Wed, 31 Jul 2002 12:18:31 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id MAA19228
	for <rddp@optimus.ietf.org>; Wed, 31 Jul 2002 12:18:29 -0400 (EDT)
Received: from rp.asomi.net (rp.asomi.net [216.55.142.101])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA04958
	for <rddp@ietf.org>; Wed, 31 Jul 2002 12:17:21 -0400 (EDT)
Received: from 192.168.0.2 (g4.asomi.net [192.168.0.2])
	by rp.asomi.net (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id g6VDnEZ12449
	for <rddp@ietf.org>; Wed, 31 Jul 2002 08:49:14 -0500
Date: Wed, 31 Jul 2002 11:18:27 -0500
From: Caitlin Bestler <caitlinb@rp.asomi.net>
Subject: RE: [rddp] comments on draft-black-rdma-concerns-00.txt 
To: rddp@ietf.org
X-Priority: 3
In-Reply-To: <r01050300-1015-FC8F748FA49B11D69E1C003065D48EE0@[192.168.0.2]>
Message-ID: <r01050300-1015-25432C14A4A111D69E1C003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org
Content-Transfer-Encoding: 7bit

On 7/31/02, Caitlin Bestler wrote:


>This needs to be combined with:
>
>    Any use of an STag on a connection other than the one
>    on which it was delivered MUST be explicitly authorized
>    by the Upper Layer Protocol.
>    


Very shortly after hitting 'send' a more concise wording hit
me over the head:

    An exported STag has ONLY the meaning conveyed by its
    exporter. There are no valid presumptions about the
    scope or duration of its validity beyond those supplied
    through the ULP used to convey the STag.

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Wed Jul 31 15:47:37 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA15341
	for <rddp-archive@odin.ietf.org>; Wed, 31 Jul 2002 15:47:37 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id PAA05674
	for rddp-archive@odin.ietf.org; Wed, 31 Jul 2002 15:48:46 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA05639;
	Wed, 31 Jul 2002 15:48:01 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id PAA05608
	for <rddp@optimus.ietf.org>; Wed, 31 Jul 2002 15:48:00 -0400 (EDT)
Received: from rp.asomi.net (rp.asomi.net [216.55.142.101])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA15316
	for <rddp@ietf.org>; Wed, 31 Jul 2002 15:46:49 -0400 (EDT)
Received: from 192.168.0.2 (g4.asomi.net [192.168.0.2])
	by rp.asomi.net (8.11.3/8.11.2/SuSE Linux 8.11.1-0.5) with ESMTP id g6VHIfZ13109;
	Wed, 31 Jul 2002 12:18:41 -0500
Date: Wed, 31 Jul 2002 14:47:54 -0500
From: Caitlin Bestler <caitlinb@rp.asomi.net>
To: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
cc: rddp@ietf.org
X-Priority: 3
In-Reply-To: <200207310127.g6V1RjXl963385@jurassic.eng.sun.com>
Message-ID: <r01050300-1015-67871A74A4BE11D69E1C003065D48EE0@[192.168.0.2]>
MIME-Version: 1.0
Content-Type: text/plain; Charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Mailer: Mailsmith 1.5.3 (Blindsider)
Content-Transfer-Encoding: 7bit
Subject: [rddp] Re: STag size (was rdma-concerns)
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org
Content-Transfer-Encoding: 7bit

On 7/30/02, H.K. Jerry Chu wrote:


>
>It seems that the last defense here is a good selection of
>STags to export. 32 bit STag seems too small for security
>purpose. In the least one should analyze the implication
>on security and recommend some algorithm for selecting
>STags that is better against spoofing. E.g. a simple tag
>index into a page table will not be random at all and be
>trivial to spoof. (But a randomized STag may be hard to
>implement on NICs efficiently.) This is similar to
>hijacking a TCP connection except that the latter still
>seems much hard. One must guess the correct seq# and port
># at the right time and also manage to shut out the other
>end of the TCP connection... But here once a valid STag is
>guessed no other protection is there. On a busy system the
>working set of transient network buffers could be in many
>megabyte range making the probability of guessing a valid
>32 bit STag >> 0. Even with IPsec, a malicious user on an
>authenticated system or a buggy application may stomp on
>some random place in remote memory causing data corruption
>that will be impossible to debug.
>
>Rgds,
>


If RDDP is carried over a secure transport then the density
of STags is not relevant. Authentication is provided by the
transport itself.

If RDDP is not carried over a secure transport the size of
the STag is almost irrelevant because it is easily observed,
and almost always used multiple times. It is just as easy to
observe 64 bits, or even 256 bits, as 32 bits.

Providing a valid hamming distance is desirable to minimize
the risk of unintentional collisions.  But, I believe that
32 bits is more than adequate for that purpose.

The local interface can enhance that even more in two ways,
both of which are fully compatible with everything defined
about STags to date:

-- define STags only for the connections on which they are
   valid. 2**32 is an exteremely large number of STags for
   any single session.
   
-- Define an S-Key like sequence such that a difference STag
   must be computed for each put based upon a shared secret,
   but no single STag is used more than once. (Hey, we never
   said that ULP conveyance of the STag had to be *simple*).
   
   

I do not believe the added benefit of a larger STag
justifies introducing a different size than is used for
InfiniBand.

The first option can be translated to an IB like "verbs"
layer as allowing creation of "R-Protection Domains", i.e.
low cost Protection Domains that are different only for
checking remote connections. They access the same memory.
Essentially the same relationship as between R-Key and
L-Key.

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Wed Jul 31 19:46:26 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA21954
	for <rddp-archive@odin.ietf.org>; Wed, 31 Jul 2002 19:46:26 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id TAA18207
	for rddp-archive@odin.ietf.org; Wed, 31 Jul 2002 19:47:33 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA18179;
	Wed, 31 Jul 2002 19:47:14 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id TAA18147
	for <rddp@optimus.ietf.org>; Wed, 31 Jul 2002 19:47:13 -0400 (EDT)
Received: from mxic2.corp.emc.com (mxic2.us.dg.com [128.221.31.40])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA21942
	for <rddp@ietf.org>; Wed, 31 Jul 2002 19:46:05 -0400 (EDT)
From: Black_David@emc.com
Received: by mxic2.corp.emc.com with Internet Mail Service (5.5.2653.19)
	id <PJ6WQKB8>; Wed, 31 Jul 2002 19:46:38 -0400
Message-ID: <277DD60FB639D511AC0400B0D068B71E0564C152@CORPMX14>
To: rddp@ietf.org
Date: Wed, 31 Jul 2002 19:46:35 -0400
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
Subject: [rddp] rddp Yokohama minutes
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org

Since there have been no comments, the DRAFT minutes
are the final minutes.  Thanks, --David

> -----Original Message-----
> From: Black_David@emc.com [mailto:Black_David@emc.com]
> Sent: Tuesday, July 23, 2002 3:38 PM
> To: rddp@ietf.org
> Subject: [rddp] rddp Yokohama DRAFT minutes
> 
> 
> Thanks to Mike Smith for taking notes.  Here are
> the draft minutes.  Please comment either privately
> or to the list - the final minutes will be sent to
> the secretariat around the end of this week.
> 
> Thanks,
> --David
> 
> IETF Remote Direct Data Placement (rddp) WG - DRAFT
> Meeting minutes - July 18, 2002, Yokohama, Japan
> -------------------------------------------------
> 
> Chair: David L. Black (black_david@emc.com)
> 
> -- Agenda Bashing and Administrivia
> 
> rddp is now an IETF Working Group, formed based on the ROI
> (RDMA Over the Internet) BOFs in Salt Lake City and Minneapolis.
> 
> The mailing list is up and running at rddp@ietf.org.  To subscribe,
> send email to rddp-request@ietf.org.  Use of the rdma list at Yahoo
> Groups for IETF work has ceased.
> 
> -- Charter and Goal/Milestone review and discussion
> 
> The official WG charter is available at:
> 
> 	http://www.ietf.org/html.charters/rddp-charter.html
> 
> The four work items on the charter encompass the entire scope of
> the WG's activities.  TCP framing was deliberately omitted from that
> list - TCP framing work belongs in the Transport Area (tsvwg)
> WG. The rddp WG is not permitted to standardize changes to transport
> protocols - it may recommend (minor) SCTP specification changes to
> the tsvwg WG, but TCP specification changes are not permitted.
> The rddp WG has a rechartering scheduled for March 2003 whose
> outcome (e.g., whether the WG will be allowed to continue) will
> be based to a large extent on the progress made between now and then.
> 
> The rddp WG is expected to work on mapping its functionality to both
> SCTP and TCP - see the charter.  Work to apply RDDP mechanisms to
> protocols other than SCTP and TCP is outside the current charter -
> those interested in such work, including generic IP-layer mechanisms
> applicable to all protocols that use IP, should submit Internet-Drafts
> as a basis for further discussion.  The planned WG draft mapping RDDP
> to TCP is currently intended to become an Informational RFC 
> rather than
> a standards track RFC - any decision to change this is best done
> with a draft in hand, so consideration will be deferred to the rddp
> WG rechartering currently scheduled for March 2003.  On the
> subject of compatibility with proprietary interconnects/protocols
> that already support RDDP-like functionality (often called RDMA), the
> guidance from the Area Director is that the WG should not go out of
> its way to be incompatible, but likewise, heroic efforts motivated
> solely by such compatibility are also inadvisable.
> 
> The RDMA Consortium (www.rdmaconsortium.org) was brought up.  This
> industry consortium was organized to work on providing functionality
> over TCP, and has a goal of transferring its work to the IETF 
> and putting
> itself out of business.  The RDMA consortium was formed at a 
> time where
> there was some confusion over whether and to what extent the IETF
> would permit RDDP work over TCP - that confusion has now been resolved
> by the rddp WG charter (such work is permitted, but SCTP comes first).
> The WG chair will work with the RDMA consortium towards their goal
> of moving their work into the IETF.  It should be understood that
> "the fix is not in" - while the RDMA Consortium is very 
> likely to submit
> drafts describing its work, that does not preclude others from writing
> drafts on the same or similar topics for consideration by the WG.
> 
> -- RDMA Concerns Draft (draft-black-rdma-concerns-00.txt)
> 
> This draft was based on concerns arising from the Minneapolis BOF
> and evolved to its current form via discussion at an end2end Research
> Group meeting.  This draft is input to the rddp WG, and is 
> not expected
> to become an RFC.
> 
> The topic of potential Intellectual Property Rights issues was raised.
> The only official way to notify the IETF of such issues is to send an
> IPR notice to the IETF - mailing list discussions are not "official
> notice".  If anyone knows of an IPR claim for which they 
> believe a notice
> should have been sent to the IETF but was not, a private email should
> be sent to the WG chair as an initial step.
> 
> An issue was raised about the absence of "application 
> integrity" from the
> RDMA concerns draft - in addition to "system integrity" concerns, the
> overwrites made possible by RDDP functionality open up 
> potential attacks on
> applications via overwriting data between the time that an application
> checks the data and the time that the application makes use 
> of the data.
> This concern will be included in the -01 version of the RDMA 
> concerns draft.
> 
> -- Problem Statement and Requirements drafts
> 	(draft-romanow-rdma-over-ip-problem-statement-00.txt)
> 	(draft-talpey-rdma-over-ip-requirements-00.txt)
> 
> -01 versions of both drafts with minor revisions should hit the
> Internet-Draft servers shortly after the Yokohama meeting week.
> 
> The rddp WG is tasked to produce a problem statement/architecture
> draft along with specifications of an RDDP protocol and an 
> RDDP control
> protocol (for RDMA functionality) and mappings of those protocols to
> SCTP and TCP .  The problem statement draft will be expanded with
> architecture text to become the first of these drafts (one possible
> source is draft-bailey-roi-ddp-rdma-arch-00.txt).  Randy Stewart has
> a draft (draft-stewart-sctp-roi-00.txt) that could serve as the basis
> for the mapping to SCTP.
> 
> There was a discussion about the relationship between RDDP 
> operation sizes
> (as invoked by protocols above RDDP) and transport segment 
> sizes (based on
> network Path MTU).  These sizes ought to be independent, which entails
> segmentation and reassembly in the RDDP layer with the goal 
> of providing
> an RDDP header in each transport segment; SCTP's ability to 
> operate without
> SCTP fragmentation helps, but there are some unavoidable 
> situations in the
> face of Path MTU changes where transport segments may have to be sent
> without an RDDP header in each segment. 
> 
> -- Other Business and/or Discussion
> 
> The division of work for applying RDDP to various protocols 
> will probably
> be similar to the way that the IETF handles MIBs - the rddp WG will do
> the basic work to define the RDDP mechanisms, and then other WGs will
> handle application/mapping to their respective protocols.  For the
> specific case of SRP (SCSI RDMA Protocol), the IP Storage (ips) WG is
> one possible place to do that application/mapping work even though
> the protocol is specified by T10.
> 
> For interface specifications (e.g., application interface to RDDP
> implemented in hardware), drafts are welcome (including the envisioned
> division of functionality between hardware and software), but the IETF
> has not generally been enthusiastic about specifying APIs and 
> the like,
> and does not use API specifications to drive protocol definitions. 
> The RDMA Consortium is currently working on verbs (abstract 
> API definition),
> and it may be appropriate to allow that work to continue there rather
> than transferring it to the IETF.  
> 
> _______________________________________________
> rddp mailing list
> rddp@ietf.org
> https://www1.ietf.org/mailman/listinfo/rddp
> 

_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



From daemon@optimus.ietf.org  Wed Jul 31 23:13:05 2002
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged))
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA25387
	for <rddp-archive@odin.ietf.org>; Wed, 31 Jul 2002 23:13:04 -0400 (EDT)
Received: (from daemon@localhost)
	by optimus.ietf.org (8.9.1a/8.9.1) id XAA27003
	for rddp-archive@odin.ietf.org; Wed, 31 Jul 2002 23:14:14 -0400 (EDT)
Received: from optimus.ietf.org (localhost [127.0.0.1])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id XAA26974;
	Wed, 31 Jul 2002 23:12:15 -0400 (EDT)
Received: from ietf.org (odin [132.151.1.176])
	by optimus.ietf.org (8.9.1a/8.9.1) with ESMTP id XAA26941
	for <rddp@optimus.ietf.org>; Wed, 31 Jul 2002 23:12:12 -0400 (EDT)
Received: from kathmandu.sun.com (kathmandu.sun.com [192.18.98.36])
	by ietf.org (8.9.1a/8.9.1a) with ESMTP id XAA25374
	for <rddp@ietf.org>; Wed, 31 Jul 2002 23:11:02 -0400 (EDT)
Received: from jurassic.eng.sun.com ([129.146.17.55])
	by kathmandu.sun.com (8.9.3+Sun/8.9.3) with ESMTP id VAA12772;
	Wed, 31 Jul 2002 21:12:11 -0600 (MDT)
Received: from taipei (taipei.Eng.Sun.COM [129.146.85.178])
	by jurassic.eng.sun.com (8.12.4+Sun/8.12.4) with SMTP id g713CBJK320196;
	Wed, 31 Jul 2002 20:12:11 -0700 (PDT)
Message-Id: <200208010312.g713CBJK320196@jurassic.eng.sun.com>
Date: Wed, 31 Jul 2002 20:12:02 -0700 (PDT)
From: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Reply-To: "H.K. Jerry Chu" <Jerry.Chu@eng.sun.com>
Subject: RE: [rddp] comments on draft-black-rdma-concerns-00.txt 
To: Paul.Culley@hp.com
Cc: rddp@ietf.org
MIME-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
Content-MD5: RygPvu0CIQOT3CIxSVFF3w==
X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.4.6_06 SunOS 5.8 sun4u sparc 
Sender: rddp-admin@ietf.org
Errors-To: rddp-admin@ietf.org
X-Mailman-Version: 1.0
Precedence: bulk
List-Id: IETF Remote Direct Data Placement (rddp) WG <rddp.ietf.org>
X-BeenThere: rddp@ietf.org

>I suspect that many folks are expecting STags to have a NIC wide scope, that 
>is, any connection on the NIC can use any STag.  That definitely opens up 
>guessing attacks.  While likely implementations MAY do that, a much better 
>choice will be to limit the STag scope to an individual connection.

I thought about that too. But I came to realize this is logically
equivalent to widening the STag, except that widening STags has the
benefit of making NIC's job easier. NICs only need to deal with STags,
which have the dual "security key" function proposed.

>
>If connection scope is chosen, then an attacker must both "hijack the
>TCP connection" and guess the correct STag.

What you said is true only if the connection scope is enforced by the 
DDP-capable NICs before data is placed. This almost requires TOE in
NICs, which is a difficult hoop to jump, both technically and economically.

Also a STag with a scope limited to a connection almost implies the
same value won't be used in any other connections. Randomizing the
value of STags seems a much simpler way to guard against forgery.

>This is where careful use of IPSEC can help; it can make it nearly
>impossible to hijack the connection.
>
>For an example of how the STag scope might be made variable, see VIA Ptags, or 
>InfiniBand Protection Domains.

I took a look at IBA spec on the protection domain, but can't see how it
can help unless the protection domain # is passed on the wire, or is tied
to some connection # and the NIC knows about the mapping (so it can
decide a STag is bad and throw it away). But the last one is equivalent
to limiting STag to a connection scope, which requires TOE on the NICs...

Jerry Chu
Sr. Staff Engineer
Solaris Networking
Sun Microsystems, Inc.
(650) 786-5146

>Note that I don't believe that these mechanisms belong in a wire level 
>specification; but it might be instructive to provide a pointer to, or include 
>a "verbs" level description of these in the security sections of the RDDP 
>document.
>
>Paul Culley
>HP Sr. Fellow
>281-514-5543
>
>
>-----Original Message-----
>From: H.K. Jerry Chu [mailto:Jerry.Chu@eng.sun.com]
>Sent: Tuesday, July 30, 2002 8:28 PM
>To: caitlinb@rp.asomi.net
>Cc: rddp@ietf.org
>Subject: Re: [rddp] comments on draft-black-rdma-concerns-00.txt 
>
>
>>It is NICE if the STag deregistration can be automatically
>>applied at the earliest possible moment. But it does not
>>close any security holes. The buffers are exposed until the
>>trailing SEND *and* the deregisration have completed.
>>Consolidating those two completions is a *local*
>>optimization, it has no interoperabilty implications.
>>
>>It is A REALLY GOOD IDEA to use STags exactly once, and it
>>is also A REALLY GOOD IDEA to deregister them as prompty as
>>possible. It is also A REALLY GOOD IDEA to have the fastest
>>processor possible and a few spare GB of DDRAM for your host
>>processor. Redundant power supplies are nice too.
>>
>>The point is that how many STags are available, how quickly
>>they must be reused and how efficiently they can be reused
>>are all economic tradeoffs. The market should be allowed to
>>determine those answers, not the IETF.
>
>It seems that the last defense here is a good selection of STags
>to export. 32 bit STag seems too small for security purpose. In
>the least one should analyze the implication on security and
>recommend some algorithm for selecting STags that is better against
>spoofing. E.g. a simple tag index into a page table will not be
>random at all and be trivial to spoof. (But a randomized STag may
>be hard to implement on NICs efficiently.) This is similar to
>hijacking a TCP connection except that the latter still seems much
>hard. One must guess the correct seq# and port # at the right time
>and also manage to shut out the other end of the TCP connection...
>But here once a valid STag is guessed no other protection is there.
>On a busy system the working set of transient network buffers could
>be in many megabyte range making the probability of guessing a valid
>32 bit STag >> 0. Even with IPsec, a malicious user on an
>authenticated system or a buggy application may stomp on some
>random place in remote memory causing data corruption that will be
>impossible to debug.
>
>Rgds,
>
>Jerry
>
>>
>>
>>
>>
>>
>>
>>Caitlin Bestler
>>http://www.asomi.com/CaitlinBestler.html
>
>
>_______________________________________________
>rddp mailing list
>rddp@ietf.org
>https://www1.ietf.org/mailman/listinfo/rddp
>


_______________________________________________
rddp mailing list
rddp@ietf.org
https://www1.ietf.org/mailman/listinfo/rddp



