Return-Path: <owner-ips-outgoing@ece.cmu.edu>
Received: from newbox.ece.cmu.edu (localhost [127.0.0.1])
	by newbox.ece.cmu.edu (Cyrus v2.1.12) with LMTP; Sun, 27 Jul 2003 21:32:04 -0400
X-Sieve: CMU Sieve 2.2
Received: from hazard.ece.cmu.edu (HAZARD.ECE.CMU.EDU [128.2.129.24])
	by newbox.ece.cmu.edu (Postfix) with ESMTP id 582113091
	for <ipsml@ece.cmu.edu>; Sun, 27 Jul 2003 21:32:04 -0400 (EDT)
Received: by hazard.ece.cmu.edu (Postfix, from userid 953)
	id 3C70A153; Sun, 27 Jul 2003 21:32:03 -0400 (EDT)
Received: from sos.ece.cmu.edu (SOS.ECE.CMU.EDU [128.2.129.27])
	by hazard.ece.cmu.edu (Postfix) with ESMTP
	id 3C885130; Sun, 27 Jul 2003 21:32:00 -0400 (EDT)
Received: by sos.ece.cmu.edu (Postfix)
	id 7C0DF89F4; Sun, 27 Jul 2003 21:31:42 -0400 (EDT)
Delivered-To: ips-outgoing@sos.ece.cmu.edu
Received: from osgood.ece.cmu.edu (OSGOOD.ECE.CMU.EDU [128.2.129.25])
	by sos.ece.cmu.edu (Postfix) with ESMTP id 5FE9889F2
	for <ips-outgoing@sos.ece.cmu.edu>; Sun, 27 Jul 2003 21:31:42 -0400 (EDT)
Received: by osgood.ece.cmu.edu (Postfix)
	id 40CB588; Sun, 27 Jul 2003 21:31:42 -0400 (EDT)
Delivered-To: ips-outgoing@ece.cmu.edu
Received: by osgood.ece.cmu.edu (Postfix, from userid 953)
	id 16CED91; Sun, 27 Jul 2003 21:31:42 -0400 (EDT)
Received: from sos.ece.cmu.edu (SOS.ECE.CMU.EDU [128.2.129.27])
	by osgood.ece.cmu.edu (Postfix) with ESMTP id 36B688C
	for <ips-outgoing@ece.cmu.edu>; Sun, 27 Jul 2003 21:31:39 -0400 (EDT)
Received: by sos.ece.cmu.edu (Postfix, from userid 363)
	id 0D6B689F8; Sun, 27 Jul 2003 21:31:39 -0400 (EDT)
X-Original-To: ips@sos.ece.cmu.edu
Delivered-To: ips@sos.ece.cmu.edu
Received: from osgood.ece.cmu.edu (OSGOOD.ECE.CMU.EDU [128.2.129.25])
	by sos.ece.cmu.edu (Postfix) with ESMTP id 171B589F2
	for <ips@sos.ece.cmu.edu>; Sun, 27 Jul 2003 21:31:37 -0400 (EDT)
Received: by osgood.ece.cmu.edu (Postfix)
	id 8C87488; Sun, 27 Jul 2003 21:31:36 -0400 (EDT)
Delivered-To: ips@ece.cmu.edu
Received: by osgood.ece.cmu.edu (Postfix, from userid 953)
	id 466158C; Sun, 27 Jul 2003 21:31:36 -0400 (EDT)
Received: from e33.co.us.ibm.com (e33.co.us.ibm.com [32.97.110.131])
	by osgood.ece.cmu.edu (Postfix) with ESMTP id 2574E88
	for <ips@ece.cmu.edu>; Sun, 27 Jul 2003 21:31:33 -0400 (EDT)
Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.17.195.11])
	by e33.co.us.ibm.com (8.12.9/8.12.2) with ESMTP id h6S1VVj3220624;
	Sun, 27 Jul 2003 21:31:31 -0400
Received: from d03nm694.boulder.ibm.com (d03av02.boulder.ibm.com [9.17.193.82])
	by westrelay02.boulder.ibm.com (8.12.9/NCO/VER6.5) with ESMTP id h6S1VVY5201900;
	Sun, 27 Jul 2003 19:31:31 -0600
Importance: Normal
MIME-Version: 1.0
Sensitivity: 
To: Caitlin Bestler <cait@asomi.com>
Cc: ips@ece.cmu.edu
Subject: Re: iSCSI/iWARP drafts and flow control
X-Mailer: Lotus Notes Release 5.0.7  March 21, 2001
Message-ID: <OFE75635AC.5EBA9336-ON88256D71.000534FF@us.ibm.com>
From: Mike Ko <mako@almaden.ibm.com>
Date: Sun, 27 Jul 2003 18:31:25 -0700
X-MIMETrack: Serialize by Router on D03NM694/03/M/IBM(Release 6.0.1 [IBM]|June 10, 2003) at
 07/27/2003 19:31:30,
	Serialize complete at 07/27/2003 19:31:30
Content-Type: text/plain; charset="us-ascii"
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
X-Spam-Status: No, hits=-16.3 required=5.0
	tests=BAYES_01,EMAIL_ATTRIBUTION,QUOTED_EMAIL_TEXT
	version=2.50
X-Spam-Level: 
X-Spam-Checker-Version: SpamAssassin 2.50 (1.173-2003-02-20-exp)

Caitlin, it looks like as far as iSCSI asynchronous messages are 
concerned, we both agree that "setting limits on most of the types listed 
is so easy that there is no need to negotiate the limit".  So the only 
other item is the SCSI asynchronous events, and the question yet to be 
answered is how many AENs can realistically be sent at one time.  Since 
the optional Shared Receive Queue already provides a solution to handling 
this situation without impacting the iSCSI spec, does anyone else thinks 
that the handling of AENs justify adding a negotiable item to set a hard 
limit on the number of AENs?

Mike Ko
IBM Almaden Research
San Jose, CA 95120

Sent by:        owner-ips@ece.cmu.edu
To:     Mike Ko <mako@almaden.ibm.com>
cc:     ips@ece.cmu.edu 
Subject:        Re: iSCSI/iWARP drafts and flow control




On Sunday, July 27, 2003, at 02:21 PM, Mike Ko wrote:

> From sec.3.5.2.1 of the iSCSI spec, "Asychronoous Messages are used to
> carry SCSI asyncrhonous events (AEN) and iSCSI asynchronous messages."
> Sec.10.9.1 lists the codes for Asyncrhonous Messages which include 0)
> SCSI
> Asynchronous Event, 1) target requests Logout, 2) target indicates it
> will
> drop the connection, 3) target indicates it will drop all connections
> of
> this session, and 4) target requests parameter negotiation.  Since the
> iSER layer can estimate the number of Asynchronous Messages needed in a
> typical situation for the AsyncEvent codes listed in the iSCSI spec, it
> can determine how much to overprovision when a dedicated receive queue
> is
> used.  Additionally, if one is concerned about the atypical case, the
> optional features of graceful handling and the shared receive queue as
> you
> have pointed out can be used.  So it doesn't seem to me that flow
> control
> for Asynchronous Messages are needed.  Can you provide an example
> where an
> unbounded number of Asynchronous Messages would need to be sent?

If it is so easy to bound the number of Asynchronous Messages without
specific knowledge of your peer, why not simply state that limit in
the iSER spec?

If you cannot agree on that limit for a spec, how are two peers supposed
to do so in the wild?

iWARP ULPs are required to take responsibility for flow control That
means
the Data Sink must know how many untagged messages are allowed to have
in flight. Period. It does not mean that there has to be that many
buffers pre-committed.  Just that there is an agreed upon limit.

There is already an agreed-upon limit for 99.99% of the traffic,
untagged
messages that have CmdSNs. It is almost silly to reduce the protocol
from
being truly reliable to being statistically reliable for exceptions.

Setting limits on most of the types listed is so easy that there is no
need to negotiate the limit. It would be absurd to have two "I will drop
this connection" messages in flight on the same reliable connection.

That leaves the Asynchronous Events. As that these are dependent on the
type of target it would seem to be a bit more difficult to meaningfully
state a maximum number of in-flight messages that a future device might
need to generate. So you have to include an "in-flight" limit during
setup,
and then you need to a way to know when the message is no longer
"in-flight".
The obvious solution is a response to a *later* command.



