Subject: RE: [Ips] rationale to send PDUs in increasing CmdSn on single co nnection

From: Julian Satran <[email-masked]>
Date: Sat, 1 Sep 2007 08:46:10 +0300

David - I admit I did not use the right word - I should have said not "visible" but "good" motivation. And the immediate vs. non-immediate shipping can be solved within any of the layers above. I do not even recall a heated debate about it.

Julo

-----David Sheehy <[email-masked]> wrote: -----

To: Julian Satran/Haifa/IBM@IBMIL, Parav Pandit <[email-masked]>
From: David Sheehy <[email-masked]>
Date: 08/31/2007 09:34PM
cc: [email-masked]
Subject: RE: [Ips] rationale to send PDUs in increasing CmdSn on single co nnection

and
there was no visible motivation for out of order commands on a single
connection

On the contrary, there was plenty of motivation. It was
overridden by the desire for simplified implementations over single TCP
connections as the Parav conjectured. There are DMA related efficiencies that
can be realized in situations with a mix of solicited and immediate/unsolicited
I/O traffic. The folks in favor of this optimization were outvoted by those who
wanted to simplify their single connection based
implementations.

Dave

From:
Julian Satran
[mailto:[email-masked]]
Sent:
Friday, August 31, 2007 8:54
AM
To:
Parav Pandit
Cc:
[email-masked]
Subject:
Re:
[Ips] rationale to send PDUs in increasing CmdSn on single
connection

Parav Pandit <[email-masked]>
wrote on 31/08/2007 12:52:04:

> Hi,
> RFC 3720,
section 3.2.2.1 says
>
> "On any connection, the iSCSI initiator
MUST send the
> commands in increasing order of CmdSN, except for
>
commands that are retransmitted due to digest error
> recovery and
connection recovery. "
>
> (Assuming Single TCP connection ISCSI
session)
>
> 1. I interpret above 3.2.2.1 statement as
>
SCSI layer gives SCSI commands to the ISCSI stack in
> the order of Cmd-1
and Cmd-2.
> Cmd-1 will have CmdSn = 10.
> Cmd-2 will have CmdSn =
11.
> ISCSI stack CAN send PDUs to the TCP layer in
> following
order ONLY.
> PDU-1 with Cmd-1.
> PDU-2 with Cmd-2.
Is this correct interpretation?
> Or

Yes
> 2. On a SINGLE connection can ISCSI stack send the
>
PDU-1 with Cmd-2 followed by
> PDU-2 with Cmd-1?

NO
> Assuming the answer of the question
#2 is No,
>
> 3. If there are multiple connections in a session
then
> command MAY any way reach out of order. And targets
> need to
wait for the previous expected commands.
> So targets will
receive out of order ISCSI PDUs from
> the TCP layer and ISCSI stack
handles them.
>
> So then why initiators have restriction of
sending
> command in the increasing order of CmdSn on SINGLE TCP
>
connection?

To simplify recovery and
to...
> Is it to simplify the implementation of targets
> supporting
only single TCP connection?

and there was no visible
motivation for out of order commands on a single connection

> Regards,
> Parav Pandit
____________________________________________________________________________________
Looking for a deal? Find great prices on flights and hotels with
> Yahoo!
FareChase.
http://farechase.yahoo.com/

>
>
> _______________________________________________
> Ips
mailing list
> [email-masked]
https://www1.ietf.org/mailman/listinfo/ips

_______________________________________________
Ips mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ips

---

From: Parav Pandit <[email-masked]>
Date: Fri, 31 Aug 2007 23:18:06 -0700 (PDT)

Exactly thats the reason which lead me to asko this
question.

DMA optimization is possible if we can send out of
order commands to the TCP layer on single connection.

If I take pragmatic approach, 
Any idea, currently how current single connection
target implementation is?

I mean, if current targets are smart enough (accept
out of order PDUs on single connection- liberal on
point 3.2.2.1), then initiators can use their DMA
optimization.
What do you say?

Parav

--- David Sheehy <[email-masked]> wrote:

> > and there was no visible motivation for out of
> order commands on a single connection 
>  
> On the contrary, there was plenty of motivation. It
> was overridden by the desire for simplified
> implementations over single TCP connections as the
> Parav conjectured. There are DMA related
> efficiencies that can be realized in situations with
> a mix of solicited and immediate/unsolicited I/O
> traffic. The folks in favor of this optimization
> were outvoted by those who wanted to simplify their
> single connection based implementations.
>  
> Dave
> 
>   _____  
> 
> From: Julian Satran
> Sent: Friday, August 31, 2007 8:54 AM
> To: Parav Pandit
> Cc: [email-masked]
> Subject: Re: [Ips] rationale to send PDUs in
> increasing CmdSn on single connection
> 
> 
> 
> 
> Parav Pandit <[email-masked]> wrote on
> 31/08/2007 12:52:04:
> 
> > Hi,
> >  
> > RFC 3720, section 3.2.2.1 says 
> > 
> > "On any connection, the iSCSI initiator MUST send
> the
> > recovery and connection recovery. "
> > 
> > (Assuming Single TCP connection ISCSI session)
> > 
> > the order of Cmd-1 and Cmd-2. 
> > Cmd-2 will have CmdSn = 11.
> > following order ONLY.
> > Assuming the answer of the question #2 is No,
> > 
> > need to wait for the previous expected commands.
> > 
> > So targets will receive out of order ISCSI PDUs
> from
> > the TCP layer and ISCSI stack handles them.
> > 
> > So then why initiators have restriction of sending
> To simplify recovery and to...
> > supporting only single TCP connection?
> > 
> > 
> 
> and there was no visible motivation for out of order
> commands on a single connection 
> 
> > Regards,
> > Parav Pandit
> > 
> > 
> > 
> >        
> >
>
> > Yahoo! FareChase.
> >  <http://farechase.yahoo.com/>
> >  <https://www1.ietf.org/mailman/listinfo/ips>

      ____________________________________________________________________________________
Luggage? GPS? Comic books? 
Check out fitting gifts for grads at Yahoo! Search
http://search.yahoo.com/search?fr=oni_on_mail&p=graduation+gifts&cs=bz

---

From: Julian Satran <[email-masked]>
Date: Sat, 1 Sep 2007 18:49:22 +0300

As assigning CMD sequence number is the prerogative of the initiator I do 
not  understand the point of send out of order commands. If you want to 
optimize within an independent RDMA layer at the initiator the assign the 
numbers there.  Initiators sending out-of-order packets will probably be 
flagged as bad by any of the certification packages.

Julo

01/09/07 09:18

To
David Sheehy <[email-masked]>, Julian Satran/Haifa/IBM@IBMIL
cc
[email-masked]
Subject
RE: [Ips] rationale to send PDUs in increasing CmdSn on single co nnection

Parav

---

From: Paul Koning <[email-masked]>
Date: Tue, 4 Sep 2007 09:27:42 -0400

>>>>> "Parav" == Parav Pandit <[email-masked]> writes:

 Parav> If I take pragmatic approach, Any idea, currently how current
 Parav> single connection target implementation is?

I suspect most of them are single connection.

 Parav> I mean, if current targets are smart enough (accept out of
 Parav> order PDUs on single connection- liberal on point 3.2.2.1),
 Parav> then initiators can use their DMA optimization.  What do you
 Parav> say?

Don't.

You'd be creating an initiator that violates the protocol spec.  You'd
have no reason to expect that to work at all.  Many targets will check
this and call it a protocol violation (you may end up with the
connection terminated).  Some may not notice but may end up doing the
wrong thing.

      paul

---

From: "Mallikarjun C." <[email-masked]>
Date: Tue, 4 Sep 2007 11:42:06 -0700 (PDT)

Agree with Julian on this.

RFC 3720 simply requires that once the CmdSN is
assigned to a command, it has to go out in that order
on to the wire.  Note that RFC 3720 does not mandate 
where/when CmdSN assignment must be done, other than
of course that it is done before the command goes out.
 I don't see the specific DMA optimizations that the
RFC precludes.

Mallikarjun 

--- Julian Satran <[email-masked]> wrote:

> > > SCSI layer gives SCSI commands to the ISCSI
> stack
> > in
> > > command in the increasing order of CmdSn on
> SINGLE
> > TCP
> > > connection?
> > > 

____________________________________________________________________________________
Building a website is a piece of cake. Yahoo! Small Business gives you all the tools to get online.
http://smallbusiness.yahoo.com/webhosting

---

From: Parav Pandit <[email-masked]>
Date: Wed, 5 Sep 2007 06:00:37 -0700 (PDT)

> connection terminated).  Some may not notice but may

[Parav] Thanks Paul. I certainly don't want to violate
the protocol. I was wondering how current qualified,
proven iscsi targets are behaving for this scenario.
And you confirmed the same, so will surely adhere to
the protocol.

Thanks,
Parav

____________________________________________________________________________________
Take the Internet to Go: Yahoo!Go puts the Internet in your pocket: mail, news, photos & more. 
http://mobile.yahoo.com/go?refer=1GNXIC

---

From: "Pat Thaler" <[email-masked]>
Date: Mon, 17 Sep 2007 14:41:29 -0700

The focus of comments seems to have been around single connection with
the implication that multi-connection already has to handle out of order
and the restriction is doing nothing for multi-connections sessions.
This is incorrect. Even when there are multiple connections, the
requirement to have increasing order on a connection restricts the
behavior the receiver will see. For example, when there are two
connections and packets 1 through 4 have been sent, the receiver might
see:

Connection: A      B
                   1       
                           3

                           4
                   2

But would not see:
                  A     B
                  1  
                         3
    
                         2

                  4

For a session with N connections, the receiver only has to look N places
(the front of what was received from each connection) for the next
CmdSN. If one removes the in-order per connection rule, the receiver
would have to look for the next CmdSN in the whole set of received
commands. It would be incorrect to assume that any multi-connection
iSCSI implementation would work with the restriction removed. Some may
have been implemented for the subset of out of order behavior that
occurs within the current rules. They may handle deviations from that
behavior for recovery but with less efficiency so that it is
undesireable to purposely cause that condition.

Pat

-----Original Message-----
From: Parav Pandit [mailto:[email-masked]] 
Sent: Wednesday, September 05, 2007 6:01 AM
To: Paul Koning
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: RE: [Ips] rationale to send PDUs in increasing CmdSn on single
co nnection

Thanks,
Parav

________________________________________________________________________
____________
Take the Internet to Go: Yahoo!Go puts the Internet in your pocket:
mail, news, photos & more. 
http://mobile.yahoo.com/go?refer=1GNXIC

---

From: "Deuskar, Prafulla" <[email-masked]>
Date: Wed, 19 Sep 2007 04:49:39 -0700

All,

Section 3.2.3 of RFC says that:

"The purpose of the iSCSI login is to enable a TCP connection for
 iSCSI use, authentication of the parties, negotiation of the
 session's parameters and marking of the connection as belonging to an
 iSCSI session."

When we do a login for non-leading connection do we renegotiate session
parameters?

Thanks,
Prafulla

---

From: Julian Satran <[email-masked]>
Date: Wed, 19 Sep 2007 18:19:30 +0200

Any of login's can renegotiate any parameter that is not called-out 
specifically as being "leading-only".
For the "standard keys" as defined in 3270  that is an empty set (all the 
session-wide keys are LO) but newly defined keys and vendor specific 
parameters might be renegotiable with every new connection or even during 
the life of a connection.

Julo

From:
"Deuskar, Prafulla" <[email-masked]>
To:
<[email-masked]>
Date:
19/09/07 14:12
Subject:
[Ips] Multiple connection - login

All,

Thanks,
Prafulla

---

From: William Studenmund <[email-masked]>
Date: Wed, 19 Sep 2007 09:42:22 -0700

> When we do a login for non-leading connection do we renegotiate  

No. That's why some keys are marked "Leading-Only".

Take care,

Bill

---

From: "Eddy Quicksall" <[email-masked]>
Date: Wed, 26 Sep 2007 13:27:40 -0400

With my target, if I receive CmdSN's out of order I will re-order them 
before they proceed to the SCSI layer. Since this is necessary for 
multi-connections then it just falls out for single connections.

Also, don't forget that if the Queue Algorithm Modifier is 1 (which it 
usually is) then simple commands can execute in any order regardless of the 
CmdSN (i.e. once it reaches the SCSI layer). So just give the increasing 
CmdSN and the target will, on a single connection, immediately forward the 
command to the SCSI layer and then the optimizations you are thinking of 
will most likely take place.

Eddy

----- Original Message ----- 
From: "Parav Pandit" <[email-masked]>
To: "Paul Koning" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>; 
<[email-masked]>
Sent: Wednesday, September 05, 2007 9:00 AM
Subject: RE: [Ips] rationale to send PDUs in increasing CmdSn on single 
connection