Subject: Re: ISCSI: CmdSN in non-leading login

From: "Julian Satran" <[email-masked]>
Date: Sun, 12 May 2002 08:22:19 +0300

John,

My only point was that there is enough information in the section to 
describe correct behavior.
And immediate commands do not create any blockage regardless of their 
CmdSN.

I am still trying to understand what your point was (is).

Julo

John Hufferd@IBMUS
05/11/2002 10:25 PM

        To:     Julian Satran/Haifa/IBM@IBMIL
        cc:     [email-masked], "Mark S. Edwards" <[email-masked]>, 
[email-masked]
        From:   John Hufferd/San Jose/IBM@IBMUS
        Subject:        Re: ISCSI: CmdSN in non-leading login

Julian,  (watch out long note to follow)
Sometimes our messages pass by each other in the night.  I had corrected 
my note to indicate that I had overlooked the very obvious statement that 
we kept in the draft that the Login command was immediate.  But the rest 
of the note I think is correct.

That is, the command that follows the completion of the non 
leading-connection Login should take its CmdSN from the next command 
sequence number in the "Session" stream, not the CmdSN of the Login for 
that connection. 

Here is the thinking, 

1. The second connection is started while commands are flowing on the 
Leading connection,
2. The CmdSN in the stream when the Login for the second connection is 
started is lets say 123. 
3. Since the Login command is immediate, and does not change the ExpCmdSN 
then one of three things can happen
    a. Initiator uses the same CmdSN for the next command that it sends on 
the first connection, etc. (123, 124, 125, etc.)
    OR 
    b. Initiator uses the next CmdSN 124 on the leading connection and 
thereby creates a delivery hole, for the target until the CmdSN 123 is 
finally received on the secondary connection and then all the commands 
123----125, etc, are delivered
    c. No commands are sent on any connection until the Login of the 
secondary connection is completed and the next command is given the CmdSN 
of 123
4. If 3a is done, then the first command after the delivery will use the 
next CmdSN in the stream and no blockage would have occurred.
5. If 3b is done there will be a blockage of the delivery of commands 
until the secondary Login in completed
6. If 3c is done, then the first command after the delivery will use the 
same CmdSN (123) as the Login, but the commands will have a blockage at 
the initiator until the secondary Login is completed.

So I believe, that IF the subsequent commands in a secondary connection 
has to have the same CmdSN as the Initial Login of that  connection, 
either all commands in the entire session must stop being sent on other 
connections, or commands will not be delivered until the delivery hole is 
closed at the end of the secondary Login.

If we were to handle the Login for secondary connections like other 
immediate commands,  the 3a. path would be done (the ExpCmdSN would not be 
advanced, but other commands will continue and they will advance the 
ExpCmdSN as normal, and when the next  non immediate command is sent on 
the same connection as the immediate command, it is simply given the very 
next CmdSN from the session stream.

The problem of treating the secondary connection Login exactly like the 
leading Connection, when it comes to the next command, is that, unlike the 
leading connection, other things continue to move on the other 
connections.

Therefore, in my opinion the correct way of interpreting the current 
wordage in 9.12.8 is at the point it says:
  "....and the first non-immediate command also carries the CmdSN 123.)" 
we should read that as "... the next non-immediate command IN THE SESSION 
also carries the CmdSN 123)".  In this way the words mean the same for 
both the leading connection and secondary connection.

This is how I believed the implementations have been built.

Anyway, that was the essences of my words where I suggested:
 
At the end of the first paragraph under 9.12.8, we should add, the words
"Similar rules exist for the Leading Login for a non leading connection,
that is, the next CmdSN value is used for the Leading Login of the non
leading connection, and that same number is used with all the subsequent
Login PDUs on that connection. But at completion of the Login of the
connection, the ExpCmdSN is not advanced and the subsequent commands take
the next CmdSN value in the session."

Now perhaps we can have some better words, perhaps the words but clearly 
the current word have confused some folks.  Perhaps even me :-)

.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Home Office (408) 997-6136, Cell: (408) 499-9702
Internet address: [email-masked]

Sent by:        [email-masked]
To:     John Hufferd/San Jose/IBM@IBMUS
cc:     [email-masked], "Mark S. Edwards" <[email-masked]>, 
[email-masked] 
Subject:        Re: ISCSI: CmdSN in non-leading login

John, 

The current text starts by stating that login requests are immediate. 

It then states: 

CmdSN is either the initial command sequence number of a session (for the 
first Login request of a session - the "leading" login) or the command 
sequence number in the command stream (e.g., if the leading login carries 
the CmdSN 123 all other Login requests carry the CmdSN 123 and the first 
non-immediate command also carries the CmdSN 123). 

It has all the information needed. 
I am puzzled by your comments. 

Julo 

John Hufferd/San Jose/IBM@IBMUS 
Sent by: [email-masked] 

05/11/2002 03:38 AM 
Please respond to John Hufferd 

        To:        "Mark S. Edwards" <[email-masked]> 
        cc:        <[email-masked]> 
        Subject:        Re: ISCSI: CmdSN in non-leading login 

The text as it is written has not been updated since we at one point had
the Login command marked as and Immediate command.  Therefore, it always
had to pick up the next value that would be assigned to the next non
immediate command.  Also since it was an immediate command the ExpCmdSN 
was
not advanced so it did not effect the commands in the rest of the session.
They would continue with the next CmdSN as usual.  Login is still not
suppose to advance the ExpCmdSN, so there should still not be a problem.

When we removed the flag that said that it was an immediate command, the
wording in the draft about the Initial Login was not adjusted.

I suggest that the only thing that needs to be changed is the following:
At the end of the first paragraph under 9.12.8, we should add, the words
"Similar rules exist for the Leading Login for a non leading connection,
that is, the next CmdSN value is used for the Leading Login of the non
leading connection, and that same number is used with all the subsequent
Login PDUs on that connection. But at completion of the Login of the
connection, the ExpCmdSN is not advanced and the subsequent commands take
the next CmdSN value in the session."

This is what we were all doing, and simply did not change anything with 
the
Immediate bit was dropped from the Login PDU.  I suggest we leave it with
that operational intent, since it was so long ago that we establish this
way of operating I can not remember exactly why, but would not like to be
surprised sometime in the future, and I do not see why we would have to
change it.

.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Home Office (408) 997-6136, Cell: (408) 499-9702
Internet address: [email-masked]

"Mark S. Edwards" <[email-masked]>@ece.cmu.edu on 05/10/2002 10:38:07
AM

Sent by:    [email-masked]

To:    John Hufferd/San Jose/IBM@IBMUS
cc:    <[email-masked]>
Subject:    Re: ISCSI: CmdSN in non-leading login

John,

If an initiator opens a new connection on a session and in the first login
request uses the current session CmdSN, the lifetime of that CmdSN
continues all the way through until the completion of the first
non-immediate command on that new connection.

This means that the initiator has to reserve the use of that CmdSN for the
first non-immediate command on that particular session.  That's stupid.

Especially bearing in mind that it could be pumping commands like crazy
down other open connections in the session.

There is also another side effect, if the login takes any length of time,
like computing a DH or having to make external calls to a Radius or a
Kerberos server, then because this CmdSN is outstanding, the other
connections in the session will block as soon as the command window is
filled.

The final sentence in 9.12.8 is meaningless for a non-leading
connection.  The target either has to block the whole session until the
login is completed, or run the risk of corrupting session state.

Mark

At 10:14 AM 5/10/2002 -0700, John Hufferd wrote:

>I do not see the issue, it works as is, it does not effect the first non
>immediate command unless it completes as a authorized connection. Again, 
I
>do not see the issue, in fact it works as is.
>
>.
>.
>.
>John L. Hufferd
>Senior Technical Staff Member (STSM)
>IBM/SSG San Jose Ca
>Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
>Home Office (408) 997-6136, Cell: (408) 499-9702
>Internet address: [email-masked]
>
>
>"Mark S. Edwards" <[email-masked]>@ece.cmu.edu on 05/10/2002 
09:47:53
>AM
>
>Sent by:    [email-masked]
>
>
>To:    Bill Studenmund <[email-masked]>
>cc:    <[email-masked]>
>Subject:    Re: ISCSI: CmdSN in non-leading login
>
>
>
>At 09:13 AM 5/10/2002 -0700, Bill Studenmund wrote:
> >On Fri, 10 May 2002, Mark S. Edwards wrote:
> >
> > > There are a couple of problems here for the target.  At what point 
is
a
> > > non-leading connection considered to be part of the session ?  Is it
>the
> > > moment that the login request is received with a non-zero TSIH ?  Or
is
>it
> > > only when the non-leading login succeeds and it enters full feature
>phase ?
> >
> >I think the target shouldn't let anything from a new connection
influence
> >things until after at least the security phase has been passed.
Otherwise
> >we have a security hole. After operational negotiation would probably 
be
> >best.
>
>Good, as far as I can see, a new connection should not be allowed to
>influence any session context until the connection reached full-feature
>phase.  But this is a problem if we have to follow the CmdSN rules of
>9.12.8.   See next comment.
>
>
> > > So, should an initiator set the CmdSN in the first login request to
>zero
> > > and only synchronise with the session command stream after full
feature
> > > phase is established ?  This is my preferred option.
> > >
> > > What happens if the initiator tries using the current session 
command
> > > sequence number is that whilst the login negotiation occurs, other
> > > connections within the session can be issuing new commands, so by 
the
>time
> > > that the login is finished the CmdSN exchanged in the initial 
request
>is
> > > invalid anyway.
> > >
> > > I would like to see something along the lines that for a non-leading
> > > connection, the CmdSN field MUST be zero and that the connection can
>not be
> > > considered part of the session until full feature phase is entered,
at
> > > which point any commands issued on the connection are now
synchronised
>with
> > > the session command sequence number as observed by all other 
existing
> > > connections on the session.
> >
> >I thought login counted as a command, so it got its own command number,
in
> >the stream of all other commands. ??
>
>
>For a leading connection the CmdSN, whether zero or non-zero, is regarded
>as a primer with which to set the initial session command sequence 
number,
>all the login requests exchanged in the negotiation, no matter how many
>carry the same CmdSN as does the first non-immediate command.  In other
>words, the first pdu in a leading connection does influence session 
state.
>
>So if we agree that a non-leading connection can not influence session
>state until full-feature phase, then we have to also state that the rule
in
>9.12.8 about the first non-immediate command carrying the same CmdSN as
the
>initial login request can not work for a non-leading connection.  In this
>case, the first non-immediate command must be set from the next logical
>value in the session context.  I would like the spec to have this added
and
>to explicitly say that the CmdSN in a login request for a non-leading
>connection is ignored by the target.  I'd really prefer that the actual
>value be zero, but that's not necessary from a protocol perspective if 
the
>target ignores the field.
>
>Mark.

---

From: Bill Studenmund <[email-masked]>
Date: Mon, 13 May 2002 10:57:02 -0700 (PDT)

> And immediate commands do not create any blockage regardless of their

Don't they though? Say I have a queue depth of 2, and I start a secondary
login with a CmdSN of 1000. I can then send other immediate commands at
1000, and a non-immediate command at 1000. I can then also send a
non-immeidate command at 1001. But can I send a command at 1002 before the
login has finished? If we increase the queue depth, there will still be
some other command number that would overflow it.

Say the login takes a while. Won't the login pin the queue at 1002 until
it's done? Now let the login take a while, it requires talking to say a
low kerberos server or doing a DH computation. If we really pay attention
to CmdSN, then the slow login process can effectively take the device
off-line while it completes, since we can't enqueue new commands.

Worse yet, say the secondary connection is NOT from our initiator, but
from an imposter. All the rogue has to do is get the TSIH right & guess a
CmdSN, and stall at trying to add another connection. Say do leading
negotiation, but just never send any security negotiations. The queue will
eventually become pinned until the target times out. Nice little DOS
attack; all you have to do is get the TSIH right & guess a CmdSN. You
don't have to do any security negotiation.  :-)

I'd like to suggest that we just ignore CmdSN on non-leading sessions,
beyond the fact that a well-behaved initiator should pick a CmdSN in its
current command window. We certainly shouldn't pay attention to it before
security negotiations have completed. :-) Oh also no commands should be
sent over the new connection with a CmdSN lower than the one used in the
new connection.

Take care,

Bill