Subject: Re: iSCSI: CmdSN during the Login phase

From: "Matt Wakeley" <[email-masked]>
Date: Mon, 16 Jul 2001 18:15:55 -0700

I think it's pretty obvious that it's your option #1: (the initiator supplies
the initial CmdSN in the login PDU, and for every command PDU after that, the
CmdSN gets incremented, just like in "full feature phase")

2.10.7 CmdSN says "CmdSN is either the initial number of a session..."
[Julian, this should say "initial command number of a session"]

So, the Login PDU defines the first CmdSN.

1.2.2.1 says "Commands in transit from the initiator to the target layer are
numbered by iSCSI; the number is carried by the iSCSI PDU as CmdSN".  Text
commands are commands, so they are numbered also.

It goes on to say "CmdSN - the current command Sequence Number advanced by 1
on each command shipped except for commands marked for immediate delivery."

I agree that this needs to be cleaned up. CmdSN and StatSN should work during
login just like they do during "full feature phase" - there is no reason why
the should not, and making them always work the same removes complexity and
interoperability problems.

-Matt

"Scott M. Ferris" wrote:
> 
>   At what point during the Login Phase of a leading connection (null
> TSID) does a session exist?
> 
>   Section 2.10.7 describing CmdSN for the Login PDU indicates that
> when TSID is null, CmdSN indicates the starting Command Sequence
> number for this session.
> 
>   For interoperability, it is important to define precisely when a
> session exists, so that all initiators and targets agree on when
> CmdSNs are significant, and do not reject or ignore PDUs due to
> differing assumptions of how the CmdSN numbering should be done during
> the Login Phase.
> 
> Some of the possible choices for session start are:
> 
> 1) a session starts before the initiator sends anything, and the Login
>    PDU is the first PDU of a session.
> 
> 2) a session starts when the initiator receives a (possibly partial)
>    Login Response with an accept status class and a non-zero TSID, and
>    the next PDU sent by the initiator is the first PDU of the session,
>    which uses the same CmdSN as the Login PDU.
> 
> 3) a session starts when the initiator receives a Final Login
>    Response, and the next PDU sent by the initiator is the first PDU
>    of the session, which uses the same CmdSN as the Login PDU.
> 
>   After searching through draft 6, I was unable to find anything that
> clearly indicated which if any of the above possibilities was correct,
> or, if more than one is possible, how the initiator and target are
> supposed to determine what CmdSN numbering scheme the other side is
> using for the Login Phase.
> 
>   The UNH draft-6 initiator appears to use choice #1.  The Cisco
> draft-6 initiator and target use choice #2.  This can cause the UNH
> initiator to hang when trying to login to the Cisco target, since the
> initiator may send a Text PDU with CmdSN 2, while the target is
> waiting for a PDU with CmdSN 1.
> 
>   Section 1.2.2.2 states that "Status numbering starts after
> Login. During login, there is always only one outstanding command per
> connection and status numbering is not strictly needed but may be used
> as a sanity check."
> 
>   A similar argument could be made that CmdSN is not needed during the
> Login Phase, suggesting choice #3 may be the simplest.
> 
>   I also think section 1.2.2.2 is problematic, as it appears to
> contradict itself by saying that status numbering is not used during
> login, and then follows up saying status numbering may be used during
> login.
> 
> --
> Scott M. Ferris,
> [email-masked]

---

From: "Scott M. Ferris" <[email-masked]>
Date: Mon, 16 Jul 2001 17:26:13 -0500 (CDT)

-- 
Scott M. Ferris,
[email-masked]

---

From: "Matt Wakeley" <[email-masked]>
Date: Tue, 17 Jul 2001 14:03:27 -0700

I agree with Mark's proposal.

-Matt

Mark Bakke wrote:
> 
> Given that this is being interpreted differently by enough
> implementations, I would prefer to see us take a step back
> and look at which of Scott's three options should make the
> most sense.  It looks like options #1 and #2 are what is
> currently being implemented, regardless of what's specified.
> 
> However, a connection technically can't really be part of a
> session (sharing its work load) until it has completed a
> successful login phase.  There's also little reason for
> numbering until the login phase is complete, since the login
> and text responses during negotiation are synchronous, and
> pertain only to the connection, rather than the session.
> 
> I would recommend stating that a connection joins (or becomes,
> if it's the first one) a session after the login response
> with the F bit set is sent (on a target), or after the login
> response with the F bit set is received (on an initiator),
> and that CmdSN is not necessary until such a time, especially
> since CmdSN is per-session, not per-connection.  This is
> identical to Scott's option 3.
> 
> I think that defining it this way would remove any further
> ambiguity on when a connection is part of a session.  Since
> it appears that most implementations will have to modify their
> login code anyway, this shouldn't be too bad.
> 
> --
> Mark

---

From: Mark Bakke <[email-masked]>
Date: Tue, 17 Jul 2001 15:48:57 -0500

--
Mark

-- 
Mark A. Bakke
Cisco Systems
[email-masked]
763.398.1054

---

From: "Robert D. Russell" <[email-masked]>
Date: Wed, 18 Jul 2001 10:21:14 -0400

Mark:

I respectfully disagree with your recommendation to go with option 3.

The experience here at the plugfest seems to be that option 1 is
easiest to work with since it involves no special case code --
the cmdsn is incremented on every command sent.  Period.
A simple statement to that effect in the standard is much, much
prefered than a long chain of "technical" reasoning that readers have to
follow through to the end to infer what the final result should be.

I believe simplicity is the key to interoperability, and the standard
mechanisms should be designed to be simple.  There is indeed a reason
to number commands during login -- because then they are treated the
same as any other commands and you don't need special purpose code
during login to deal with that part of shipping the commands.
I.e., you keep it uniform and simple, no special cases.
This allows code to be reused and debugged once.

As a final point, most implementations will NOT have to be modified
to implement option 1 -- they now already do that.

Bob Russell
InterOperability Lab
University of New Hampshire
[email-masked]
603-862-3774

---

From: "Matt Wakeley" <[email-masked]>
Date: Wed, 18 Jul 2001 13:14:56 -0700

Robert,

The question is, in the case of adding connections to (form) a multiple
connection session, what CmdSN values do you use to start the new connection? 
If you use the next in line CmdSN for the session, you block all the SCSI
commands the existing connections are sending while negotiating this new
connection.

-Matt

---

From: "Robert D. Russell" <[email-masked]>
Date: Thu, 19 Jul 2001 11:34:09 -0400

> commands the existing connections are sending while negotiating this new

I don't understand how the new negotiation blocks the existing connections
in this case.  The login negotiation involves exchanges of login, text
commands, text responses and login responses that all share the same
Initiator Task Tags, but NOT the same CmdSN -- each new command in this
negotiation process carries a new CmdSN, and other connections can get
their own CmdSN values without waiting for the full negotiation process
to complete.

Consider the following:

1. The TCP connection is established completely "in parallel" to the
   existing connections and has no effect on them.

2. If IPsec or other non-iSCSI security is being used, it is negotiated
   "in parallel" to the existing connections and has no effect on them.

3. The initiator now needs to send the login on the new connection, and
   for that it selects the "next" CmdSN, increments it by 1, and sends
   the login command.  The next command on any other connection will
   use the incremented CmdSN and increment it in turn, etc.  This command
   can still be sent on the other connection.  It is true that the target
   cannot process this other command until the target has sent the first
   login response to the login command (because of the sequencing imposed
   by the CmdSN).  However, the other connection is NOT blocked during the
   complete login negotiation phase, because each subsequent text command
   sent by the initiator uses the next CmdSN and is processed by the
   target AFTER any intervening commands on other connections.

   The commands in the login negotiation all have the same
   Initiator Task Tag, but not the same CmdSN.  Therefore they do not
   prevent other connections from consuming CmdSN values.  In fact, it is
   possible for several logins for additional connections to be going on
   simultaneously while commands are still flowing (and being processed)
   on the existing connections.

   Of course, the time needed by the target to process login and text
   commands should be kept to a minimum, which just reinforces the need
   to simplify the login negotiation process as much as possible.

Bob Russell