Subject: RE: iscsi : numerical negotiation wording is ambiguous

From: "Julian Satran" <[email-masked]>
Date: Wed, 3 Oct 2001 15:26:58 +0300

An example:
 08.txt logic
i->t DataPDULength=16k
t->i DataPDULength= 24k

Selected value is 16k 

i->t DataPDULength=16k
t->i DataPDULength= 8k

Selected value is 8k

t->i DataPDULength=16k
i->t DataPDULength= 24k

Selected value is 16k 

t->i DataPDULength=16k
i->t DataPDULength= 8k

Selected value is 8k

-----

Responder selects logic

i->t DataPDULength=16k
t->i DataPDULength= 24k

Error - Initiator Reject - Closes connection

i->t DataPDULength=16k
t->i DataPDULength= 8k

Selected value is 8k

t->i DataPDULength=16k
i->t DataPDULength= 24k

Error Target has to terminate with parameter error

t->i DataPDULength=16k
i->t DataPDULength= 8k

Selected value is 8k

"Dev - Platys" <[email-masked]>
01-10-01 23:46
Please respond to "Dev - Platys"

        To:     "Julian Satran" <[email-masked]>, <[email-masked]>
        cc: 
        Subject:        RE: iscsi : numerical negotiation wording is ambiguous

Julian,
 
I am not sure why a 'REJECT' is required.
Can you please explain why is this required?
 
I am with Santosh in this.
 
>There is NO need for any REJECT in the above >case. If the initiator
>is'nt satisfied with the value returned by the >originator and cannot
>function with the negotiated values, it can simply >close the TCP
>connection. There is no need to send any >REJECT.

Thanks
 
Deva
Adaptec

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of Julian Satran
Sent: Monday, October 01, 2001 1:28 PM
To: [email-masked]
Subject: Re: iscsi : numerical negotiation wording is ambiguous

Santosh, 

We are getting nowhere. Even if requester is doing it's stuff - the 
requester will have to check and be prepared for a mistake. The coding 
will require a reject. 

Julo 

Santosh Rao <[email-masked]> 
Sent by: [email-masked] 
01-10-01 20:37 
Please respond to Santosh Rao 
        
        To:        [email-masked] 
        cc:        "Sanjeev Bhagat (TRIPACE/Zoetermeer)" 
<[email-masked]>, Julian Satran/Haifa/IBM@IBMIL 
        Subject:        Re: iscsi : numerical negotiation wording is 
ambiguous 

Julian & Sanjeev,

Responding to both your mails......

Julian :

I think you may have mis-interpreted my comments. I believe Sanjeev has
clarified the intent of my suggestions. 

I am *not* suggesting that the responder send back its values and these
be blindly imposed on the originator. On the contrary, my suggestion is
that the computation of the result of the negotiation (higher or lower
of the 2 values) be only performed by the responder and sent back to the
originator.

The result of the negotiation is the same in both cases and there is no
REJECT required in my case nor yours. The difference is the advantages
I've stated in my model. 

Sanjeev, in response to your comment :

" [Sanjeev Bhagat (TRIPACE/Zoetermeer)] Although there is
>  no reject , but it can be a problem in future .
>  Consider your example of DataPDULength in your own
>  message. Suppose offering party sends a value of 16,384
>  (this is also lowest value it can send) and responding
>  party responds with 8,192. In this case the offering
>  party will have to reject the negotiation. So a reject
>  message is needed in this case also. "

There is NO need for any REJECT in the above case. If the initiator
is'nt satisfied with the value returned by the originator and cannot
function with the negotiated values, it can simply close the TCP
connection. There is no need to send any REJECT.

Thanks,
Santosh

> "Sanjeev Bhagat (TRIPACE/Zoetermeer)" wrote:
> 
> Thanks Julian, please find my further comments in the message
> 
>      -----Original Message-----
>      From: Julian Satran [mailto:[email-masked]]
>      Sent: Sunday, September 30, 2001 6:07 PM
>      To: [email-masked]
>      Subject: RE: iscsi : numerical negotiation wording is
>      ambiguous
> 
>      Sanjeev,
> 
>      I am not sure clear I made the tiny diference between what I
>      say and what Santosh said:
> 
>      Santosh says:
> 
>        1. a requester sends A=valuex
>        2. a responder sends B=valuey
>        3. the responder assumes that the value y is the correct
>           value and so does an external observer
>           [Sanjeev Bhagat (TRIPACE/Zoetermeer)] I would rather
>           say it this way the responder computes the value with
>           its own supported values and responds with a value y
>           which the responder thinks is correct and so does an
>           external observer
>        4. the requester checks that valuey is conformant (he will
>           not believe it) and will reject it if not conformant
>           else it will accept it
>           [Sanjeev Bhagat (TRIPACE/Zoetermeer)] Although correct,
>           but it is highly unlikely for the responder to reject
>           the final result because the rule states that (lower or
>           higher of two will be the result) and so the offering
>           party should be able to accept the lower or higher
>           range as defined by rule. In case the key is dependent
>           upon some range of fixed values then the negotiation
>           should be performed as list negotiation and not 
>           numerical negotiation.
> 
>           This might be what you "conventionally" do - I don't
>           and that shows that convention like morals are a matter
>           of geography :-)
>           [Sanjeev Bhagat (TRIPACE/Zoetermeer)] :-)
> 
>           The advantage of this set of rules is that it allows an
>           external observer to know what was selected without
>           knowing the rules
>           [Sanjeev Bhagat (TRIPACE/Zoetermeer)] Even in this
>           case, I guess that the external observer has to know
>           the rules to double check the value is correct or not
>           The disadvantage is that messages have to be "built",
>           an additional reject states exists and MOST IMPORTANT
>           you need both messages
> 
>           In what I said:
> 
>           1. The requester sends A=valuex
>           2. The Responder has to send either nothing (if valuex
>           is enough on both sides to compute the result like in
>           the case in which the function is a Boolean AND and the
>           value is NO) or valuey
>           3. Both the requestor and responder have to compute the
>           value (they have to know the rules anyhow) and so does
>           the external observer
> 
>           The disadvantage is that the external observer has to
>           know the rules
>           The advantage is that there is no reject, in binary
>           negotiations you can go away with shorter negotiations
>           and you can set strings at fixed values.
>           [Sanjeev Bhagat (TRIPACE/Zoetermeer)] Although there is
>           no reject , but it can be a problem in future .
>           Consider your example of DataPDULength in your own
>           message. Suppose offering party sends a value of 16,384
>           (this is also lowest value it can send) and responding
>           party responds with 8,192. In this case the offering
>           party will have to reject the negotiation. So a reject
>           message is needed in this case also.
> 
> 
>      Sanjeev
>       "Sanjeev Bhagat
>       (TRIPACE/Zoetermeer)"                    To:        "'Santosh 
Rao'"
>       <[email-masked]>            <[email-masked]>, Julian
>       Sent by: [email-masked]   Satran/Haifa/IBM@IBMIL
>                                                cc: [email-masked]
>       30-09-01 16:32                           Subject:        RE: iscsi 
:
>       Please respond to "Sanjeev       numerical negotiation wording is
>       Bhagat (TRIPACE/Zoetermeer)"     ambiguous
> 
> 
> 
>      All,
> 
>      I agree with Santosh that the responding party must respond
>      the result of
>      the negotiation as compared with the value from offering
>      party. All
>      negotiations in SCSI like (sync, disconnect etc) are also
>      done the same way.
>      I also object to Julian's reason of a simple minded target
>      which wants to
>      send certain fixed values only. In case a simple minded
>      target identifies a
>      value which it cannot support or is less than the value a
>      target can
>      support, then there should be a method for target to reject
>      such a
>      negotiation and the default values be accepted on both side
>      as a result of
>      negotiation.
> 
>      1 Because even if simple minded target sends its fixed value
>      which is
>      greater than the value sent by offering party then the final
>      result of
>      negotiation will be taken as ( lesser of the two) and in
>      this case target
>      which which cannot support the lower value will produce some
>      illegal
>      results.
> 
>      2. if simple minded target wants to send its own value and
>      wants it to be
>      accpeted then the responding party is not negotiating but
>      forcing the result
>      on initiator(this method should not be allowed unless
>      explicitly mentioned).
> 
>      however if there is another proposal of numeric negotiation
>      in which the
>      responding party can force its result to be over ridden by
>      the offering
>      party's result then it is acceptable for offering party and
>      responding party
>      to send there own supported key-value result and it can then
>      be computed at
>      offering party's end.
> 
>      IMP: (May be I am missing something here) I do not see how a
>      numeric
>      negotiation can be rejected. Is it possible to reject such
>      kind of
>      negotiaion?
> 
>      Sanjeev
> 
>      -----Original Message-----
>      From: Santosh Rao [mailto:[email-masked]]
>      Sent: Friday, September 28, 2001 11:15 PM
>      To: Julian Satran
>      Cc: [email-masked]
>      Subject: Re: iscsi : numerical negotiation wording is
>      ambiguous
> 
>      Julian & All,
> 
>      I request the group to take a close look at this negotiation
>      process
>      again and [re-]evaluate the alternative being proposed.
> 
>      Further, it appears that the login stage negotiation  is
>      also following
>      the same algorithm as the login key negotiation, wherein
>      originator &
>      responder offer their respective values and both sides need
>      to determine
>      the result of the negotiation. i.e. both initiator and
>      target need to
>      compare their NSG with the other party's NSG and pick the
>      lower of the
>      2.
> 
>      I suggest that both the login key negotiation and the login
>      stage
>      negotiation follow the policy that the originator offers a
>      value and the
>      responder picks the result of the negotiation based on (the
>      offered
>      value & its own value). The value picked by the responder is
>      sent back
>      to the originator.
> 
>      This model has the following advantages :
> 
>      1) Only one side picks the result of the negotiaton. Hence,
>      the 2 sides
>      cannot go out of sync on the value picked.
> 
>      2) The originator knows the negotiated result at the the
>      responder since
>      the responder sends back the result of the negotiation.
> 
>      3) This model is easier to debug because of (1) & (2).
> 
>      4) Less code since only 1 party (responder) needs to perform
>      the
>      computation to pick the result of the negotiation.
> 
>      5) This scheme leaves less room for interop problems and
>      mis-interpretation since it is the more familiar negotiation
>      technique
>      that is in use in most other mass storage protocols. (ex :
>      FC PLOGI, FC
>      PRLI, etc). From a first reading of the current negotiation
>      scheme, it
>      is'nt readily apparent that the currently defined model is
>      different
>      from the above and requires both sides to be picking the
>      result of the
>      negotiation, instead of just the responder.
> 
>      Comments ?
> 
>      Thanks,
>      Santosh
> 

-- 
##################################
Santosh Rao
Software Design Engineer,
HP-UX iSCSI Driver Team,
Hewlett Packard, Cupertino.
email : [email-masked]
Phone : 408-447-3751
##################################

---

From: Sandeep Joshi <[email-masked]>
Date: Wed, 03 Oct 2001 10:29:01 -0400

The first & third example below seem illogical. 

The responder should not be sending 24K if the initial
sender has chosen a value of 16K, since there is no
possibility of that 24K being accepted (unless we want
three way exchanges for every negotiation??)   

The problem of a Reject message does not arise since the 
responder should only be choosing something less than 16K.
So I guess this puts me in the "responder selects logic" 
camp.

-Sandeep

Julian Satran wrote:
>   "Dev - Platys"
>   <[email-masked]>                To:        "Julian Satran"
>                            <[email-masked]>,
>   01-10-01 23:46           <[email-masked]>
>   Please respond to "Dev -         cc:
>   Platys"                          Subject:        RE: iscsi :
>                            numerical negotiation wording is ambiguous
> 
> 
> 
> Julian,
> 
>   <[email-masked]>        To:        [email-masked]
>   Sent by:                     cc:        "Sanjeev Bhagat
>   [email-masked]   (TRIPACE/Zoetermeer)" <[email-masked]>,
>                         Julian Satran/Haifa/IBM@IBMIL
>   01-10-01 20:37               Subject:        Re: iscsi : numerical
>   Please respond to     negotiation wording is ambiguous
>   Santosh Rao
> 
> 
> Julian & Sanjeev,
> 
>  [email-masked]
> >       30-09-01 16:32                           Subject:        RE:
> iscsi :

---

From: Santosh Rao <[email-masked]>
Date: Wed, 03 Oct 2001 09:26:04 -0700

Julian,

Another value-add of the "responder picks the negotiation result model"
is that the initiator can use the following "query based" negotiation
model to always use the values the target is capable of offering :

I -> T : DataPDULength=?
T -> I : DataPDULength=64K

Both sides agree to use a DataPDULength of 64K. 

In the "both sides pick negotiation result" model, the above cannot be
achieved, since the initiator has to always offer a value.

Thus, in the "both sides pick negotiation result" model , the above
example would be :
I -> T : DataPDULength=8K
T -> I : DataPDULength=64K
I -> T : DataPDULength=64K
T -> I : DataPDULength=64K

Both sides settle at 64K.

I suggest that we allow the above "query based model", since this is
more efficient to use when an initiator has no key limitations and would
like to use the value a target can offer. In order to allow the "query
based model", you would need to state in the draft that a key value of
"?" over-rides the default, implying the target offered value would be
the result of the negotiation.

In particular, the "query based model" is quite useful when an initiator
wishes to function with the target's maximal supported values for keys
like DataPDULength, FirstBurstSize, MaxBurstSize, MaxOutStandingR2T,
LogoutLoginMinTime, LogoutLoginMaxTime, etc without expressing any
limitations on its own key value.

Comments ?

Thanks,
Santosh

---

From: Paul Koning <[email-masked]>
Date: Wed, 3 Oct 2001 10:43:13 -0400

> Error Target has to terminate with parameter error

In most protocols, "responder selects" is used.  And the rule is that
the responder MUST respond with a value acceptable to both sides.

Therefore, if the negotiation rule for DataPDULength is to pick the
smaller of the two values preferred by the two sides, your first and
third examples would be defective implementations.  Often, protocol
specs don't say what to do with defective implementations; a typical
answer is to terminate the conversation, which is a sensible answer.
Sending rejects is not that useful for defective implementations,
since there is no reason to assume that they will all of a sudden
start to act correctly.  Also, it is useful to build in protocol
mechanisms for bit errors, memory errors, dropped packets, etc. -- but
it is generally NOT useful to build mechanisms that apply only when
the other side has a software bug.  I agree with Deva that the
appropriate response to a protocol violation is to terminate the
conversation. 

So for "responder selects" examples 1 and 3 should have read like
this:

i->t DataPDULength=16k
t->i DataPDULength= 16k		(t wants 24 but 16 is the agreed value)

...

t->i DataPDULength=16k
i->t DataPDULength= 16k		(i wants 24 but 16 is the agreed value)

     paul

---

From: Santosh Rao <[email-masked]>
Date: Wed, 03 Oct 2001 09:14:50 -0700

Julian,

I believe you may be [still] mis-interpreting my proposed change and
that may be the basis for your rejecting the proposal. Let me re-iterate
once again the clarification I 
made in :
http://www.pdl.cmu.edu/mailinglists/ips/mail/msg06924.html

The "responder picks the negotiation result" scheme proposes just what
the title states. The responder of a negotiation computes the result of
a negotiation and sends back its computed redult as its response to the
originator, instead of sending back its supported value. 

Note that the above proposal is also consistent with the manner in which
list negotiation is performed today. With list negotiation, the
responder picks a value from the originating party's offered list and
sends this value back. I am proposing that a similar scheme be used for
all negotiations, [numerical, list & binary]. In all types of
negotiation, the responding party always send back the result of the
negotiation based on its computation.

I also suggest that the same model be applied to the negotiation of
login stages. i.e. The target respond with its NSG set to the next stage
both sides can switch to, instead of each side offering their NSG's and
both sides picking the lower NSG. This keeps all forms of negotiation
consistent in their model and simpler to understand.

I have made corrections inline to the below examples. Further, I have a
question about how/when a target would be originating ANY operational
parameters ? Is it not the case that for all operational parameters, we
have EITHER a default defined or mandated that the initiator send this
key. Given the above, the target would never be originating a key. It
would always be responding to an initiator's offered value for a key.
This offered value may either be explicit or implicit thru a default
specification.

I hope I'm doing a little better at explaining myself :-) Or, are we
still talking past each other. (?). More inline....

Thanks,
Santosh

> Error - Initiator Reject - Closes connection

The above example is incorrect. It should read :
i -> t : DataPDuLength=16K
t -> i : DataPDULength=16K (target supports 24K, initiator supports 16K.
lower of the 2 is sent back as the result of the negotiation.)

> Error Target has to terminate with parameter error

Please see my note above on target always being the responder. Hence,
the above example should actually read :
i -> t : (no explicit value offered for DataPDULength. Default of 8K
assumed).
t -> i : DataPDULength=8K. (tgt supports 16k, initiator asks for 8K.
result of 8K is sent back.)

Thus, for all operational keys, the negotiation must end with a t->i
exchange. 

Also, I suggest that a target ALWAYS respond to any key that the
initiator explicitly send it. This keeps the negotiation result more
explicitly known to both sides, as opposed to target silently dropping
any binary negotiation key if it was offered as a "no".

---

From: "Dev - Platys" <[email-masked]>
Date: Wed, 3 Oct 2001 09:14:14 -0700

Santosh,

The point is, if the target is not capable of supporting 16K (it is just an
example that is being discussed), then it indicates that is what it wants.
Initiator can close the connection. (No reject is necessary though).

However, if the target knows it can support the initiator negotiated value,
it could return it.

Again, Yes, implicitly it becomes the responder selects logic. 

Deva
Adaptec

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of
Sandeep Joshi
Sent: Wednesday, October 03, 2001 7:29 AM
To: [email-masked]
Subject: Re: iscsi : numerical negotiation wording is ambiguous

-Sandeep

---

From: Santosh Rao <[email-masked]>
Date: Wed, 03 Oct 2001 10:31:31 -0700

Deva, Julian & All,

I think we have a more fundamental problem here, that spans beyond the
issue of which negotiation model should be used for numerical & binary
key negotiation. This problem needs to be addressed seperately.

The DataPDULength can and should be allowed to be different in each
direction. I -> T direction PDUs should be allowed to use a different
PDU Length than T -> I direction PDUs. 

Each side should be allowed to specify the DataPDULength it will be
using and there should be no attempt to negotiate this value. Rather,
the key is exchanged in either direction to inform the other side as to
what sized PDUs it should expect.

This is somwhat akin to a protocol's MTU definition or a max_frame_size
definition (ex : FC PLOGI Receive Data Field Size in its common service
parameters.). THese values are allowed to be different going in each
direction, allowing for implementation specific quirks that restrict the
use of certain values. (ex : some DMA bugs may require implementations
to have their DataPDULength to be a cache line size shorter, etc.)

I suggest the definition of the DataPDULength key be re-phrased taking
the above into account.

Regards,
Santosh

Dev - Platys wrote:
> > Julian Satran

---

From: "Dev - Platys" <[email-masked]>
Date: Wed, 3 Oct 2001 10:59:42 -0700

Santosh,

I am NOT for this, as I feel that we are complicating the protocol which
has to be simplistic. I understand where you are coming from (analogous to
MTU) etc.

I guess that the example you state is wrong as the evolution of protocol
specification should not be centered around specific hardware
implementations
and should be the other way around.

Deva

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Wednesday, October 03, 2001 10:32 AM
To: Dev - Platys; Julian Satran
Cc: Sandeep Joshi; [email-masked]
Subject: iscsi : DataPDULength can differ in each direction.

Deva, Julian & All,

Regards,
Santosh

---

From: "Ayman Ghanem" <[email-masked]>
Date: Wed, 3 Oct 2001 13:38:23 -0500

I think this will add unnecessary complexity, specially for data CRCs having
to be calculated based on two different PDU sizes for Reads and Writes.
Also, one end may choose to do data digests in software. If the other end
decides to use a really large PDU length (and the first doesn't have a say
about it), this could be a problem.

-Ayman

> Sent: Wednesday, October 03, 2001 12:32 PM

---

From: Santosh Rao <[email-masked]>
Date: Fri, 05 Oct 2001 09:09:11 -0700

Julian,

I have another question on the subject of login negotiation, which was
brought up earlier in a previous mail. I've enclosed the mail below
[with irrelevant portions snipped out.]

Could you please clarify if the below described behaviour is possible ?

Thanks,
Santosh

---

From: Sandeep Joshi <[email-masked]>
Date: Fri, 05 Oct 2001 12:20:15 -0400

Another alternative is to let the the initiator send 
"DataPDULength=0".  The target can then select & send
a lower value it prefers.

See Appendix D
  A value of 0 indicates no limit (the largest possible number)

This would work for DataPDULength, MaxBurst and FirstBurst.

-Sandeep