Subject: RE: iSCSI: Negotiation clarifications still needed

From: "John Hufferd" <[email-masked]>
Date: Fri, 24 May 2002 23:14:36 -0700

With the risk of sounding fickle, I can also buy this idea, and your
wordage.

.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Home Office (408) 997-6136, Cell: (408) 499-9702
Internet address: [email-masked]

Julian Satran/Haifa/IBM@[email-masked] on 05/24/2002 10:26:08 PM

Sent by:    [email-masked]

To:    [email-masked]
cc:    [email-masked], [email-masked]
Subject:    RE: iSCSI: Negotiation clarifications still needed

Pat your proposed 2b may be what we are looking for - i.e. a responder may
not originate a key if it has an incomplete key text.

The text we may want to add to section 4.2 is:

Key=value pairs may span PDU boundaries. A responder having a received
partial key text MUST refrain from originating any new key=value
negotiations until it has no incomplete key text. This way one avoids
having both negotiating entities assuming the originator role in a
negotiation.

Julo

    pat_thaler@a                                                           
    gilent.com                  To:        [email-masked], Julian       
                        Satran/Haifa/IBM@IBMIL                             
    05/25/2002                  cc:        [email-masked]                 
    12:16 AM                    Subject:        RE: iSCSI: Negotiation     
    Please              clarifications still needed                        
    respond to                                                             
    pat_thaler                                                             

Martins,

Comments referenced by the same items Martins used.

1. Julian sent an email saying he would put the text
I proposed in (though the text you quoted is not the
whole text).

2. I think that the principle we have been using on
text negotiation was that each key negotion is a
separate item. Your proposal would be counter to that
and I don't think it would be an improvement. The
target should be allowed to respond to any complete
key-value pair it has received. When a key-value
pair is straddling the PDU bondary, then it shouldn't
respond to that key until the complete key-value pair
has been received.

There is one potential corner case issue that should
to be covered. Targets can initiate keys. If key-value
pairs didn't straddle PDU boundaries, then ensuring
that there is a clear originator for each offered key
is easy. You can originate any key that you haven't
received an offer of from your partner. But now keys
can straddle PDUs. If the text between the last separater
and the end of the PDU is Ma, then you don't know what
key your partner has started to offer. If the partner
was starting to offer MaxBurstSize and you offer it in
the next PDU of the exchange, both sides may think they
are originator.

I suggest one of the following
a) don't allow keys to straddle PDU boundaries.
b) don't allow originating a key when the last login
PDU ended in a partial key.
c) don't allow offering a key where the start of the key
matches a partial key at the end of the last login
PDU.

3. Yes, we noticed a little while ago that losing a
packet at the end of negotiation could hang things up
though the concern is mainly for a full feature phase
negotiation. Looking at 6.8, any timeout during
negotiation causes the login and its TCP connection to
be terminated. The whole negotiation process (see the
point about origninators in 2) depends upon a one-by-one
exchange of PDUs. PDU loss has to terminate it.

Therefore, the target commits to the end of login
as described for T5 target. It has sent the final
login response with a status of zero. Moves to S5.
If the login response doesn't get to the initiator,
then either the initiator will close the connection
due to the timeout. Since the target is in S4, loss
of the transport connection will cause it to go to
S8 and R1 of the cleanup state machine. It presumably
will not take the M2 transition because the intiator
isn't going to do cleanup for a connection it thinks
wasn't in full feature phase. It will take M1 due to
timeout - not elegant but good enough.

The concern was Full Feature Phase negotation. Until
negotiation ends, it can be reset and no values change.
When the target sends the last Text Response PDU, then
it thinks negotiation has ended and it applies the
new values. If that PDU doesn't reach the initiatior,
then it terminates the entire negotiation and continues
to use the old values. The two ends are using different
values.

We decided to not raise this as an issue because it is
such a corner case - we are operating over a reliable
connection so PDUs shouldn't be lost (unless the whole
path goes down in which case it doesn't matter). Also,
there are few values exchanged during full feature so
it isn't worthwhile to add complexity.

Regards,
Pat

-----Original Message-----
From: Martins Krikis [mailto:[email-masked]]
Sent: Friday, May 24, 2002 12:39 PM
To: Julian Satran
Cc: [email-masked]
Subject: iSCSI: Negotiation clarifications still needed

The previous thread went on too long, but since it
has now quieted down, I'll conjecture that the
following are the aspects that still need to be
addressed.

1. Not everybody seemed to have noticed that it is
  NOT legal to send the same key again, if it
  has once been negotiated (including negotiations
  that end with a reserved value (Reject,
  Irrelevant or NotUnderstood).

I think it would benefit the draft to add the
sentence that Pat proposed to paragraph 5 or page
72 in 12-92: "Sending the key again would be a
re-negotiation". That I think would make it crystal
clear.

2. When the key=value pairs that Originator is
  sending are broken across multiple PDUs, it
  is not clear whether the Responder may start
  responding to the keys as soon as it receives
  them or whether it should send blank PDUs back
  (as in the example on page 164 of 12-92) until
  it gets a PDU, the data part of which ends in
  a NUL byte (thus signaling that there are no
  broken key=value pairs at the end of it).

I am proposing that the draft should make it explicit
that only blank PDUs are to be sent. This allows
decoupling of key=value generation from
their encapsulation in PDUs (i.e., the generating
logic need not worry about whether a key=value
pair will fit and go out in this PDU or has to
be retained to go out in the next). I can explain
in detail why this is important (it has to do
with teh possibility of receiving the "just-about
outgoing" keys) but I'm keeping this "brief".

Furthermore, it is my feeling that instead of
checking the last bytes of a PDU for NUL, it
would be better if the end-of-data was marked by
a flag in the header. This way encapsulation will
be simpler---just put as much data in the PDU as
fits there and raise the flag if it isn't all,
instead of checking whether it ends in a NUL and
possibly shortening data to make it not end like
that.

3. There is an opinion that on page 73 of 12-92,
  the phrase that says "or the responder may
  select an admissible value" is in contradiction
  to the very next sentence. There is also an
  opinion that this phrase is entirely unnecessary
  and detrimental to achieving broad
  interoperability (I call it "cutting slack to
  misbehaving or incompatible originators").

I don't have a suggestion since I consider the
"feature" that this phrase allows of little
importance to a properly built iSCSI node.

4. This is new. When doing Text Request/Response
negotiations (i.e., in FFP), it seems
that the Initiator commits to the new values
when it receives a response from the Target with
the F bit set. It is unclear when the Target should
commit. Should it switch to using the new values
as soon as it sends its response with the F-bit
set, or should it do so only when it knows that
the Initiator received its response?

Commiting right away is simpler and since responses
with F-bits set have TTT=0xffffffff and thus
may not be reset, sounds plausible. If the
values have importance on the next reception,
it may also be important to commit timely. However,
what if the Initiator doesn't get this response?
Target now has committed, Initiator hasn't. Committing
later puts the burden on Initiator to send something
effectively telling "I've received your final
response". Otherwise the Target will time out and
not commit. This response can get lost too.

Basically, it is beginning to look a bit like
(what was it called?) "distributed consensus problem"?
I think it goes like this:

Two generals that are on oposite sides of the
enemy want to synchronize their attack, and
start sending messengers through with messages
like
 "attack at dawn->",
 "<-ok, attack at dawn",
 "I know you know we attack at dawn->",
 "<-, I know you know I know we attack at dawn",
 etc., etc., ...
But at no point can they commit yet...

Is anybody else worried about this?
Anyway, so when should a target commit? Page 83
of 12-92 is the relevant reference.

Thanks,

 Martins Krikis

Disclaimer: these opinions are my own and may
           not be those of my employer.

__________________________________________________
Do You Yahoo!?
LAUNCH - Your Yahoo! Music Experience
http://launch.yahoo.com

---

From: "THALER,PAT (A-Roseville,ex1)" <[email-masked]>
Date: Tue, 28 May 2002 10:49:56 -0600

Julian,
 
I think the text looks fine except for the word "responder" which is difficult because one is a "responder" or an "originator" per key and having received a partial key, one doesn't know whether one is a responder or an originator yet. It would be better to replace "responder" with "initiator or target"
 
Pat
 
-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Friday, May 24, 2002 10:26 PM
To: [email-masked]
Cc: [email-masked]; [email-masked]
Subject: RE: iSCSI: Negotiation clarifications still needed

Pat your proposed 2b may be what we are looking for - i.e. a responder may not originate a key if it has an incomplete key text. 

Key=value pairs may span PDU boundaries. A responder having a received partial key text MUST refrain from originating any new key=value negotiations until it has no incomplete key text. This way one avoids having both negotiating entities assuming the originator role in a negotiation. 

Julo 

	[email-masked] 

05/25/2002 12:16 AM 
Please respond to pat_thaler 

        To:        [email-masked], Julian Satran/Haifa/IBM@IBMIL 
        cc:        [email-masked] 
        Subject:        RE: iSCSI: Negotiation clarifications still needed 

Martins,

Regards,
Pat

-----Original Message-----
From: Martins Krikis [mailto:[email-masked]]
Sent: Friday, May 24, 2002 12:39 PM
To: Julian Satran
Cc: [email-masked]
Subject: iSCSI: Negotiation clarifications still needed

Thanks,

 Martins Krikis

---

From: "Julian Satran" <[email-masked]>
Date: Tue, 28 May 2002 20:56:40 +0300

fine - thanks - Julo

"THALER,PAT (A-Roseville,ex1)" <[email-masked]>
05/28/2002 07:49 PM
Please respond to "THALER,PAT (A-Roseville,ex1)"

        To:     Julian Satran/Haifa/IBM@IBMIL, "THALER,PAT (A-Roseville,ex1)" 
<[email-masked]>
        cc:     [email-masked], [email-masked]
        Subject:        RE: iSCSI: Negotiation clarifications still needed

Julian,
 
I think the text looks fine except for the word "responder" which is 
difficult because one is a "responder" or an "originator" per key and 
having received a partial key, one doesn't know whether one is a responder 
or an originator yet. It would be better to replace "responder" with 
"initiator or target"
 
Pat
 
-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Friday, May 24, 2002 10:26 PM
To: [email-masked]
Cc: [email-masked]; [email-masked]
Subject: RE: iSCSI: Negotiation clarifications still needed

Julo 

[email-masked] 
05/25/2002 12:16 AM 

Martins,

Regards,
Pat

-----Original Message-----
From: Martins Krikis [mailto:[email-masked]]
Sent: Friday, May 24, 2002 12:39 PM
To: Julian Satran
Cc: [email-masked]
Subject: iSCSI: Negotiation clarifications still needed

Thanks,

 Martins Krikis

---

From: [email-masked]
Date: Tue, 28 May 2002 12:05:59 -0600

Martins,

One doesn't have to have a more complex value for key state 
because one isn't required to initiate keys while one has a 
partial key-value. If one wishes to have a simpler implementation, 
then one could have a flag for whether the last received PDU ended 
with a complete key-value. If it did, one can send key responses 
and originate keys. If it didn't, one could send key responses. 

Even an implementation that does originate key-value pairs 
doesn't need to change its key state because we know there 
is never more than one key-value in an incomplete state. 
Therefore, one could check any key that one would like to 
originate against a partially received key-value before 
originating it.

Pat

-----Original Message-----
From: Martins Krikis [mailto:[email-masked]]
Sent: Friday, May 24, 2002 5:45 PM
To: Mallikarjun C.; Julian Satran
Cc: [email-masked]
Subject: Re: iSCSI: Negotiation clarifications still needed

Mallikarjun wrote:

> 2. Seems to me that we need to stipulate that only
> key values 
>     may straddle PDU boundaries - key names (ideally
> inclusive 
>     of "=") shall not.  That should avoid requiring
> blank PDUs.

Definitely inclusive of "=". Otherwise there are
no strict guarantees that the whole key has been
received (we may have keys that are prefixes to
other keys in future, and vendor keys may be like
that).

I can implement it and live with it if that's what
people decide on. I just want people to understand
that it is conceptually a more complex aproach than
requiring strictly blank PDUs and having a data-end
flag. (Even if we don't add a new flag and decide
data-end by looking for NUL-s at the end, we get
a cleaner scheme.) Allowing non-blank PDUs  gives 
better bandwith utilization under
presumably rare circumstances (all pairs not fitting),
but introduces more complexity for your everyday case.

If we only stipulate that no key ever gets broken,
then there is the following "clumsiness" required.
When "originating" a key=value pair, it has to be
checked whether the key will fit fully in the PDU
or not, and if not, then "originating" must be
postponed. We also need either a more elaborate
state for the key (to mark key as received but
value as not; so it is not processable yet, nor
originatable anymore), or for each key=value pair
being originated, key has to be checked against
the buffer that would hold the "leftover stuff"
from the just received PDU, otherwise we may
illegally originate it. Doable, but not clean.

I still think that letting the whole set-of-pairs
go through (no matter over how many PDUs spread)
before the other side is allowed to process them
is cleanest. 

Martins Krikis, Intel Corp.

Disclaimer: these opinions are my own and may
            not be those of my employer

---

From: "Julian Satran" <[email-masked]>
Date: Tue, 28 May 2002 22:34:18 +0300

Mallikarjun,

Thhe current text and your proposed text say basically the same think. If 
you feel that originating does not convey the meaning of originator I will 
rephrase it.

I will not consider key-name and text-value as they would require a long 
re-reading of the text.

Julo

"Mallikarjun C." <[email-masked]>
05/28/2002 10:02 PM
Please respond to "Mallikarjun C."

        To:     <[email-masked]>, Julian Satran/Haifa/IBM@IBMIL
        cc:     <[email-masked]>, <[email-masked]>
        Subject:        Re: iSCSI: Negotiation clarifications still needed

Julian,

I generally agree with the intent of the wording.  A couple of comments -

- I assume you meant "text-value of a received key-name" by "key text"
  (the only occurrence of "key text" is here).   I suggest we use the 
definitions
  on pages 68 & 69.

  [ This is somewhat unrelated.  But with the advent of formal terminology 
now,
     I'd actually recommend that we replace all "key=value" usages in the 
text with
     "key-name=text-value".]

- IMHO (if my above assumption is true), the proposed rule is overly 
constraining.
    o The originator would only have to refrain from originating new keys 
if a partial
       *key-name* is received.  IOW, doesn't have to wait for the entire 
text-value
       to arrive (though it may).

    o The "no incomplete key text" should not imply that the responder is 
also
       forbidden from responding to other (complete) text-value offers 
that it had
       already received.  As a pathological case, if every PDU begins from 
the middle
       of one text-value and concludes in the middle of the next 
text-value, the responder
       should not be required to sit out until all the proposals are 
received in full
       (though it may).

Here's a strawman with some wordsmithing.

Key=value pairs may span PDU boundaries. A responder having a received 
partial key-name or not received the
following "=" in a PDU MUST refrain from originating any new key=value 
negotiations until it had received the
key-name in full and the following "=" . In this case, the receiving 
entity can only assume the role of a
responder for this key.  This way one avoids having both negotiating 
entities assuming the originator role in
a negotiation.

Thanks.
--
Mallikarjun

Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions
Hewlett-Packard MS 5668
Roseville CA 95747
[email-masked]

---

From: [email-masked]
Date: Tue, 28 May 2002 14:19:59 -0600

Julian,
 
Thinking about the subject of long keys and short PDUs, I realized that all keys are either:
 
Not allowed in full feature phase;
Sorter than 512 (the minimum size of maxRecvPDUlengt); or
Declartive.
 
In security negotiation and login negotiation the default of 8196 is used for PDU size so
very short PDUs do not affect those keys that are not allowed in full feature phase. 
 
If one was willing to check that a key-value string would fit in the PDU before commiting
to start it in the PDU, then there is no need to allow non-declaritive key-value pairs to span 
PDUs. Declaritive key-value pairs can span PDUs because they don't enter into the decision
on what keys can be originated by the other side. 
 
One might still allow non-declaritive key-value pairs to straddle PDU boundaries because 
one doesn't want to check that the next key-value fits in the PDU before starting to place it there 
or because one doesn't want the rule to be different for declaritive and non-declaritive PDUs but
I don't think that decision is forced upon us by very short PDUs.
 
Regards,
Pat
 
-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Friday, May 24, 2002 10:10 PM
To: Mallikarjun C.
Cc: [email-masked]; Martins Krikis
Subject: Re: iSCSI: Negotiation clarifications still needed

I agree with Mallikarjun on all but one point - requiring a key not to straddle PDU boundaries. 
This list had a long debate on the subject. We thought (initially) that not having any pair straddle PD boundaries is the best thing to do and we had it untill it became obvious that we have to support very short PDUs. 
The we went for allowing spanning and the scheme we had in  mind was a sender preparing its strings and chopping them into PDUs without any parsing and state. What you suggest now is parsing at send and that may add unnecessary complexity - and I really don't see why. 

Julo 

05/25/2002 02:49 AM 

        To:        "Martins Krikis" <[email-masked]>, Julian Satran/Haifa/IBM@IBMIL 
        cc:        <[email-masked]> 
        Subject:        Re: iSCSI: Negotiation clarifications still needed 

Martins,

Comments in the same order.

1. Julian may be adding wording for this.

2. Seems to me that we need to stipulate that only key values 
   may straddle PDU boundaries - key names (ideally inclusive 
   of "=") shall not.  That should avoid requiring blank PDUs.

3. I don't have a strong opinion on this.  I don't believe this can 
   cause interoperability problems; OTOH, I don't care if the responder
   isn't that liberal.

4. Julian and I discussed this scenario a while ago.  As you precisely
   described, only a non-stop dialogue can be reliable and we can't
   afford a non-stop dialogue.  I suppose one can recover the StatSN
   with a SNACK in FFP, or terminate the connection upon a timeout.
--
Mallikarjun

Mallikarjun Chadalapaka

---

From: Martins Krikis <[email-masked]>
Date: Tue, 28 May 2002 21:53:23 -0500

> It's to avoid corner cases while doing what you say above, "preparing 
> its
> strings and chopping them into PDUs."
> 
> The simplest thing in my mind is to:
> 
> 1) read the other side's offers into a buffer
> 
> 2) parse that buffer generating responses to another buffer
> 
> 3) send the output buffer
> 
> Does that make sense? You don't have to agree, just does it make sense?
> 
> The problem I see with what Pat described is that if you are in
> operational negotiation and have more than 1 PDU worth of data, you 
> will
> be receiving input while you're in step 3. You have to be able to 
> re-enter
> steps 1 & 2 before you can finish step 3.
> 
> <PAT> I think that is over stating the complexity. During negotiation,
> the other side can only have one PDU outstanding at a time. That is 
> explictly
> stated for TextNegotiation PDUs. For LoginPDUs, it isn't explicit but 
> since
> they are immediate and the initiator can only count on the target 
> having one
> immediate buffer for the connection it can't send another LoginRequest 
> until
> it gets the LoginResponse without risking that a PDU will be dropped. 
> Also
> allowing anything other than one oustanding LoginPDU at a time would 
> open 
> the possibility that both sides offered the same key. Therefore, a 
> target
> won't get the next PDU until after it sends the response PDU (or 
> vice-versa
> for an initiator). A PDU won't come in while a PDU is being sent from 
> the
> outgoing negotiation buffer.

Pat, I must interrupt you here. (A subliminal part of this exercise
is to show how annoying interruptions are :-).) In order to argue your 
point I think you're intentionally trying to misunderstand Bill. 
He didn't talk about getting a new PDU while sending his PDU. 
He talked about getting an incoming non-empty PDU while 
sending his data _buffer_. This buffer could be much 
larger than a PDU. You cannot deny that this may happen
with the scheme that you're arguing for. It cannot happen with the
scheme that we're proposing, since only empty PDUs would be coming
back while we're sending our (possibly much larger than PDUs) data
buffers.

> Once you send the PDU, there may still be some data left in the output
> buffer but it doesn't seem complex to switch at that point to waiting 
> to
> receive and parse the next received PDU. 

If you have noticed that a PDU is full, it is certainly possible to
switch to a receiving mode. Yet it is even simpler not to have to notice.

> One had better be able to switch
> from one process to another anyway as most implementations will handle 
> multiple connections which can be in diffent phases. 

I don't see the relevance of this argument.

> If you are willing
> to buffer all your output into a buffer and wait to send it until the 
> other
> side has stopped sending new parameters, then all you need to do to 
> change
> that to an implementation that sends what it has in the output buffer 
> as 
> it goes along is to have pointer into the buffer that indicates the 
> next
> byte to be sent. <PAT>

Not true, because as I'll start sending it, the other side may
send me new offers, thus preempting me from making similar offers
myself and instead requiring _responses_.

> The bit about sending empty PDUs is an effort to avoid that. We are 
> either
> in a sending-negotiation-text mode, or a parsing mode. We don't have to
> worry about strings being partially there, since we don't process until
> they are. We also don't have to worry about if our key/value pairs
> cross a PDU. The parser/negotiatior is MUCH simpler; it doesn't have to
> deal with all of that.
> 
> Letting the target answer while the initiator is trying to send an
> extended PDU adds the following complications (AFAICT):
> 
> 1) initiator has to make sure a key and the '=' don't get split on a 
> PDU
> boundry.
> 
> <PAT> One doesn't have to do that anyway. We can chose the rule to not
> offering a new (non-declarative key) when one has received a partial 
> key
> which seemed to be acceptable to a number of people. <PAT>

First I haven't yet seen anybody but you and myself speak about the 
declarative/non-declarative variation of this theme yet, so I wouldn't
call it "acceptable to a number of people" yet, at least not if
you include the parenthesized part. Overall, however,
a truly aware Sender most likely will check whether a key and =
got split or not. Because if it was split, and the other side
originates a new key, it is a violation. It would be nice to catch it.
And it definitely must check whether key=value fit fully or not.

> 2) if an initiator fills up a PDU, it has to stop processing & wait for
> the response. It also has to remember where it is so that it can finish
> sending that key/value next time.
> 
> <PAT> It has to remember where it was and finish the key/value next 
> time
> anyway as it can't send again until it gets a response. <PAT>

The difference is that in the scheme we're proposing we'd leave the
unsent key=value pairs sitting in the "leftover" buffer, and happily mark 
all those keys individually as "sent". So we'll consider them
processed, even though they may not have been put on the wire yet.
With your scheme, you may not mark them as sent, so there is also 
little point having them stay in the buffer. The only thing that 
can safely stay in the buffer is the remainder of the last key=value pair, 
if it was sent partially.

> 3) The target has to detect a split key/value pair on in-bound, and a)
> store the received text, and b) remember not to offer said key or any
> others.
> 
> <PAT> detecting an incomplete key is reasonably easy. And the target 
> can
> chose to do the simplist thing which is not to offer new keys when a 
> partial key is outstanding. <PAT>

Yes, easy. Still harder than never dealing with an incomplete key at all.

> 4) The target ALSO has to make sure ITS response doesn't fill up a PDU. 
> If
> it does, it has to remember where IT is and that it has more to send.
> 
> <PAT> In your approach the target can have a buffer with more than 
> one PDU worth of response in it. Once it starts sending from that 
> buffer
> it has to remember where it is and that it has more to send. This is
> not any different. There is a buffer with pointers into for end and
> last byte sent (or next byte to send). <PAT>

Yes, it is different, because keys can be marked "sent" and happily
wait their physical send-out in the buffer. With your scheme you
can't put them in a buffer and forget, because incoming data may
force you to go back and start mangling the values you had assigned
them...

> That's a lot of complexity and saved state. At least in my mind.
> 
> <PAT> I don't see much complexity difference here.

There is quite some difference in complexity, here. I think you're
trying not to see it.

<snip>

> <PAT> I agree. This means that plug fests probably haven't hit these 
> cases. <PAT>

That's why there is the famous Bill's experiment---set the PDU size
to 64 bytes and try negotiating.

> Does the statement of the perceived problem make sense, even if the
> opinion of the WG is to not use the empty PDUs? I would feel much
> comfortable with deciding to not use empty PDUs if folks said they
> understand the above point and they choose to address the problems
> differently.
> 
> <PAT> I want to point out that there are times in the protocol when 
> one will have to send an empty PDU. For instance, when getting a
> security key that is longer than a the Max PDU size (or even less
> than that - the sender isn't required to send Max size PDUs) or when 
> a device has no key-value pairs to send but its partner hasn't set
> the F bit. The issue is whether to require that there be a process
> for handing the right to originate keys (or in your proposal to send
> keys at all) keys from initiator to target and back which adds 
> its own complexity. I think the complexity will be
> of the same order as the existing situation. <PAT>

I think in both cases we do have a process to hand the right to
originate/send to the other side and back and forth again. We are
not trying to take this process away. The question is should the
"handing over of the right (to originate/send)" happen after
each PDU sent or should it happen after a data _buffer_ (possibly
containing more key=value pairs than fit in a PDU) sent?

Again a simple comparison to real life... Take the first text
in this message written by me. How would you have liked to delete
everything beneath it and to send the rest of your comments
in a new message? Again, include only as little as there is
until my next comment. That's when I'd interrupt you again
and potentially mess with what you were planning to say. 
Pretty frustrating, wouldn't it be? OK, to be fair, I couldn't 
do it as often as I could want to, since you'd be entitled to one 
PDU worth of uninterrupted text in iSCSI. But it isn't as clean
as "letting the speaker finish" (not the lecture, just the
"complete thought").

Disclaimer: these opinions are mine and may not be those of my employer.

---

From: [email-masked]
Date: Wed, 29 May 2002 17:15:57 -0600

From: Bill Studenmund 
<snip>
What do we want to decide?

Bill, 

As I mention in the response to John, I would like to see proposed text for specifying the hand-off.

I think one could approach this by handing back and forth the right to make negotiation offers and leave to the implementations whether they send empty text negotiations or responses and declarations when they don't have the right to make negotiation offers. I think the hand-off should be based on a bit flag rather than based on whether the last PDU received ended in a complete key-value pair.

Once people have had a chance to see the proposal, I would like to hear opinions beyond those of the few people who have been active in this discussion.

Regards,
Pat

---

From: "John Hufferd" <[email-masked]>
Date: Wed, 29 May 2002 20:23:02 -0700

I take it that we are all happy with the current C bit stuff (if it also is
in Login).  I think that is the hand off token.

.
.
.
John L. Hufferd

[email-masked]@ece.cmu.edu on 05/29/2002 04:15:57 PM

To:    [email-masked], [email-masked]
cc:    Julian Satran/Haifa/IBM@IBMIL, [email-masked], [email-masked],
       [email-masked]
Subject:    RE: iSCSI: Negotiation clarifications still needed

From: Bill Studenmund
<snip>

Bill,

As I mention in the response to John, I would like to see proposed text for
specifying the hand-off.

I think one could approach this by handing back and forth the right to make
negotiation offers and leave to the implementations whether they send empty
text negotiations or responses and declarations when they don't have the
right to make negotiation offers. I think the hand-off should be based on a
bit flag rather than based on whether the last PDU received ended in a
complete key-value pair.

Once people have had a chance to see the proposal, I would like to hear
opinions beyond those of the few people who have been active in this
discussion.

Regards,
Pat

---

From: [email-masked]
Date: Thu, 30 May 2002 11:42:58 -0600

I was in a meeting all yesterday so I haven't had a chance to check it yet. I
expect to have an answer later today.
Pat

-----Original Message-----
From: John Hufferd [mailto:[email-masked]]
Sent: Wednesday, May 29, 2002 8:23 PM
To: [email-masked]
Cc: [email-masked]; [email-masked]; Julian Satran;
[email-masked]; [email-masked]; [email-masked]
Subject: RE: iSCSI: Negotiation clarifications still needed

.
.
.
John L. Hufferd

From: Bill Studenmund
<snip>

Bill,

Regards,
Pat

---

From: "Julian Satran" <[email-masked]>
Date: Fri, 31 May 2002 07:10:29 +0300

One piece of information that will help you parsing. 
In fact the fact that convinced me to change position is realizing that 
adding a 0 byte after the last key=value
is not enough to distinguish the last key=value you have to have at least 
two in the absence of the C bit.

As for the rest I will make the text adjustements over (my) weekend.

Julo

[email-masked]
05/30/2002 10:07 PM

        To:     John Hufferd/San Jose/IBM@IBMUS
        cc:     [email-masked], Julian Satran/Haifa/IBM@IBMIL, 
[email-masked], [email-masked], [email-masked]
        Subject:        RE: iSCSI: Negotiation clarifications still needed

I can't parse (on page 68):

A value is whatever follows the = that follows the key-name up to a
zero-byte delimiter that separates a key=value pair from the next or
up to the end of data (for the last key=value pair if the PDU C flag
bit is set to 0).

I think I know what he meant but I can't get the text to parse clearly.
It is trying to do too much in one sentence.
How about:
A value is whatever follows the = that follows the key-name up to a
zero-byte delimiter. The zero-byte delimiter separates a key=value 
pair from the next and follows the last key=value pair when the PDU C flag
bit is set to 0.

The text on the bottom of page 72 is sufficient to produce the behavior
we want but it doesn't make clear the reason for having the C bit. We
don't need the C bit to know that the last value was incomplete. We need
it to allow us to prepare a multi-PDU buffer of key-value pairs for 
transmission.
How about:
Key=value pairs may span PDU boundaries. 

The C flag bit allows an initiator or target to prepare a set of
key values larger than the PDU size for transmission and send the set in
multiple 
PDUs ensuring that it will not receive key-value pairs while those PDUs 
are 
being transmitted. An initiator or target wishing to do this indicates 
that
more
text follows by setting the C flag bit in the Text Request or Text
Response to 1. A target or initiator receiving a Text Request or 
Text Response, respectively, with the C flag bit set to 1 MUST answer with 
a
Text Response or Text Request with no data segment (DataSegmentLength
0). A Text Request or Text Response PDU having the C flag
bit set to 1 MUST NOT have the F bit set to 1.

In 4 the bit is called the C flag bit. In 9.10 and 9.11, it is called the 
C
bit.
It should be called the same thing everywhere. Otherwise it becomes hard 
to
search for. As noted earlier the bit also needs to be added to 9.12 and
9.13.

With that done, I am content with the text though much the same thing
could be accomplished by enlarging PDU size during negotiation. The C bit 
just builds a super-PDU with pacing by the partner.

Regards,
Pat

-----Original Message-----
From: John Hufferd [mailto:[email-masked]]
Sent: Wednesday, May 29, 2002 8:23 PM
To: [email-masked]
Cc: [email-masked]; [email-masked]; Julian Satran;

I take it that we are all happy with the current C bit stuff (if it also 
is
in Login).  I think that is the hand off token.

.
.
.
John L. Hufferd

From: Bill Studenmund
<snip>

Bill,

As I mention in the response to John, I would like to see proposed text 
for
specifying the hand-off.

I think one could approach this by handing back and forth the right to 
make
negotiation offers and leave to the implementations whether they send 
empty
text negotiations or responses and declarations when they don't have the
right to make negotiation offers. I think the hand-off should be based on 
a
bit flag rather than based on whether the last PDU received ended in a
complete key-value pair.

Regards,
Pat