Subject: Re: iSCSI: Negotiation clarifications still needed

From: "Julian Satran" <[email-masked]>
Date: Sat, 25 May 2002 08:10:25 +0300

I agree with Mallikarjun on all but one point - requiring a key not to 
straddle PDU boundaries.
This list had a long debate on the subject. We thought (initially) that 
not having any pair straddle PD boundaries is the best thing to do and we 
had it untill it became obvious that we have to support very short PDUs.
The we went for allowing spanning and the scheme we had in  mind was a 
sender preparing its strings and chopping them into PDUs without any 
parsing and state. What you suggest now is parsing at send and that may 
add unnecessary complexity - and I really don't see why.

Julo

"Mallikarjun C." <[email-masked]>
05/25/2002 02:49 AM
Please respond to "Mallikarjun C."

        To:     "Martins Krikis" <[email-masked]>, Julian Satran/Haifa/IBM@IBMIL
        cc:     <[email-masked]>
        Subject:        Re: iSCSI: Negotiation clarifications still needed

Martins,

Comments in the same order.

1. Julian may be adding wording for this.

2. Seems to me that we need to stipulate that only key values 
    may straddle PDU boundaries - key names (ideally inclusive 
    of "=") shall not.  That should avoid requiring blank PDUs.

3. I don't have a strong opinion on this.  I don't believe this can 
    cause interoperability problems; OTOH, I don't care if the responder
    isn't that liberal.

4. Julian and I discussed this scenario a while ago.  As you precisely
    described, only a non-stop dialogue can be reliable and we can't
    afford a non-stop dialogue.  I suppose one can recover the StatSN
    with a SNACK in FFP, or terminate the connection upon a timeout.
--
Mallikarjun

Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions
Hewlett-Packard MS 5668 
Roseville CA 95747
[email-masked]

----- Original Message ----- 
From: "Martins Krikis" <[email-masked]>
To: "Julian Satran" <[email-masked]>
Cc: <[email-masked]>
Sent: Friday, May 24, 2002 12:38 PM
Subject: iSCSI: Negotiation clarifications still needed

> The previous thread went on too long, but since it
> has now quieted down, I'll conjecture that the
> following are the aspects that still need to be
> addressed.
> 
> 1. Not everybody seemed to have noticed that it is
>    NOT legal to send the same key again, if it
>    has once been negotiated (including negotiations
>    that end with a reserved value (Reject, 
>    Irrelevant or NotUnderstood).
> 
> I think it would benefit the draft to add the
> sentence that Pat proposed to paragraph 5 or page
> 72 in 12-92: "Sending the key again would be a
> re-negotiation". That I think would make it crystal
> clear.
> 
> 2. When the key=value pairs that Originator is
>    sending are broken across multiple PDUs, it
>    is not clear whether the Responder may start
>    responding to the keys as soon as it receives
>    them or whether it should send blank PDUs back
>    (as in the example on page 164 of 12-92) until
>    it gets a PDU, the data part of which ends in
>    a NUL byte (thus signaling that there are no
>    broken key=value pairs at the end of it).
> 
> I am proposing that the draft should make it explicit
> that only blank PDUs are to be sent. This allows
> decoupling of key=value generation from
> their encapsulation in PDUs (i.e., the generating
> logic need not worry about whether a key=value
> pair will fit and go out in this PDU or has to
> be retained to go out in the next). I can explain
> in detail why this is important (it has to do
> with teh possibility of receiving the "just-about
> outgoing" keys) but I'm keeping this "brief".
> 
> Furthermore, it is my feeling that instead of
> checking the last bytes of a PDU for NUL, it
> would be better if the end-of-data was marked by
> a flag in the header. This way encapsulation will
> be simpler---just put as much data in the PDU as
> fits there and raise the flag if it isn't all,
> instead of checking whether it ends in a NUL and
> possibly shortening data to make it not end like
> that.
> 
> 3. There is an opinion that on page 73 of 12-92,
>    the phrase that says "or the responder may
>    select an admissible value" is in contradiction
>    to the very next sentence. There is also an
>    opinion that this phrase is entirely unnecessary
>    and detrimental to achieving broad 
>    interoperability (I call it "cutting slack to
>    misbehaving or incompatible originators").
> 
> I don't have a suggestion since I consider the
> "feature" that this phrase allows of little 
> importance to a properly built iSCSI node.
> 
> 4. This is new. When doing Text Request/Response
> negotiations (i.e., in FFP), it seems 
> that the Initiator commits to the new values
> when it receives a response from the Target with
> the F bit set. It is unclear when the Target should
> commit. Should it switch to using the new values
> as soon as it sends its response with the F-bit
> set, or should it do so only when it knows that
> the Initiator received its response? 
> 
> Commiting right away is simpler and since responses
> with F-bits set have TTT=0xffffffff and thus
> may not be reset, sounds plausible. If the
> values have importance on the next reception,
> it may also be important to commit timely. However,
> what if the Initiator doesn't get this response? 
> Target now has committed, Initiator hasn't. Committing
> later puts the burden on Initiator to send something
> effectively telling "I've received your final
> response". Otherwise the Target will time out and
> not commit. This response can get lost too.
> 
> Basically, it is beginning to look a bit like
> (what was it called?) "distributed consensus problem"?
> I think it goes like this:
> 
> Two generals that are on oposite sides of the
> enemy want to synchronize their attack, and
> start sending messengers through with messages
> like 
>   "attack at dawn->", 
>   "<-ok, attack at dawn",
>   "I know you know we attack at dawn->",
>   "<-, I know you know I know we attack at dawn",
>   etc., etc., ...
> But at no point can they commit yet...
> 
> Is anybody else worried about this?
> Anyway, so when should a target commit? Page 83
> of 12-92 is the relevant reference.
> 
> Thanks,
> 
>   Martins Krikis
> 
> Disclaimer: these opinions are my own and may
>             not be those of my employer.
> 
> 
> __________________________________________________
> Do You Yahoo!?
> LAUNCH - Your Yahoo! Music Experience
> http://launch.yahoo.com
>

---

From: Bill Studenmund <[email-masked]>
Date: Tue, 28 May 2002 12:12:33 -0700 (PDT)

> add unnecessary complexity - and I really don't see why.

I'm going to try and explain why; to answer the question in the end of the
above paragraph.

It's to avoid corner cases while doing what you say above, "preparing its
strings and chopping them into PDUs."

The simplest thing in my mind is to:

1) read the other side's offers into a buffer

2) parse that buffer generating responses to another buffer

3) send the output buffer

Does that make sense? You don't have to agree, just does it make sense?

The problem I see with what Pat described is that if you are in
operational negotiation and have more than 1 PDU worth of data, you will
be receiving input while you're in step 3. You have to be able to re-enter
steps 1 & 2 before you can finish step 3.

The bit about sending empty PDUs is an effort to avoid that. We are either
in a sending-negotiation-text mode, or a parsing mode. We don't have to
worry about strings being partially there, since we don't process until
they are. We also don't have to worry about if our key/value pairs
cross a PDU. The parser/negotiatior is MUCH simpler; it doesn't have to
deal with all of that.

Letting the target answer while the initiator is trying to send an
extended PDU adds the following complications (AFAICT):

1) initiator has to make sure a key and the '=' don't get split on a PDU
boundry.

2) if an initiator fills up a PDU, it has to stop processing & wait for
the response. It also has to remember where it is so that it can finish
sending that key/value next time.

3) The target has to detect a split key/value pair on in-bound, and a)
store the received text, and b) remember not to offer said key or any
others.

4) The target ALSO has to make sure ITS response doesn't fill up a PDU. If
it does, it has to remember where IT is and that it has more to send.

That's a lot of complexity and saved state. At least in my mind.

That's the advantage I see of blank PDUs; we avoid all of the above
complications.

Truth be told, the corner cases probably don't apply at present. What we
have (and are talking about) will work for security negotiation (we have
to wait for the key to make it across) and for operational negotiation
(since I don't think we have 8k worth of negotiation data). So for now the
common case (no large vendor extentions) isn't going to tickle this set of
problems.

Does the statement of the perceived problem make sense, even if the
opinion of the WG is to not use the empty PDUs? I would feel much
comfortable with deciding to not use empty PDUs if folks said they
understand the above point and they choose to address the problems
differently.

Thoughts?

Take care,

Bill