Subject: Re: [iSCSI]: Key negotiation procedure proposal

From: "Julian Satran" <[email-masked]>
Date: Fri, 24 May 2002 07:43:43 +0300

All this thread is based on the assumption that discovery of what other 
side capabilities are is a good thing to have in the protocol.
Except for one item the group has decided a long time ago against it.
The messages consume an inordinate amount of time from all of us - and we 
are looking for things to fix (big or small).
If you think that negotiations have to be done some other way that is 
essentially better - please discuss this between yourselves and  submit an 
ID.
If something is badl in what we do now please be as concrete and brief as 
possible in outlining what is wrong and all of us will certainly listen - 
as we did in the past.

Julo

Martins Krikis <[email-masked]>
Sent by: [email-masked]
05/24/2002 12:33 AM
Please respond to Martins Krikis

        To:     [email-masked]
        cc: 
        Subject:        Re: [iSCSI]: Key negotiation procedure proposal

Bill, Luben,

Sorry, my previous response was already somewhat dated. Hard to keep
up with you. :-)

> Regular implementations:
> 
> Core rule: A negotiation of a key=value pair is
> complete/finished when both the Originator and Responder
> have sent their values (non-reserved keywords).
> 
> The core rule implies the the following:
> 
> If a Responder replies with Reject, then the Originator
> SHOULD NOT renegotiate the rejected key.

Does it mean it should not send or does it mean "should not send
with a non-reserved value"?

> If a Responder replies with Reject, it SHOULD send its value
> of that key in the same reply PDU to the Originator after the
> key=Reject pair.

There is the problem that it may not fit. If we leave it for the
next round, we have to remember that the state for this key is such.
In order to avoid this problem, we could use the key only once,
but use a list of values: first element "Reject", next elements
whatever was the value (or value list) that the Responder would
like. But, that's another big complication, so I am NOT proposing it.

> If an Originator finds the response to an offered key=value
> pair to be unacceptable, it SHOULD send Reject and close the
> connection.

But then the Responder (who both Reject-ed and responded to the key)
must remember that it not only sent the key but also rejected the
key, i.e., sent a value that does not necessarily fit in the selection
rules and which is not guaranteed to be liked by the Originator,
because if it is not liked, this Originator most likely will
send a Reject, which is not to be treated as a no-renegotionation
violation but just as a harmless little Reject. 

Thus, I think we need more state for this. But we gain the knowledge
of what values the other side likes. I like the latter feature
of course, but hate introducing more state. Plus I was getting
to feel happy about the current situation.

If the goal of this is to have a way to figure out what the other
side likes, perhaps it is simpler to reinstate the "key=?"
syntax, with the changed semantics, that the Responder must
send its acceptable values (not current). This would not count
as a start of a new negotiation, of course, so no extra state here.

I.e., 

O: k=u,v,w                   (I want (in order of preference) u, v or w)
R: k=Reject                  (But I hate all of those)
O: k=?                       (What do you like then?)
R: k=x,y,z                   (I like (in order of preference) x, y and z)

  Martins Krikis, Intel Corp.

Disclaimer: these opinions are mine and may not be those of my employer.

---

From: Martins Krikis <[email-masked]>
Date: Fri, 24 May 2002 12:38:37 -0700 (PDT)

The previous thread went on too long, but since it
has now quieted down, I'll conjecture that the
following are the aspects that still need to be
addressed.

1. Not everybody seemed to have noticed that it is
   NOT legal to send the same key again, if it
   has once been negotiated (including negotiations
   that end with a reserved value (Reject, 
   Irrelevant or NotUnderstood).

I think it would benefit the draft to add the
sentence that Pat proposed to paragraph 5 or page
72 in 12-92: "Sending the key again would be a
re-negotiation". That I think would make it crystal
clear.

2. When the key=value pairs that Originator is
   sending are broken across multiple PDUs, it
   is not clear whether the Responder may start
   responding to the keys as soon as it receives
   them or whether it should send blank PDUs back
   (as in the example on page 164 of 12-92) until
   it gets a PDU, the data part of which ends in
   a NUL byte (thus signaling that there are no
   broken key=value pairs at the end of it).

I am proposing that the draft should make it explicit
that only blank PDUs are to be sent. This allows
decoupling of key=value generation from
their encapsulation in PDUs (i.e., the generating
logic need not worry about whether a key=value
pair will fit and go out in this PDU or has to
be retained to go out in the next). I can explain
in detail why this is important (it has to do
with teh possibility of receiving the "just-about
outgoing" keys) but I'm keeping this "brief".

Furthermore, it is my feeling that instead of
checking the last bytes of a PDU for NUL, it
would be better if the end-of-data was marked by
a flag in the header. This way encapsulation will
be simpler---just put as much data in the PDU as
fits there and raise the flag if it isn't all,
instead of checking whether it ends in a NUL and
possibly shortening data to make it not end like
that.

3. There is an opinion that on page 73 of 12-92,
   the phrase that says "or the responder may
   select an admissible value" is in contradiction
   to the very next sentence. There is also an
   opinion that this phrase is entirely unnecessary
   and detrimental to achieving broad 
   interoperability (I call it "cutting slack to
   misbehaving or incompatible originators").

I don't have a suggestion since I consider the
"feature" that this phrase allows of little 
importance to a properly built iSCSI node.

4. This is new. When doing Text Request/Response
negotiations (i.e., in FFP), it seems 
that the Initiator commits to the new values
when it receives a response from the Target with
the F bit set. It is unclear when the Target should
commit. Should it switch to using the new values
as soon as it sends its response with the F-bit
set, or should it do so only when it knows that
the Initiator received its response? 

Commiting right away is simpler and since responses
with F-bits set have TTT=0xffffffff and thus
may not be reset, sounds plausible. If the
values have importance on the next reception,
it may also be important to commit timely. However,
what if the Initiator doesn't get this response? 
Target now has committed, Initiator hasn't. Committing
later puts the burden on Initiator to send something
effectively telling "I've received your final
response". Otherwise the Target will time out and
not commit. This response can get lost too.

Basically, it is beginning to look a bit like
(what was it called?) "distributed consensus problem"?
I think it goes like this:

Two generals that are on oposite sides of the
enemy want to synchronize their attack, and
start sending messengers through with messages
like 
  "attack at dawn->", 
  "<-ok, attack at dawn",
  "I know you know we attack at dawn->",
  "<-, I know you know I know we attack at dawn",
  etc., etc., ...
But at no point can they commit yet...

Is anybody else worried about this?
Anyway, so when should a target commit? Page 83
of 12-92 is the relevant reference.

Thanks,

  Martins Krikis

Disclaimer: these opinions are my own and may
            not be those of my employer.

__________________________________________________
Do You Yahoo!?
LAUNCH - Your Yahoo! Music Experience
http://launch.yahoo.com

---

From: Bill Studenmund <[email-masked]>
Date: Fri, 24 May 2002 12:53:57 -0700 (PDT)

> side capabilities are is a good thing to have in the protocol.

We must be reading different threads. While I've seen discovery mentioned
and discussed, it's not been a base assumption of what I understood the
thread was about.

> are looking for things to fix (big or small).

The thread was about the fact people think that login negotiation needs
fixing. It could be the negotiation methods need changing (as in Lubin's
suggestion), or the text needs changing to something more like what John
wrote.

The point though is that the current negotiation description is broken in
the minds of a number of implementors. It sounds like it is not broken in
the minds of a number of the WG members who have been involved for a
while. So from that I conclude the spec hasn't been capturing what is in
the WG's thoughts; the spec needs work.

> essentially better - please discuss this between yourselves and  submit an

I think that would be a reasonable suggestion if we were talking about
making it spiffier & better. But we're complaining that we think it's
broken. Telling us to go off and come back with an ID does not strike me
as a suggestion which will help us converge to an answer.

Take care,

Bill

---

From: "Mallikarjun C." <[email-masked]>
Date: Fri, 24 May 2002 16:49:26 -0700

Martins,

Comments in the same order.

1. Julian may be adding wording for this.

2. Seems to me that we need to stipulate that only key values 
    may straddle PDU boundaries - key names (ideally inclusive 
    of "=") shall not.  That should avoid requiring blank PDUs.

3. I don't have a strong opinion on this.  I don't believe this can 
    cause interoperability problems; OTOH, I don't care if the responder
    isn't that liberal.

4. Julian and I discussed this scenario a while ago.  As you precisely
    described, only a non-stop dialogue can be reliable and we can't
    afford a non-stop dialogue.  I suppose one can recover the StatSN
    with a SNACK in FFP, or terminate the connection upon a timeout.
--
Mallikarjun

Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions
Hewlett-Packard MS 5668 
Roseville CA 95747
[email-masked]

---

From: Martins Krikis <[email-masked]>
Date: Fri, 24 May 2002 17:45:09 -0700 (PDT)

>     of "=") shall not.  That should avoid requiring

Definitely inclusive of "=". Otherwise there are
no strict guarantees that the whole key has been
received (we may have keys that are prefixes to
other keys in future, and vendor keys may be like
that).

I can implement it and live with it if that's what
people decide on. I just want people to understand
that it is conceptually a more complex aproach than
requiring strictly blank PDUs and having a data-end
flag. (Even if we don't add a new flag and decide
data-end by looking for NUL-s at the end, we get
a cleaner scheme.) Allowing non-blank PDUs  gives 
better bandwith utilization under
presumably rare circumstances (all pairs not fitting),
but introduces more complexity for your everyday case.

If we only stipulate that no key ever gets broken,
then there is the following "clumsiness" required.
When "originating" a key=value pair, it has to be
checked whether the key will fit fully in the PDU
or not, and if not, then "originating" must be
postponed. We also need either a more elaborate
state for the key (to mark key as received but
value as not; so it is not processable yet, nor
originatable anymore), or for each key=value pair
being originated, key has to be checked against
the buffer that would hold the "leftover stuff"
from the just received PDU, otherwise we may
illegally originate it. Doable, but not clean.

I still think that letting the whole set-of-pairs
go through (no matter over how many PDUs spread)
before the other side is allowed to process them
is cleanest. 

Disclaimer: these opinions are my own and may
            not be those of my employer