Subject: RE: iSCSI: Negotiation clarifications still needed

From: [email-masked]
Date: Tue, 28 May 2002 18:48:26 -0600

Bill,

My comments are inserted in line bracketed with my initials. 

Pat

-----Original Message-----
From: Bill Studenmund [mailto:[email-masked]]
Sent: Tuesday, May 28, 2002 12:13 PM

<snip>
It's to avoid corner cases while doing what you say above, "preparing its
strings and chopping them into PDUs."

The simplest thing in my mind is to:

1) read the other side's offers into a buffer

2) parse that buffer generating responses to another buffer

3) send the output buffer

Does that make sense? You don't have to agree, just does it make sense?

The problem I see with what Pat described is that if you are in
operational negotiation and have more than 1 PDU worth of data, you will
be receiving input while you're in step 3. You have to be able to re-enter
steps 1 & 2 before you can finish step 3.

<PAT> I think that is over stating the complexity. During negotiation,
the other side can only have one PDU outstanding at a time. That is explictly
stated for TextNegotiation PDUs. For LoginPDUs, it isn't explicit but since
they are immediate and the initiator can only count on the target having one
immediate buffer for the connection it can't send another LoginRequest until
it gets the LoginResponse without risking that a PDU will be dropped. Also
allowing anything other than one oustanding LoginPDU at a time would open 
the possibility that both sides offered the same key. Therefore, a target
won't get the next PDU until after it sends the response PDU (or vice-versa
for an initiator). A PDU won't come in while a PDU is being sent from the
outgoing negotiation buffer.

Once you send the PDU, there may still be some data left in the output
buffer but it doesn't seem complex to switch at that point to waiting to
receive and parse the next received PDU. One had better be able to switch
from one process to another anyway as most implementations will handle 
multiple connections which can be in diffent phases. If you are willing
to buffer all your output into a buffer and wait to send it until the other
side has stopped sending new parameters, then all you need to do to change
that to an implementation that sends what it has in the output buffer as 
it goes along is to have pointer into the buffer that indicates the next
byte to be sent. <PAT>

The bit about sending empty PDUs is an effort to avoid that. We are either
in a sending-negotiation-text mode, or a parsing mode. We don't have to
worry about strings being partially there, since we don't process until
they are. We also don't have to worry about if our key/value pairs
cross a PDU. The parser/negotiatior is MUCH simpler; it doesn't have to
deal with all of that.

Letting the target answer while the initiator is trying to send an
extended PDU adds the following complications (AFAICT):

1) initiator has to make sure a key and the '=' don't get split on a PDU
boundry.

<PAT> One doesn't have to do that anyway. We can chose the rule to not
offering a new (non-declarative key) when one has received a partial key
which seemed to be acceptable to a number of people. <PAT>

2) if an initiator fills up a PDU, it has to stop processing & wait for
the response. It also has to remember where it is so that it can finish
sending that key/value next time.

<PAT> It has to remember where it was and finish the key/value next time
anyway as it can't send again until it gets a response. <PAT>

3) The target has to detect a split key/value pair on in-bound, and a)
store the received text, and b) remember not to offer said key or any
others.

<PAT> detecting an incomplete key is reasonably easy. And the target can
chose to do the simplist thing which is not to offer new keys when a 
partial key is outstanding. <PAT>

4) The target ALSO has to make sure ITS response doesn't fill up a PDU. If
it does, it has to remember where IT is and that it has more to send.

<PAT> In your approach the target can have a buffer with more than 
one PDU worth of response in it. Once it starts sending from that buffer
it has to remember where it is and that it has more to send. This is
not any different. There is a buffer with pointers into for end and
last byte sent (or next byte to send). <PAT>

That's a lot of complexity and saved state. At least in my mind.

<PAT> I don't see much complexity difference here. If one wanted to make
negotiation as simple as possible, one would make the Initiator the
originator and have it solicit declarations with ?. When the group 
decided it wanted the flexibility of not offering keys if one was
going to use the default value and letting the target offer such a key
to force the negotiation of it, then a certain amount of complexity 
was introduced. <PAT>

That's the advantage I see of blank PDUs; we avoid all of the above
complications.

Truth be told, the corner cases probably don't apply at present. What we
have (and are talking about) will work for security negotiation (we have
to wait for the key to make it across) and for operational negotiation
(since I don't think we have 8k worth of negotiation data). So for now the
common case (no large vendor extentions) isn't going to tickle this set of
problems.

<PAT> I agree. This means that plug fests probably haven't hit these 
cases. <PAT>

Does the statement of the perceived problem make sense, even if the
opinion of the WG is to not use the empty PDUs? I would feel much
comfortable with deciding to not use empty PDUs if folks said they
understand the above point and they choose to address the problems
differently.

<PAT> I want to point out that there are times in the protocol when 
one will have to send an empty PDU. For instance, when getting a
security key that is longer than a the Max PDU size (or even less
than that - the sender isn't required to send Max size PDUs) or when 
a device has no key-value pairs to send but its partner hasn't set
the F bit. The issue is whether to require that there be a process
for handing the right to originate keys (or in your proposal to send
keys at all) keys from initiator to target and back which adds 
its own complexity. I think the complexity will be
of the same order as the existing situation. <PAT>

Thoughts?

Take care,

Bill

---

From: Bill Studenmund <[email-masked]>
Date: Tue, 28 May 2002 18:51:42 -0700 (PDT)

> My comments are inserted in line bracketed with my initials.

Thank you. I appreciate the time & attention you're putting into this.

> outgoing negotiation buffer.

I agree. I didn't meant to imply we'd do something other than one PDU goes
each direction at a time. You're right that to do THAT would be VERY
messy.

> it goes along is to have pointer into the buffer that indicates the next

I guess the question is where were you when you noticed that you had to
stop? How much state do you need to save to pick things up at that point.
What kind of design permits you to pick things back up at that point?
While negotiations spanning PDUs will mean there is always some state to
keep, the thing about empty PDUs is that there is no state in the parser
code that needs saving; just where are we in the buffer we are sending or
receiving.

> which seemed to be acceptable to a number of people. <PAT>

Maybe I'm confused. I thought I was talking about the case where the
initiator is just realizing that what it wants to send won't fit. i.e.
when we first realize, "oh, we're going to need to split this up." This is
before the other side realizes the PDU is split.

I was sloppy with my use of initiator and target. Sorry. :-(

> last byte sent (or next byte to send). <PAT>

The one difference is that we could end up having both a partially-full
input buffer to parse (because the info sent to us didn't fit a PDU) and
a partially-full output buffer (since our output overflowed a PDU too).
While you're right that the negotiation phase will work around this,
having both buffers open at once is just a bit more complicated.

> to force the negotiation of it, then a certain amount of complexity

To quote code, what I'm envisioning is w/ the Empty PDU idea, things like
PARAM_TEXT_PARSE_ELSE() in the intel code, and scan_input_and_process() in
the UNH code can be used as-is. Without the empty PDU idea, they need to
save more state. Maybe not much, but more.

> <PAT> I agree. This means that plug fests probably haven't hit these

Agreed. That's why I suggested that experiment. Just crank the PDU data
size you (and this "you" is the whole list) use in login negotiations to
something really small, like 128.  Obviously you'd never ship such a
thing, but just see what your code does, and what you think of what your
code does.

You'll find out real quick how well your code does in the case we're
talking about here.

> of the same order as the existing situation. <PAT>

Well, then we have a difference of opinion. :-)

So as John puts it we have moved to the personal opinion stage. :-)

What do we want to decide?

Take care,

Bill