Subject: RE: iSCSI: Login negotiation space

From: [email-masked]
Date: Mon, 1 Jul 2002 16:56:09 -0600

Julian,

4.1 contains a requirement that initiators and targets support receiving at least 16384 bytes of key=value data (when not supporting very long authentication items). This is overkill and is making drivers larger than they need to be. 

My calculations are that Security negotiation using the CHAP method (including the Any-stage keys) takes less than 1600 bytes. Operational negotiation requires 1276 bytes (also including the Any-stage keys). All the operational plus security keys are 1911 bytes.

Therefore an implementation could hold all the necessary keys in 2 Kbytes. And, the implementation doesn't need to keep the security keys after the security negotiation is done so it could clear some of that out to make room for future keys or vendor specific keys. 16 K bytes is about 10 times the necessary storage in that case.

The number of negotiation bytes that a device is required to support should be reduced substantially - preferably to 2048 bytes.

Regards,
Pat

The numbers:

  43		AuthMethod
           CHAP keys 
 264		Name (no defined size limit - using 255)
  11		Algorithm
  11		Identifier
 264		Challenge
  42		Response (for MD5)
_____
 635 Chap negotiation length
 892 Any-stage keys
----
1527

Operational negotiation keys
  24		HeaderDigest
  24        DataDigest
  49        MaxConnections
 270		TargetName* or InitiatorName*
 271		TargetAlias* or InitiatorAlias*
 273        TargetAddress*
  55		TargetPortalGroupTag*
  15		InitialR2T
  19		BidiInitialR2T
  18		ImmediateData
  34		MaxRecvDataSegmentLength
  24		MaxBurstLength
  26		FirstBurstLength
  24		DefaultTime2Wait
  26		DefaultTime2Retain
  25		MaxOutstandingR2T
  18		DataPDUInOrder
  24		DataSequenceInOrder
  24		ErrorRecoveryLevel
  23		SessionType*
------
 892 Any-stage keys
 384 Not Any-stage
------
1276 Operational keys

---

From: "Julian Satran" <[email-masked]>
Date: Tue, 2 Jul 2002 10:10:21 +0300

Pat,

A factor of 10 over what is needed today is what I would call a 
conservative design.
Limiting the support to 2k would be extremely unwise.
And 16 is a tiny amount of memory - of no concern to software 
implementation and to most hardware implementations.

Julo

[email-masked]
07/02/2002 01:56 AM
Please respond to pat_thaler

        To:     Julian Satran/Haifa/IBM@IBMIL
        cc:     [email-masked]
        Subject:        RE: iSCSI: Login negotiation space

Julian,

4.1 contains a requirement that initiators and targets support receiving 
at least 16384 bytes of key=value data (when not supporting very long 
authentication items). This is overkill and is making drivers larger than 
they need to be. 

My calculations are that Security negotiation using the CHAP method 
(including the Any-stage keys) takes less than 1600 bytes. Operational 
negotiation requires 1276 bytes (also including the Any-stage keys). All 
the operational plus security keys are 1911 bytes.

Therefore an implementation could hold all the necessary keys in 2 Kbytes. 
And, the implementation doesn't need to keep the security keys after the 
security negotiation is done so it could clear some of that out to make 
room for future keys or vendor specific keys. 16 K bytes is about 10 times 
the necessary storage in that case.

The number of negotiation bytes that a device is required to support 
should be reduced substantially - preferably to 2048 bytes.

Regards,
Pat

The numbers:

  43                             AuthMethod
           CHAP keys

---

From: [email-masked]
Date: Tue, 2 Jul 2002 13:02:59 -0600

Julian,
 
A factor of 10 is way overkill. It isn't just 16 k bytes of extra memory. It is n * 16k where n is the number of simultaneous logins the implementation supports - it adds up. The system vendors we work with want to use the memory to do useful work and demand that driver sizes be kept small -- they want the memory available to do "real" work. 
 
If 2k doesn't leave enough headroom, then we could live with somewhat more like 4k. 
 
If future features added to iSCSI require more space, the drivers that support that can allocate more memory. It is an internal parameter that can be changed when the need arises. There is no future interoperability need to make the buffer oversized in current implementations.
 
Regards,
Pat
 
-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Tuesday, July 02, 2002 12:10 AM
To: [email-masked]
Cc: [email-masked]
Subject: RE: iSCSI: Login negotiation space

Pat, 

A factor of 10 over what is needed today is what I would call a conservative design. 
Limiting the support to 2k would be extremely unwise. 
And 16 is a tiny amount of memory - of no concern to software implementation and to most hardware implementations. 

Julo 

	[email-masked] 

07/02/2002 01:56 AM 

Julian,

Regards,
Pat

The numbers:

 43                                  AuthMethod
          CHAP keys

---

From: [email-masked]
Date: Tue, 2 Jul 2002 16:08:23 -0600

Julian,
 
There is a separate number for cases where very long authentication items are supported and I haven't asked to change that number. 
 
Remember the text is:
Any iSCSI target or initiator MUST support receiving at least 16384 bytes of key=value data in a negotiation sequence except when indicating support for very long authentication items by offering or selecting authentication methods such as public key certificates in which case they MUST support receiving at least 64 kilobytes of key=value data.

Changing the 16384 number on the first line to 4096 will have no effect on the 64 kilobytes limit on the last line and implementations desiring to support very long authentication items are covered.
 
4k works fine for the required CHAP authentication and there is no reason to require 16k. Controlling memory by delaying negotiation with some parties as memory fills is an unnecessary complication which can result in deadlocks. 
 
If one does what you suggest and throttles keys "as you have unprocessed keys exceeding the mount of memory you want to keep aside", then you may get to that point but find that none of the keys you have are processable - the C bit is set and you need to get some more packets before you can process. 
 
One could throttle just some logins when one has enough memory left to complete some but not all of the negotiations, but there is a risk of deadlock. A simple example: Target X decides to stall negotiation from Initiator B until negotiation from Initiator A completes freeing buffer. Initiator A has stalled negotiation with Target X until negotiation with Target Y completes freeing buffer. Target Y has stalled negotiation with Initiator A until negotiation with Initiator B completes. Initiator B has stalled negotiation with Target Y until negotiation with Target X completes. They are all stalled until one or more of the logins times out.
 
It is much safer and simpler to set the required memory size to a reasonable limit for the mandatory negotiation items plus some comfortable room for expansion.
 
Regards,
Pat
 
-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Tuesday, July 02, 2002 12:36 PM
To: [email-masked]
Subject: RE: iSCSI: Login negotiation space

Pat, 

4k is absurd. We have the security guys asking for 64k certificates! 
And the PDUlength default is 8k! 
Besides you can control the total memory consumed by throttling login as soon as you have unprocessed keys exceeding the mount of memory you want to keep aside (you control the login progress by delaying requests/responses). 

Julo 

	[email-masked] 

07/02/2002 10:02 PM 

        To:        Julian Satran/Haifa/IBM@IBMIL, [email-masked] 
        cc:        [email-masked] 
        Subject:        RE: iSCSI: Login negotiation space 

Julian, 

  
  
Regards, 
Pat 
  
-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Tuesday, July 02, 2002 12:10 AM
To: [email-masked]
Cc: [email-masked]
Subject: RE: iSCSI: Login negotiation space

Pat, 

Julo 

	[email-masked] 

07/02/2002 01:56 AM 

Julian,

Regards,
Pat

The numbers:

43                                  AuthMethod
         CHAP keys

---

From: "Julian Satran" <[email-masked]>
Date: Wed, 3 Jul 2002 18:34:19 +0300

Pat,

4k does not make too much sense either with default PDU size of 8k.
Are you suggesting we limit this too to a lower value?
Your deadlock example while academically correct is not very interesting 
as I assume that throttling can be done at the first buffer level.

Julo

[email-masked]
Sent by: [email-masked]
07/03/2002 01:08 AM
Please respond to pat_thaler

        To:     "Julian Satran" <[email-masked]>, [email-masked]
        cc: 
        Subject:        RE: iSCSI: Login negotiation space

Julian,
 
There is a separate number for cases where very long authentication items 
are supported and I haven't asked to change that number. 
 
Remember the text is:
Any iSCSI target or initiator MUST support receiving at least 16384 bytes 
of key=value data in a negotiation sequence except when indicating support 
for very long authentication items by offering or selecting authentication 
methods such as public key certificates in which case they MUST support 
receiving at least 64 kilobytes of key=value data.
Changing the 16384 number on the first line to 4096 will have no effect on 
the 64 kilobytes limit on the last line and implementations desiring to 
support very long authentication items are covered.
 
4k works fine for the required CHAP authentication and there is no reason 
to require 16k. Controlling memory by delaying negotiation with some 
parties as memory fills is an unnecessary complication which can result in 
deadlocks. 
 
If one does what you suggest and throttles keys "as you have unprocessed 
keys exceeding the mount of memory you want to keep aside", then you may 
get to that point but find that none of the keys you have are processable 
- the C bit is set and you need to get some more packets before you can 
process. 
 
One could throttle just some logins when one has enough memory left to 
complete some but not all of the negotiations, but there is a risk of 
deadlock. A simple example: Target X decides to stall negotiation from 
Initiator B until negotiation from Initiator A completes freeing buffer. 
Initiator A has stalled negotiation with Target X until negotiation with 
Target Y completes freeing buffer. Target Y has stalled negotiation with 
Initiator A until negotiation with Initiator B completes. Initiator B has 
stalled negotiation with Target Y until negotiation with Target X 
completes. They are all stalled until one or more of the logins times out.
 
It is much safer and simpler to set the required memory size to a 
reasonable limit for the mandatory negotiation items plus some comfortable 
room for expansion.
 
Regards,
Pat
 
-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Tuesday, July 02, 2002 12:36 PM
To: [email-masked]
Subject: RE: iSCSI: Login negotiation space

Pat, 

4k is absurd. We have the security guys asking for 64k certificates! 
And the PDUlength default is 8k! 
Besides you can control the total memory consumed by throttling login as 
soon as you have unprocessed keys exceeding the mount of memory you want 
to keep aside (you control the login progress by delaying 
requests/responses). 

Julo 

[email-masked] 
07/02/2002 10:02 PM 

Julian, 
  
A factor of 10 is way overkill. It isn't just 16 k bytes of extra memory. 
It is n * 16k where n is the number of simultaneous logins the 
implementation supports - it adds up. The system vendors we work with want 
to use the memory to do useful work and demand that driver sizes be kept 
small -- they want the memory available to do "real" work. 
  
If 2k doesn't leave enough headroom, then we could live with somewhat more 
like 4k. 
  
If future features added to iSCSI require more space, the drivers that 
support that can allocate more memory. It is an internal parameter that 
can be changed when the need arises. There is no future interoperability 
need to make the buffer oversized in current implementations. 
  
Regards, 
Pat 
  
-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Tuesday, July 02, 2002 12:10 AM
To: [email-masked]
Cc: [email-masked]
Subject: RE: iSCSI: Login negotiation space

Pat, 

Julo 

[email-masked] 
07/02/2002 01:56 AM 

Julian,

Regards,
Pat

The numbers:

43                                  AuthMethod
         CHAP keys 
264                                  Name (no defined size limit - using 
255)
11                                  Algorithm
11                                  Identifier
264                                  Challenge
42                                  Response (for MD5)
_____
635 Chap negotiation length
892 Any-stage keys
----
1527

---

From: "John Hufferd" <[email-masked]>
Date: Wed, 3 Jul 2002 23:34:02 -0700

Pat,
I guess I do not see a real problem here.  This is a buffering design
issue, which if done well and dynamically, does not have a problem.  It is
not clear that anything needs to change.  16k is not very large (and many
times this is divided into even smaller segments that are taken from a pool
dynamically).   And I have not seen the case where so many logins occurring
at the same time that an real issue exists.

Also, the issue of everyone stalling out, is in my opinion a matter of bad
design.

I do not believe there is an important issue here since regardless of what
size you come up with, the theory of problems occurring with a bad design
is the same.  At some point there is always a limit of resources, and the
design is what causes it to occur at one time versus another.  I do not
believe, that the fact that the 16k value is not fully used each time is an
important issue.  If the buffering is done well, there is very little
unused space during high utilization.

Also, I do not believe there is an important interlock issue.  That is
because we are talking about a large number of sessions being logged in, or
there is not a problem to begin with.  Therefore, to think that there are
many that are so tied up with the C bit handling that they never free any
space, is not a situation that I find probable.  There will always be some
logins that will be finishing up.  If the one in a billion chance occurs
that they run out of buffers in a well managed buffer pool, then "out of
resources" error is approprate.  Again, changing from 16k to a smaller
value, is not an important issue, since you can always dynamically allocate
smaller buffer segments on your way to meet a 16k need.

.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Home Office (408) 997-6136, Cell: (408) 499-9702
Internet address: [email-masked]

[email-masked]@ece.cmu.edu on 07/03/2002 11:09:20 AM

Julian,

I wouldn't object to  the default PDU size being lowered, but that is not
what I am  suggesting.

There is no conflict between having a default PDU  size of 8k during login
and having a requirement to support at least 4096 bytes  of key=value data
in a negotiation sequence. PDUs don't have to be full. The  situation is no
worse than having a MaxRecvDataSegmentLength of 8k when  MaxBurstLength is
4k or having a 4 kbyte write with an 8kb byte  MaxRecvDataSegmentLength.

The  default MaxRecvDataSegmentLength at 8k  would still be useful when
implementations were supporting very long  authentication items. Also, a
vendor might suport a larger negotiaton space for  vendor specific keys.
Such an implmentation could send a key like  X-com.acme.HugeKeySet=yes. If
the partner responds X-com.acme.HugeKeySet=yes,  then it can proceed to
send the hugh key set knowing that its partner supports  it and therefore
has allocated space for it. If it gets a no or NotUnderstood  response, it
doesn't send the extra keys.

I disagree  with your statement about deadlock. It is a real issue when
throttling is  engaged.

Regards,
Pat

-----Original Message-----
From: Julian Satran  [mailto:[email-masked]]
Sent: Wednesday, July 03, 2002 8:34  AM
To: [email-masked]
Cc:  [email-masked]
Subject: RE: iSCSI: Login negotiation  space

Pat,

Julo

---

From: "Julian Satran" <[email-masked]>
Date: Fri, 5 Jul 2002 21:25:25 +0300

Pat,

I had trouble understanding the "basics" underneath your claim that 16k is 
excessive.
If you are concerned about many sessions having to provide this space - 
the targets supporting many sessions 
 must anyhow have many resources to do so (i.e., they are high end 
devices) .

The vast majority of those will go for the TCP window. the little that has 
to go for session management will either be in some host space or will be 
shared with other buffers for the session. 

As for throttling - if you have the 16k from the outset of a ngegtotiation 
I do not see how your real dealock can occur.

I am not sure that this thread is expresses a decent level of maturity but 
if 16k is really of concern I suggest we get it down to one PDU i.e., 8k 
and conclude the thread before time runs out.

And only to add a grain of salt to all this the 16 was not arbitrarily 
chosen. It is based on a very simple traffic model and on the assumption 
that a socket consumes around 6k in idle state.

Julo

[email-masked]
07/03/2002 09:09 PM

Julian,

 

Regards,
Pat
 
-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Wednesday, July 03, 2002 8:34 AM
To: [email-masked]
Cc: [email-masked]
Subject: RE: iSCSI: Login negotiation space

Pat, 

Julo