Subject: Text request/response spanning - security issue?

From: Luben Tuikov <[email-masked]>
Date: Thu, 28 Mar 2002 19:12:47 -0500

A key=value pair can span multiple Text Request/Response PDU's.

A rougue/badly implemented T/I can use this ``feature''
to completely drain the I/T resources and stall its
operation.

I.e. the node will keep the data and wait indefinitely until
0x00 in order to process the request. If 0x00 is never
received, the node will eventually run out of memory.

If such an implementation is in kernel space,
then after such an attack, the only solution
is the big red button.

-- 
Luben

---

From: "Mallikarjun C." <[email-masked]>
Date: Thu, 28 Mar 2002 16:37:50 -0800

Luben,

> to completely drain the I/T resources and stall its

I assume you are describing a DoS attack on an iSCSI peer -
it isn't exactly limited only to this feature, there are several other
ways - not sending a Login Request at all after the TCP connection
establishment, not setting the F-bit/T-bit etc.  will all result in this
problem.

The expectation is that implementations will set the right timeouts
to detect and get out of these conditions.  The state transitions 
(chapter 5) allow these timeouts as legal events that could cause a 
Login failure.  Also take a look at section 6.8, which deals with 
timeouts in text negotiations.
--
Mallikarjun

Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions Organization
Hewlett-Packard MS 5668 
Roseville CA 95747
[email-masked]

---

From: Luben Tuikov <[email-masked]>
Date: Thu, 28 Mar 2002 20:40:23 -0500

> timeouts in text negotiations.

This isn't a timeout issue at all (but it is a DoS attack).

The Text Requests can come at the right time,
and none of them may have a 0x00 in it's payload
to indicate end of key=value pair. This will
drain the node's memory.

There needs to be a limitaion (somehow) on how long
a key=value could ever be (spanning or not).
Negotiated or not.

"Buffer overrun" rings a bell.

-- 
Luben

---

From: "Mallikarjun C." <[email-masked]>
Date: Thu, 28 Mar 2002 18:03:38 -0800

> This isn't a timeout issue at all

Okay, I guess I was sidetracked by "wait indefinitely " in 
your note.

> and none of them may have a 0x00 in it's payload

Since we define the max key size in chapter 4, I take it
that you are concerned about very long lists causing buffer
overruns even while each of the options being within legal size
limits?  I haven't checked the draft, but it appears that this
may need clarification.
--
Mallikarjun

Mallikarjun Chadalapaka

---

From: Luben Tuikov <[email-masked]>
Date: Thu, 28 Mar 2002 22:37:28 -0500

"Mallikarjun C." wrote:
> limits?

Yes, exactly.

Furthermore an individual value encoded representation
can have any length -- the draft only specifies that
its _decoded_ representation is <= 255 bytes.

That is "rogue=base64 of length 1e30,bye,bye" will make a node
die. (You get the meaning.)

Since a "KEY=VALUE" cannot contain 0x00 (since it separates it
from other assignments), we need to consider the entity
"KEY=VALUE" as a whole and impose restrictions on it
as a whole. (Reason 1)

We cannot control the format of the VALUE (above),
as companies will add their own keys. (Reason 2)

Thus we need to restrict the "KEY=VALUE" as a whole,
its internals past iSCSI are up to the implementations/
companies which add them.

If we impose restrictions on "KEY=VALUE" then we need not
impose restrictions on the size of KEY or VALUE separately,
just that KEY cannot be an empty sequence.

The node should know in advance how big of a span a
"KEY=VALUE" will be in order to 1) reject it (out of resources)
or 2) prepare for its arrival (whatever this means).

-- 
Luben

---

From: Paul Koning <[email-masked]>
Date: Fri, 29 Mar 2002 08:12:33 -0500

>>>>> "Luben" == Luben Tuikov <[email-masked]> writes:

 Luben> ...We cannot control the format of the VALUE (above), as
 Luben> companies will add their own keys. (Reason 2)

 Luben> Thus we need to restrict the "KEY=VALUE" as a whole, its
 Luben> internals past iSCSI are up to the implementations/ companies
 Luben> which add them.

 Luben> If we impose restrictions on "KEY=VALUE" then we need not
 Luben> impose restrictions on the size of KEY or VALUE separately,
 Luben> just that KEY cannot be an empty sequence.

 Luben> The node should know in advance how big of a span a
 Luben> "KEY=VALUE" will be in order to 1) reject it (out of
 Luben> resources) or 2) prepare for its arrival (whatever this
 Luben> means).

I would argue that an implementation can, and should, have its own
protective limits no matter what the standard may have to say about
it.  If a well-crafted sequence of messages crashes an implementation,
the blame goes to the implementation, not to the standard.

But I do agree that the standard needs to say more.  Given that
resources and needs may vary, it seems overly restrictive to place a
hard upper bound on the overall size.  What I would propose instead is
that the standard specify an overall size that all implementations
MUST support.  Larger sizes may be accepted if the implementation has
the needed memory -- which allows implementations that have special
requirements to deal with that within the standard -- but a conforming
implementation would be entitled to reject such large negotiations.

One question: are we concerned here with an individual "key=value" or
with all of the key=value pairs in the text messages taken together?
I can see reasons to worry about the entire set of key=value pairs, so
having a size bound (as in "everyone MUST support at least this much")
on that would take care of the entire question in one step.

       paul

---

From: Bill Studenmund <[email-masked]>
Date: Fri, 29 Mar 2002 10:45:24 -0800 (PST)

> on that would take care of the entire question in one step.

Why don't we negotiate a size? We have a default max size for individual
key=value items, and a max for the entire set. Either the target or the
initiator can try to negotiate it up, but has to deal with the other side
saying no. You can't negotiate it below the default max that we decide on.

The main thing about making it a negotiated value is both sides can know
what the other can do. We won't get surprise errors as we tripped over an
undisclosed limit one side had.

Take care,

Bill

---

From: Paul Koning <[email-masked]>
Date: Fri, 29 Mar 2002 14:50:16 -0500

> undisclosed limit one side had.

But we're talking about limits of the negotiation process itself.
Yes, you can renegotiate after login, but login is the primary
negotiation point.  I think a fixed minimum requirement is more
straightforward. 

	 paul

---

From: Paul Koning <[email-masked]>
Date: Fri, 29 Mar 2002 15:27:22 -0500

Excerpt of message (sent 29 March 2002) by Bill Studenmund:
> While having a minimum required is good, if we don't have a way to
> negotiate a larger value, how can we really use a larger value? So if we
> can't negotiate the largest size we allow for key=value items and for the
> set, aren't you really suggesting we just pick a number and that's it?
>
> What's the alternative? Send something too large and either crash the
> other side or have some 'I'm confused' error come back? At least with
> negotiation, each side will know what it can and can't send.

If you crash, you have a bug.  I was assuming a reject that says "your
text is too long".
 
> So here's the suggestion again. We start negotiation with a default value
> for the largest key=value item that can be sent, and the largest set of
> items that can be sent. These defaults are the minimum required that you
> mention. If we want, either side can try to negotiate these values larger.
> If negotiation suceeds, then future steps of negotiation can use the
> larger values. Negotiation can't lower the values below the minimum
> required.

That sounds workable but I don't see the justification for that extra
complexity.  Luben brought up the original issue as a denial of
service concern.  Is there a plausible real-world example where
someone will want to send a meaningful negotiation exchange that's
"very large"?  I don't know of one.  If there isn't a real need for
the complexity, let's make the minimal fix needed, which is to say
"everyone must support at least x" without negotiating x.

	    paul

---

From: Bill Studenmund <[email-masked]>
Date: Fri, 29 Mar 2002 12:12:24 -0800 (PST)

> > Why don't we negotiate a size? We have a default max size for individual
                                             ^^^^^^^^^^^
This is the fixed minimum you mention below, under a different name. I
call it a max since it is the max size of negotiation items it covers.

> But we're talking about limits of the negotiation process itself.

I understand.

I am suggesting that durning negotiation we negotiate parameters covering
the very negotiations we are in. That means that some key=value items can
only be sent after a given size has been negotiated.

Take care,

Bill

---

From: Luben Tuikov <[email-masked]>
Date: Fri, 29 Mar 2002 21:40:05 -0500

Paul Koning wrote:
> it.

Yes and no.

Yes, since a well written implementation
should indeed check buffer limits (if they are known
beforehand).

No, because a computer program is a theorem proof;
the theorem being the specification, which is the iSCSI
draft/RFC. And if the specification is faulty, then
there is no way the program can work.

>  If a well-crafted sequence of messages crashes an implementation,

Yes, but I can crash a conforming to the iSCSI spec implementation,
by sending it an extremely long "KEY=VALUE" sequence.
That is, the implementation will have to keep a state
until 0x00 is received so that it can _then_ interpret it.

> with all of the key=value pairs in the text messages taken together?

Answer: limit ONLY a "KEY=VALUE" sequence. Either by
negotiation per assignment or global.

Reason 1: The reason is that such a sequence can span PDUs.
If it didn't, then it is limited by the PDU size.

Reason 2: Multiple "KEY=VALUE" pairs that span PDUs have NO
burden on memory because as soon as a 0x00 is received
_that_ key=value assignment is processed and the memory
where it is kept is _freed_. (something like a conveyer belt)

BUT as noted in reason 1, you support a KEY=VALUE assignment
to span multiple PDU's AND I send you one, you'll have to
``wait'' or ``keep state'' until I send you 0x00
and by that time you can run out of memory.

For a reference see char *gets(char *s) which has exactly
the same problem I'm describing here.

-- 
Luben

---

From: Luben Tuikov <[email-masked]>
Date: Fri, 29 Mar 2002 23:24:02 -0500

> larger values. Negotiation can't lower the values below the minimum

I like this.

Have a maximium default, and if a node needs a larger
value, then let it negotiate it. Futhermore, let the
negotiation be dynamic, that is, if later on
a node needs yet a larger size for an assignment,
let it negotiate it _again_.

Question: I don't see the point in MaxRecvPDULength
be _per_direction_. This is unnecessary and adds extra
complexity. Furthermore since its value is a minimum
of other values, it will be better if it is
only _per_connection_, both directions.

-- 
Luben

---

From: Luben Tuikov <[email-masked]>
Date: Fri, 29 Mar 2002 23:07:30 -0500

Paul Koning wrote:
> 
> If you crash, you have a bug.
>

No. Paul, as the draft currently stands, there is no limit
on a single assignment "KEY=VALUE" and it can span multiple
PDU's -- that is, the specification is faulty (the theorem),
_then_ the program crashes (proof doesn't work).

That is, a program satisfies the specification and
upon a legitimate input it crashes the computer.

> Is there a plausible real-world example where
> "very large"?

Why wouldn't there be? This gets tricky especially if
companies add their own variables and allow users to
set their values... This way I can send _anything_ over
the wire, being a value assignment.

-- 
Luben