Subject: iSCSI: target resource "leak"/logout command

From: Pierre Labat <[email-masked]>
Date: Tue, 09 Jan 2001 02:15:59 -0800

Julian,

There is a corner case where
- a [group of] command completion[s]  will never be acked (ExpStatRN)
- the corresponding task[s] are not "retried" because they are perfectly

   completed from the initiator point of view and
- the session continues to work fine.
Thus the resource for the task[s] may never
be released on the target (the target waits for ExpStatRN acking
the completion before releasing task resources) because
on the target side the ack of the completion is not received
and on the initiator will not retry the command.

A resource (memory) "leak" happens on the target.

First i described the scenario, then the solution.

Scenario
=======
Session with 2 TCP connections 1 and 2

1)  A command completion (StatRN=10) is received by
      the initiator (over cx 1) for the command 5.

2) The initiator sends back ExpStatRN=11 with a command over  TCP cx 1

3) The TCP cx 1 drops and the command 5 (so ExpStatRN=11) will never
   make it to the target.

4) The command 5 is not retried on the connection 2 because for the
initiator
      it is perfectly completed.

5) The target never gets the completion acked and doesn't know when
    to release resources.

Solution
=======

- Add the field ExpStatRN in the logout command.
- The initiator puts in this field the last value of ExpStatRN of the
failed
    cx. (Not the ExpStatRN value of the connection used to transport the
logout)
- The target when receiving the logout, reads ExpStatRN and is able to
free up
   the resources.

Regards,

Pierre

---

From: Pierre Labat <[email-masked]>
Date: Tue, 09 Jan 2001 23:58:09 -0800

sip - Santosh Rao wrote:

>     +----------------------------------------------------------+
>     | This is a broadcast message.  Any reply to this          |
>     | message will broadcast to the entire distribution.       |
>     | To reply only to the individual submitter, send mail     |
>     | directly to < mailto:[email-masked] >
>     +----------------------------------------------------------+
>
> This is a multi-part message in MIME format.
>
>
------------------------------------------------------------------------

> Pierre,
>
> Some comments embedded below. Perhaps, Doug or some of the other
target
> folks can add more comments here.
>
> Thanks,
> Santosh
>
> Pierre Labat wrote:
>
> > Scenario
> > =======

> >    make it to the target.
>
> A connection drop is followed by the initiator migrating its
outstanding
> commands to a new connection and issuing a Logout on the CID of the
failed
> connection. The logout will cause the target to clean up all its
resources.

The logout doesn't cause the target to cleanup all its resources. The
target
MAY keep the resources corresponding to the task pending
on the failed connection. If we don't assume that we can throw away
the "retry" strategy. The logout indicates to the target that it has
to throw away all that is incoming on the failed connection and
that the command pending will be retried (or aborted) from
another connection.

>
>
> In any cases, no self respecting target will [and can afford to] keep
its
> I/O buffers and SEST around forever after sending the Status. They
will run
> out of SEST type of resources at the target end. The typical target
may
> start a timer once it sends the SCSI Status PDU for a SCSI command and
on
> timeout would clean up the SEST entry and release the I/O buffers for
that
> timed out I/O which did not get an updated ExpStatRN.

I know they are such timeouts, but it is better to free up the resources

sooner
on the target by designing  a robust protocol. Timers have to be the
"last resort".

>
>
> Further, on such a timer pop, the target may choose to use a NOP-OUT
with
> the "Ping" bit set to test the connection.
>
> However, this starts a new issue here. Once again, the philosophy of
StatRN
> here seems to be to optimize the recovery paths  and affect the
performance
> paths !!

The only goal of StatRN is to fasten a recovery using "retry".

>
>
> How does the draft cover the situation when an initator has no further

> outbound traffic on whch it can send an update to ExpStatRN ? If there
was
> a burst of traffic from one initiator followed by a period of
quiescence,
> it could end up not sending ExpStatRN update to the target and this
would
> eat up resources on the target, affecting other initiators that may
still
> be talking to it.

The target sends a NOP-in with the P bit set.

Regards,

Pierre