Subject: RE: iSCSI: freeing resources during data-in and r2t

From: Eddy Quicksall <[email-masked]>
Date: Wed, 17 Jul 2002 12:14:32 -0400

I think see a slight inefficiency in how resources are freed when multiple
data-in and r2t's PDU's are used when ErrorRecoverLevel > 0 or TCP ACK is
not available to the iSCSI layer.
 
Data-in and R2T's use DataSN and R2TSN. To free those resources, ExpStatSN
is used.
 
But if several R2T's and/or Data-in PDU's were used, resources used for
those PDU's are tied up for the entire command.
 
Since other commands could be received during this time, it would be nice if
those commands could contain information that would tell the target that the
R2T and/or Data-in PDU's have been received.
 
I know a radical change at this point is probably a bad idea but I just
wanted to throw out the idea ... if the StatSN/ExpStatSN were changed to
something like RespSN/ExpRespSN and if everything going from the target to
the initiator carried a new RespSN, then the resources could be freed up
sooner.
 
I would hate to use the A bit for this because it causes more traffic.
 
Eddy
mailto:[email-masked]

---

From: [email-masked]
Date: Wed, 17 Jul 2002 21:56:33 -0400

Eddy,
 
The A bit in Data-In plus Data ACK (SNACK type 2 in -14, Julian's
renumbering of the SNACK types in the working version of -15 is/will
be/has been undone) already provides the intermediate resource freeing
for Data-In.  This is only available for ErrorRecoveryLevel >= 1
We've never thought that R2Ts would consume enough resources to
be worth putting in support for intermediate resource
freeing of them.  Adding another sequence number at this late date
to deal with a "slight inefficiency" when the A bit is used properly
(SNACK-Data ACK PDUs vs. a piggybacked count) does not seem like a
good idea.
 
Thanks,
--David

-----Original Message-----
From: Eddy Quicksall [mailto:[email-masked]]
Sent: Wednesday, July 17, 2002 12:15 PM
To: Mallikarjun C. (E-mail)
Cc: ips@ece. cmu. edu (E-mail)
Subject: RE: iSCSI: freeing resources during data-in and r2t

 

 
 
Eddy

---

From: Eddy Quicksall <[email-masked]>
Date: Fri, 19 Jul 2002 08:43:32 -0400

Yes, I agree that it is probably too late to think about this. But just to
put you in perspective, I was referring more to the data-in, not the R2T's
and the A bit causes extra traffic.
 
I mostly wanted to point this out, I am not really asking for a change
unless the group thinks it is relevant. If no one has an issue (and I really
don't), then we can drop the thread.
 
Eddy

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Wednesday, July 17, 2002 9:57 PM
To: [email-masked]
Cc: [email-masked]
Subject: RE: iSCSI: freeing resources during data-in and r2t

Eddy,

Thanks,
--David

-----Original Message-----
From: Eddy Quicksall [mailto:[email-masked]]
Sent: Wednesday, July 17, 2002 12:15 PM
To: Mallikarjun C. (E-mail)
Cc: ips@ece. cmu. edu (E-mail)
Subject: RE: iSCSI: freeing resources during data-in and r2t

 

 
 
Eddy

---

From: "Julian Satran" <[email-masked]>
Date: Sat, 20 Jul 2002 03:43:32 +0300

Eddy,

What resources?  R2Ts for which the initiator has finished sending data 
can be discarded.
Targets build R2Ts as needed.
So what resources do you have in mind?  Data Buffers at initiator? 
According to SAM2 you have to keep them up to the bitter end unless
you request delivery strictly in order in which as you are better off 
issuing in sequence short writes if you want to allow recovery.
If that is what you have in mind there is a basic asymmetry build in SCSI 
(and others) that the command issuers has
the resource to keep the command going while the target (that can't plan 
the commands hitting it) has to be taken care of.

iSCSI is not worsening this (neither improving on it).

Regards,
Julo

Eddy Quicksall <[email-masked]>
Sent by: [email-masked]
07/19/2002 03:43 PM

        To:     [email-masked]
        cc:     [email-masked]

I mostly wanted to point this out, I am not really asking for a change 
unless the group thinks it is relevant. If no one has an issue (and I 
really don't), then we can drop the thread.
 
Eddy
-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Wednesday, July 17, 2002 9:57 PM
To: [email-masked]
Cc: [email-masked]
Subject: RE: iSCSI: freeing resources during data-in and r2t

Eddy,

Thanks,
--David
-----Original Message-----
From: Eddy Quicksall [mailto:[email-masked]]
Sent: Wednesday, July 17, 2002 12:15 PM
To: Mallikarjun C. (E-mail)
Cc: ips@ece. cmu. edu (E-mail)
Subject: RE: iSCSI: freeing resources during data-in and r2t

 
 
Since other commands could be received during this time, it would be nice 
if those commands could contain information that would tell the target 
that the R2T and/or Data-in PDU's have been received.

 
Eddy

---

From: [email-masked]
Date: Mon, 22 Jul 2002 15:33:03 -0400

> As I said to David, it is probably too late for a change here but
> it seems like we could have used StatSN (actually with a different name)
> as a counter of everything going from the target to the initiator.
> That way, the target does not have to associate the TCP ACK in order
> to tell if the initiator got a PDU.

A word of caution - the TCP ACK indicates that TCP delivered the bytes
- if CRC digests are in use, a digest error could cause those bytes not
to form a usable PDU.

I view the A bit as an 80% solution that addresses the major portion
of the need for targets to free resources prior to command completion,
and I think Eddy's in reluctant semi-violent agreement that no change
should be made here.

Thanks,
--David

-----Original Message-----
From: Eddy Quicksall [mailto:[email-masked]]
Sent: Monday, July 22, 2002 3:12 PM
To: Julian Satran
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: RE: iSCSI: freeing resources during data-in and r2t

There are other resources besides the buffers that are used to transmit the
data and it would be nice if those could be freed as soon as I know the
initiator has them.
 
As I said to David, it is probably too late for a change here but it seems
like we could have used StatSN (actually with a different name) as a counter
of everything going from the target to the initiator. That way, the target
does not have to associate the TCP ACK in order to tell if the initiator got
a PDU.
 
Eddy
-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Friday, July 19, 2002 8:44 PM
To: Eddy Quicksall
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: RE: iSCSI: freeing resources during data-in and r2t

        Eddy, 

What resources?  R2Ts for which the initiator has finished sending data can
be discarded. 
Targets build R2Ts as needed. 
So what resources do you have in mind?  Data Buffers at initiator? According
to SAM2 you have to keep them up to the bitter end unless 
you request delivery strictly in order in which as you are better off
issuing in sequence short writes if you want to allow recovery. 
If that is what you have in mind there is a basic asymmetry build in SCSI
(and others) that the command issuers has 
the resource to keep the command going while the target (that can't plan the
commands hitting it) has to be taken care of. 

Regards, 
Julo 

        To:        [email-masked] 
        cc:        [email-masked] 

  
Eddy 
-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Wednesday, July 17, 2002 9:57 PM
To: [email-masked]
Cc: [email-masked]
Subject: RE: iSCSI: freeing resources during data-in and r2t

Eddy, 

Thanks, 
--David 
-----Original Message-----
From: Eddy Quicksall [mailto:[email-masked]]
Sent: Wednesday, July 17, 2002 12:15 PM
To: Mallikarjun C. (E-mail)
Cc: ips@ece. cmu. edu (E-mail)
Subject: RE: iSCSI: freeing resources during data-in and r2t

  

  
  
Eddy

---

From: Eddy Quicksall <[email-masked]>
Date: Mon, 22 Jul 2002 15:11:58 -0400

Eddy

-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Friday, July 19, 2002 8:44 PM
To: Eddy Quicksall
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: RE: iSCSI: freeing resources during data-in and r2t

        Eddy, 

Regards, 
Julo 

07/19/2002 03:43 PM 

        To:        [email-masked] 
        cc:        [email-masked] 

  
Eddy 
-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Wednesday, July 17, 2002 9:57 PM
To: [email-masked]
Cc: [email-masked]
Subject: RE: iSCSI: freeing resources during data-in and r2t

Eddy, 

Thanks, 
--David 
-----Original Message-----
From: Eddy Quicksall [mailto:[email-masked]]
Sent: Wednesday, July 17, 2002 12:15 PM
To: Mallikarjun C. (E-mail)
Cc: ips@ece. cmu. edu (E-mail)
Subject: RE: iSCSI: freeing resources during data-in and r2t

  

  
  
Eddy

---

From: Eddy Quicksall <[email-masked]>
Date: Mon, 22 Jul 2002 17:19:56 -0400

>reluctant semi-violent agreement -- yes

Eddy

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Monday, July 22, 2002 3:33 PM
To: [email-masked]
Cc: [email-masked]
Subject: RE: iSCSI: freeing resources during data-in and r2t

Thanks,
--David

-----Original Message-----
From: Eddy Quicksall [mailto:[email-masked]]
Sent: Monday, July 22, 2002 3:12 PM
To: Julian Satran
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: RE: iSCSI: freeing resources during data-in and r2t

 
Eddy
-----Original Message-----
From: Julian Satran [mailto:[email-masked]]
Sent: Friday, July 19, 2002 8:44 PM
To: Eddy Quicksall
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: RE: iSCSI: freeing resources during data-in and r2t

        Eddy, 

Regards, 
Julo 

        To:        [email-masked] 
        cc:        [email-masked] 

  
Eddy 
-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Wednesday, July 17, 2002 9:57 PM
To: [email-masked]
Cc: [email-masked]
Subject: RE: iSCSI: freeing resources during data-in and r2t

Eddy, 

Thanks, 
--David 
-----Original Message-----
From: Eddy Quicksall [mailto:[email-masked]]
Sent: Wednesday, July 17, 2002 12:15 PM
To: Mallikarjun C. (E-mail)
Cc: ips@ece. cmu. edu (E-mail)
Subject: RE: iSCSI: freeing resources during data-in and r2t

  

  
  
Eddy