Subject: RE: Addition of CmdSN in Data-out PDU

From: Robert Snively <[email-masked]>
Date: Fri, 12 Oct 2001 15:56:06 -0700

I have two small questions that would help me understand this.

How do you know that unsolicited data is expected?  By nature,
unsolicited data is received as a "maximum surprise" which can
only be determined after unpacking and parsing at least a
part of the command structure, possibly even including the
SCSI command (although there are some hints contained in the
command header information).

How can you constrain a generic system to posting the 
unsolicited data in the same order that the commands were
emitted?  In general, I would have expected the system to
be emitting command followed immediately by the corresponding
unsolicited data.  If that is not the case, it means that there
was a delay in obtaining the unsolicited data for transfer and
that the delay was sufficient to allow the insertion of commands.
If the delay is that large (and probably variable), the enforcement
of transfer of unsolicited data in the same order as the 
commands are emitted seems to me to be a significant challenge,
and certainly shouldn't be required as normal behavior.  While it
would make things simpler for targets (already challenged by
unsolicited data), it seems to me that it would make things 
much more complex for initiators.

Bob

> -----Original Message-----
> From: Somesh Gupta [mailto:[email-masked]]
> Sent: Friday, October 12, 2001 2:51 PM
> To: BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2); 'Binford, Charles';
> [email-masked]
> Subject: RE: Addition of CmdSN in Data-out PDU
> 
> 
> Matthew,
> 
> Since the unsolicited data does not follow the
> command, you need the link list. But since the
> unsolicited data must be sent in the same order
> as the commands, a link list is enough.
> 
> Let us say that you have 8 commands. The ones
> for which we expect unsolicted data are marked
> as Cnn(ud). And I have marked the unsolicted
> data PDUs as UD(nn). The (nn) with data implies
> that it is implicit and not actually carried with
> the PDU itself.
> 
> C01(ud) C02(ud) C03(ud) C04 C05 C06(ud) UD(01) --->
> --> C07(ud) UD(02) UD(03) D04 D04 D05 D05 UD(06) --->
> --> UD(07) C08(ud) UD(08) --->
> 
> After the target receives the command C01, C02, and C03
> for which it expects unsolicited data, it puts them in
> a link list. It also receives C04 and C05 for which
> unsolicited data is not expected and they don't go
> on the list. It then receives C06 for which unsolicited
> data is expected, and it is added to then end of the list.
> Then an unsolicited data PDU is received. It must go
> with the command at the head of the list which is C01.
> Use the ITT to make sure and you can then take C01 off
> the list and so on.
> 
> Somesh
> 
> 
> > -----Original Message-----
> > From: [email-masked] 
> [mailto:[email-masked]]On Behalf Of
> > BURBRIDGE,MATTHEW (HP-UnitedKingdom,ex2)
> > Sent: Friday, October 12, 2001 7:53 AM
> > To: 'Binford, Charles'; [email-masked]
> > Subject: RE: Addition of CmdSN in Data-out PDU
> >
> >
> > Charles,
> >
> > As you have described the spec states that "that 
> unsolicited data MUST be
> > sent in the same order as the commands".  This is not the same as
> > unsolicited data must follow the command associated with 
> it:  For example:
> >
> > (Cx = SCSI Command PDU, Dx = The unsolicited data PDUs. The 
> x in all the
> > example can be the ITT. It is not the CmdSN.
> >
> > This is allowed:
> >
> >   C1 C2 C3 D1 D2 C4 D3 D4
> >
> > and the target will have to use the ITT to associate the 
> data with the
> > command.
> >
> > Matthew
> >
> >
> > -----Original Message-----
> > From: Binford, Charles [mailto:[email-masked]]
> > Sent: Friday, October 12, 2001 2:50 PM
> > To: [email-masked]
> > Subject: RE: Addition of CmdSN in Data-out PDU
> >
> >
> > I have not verify version 8 is still the same, but version 07-97
> > had a rule
> > that unsolicited data MUST be sent in the same order as the 
> commands.
> >
> > Thus, there is no need for a search on the ITT.  The target 
> just needs to
> > keep of linked list of I/Os waiting on unsolicited data.  
> New commands are
> > added to the tail, any unsolicited data *should* be associated
> > with the I/O
> > at the head of the list.  The ITT is used as a sanity check and
> > you're done.
> >
> > What am I missing?
> >
> > Charles Binford
> > Pirus Networks
> > 316.315.0382 x222

---

From: "Somesh Gupta" <[email-masked]>
Date: Fri, 12 Oct 2001 16:36:54 -0700

> From: Robert Snively [mailto:[email-masked]]
> Sent: Friday, October 12, 2001 3:56 PM
> To: [email-masked]'; BURBRIDGE,MATTHEW
> (HP-UnitedKingdom,ex2); 'Binford, Charles'; [email-masked]

  The F bit in the cmd PDU indicates whether unsolicited data
  PDUs should be expected for a particular SCSI command or
  not. There can be multiple unsolicited data PDUs for a
  particular command, and the last unsolicited data PDU
  for a command should have the F bit set. I assume that
  we use TTT = 0xffffffff to determine that this is an
  unsolicited data PDU. So yes, it would not be possible
  to do unsynchronized parallel processing of a packet
  containing a command and a packet containing unsolicited
  data for it.
> emitted? 

  That is a protocol requirement.

> In general, I would have expected the system to

  I am not quite sure why we have the restriction. Julian
  should be able to shed more light on it. I think Costa
  had some deadlock scenario if ordering was not enforced
  (but I think that was before a command credit mechanism
  was mandated). However with multiple data PDUs, if some
  sort of ordering/completion rules are not mandated, there
  will be deadlock.

  One scenario that you do not mention is where an adapter
  might allow commands to pass data to reduce latency.

---

From: Santosh Rao <[email-masked]>
Date: Fri, 12 Oct 2001 16:34:15 -0700

> command header information).

Un-solicited data is expected when the scsi command pdu does not have
the F bit set. When you crack open the header and find the F bit not
set, you add the command control block onto this additional
"un-solicited data linked list" that is being discussed.

> much more complex for initiators.

I agree. I don't quite understand why the restriction exists, to enforce
ordering of un-solicited data to be the same as the order of the
corresponding commands.

A dis-advantage of enforcing such strict ordering is that when the F bit
data-out PDU of one of the un-solicited data sequences is dropped, the
order of data received at the target violates the command order.
Currently, the spec requires session recovery to be performed (tear down
session on order violation of un-solicited data) on this order
violation.

For those who tape fans out there who were planning on using
un-solicited data, this would be a nasty side effect, since the
"within-command-recovery" is un-usable when a digest error occurs on
un-solicited data and the session would be torn down. 

Since SNACK was included primarily for the tape application support, it
seems like it would'nt serve much purpose to such tape applications if a
digest error were to occur on un-solicited data.

I suggest that this strict ordering of data with commands rule be
removed.

Regards,
Santosh

-- 
##################################
Santosh Rao
Software Design Engineer,
HP-UX iSCSI Driver Team,
Hewlett Packard, Cupertino.
email : [email-masked]
Phone : 408-447-3751
##################################