Subject: Re: comments on draft-balakrishnan-cm-02.txt

From: Vern Paxson <[email-masked]>
Date: Wed, 05 Jul 2000 13:42:49 PDT

Thanks for addressing these points, looks good.  One comment:

> > >    does not transmit any data upon a cmapp_send() callback invocation,
> > >    it SHOULD call cm_notify(dst, 0) to allow the CM to permit other
> > >    flows in the macroflow to transmit data.
> > 
> > Why give it any sort of leeway here?  If the the app doesn't call
> > cm_notify(), what ensures that the bookkeeping will stay consistent?
> > (Or is the SHOULD to emphasize that correct operation of the controller
> > shouldn't rely on the didn't-send notification, because it's an easy
> > mistake for the app to make?)
> 
> Notice that this is only for a non-transmission.  If the app doesn't call 
> cm_notify(), it isn't the end of the world.  All other cm_notify()'s are done 
> from the IP output routine.

My concern is that some bookkeeping could get inconsistent if the app doesn't
call cm_notify() when it decides not to send.  In particular, I'm wondering
whether an app could "save up" several credits from cmapp_send() and then
dump them all into the network later; I'd like to ensure that that doesn't
happen.  Is this a problem?

		Vern

---

From: Hari Balakrishnan <[email-masked]>
Date: Wed, 05 Jul 2000 23:25:57 -0400

Vern,

You bring up a good point, which we ought to have brought up as a discussion 
issue.

There are two cases:

(i) A CM app hoards credits and eventually "cashes in" on them all in a burst.
This is the case you bring up.

(ii) One or more CM apps on a macroflow crashes/disappears/goes-AWOL.

Case (i) is relatively easy to handle.  We do not issue subsequent cmapp_send() 
calls to a stream while one is pending.  In our implementation, this would be 
done by adding a "pending" flag on the cm_stream struct that is cleared by 
cm_notify().  Notice that this means that an app MUST do a cm_notify(0) is it 
decides not to send on a previous cmapp_send().  The current draft is incorrect 
(SHOULD instead of MUST) on this count.

Case (ii) is trickier; crashes always are :}.  This requires a timeout before 
that grant can be given to other streams on the macroflow, and before state can 
be cleaned up.  It's hard to see what the correct timeout should be, but having 
said that, I don't think it really matters what the timeout is.

I would reckon something like "if a stream hasn't used a grant in one RTT, 
pretend it's been used up."  It's not clear to me when to clean up state for an 
application that may have crashed.  Obviously if the process has crashed it's 
not an issue, but I don't know if that's enough of a check, especially for 
in-kernel protocols that may get wedged every so often.

We'll think about this some more.

If anyone on the list has any ideas, we'd appreciate hearing them!

Thanks,
Hari