Subject: Re: Next draft of ECM WG congestion manager document

From: Srinivasan Seshan <[email-masked]>
Date: Tue, 17 Oct 2000 14:42:49 -0400

> 
> Joe Touch wrote:
> > 
> > On Thu, 03 Aug 2000 00:33:20 PDT, Vern Paxson wrote:
> > 
> > 
> > The issue here is to make the cm_request(id) call, which currently takes no
> > other arguments, richer.  In particular, the suggestion is:
> >         cm_request(id, npkts, size)
> > 
> > I tried to articulate (at the last IETF) why this may not be a good idea.
> > Here's the reasoning:
> > 
> > 1. The current API is not as inefficient as one might believe.  It may be
> > tempting to think that the current API causes a user-kernel crossing for ea
> > packet.  That's not necessarily true.  A user-level library that implements
> > API can package multiple such calls into one system call.  For in-kernel ap
> > like TCP, cm_request(id) is only a function call.  And if you believe that 
> > function call is expensive, inlining it (and inlining the in-kernel functio
> > callback) is pretty straightforward.
> 
> It may be difficult to inline functions in loadable kernel modules.
> 

Agreed..

> Further, beyond the function call overhead, there is the per-call
> processing overhead. Taking this overhead (or any overhead) on 
> a per-packet basis is asking for substantial performance penalty.
> 

However, the amount of per packet processing done within the CM is minimal 
compared to the other per packet processing done by a host (e.g. it is 
significantly less than the per packet processing done by TCP currently). As a 
result, it should not be a significant penalty.

> > 2. If we add a "MAY" clause for cm_request(id, npkts, size), it MUST requir
> > that applications negotiate the specific supported mode in the API.
> 
> The proposed extension to the API contains two parameters:
> 
> 	size:	this is consistent with the byte-counting proposal
> 		for TCP.
> 
> 	npkts:	this allows per-API-call processing to occur on
> 		timescales coarser than per-packet. The issue is 
> 		being able to amortize the processing on the API;
> 		inlining isn't sufficient to provide amortization.
> 
> An application that desires to use the current interface can use
> size=MTU and npkts=1.
> 

The concern here is about the "MAY". This is bad wording to have in an API 
document. The issue is not that your API can support simpler interactions but 
that the MAY forces all applications to support both versions of API. E.g. I 
write an application that has cm_request( id, 10 pkts, 10KB)... Because of the 
MAY, I must now write a version of the application that uses the 
cm_request(id) only and a set of configure scripts that chooses between the 
two. If the wording should use MUST, I believe that the we should (at least 
for now) stick with the simpler interface.

> > I am not sure at all that complicating the API
> > to accommodate this, in the absence of significant experience that dictates
> > that it's necessary, is a good idea. 
> 
> That experience is in abundance; per-packet overhead should be avoided,
> especially where not otherwise necessary.

... and where you do not give up any other benefits as a result. One important 
 aspect of the CM, is that it is supposed to effectively shape/control the 
transmission of data. By allowing application to send bursts of data it gives 
up much of this control. In addition, scheduling such variable sized bursts 
may also prove to be a non-trivial problem. This seems to have much greater 
future implications than the per-packet processing overheads.

> 
> If you want a particular system not to support it (i.e., to require
> npkts to be 1), that can be decided on a per-system basis. Assuming how
> many bytes are in a packet or how many packets should be sent between 
> API calls is the assumption to be avoided.
> 

Srini and Hari

---

From: Joe Touch <[email-masked]>
Date: Tue, 17 Oct 2000 12:29:20 -0700

> result, it should not be a significant penalty.

I've heard these arguments before. In the absence of evidence that CM
is not a significant penalty (i.e., emperical evidence of the
performance
of running code), it is wise to err on the side of caution here.

> for now) stick with the simpler interface.

The solution is to not provide the cm_request(id) format;
as you point out, any provision for two interfaces is asking for
trouble.
The better solution is the more general interface. Applications that
can make due with the simpler version can call it as I indicated, or
use a macro in the code of the API.

I.e., then the cm_request(id, npkts, size) should be the MUST;
the simpler version should be the MAY.

> future implications than the per-packet processing overheads.

There are two issues here:

	1. size
		assuming MTU is imprecise anyway

	2. npkts
		making this larger does increase granularity of the API
		whether an implementation of the API supports this
		should be up to the API

		it is overrestrictive to assume that CM must be tickled
		for every packet going out the door, regardless of the size

		there are numerous cases where such bursts are desirable,
		and might be impeded by the current interface, e.g.,	
		bursts in gigabit ethernet, and the small-scale burstiness of
		TCP (sending multiple packets per ACK).

Joe