Subject: [midcom] HTTP as the Midcom protocol

From: Mark Baker <[email-masked]>
Date: Mon, 4 Feb 2002 11:41:49 -0500 (EST)

I've had a read through the requirements, and I'd suggest that HTTP
makes a pretty good candidate for a MIDCOM protocol.  The only
requirement that it currently has trouble meeting, and so would require
a small extension to meet, is 2.1.6 (assuming that status report is to
be asynchronous with respect to the agent).  WEBI is considering a
similar extension that they would use for cache invalidation, but it can
easily be used for other problems such as this one.

Many of the requirements in 2.2 would also not be directly satisfied by
HTTP, but would instead be satisfied by a data format (MidcomML?) that
would be transferred with HTTP.

Granted, HTTP 1.1 also does a lot more than what MIDCOM needs, but its
value is that it's already deployed and that there are a lot of well
tested, optimized, and secured third party implementations available.

MB
-- 
Mark Baker, Chief Science Officer, Planetfred, Inc.
Ottawa, Ontario, CANADA.      [email-masked]
http://www.markbaker.ca   http://www.planetfred.com

_______________________________________________
midcom mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/midcom

---

From: "Gilad Ben-Yossef" <[email-masked]>
Date: Mon, 4 Feb 2002 21:48:25 +0200

Hi all,

As this is my first post I believe I ought to introduce myself: my name is Gilad Ben-Yossef, and I'm the a software team leader with Kagoor Networks R&D arm.

> easily be used for other problems such as this one.

Since the subject of adopting already existing protocols has already surfaced, I'd like to suggest something that I kept thinking to myself while reading the workgroup RFCs: use SOCKS (and possible extensions) as the MIDCOM protocol. Like HTTP it is already deployed, used and debugged. Unlike HTTP, it already implements more then just the transport layer and actually is being used in a context very similar to the group objectives. Even more important: many clients and some MIDCOM equipment already implements and support SOCKS5 today. An an added bonus SOCK5 is an IETF standart.

Before the advent of NAT that deals with "generic" protocols, SOCKS was one of the leading ways to deal with the same problems we are facing now of private address spaces. There's no reason not to use it now to solve the remaining problem of protocols which simple NAT does not cover.

Of course, several extensions would be needed, for example to deal with odd/even port numbers issue but the basic infrastructure is already there.

More on SOCKS5 ca be found here: http://www.socks.nec.com/socksprot.html

Remarks are very much welcome.

Gilad.

-- 
Gilad Ben-Yossef <[email-masked]>
Tel: +972(9)9717330 | Fax: +972(9)9717334   | Cel: +972(54)756701
Kagoor Networks ltd | http://www.kagoor.com |

---

From: Melinda Shore <[email-masked]>
Date: Mon, 04 Feb 2002 13:50:47 -0500

At 09:48 PM 2/4/02 +0200, Gilad Ben-Yossef wrote:
>Since the subject of adopting already existing protocols has already surfaced, I'd like to suggest something that I kept thinking to myself while reading the workgroup RFCs: use SOCKS (and possible extensions) as the MIDCOM protocol. 

It's obviously an attractive candidate, but it would have to be
extended quite a bit - more than many others.  Also, I spent quite
some time talking with the SOCKS folks before starting work on
midcom, and they weren't at all receptive to the prospect of
addressing NAT issues.  I think the working group is on the cusp
of closing down.

This is by way of background.  If you feel that SOCKS is a superior
protocol for this application, you should feel free to 1) advocate
its use by preparing some text describing its applicability, and 2)
start working being able to describe the necessary protocol extensions 
(UDP support, NAT, etc.).

Melinda

---

From: Mark Baker <[email-masked]>
Date: Mon, 4 Feb 2002 13:52:30 -0500 (EST)

> Since the subject of adopting already existing protocols has already surfaced, I'd like to suggest something that I kept thinking to myself while reading the workgroup RFCs: use SOCKS (and possible…

I'm very glad to hear more support for using existing protocols.

> Like HTTP it is already deployed, used and debugged. Unlike HTTP, it already implements more then just the transport layer 

I don't have much of an opinion about the use of SOCKS, but I would like
to point out that HTTP is an application protocol, not a transport
protocol.

MB

---

From: Martin Stiemerling <[email-masked]>
Date: Thu, 07 Feb 2002 11:34:35 +0100

Hi,

comments are inline.

Cheers
Martin

> HTTP, but would instead be satisfied by a data format (MidcomML?) that

But in this case you have to define the MidcomML as well with states, 
messages and so on. Furthermore you have a huge protocol overhead due to 
the use of HTTP. This is not a really small protocol anymore.

> tested, optimized, and secured third party implementations available.

Martin Stiemerling

NEC Europe Ltd. -- Network Laboratories      [email-masked]
IPv4: http://www.ccrle.nec.de             IPv6: http://www.ipv6.ccrle.nec.de

---

From: Mark Baker <[email-masked]>
Date: Thu, 7 Feb 2002 10:55:19 -0500 (EST)

> messages and so on.

This work would have to be done anyway, whether it's an ML or a MIB
or whatever.  Right?

> Furthermore you have a huge protocol overhead due to 

Just to be clear, by "huge protocol overhead", do you mean bloat in
the protocol at runtime, or bloat in the specification (or something
else?)?

If the former, there is some degree of truth to this since it's a text
based protocol and sometimes transfers header information that isn't
used.  But I've used it extensively, even in wireless environments
(~1200bps), and I've never regretted the decision to use it.

If the latter, that's also true, RFC 2616 is a large specification.
But it's already published, reviewed, and been validated by dozens of
commercial and open source implementations available today.  I don't
care how small a new protocol might be, it will take it years to reach
the maturity of HTTP.

If it's a given that you have to implement the chosen protocol from
scratch, then I'd agree that this is a consideration.  But if middlebox
vendors are concerned about time-to-market, and about ensuring the
most secure environment possible, I think they would all want to have
the opportunity to reuse somebody else's battle-hardened implementation
of an existing protocol.

MB

---

From: Jonathan Rosenberg <[email-masked]>
Date: Thu, 07 Feb 2002 11:10:18 -0500

If the only requirement for midcom were the ability to move data from
the agent to the middlebox, and possibly send data from the middebox
back to the agent, then HTTP might be a good fit (although asynchronous
push has never been part of its operation). However, based on these
requirements, I think  you might find that any protocol that exists
today will work. Most that I know of can move data from A to B and back.
How about SIP! Its arguably better than HTTP since it better handles
asynchonous messages in reverse. Of course, we shouldn't forget SOAP,
the catch-all for all protocols. How about BEEP? It would work too. FTP
might work. Hey, we could even have the agent telnet to the middlebox,
and exhcnage commands over telnet! Telnet is nice and thin. 

I'm amazed at how much time people worry about the underlying framework
protocol, which in the case of HTTP would do like 2% of the real work,
instead of worrying about the real protocol mechanisms. How do we
represent pinholes? Nat bindings? How do we group resources and allow
for them to be removed as a whole? How do resource timeouts work? What
kind of resource identifiers do we need? These are the real challenges
and the important work. Arguing whether to carry all of that on HTTP as
opposed to DIAMETER as opposed to MEGACO seems like a big waste of time.
It will also be a discussion which will likely take forever, since such
debates are usually based on anecdotes like "its more mature" which are
useless in making sound technical decisions. 

Why don't we worry about specifying the detailed semantics (NOT SYNTAX)
of the protocol, without assuming an underlying framework, and once the
entire protocol is specified, behaviorially, it will be a trivial
process to see exactly how each of those behaviors would map onto an
existing framework or protocol. I suspect most frameworks will provide a
small fraction of what is needed.

But, people seem to want to worry about the least important stuff first.
So be it. Wake me up when you are done so I can participate in the
important work.

-Jonathan R.

-- 
Jonathan D. Rosenberg, Ph.D.            72 Eagle Rock Avenue
Chief Scientist                         First Floor
dynamicsoft                             East Hanover, NJ 07936
[email-masked]                 FAX: (973) 952-5050
http://www.jdrosen.net                  PH:  (973) 952-5000
http://www.dynamicsoft.com

---

From: Martin Stiemerling <[email-masked]>
Date: Thu, 07 Feb 2002 17:41:50 +0100

Hi Mark,

> This work would have to be done anyway, whether it's an ML or a MIB

Yes, you are right. The point is that, when you start designing a 
ML/MIB/whatever you start from the beginning and thus you use http only as 
a "transporter". The real work is not the "transporter", the real work is 
how pinholes/bindings are managed and so on.

> the protocol at runtime, or bloat in the specification (or something

Sorry, I haven't been precisly enough, of course I mean both runtime 
overhead and specification.

> care how small a new protocol might be, it will take it years to reach

Yeah, but there is a trade-off between maturity and protocl thickness and 
in oppinion we have to make a lot of trade-offs anyway for the midcom 
protocol in the future.
Another point is, that http is stateles abd as far as I know we need a 
stateful protocol.

Martin

Martin Stiemerling

---

From: Martin Stiemerling <[email-masked]>
Date: Thu, 07 Feb 2002 17:46:40 +0100

Comments are in the text.

Martin

--On Donnerstag, Februar 07, 2002 11:10:18 -0500 Jonathan Rosenberg 
<[email-masked]> wrote:
[...]
> instead of worrying about the real protocol mechanisms. How do we

I agree with this point, but there should be always some room for 
discussions.

> kind of resource identifiers do we need? These are the real challenges

There are some proposal that show some solutions for handling pinholes or 
bindings. So let start the technical discussions! ;-)

---

From: Mark Baker <[email-masked]>
Date: Thu, 7 Feb 2002 11:37:37 -0500 (EST)

> Another point is, that http is stateles abd as far as I know we need a 

Ah, that's interesting.  I didn't get this from the requirements.
Could you direct me to the requirement that says, or suggests that?

MB

---

From: Mark Baker <[email-masked]>
Date: Thu, 7 Feb 2002 11:47:39 -0500 (EST)

> push has never been part of its operation).

I'm sorry, but this completely misrepresents what HTTP is.  What you
describe there is a transport protocol, not an application protocol.
To people who think it's a transport protocol, I like to point out that
HTTP has two ways of transferring data; PUT and POST.  Each means
something different.  PUT means "replace", and POST means "append".

> asynchonous messages in reverse.

SIP would also be a good protocol to consider.

Neither SOAP nor BEEP provide any application semantics by themselves,
whereas SIP and HTTP do.  So you'd have to reinvent lots of stuff.

> kind of resource identifiers do we need?

MidcomML would help you represent these things.  URIs would provide
identity to them (each pinhole gets a URI, for example, so it can be
DELETEd or its characteristics changed with PUT or POST).  The HTTP
methods would help you manipulate those resources.

MB

---

From: Keith Moore <[email-masked]>
Date: Thu, 07 Feb 2002 14:08:58 -0500

> So be it. Wake me up when you are done so I can participate in the

oh, didn't you realize that all problems are solved by HTTP and XML?
(which is just another way of saying that all problems are solved 
by adding another layer of indirection.)

I find it ironic that the main reason people want to run things over HTTP
is to get around NATs and firewalls, and now HTTP is being proposed as the
protocol to control NATs and firewalls.  In other words, HTTP has become 
its own justification.

Keith

---

From: Mark Baker <[email-masked]>
Date: Thu, 7 Feb 2002 14:30:22 -0500 (EST)

> oh, didn't you realize that all problems are solved by HTTP and XML?

(don't forget URIs)

Sarcasm aside, do you believe HTTP is inappropriate for *this* problem,
and if so, which requirement do you think it doesn't meet?

> by adding another layer of indirection.)

Well, no, it isn't.  It's a way of saying that if you create a general
enough coordination language, then a whole lot of problems can be solved
*within* the constraints of that language, not as a layer on top (what's
on top of the application layer anyhow?).

MB

---

From: Andrew Molitor <[email-masked]>
Date: Thu, 7 Feb 2002 13:34:28 -0600 (CST)

> and if so, which requirement do you think it doesn't meet?

	Yes. Performance.

	When I am attempting to process 1,000 transactions per
second on the embedded processor controlling my middlebox, the
last thing I need to do is laboriously decode 4K of garp
shooting out of a TCP socket, per transaction.

	HTTP is, after a fashion, universal. It's universal in the
same way as a Turing machine built from great oaken gears and
levers is universal.

---

From: Keith Moore <[email-masked]>
Date: Thu, 07 Feb 2002 14:35:33 -0500

> and if so, which requirement do you think it doesn't meet?

HTTP contributes nothing (other than overhead and confusion) to solving 
the problem.

> on top of the application layer anyhow?).

As Jonathan pointed out, the hard part is getting the semantics right, 
and HTTP doesn't help there in the least.  (nor does XML)

I'd also claim that the last thing midcom needs is a connection setup/
teardown, plus SSL negotiation, every time some process needs to establish 
a pinhole through a NAT.  And no, persistent connections aren't a general 
solution to the problem.

Keith

---

From: Mark Baker <[email-masked]>
Date: Thu, 7 Feb 2002 15:11:06 -0500 (EST)

> and HTTP doesn't help there in the least.  (nor does XML)

It's a matter of modelling the problem domain as a set of resources
identified by URIs.  If you do that, the only application semantics you
need are GET, PUT, POST, and DELETE.

Off the top of my head, you'd need resources such as;

http://mymiddlebox.org/pinholes

A GET would return a list of current pinholes, with each pinhole being
its own resource.  Invoking GET on each pinhole would describe that
pinhole (PinholeML?), which could be changed locally, then PUT back to
commit the change.  Invoking DELETE would remove the pinhole.

> a pinhole through a NAT.

How often is that done?  The requirements don't say.  I'd be surprised
(and concerned) if pinholes were being established with a periodicity
anywhere near the time it takes to negotiate a HTTPS connection.

>  And no, persistent connections aren't a general 
> solution to the problem.

That's possible.  But I didn't see anything in the requirements that
suggested that a stateless, connection based protocol couldn't be used.
Do the requirements need updating?

MB

---

From: Keith Moore <[email-masked]>
Date: Thu, 07 Feb 2002 15:18:27 -0500

> need are GET, PUT, POST, and DELETE.

I disagree with all of the above -

- modeling the problem domain as a set of resources doesn't make
  sense when those resources interact in arbitrary ways

- identifying the resources with URIs just adds cruft

- trying to coerce the semantics of midcom's requirements to fit
  onto existing HTTP methods is probably not a good idea . 

Keith

---

From: Mark Baker <[email-masked]>
Date: Thu, 7 Feb 2002 16:52:15 -0500 (EST)

>   sense when those resources interact in arbitrary ways

Show me in RFC 2616 where it says that resource state changes can't
affect changes in the state of other resources.

>   onto existing HTTP methods is probably not a good idea . 

Coerce?  What, my /pinhole example looked kludgy?  Note that I forgot to
add that if you wanted to create a new pinhole, you could just POST a
description of a pinhole to /pinhole, and get a 201 (Created) response
back with the Location header providing the URI of the new pinhole.

I challenge you to name a midcom application semantic that cannot be
modelled and implemented in this manner.

MB

---

From: Andrew Molitor <[email-masked]>
Date: Thu, 7 Feb 2002 16:01:43 -0600 (CST)

> modelled and implemented in this manner.

	You're missing the point.

	Of course you can drive any nail you like with a bulldozer.
Nobody has claimed that HTTP wouldn't work. It's pretty obvious to
everyone that it could be made to work. The various points being made
are that:

	- bulldozers don't make very good hammers
	- hey, wouldn't it be good to figure out what nails we want to
	  drive where before we start buying bulldozers to drive them?

---

From: Keith Moore <[email-masked]>
Date: Thu, 07 Feb 2002 17:02:37 -0500

> affect changes in the state of other resources.

For some reason, I've never accepted RFC 2616 as an example of good 
protocol design. 

> Coerce?  

Yes, coerce.  It seems better to figure out what the semantics should
be than to figure out how to get the semantics to fit into a bed that
was designed for a completely different person...er, purpose.

> modelled and implemented in this manner.

"Cannot" doesn't seem like a useful criterion.  I can make you fit 
into my bed with appropriate application of chains and/or axe.  
But that doesn't mean you'll like it.

Keith

---

From: Mark Baker <[email-masked]>
Date: Thu, 7 Feb 2002 17:50:53 -0500 (EST)

> For some reason, I've never accepted RFC 2616 as an example of good 

Fine.  But I want to address your concern here that I can't use HTTP and
resource modelling because a state change to one can't affect another.
Please show me where any version of HTTP says that.  Or, if you think
it's so poorly designed that it forget to say that, please tell me why
you think that.

> But that doesn't mean you'll like it.

Well, let's see shall we.  I already covered off pinhole creation,
deletion, and modification without much bloodshed.  Pick another one.

MB

---

From: Keith Moore <[email-masked]>
Date: Thu, 07 Feb 2002 18:13:47 -0500

> resource modelling because a state change to one can't affect another.

Since that's not what I said, I won't bother responding.  
 
> deletion, and modification without much bloodshed.  Pick another one.

Let me try one more time:

Just because you carry your forehead with you everywhere you go
does not mean it's a good tool to use for hammering nails.
(even if you do manage to hammer a few nails with it) 

The challenge that this group faces is to make a good protocol for 
allowing applications to work with NATs, not to figure out a new 
way to use HTTP.

Keith

---

From: Mark Baker <[email-masked]>
Date: Thu, 7 Feb 2002 21:24:06 -0500 (EST)

> allowing applications to work with NATs, not to figure out a new 

It would *NOT* (!!!) be a new way to use HTTP.  It would be using HTTP
the way hundreds of millions of people use it each day, and the way it
was designed to be used.  Just instead of URIs for books, weblogs, news,
and porn, there'd be URIs for middleboxes, rulesets, and pinholes.  HTTP
doesn't care; they're just resources to be manipulated with the same
generic set of methods.

The Web is not your typical application, and its domain of applicability
is only "limited" to things with identity.  Witness the recent example
of somebody putting a web server in a lego brick.

Another view on this might also be helpful;

http://www.xml.com/pub/a/2002/02/06/rest.html

MB

---

From: Keith Moore <[email-masked]>
Date: Fri, 08 Feb 2002 00:07:28 -0500

> is only "limited" to things with identity. 

Whatever gave you the idea that midcom had anything to do with "the Web"?
Or, for that matter, that "the Web" requires use of HTTP?

As attractive as it might sound, I don't think it works to try to coerce
the ideas of lots of different people into a single paradigm or protocol.

But this has nothing to do with midcom, so I'll stop here.

Keith

---

From: Martin Stiemerling <[email-masked]>
Date: Fri, 08 Feb 2002 10:58:19 +0100

> Could you direct me to the requirement that says, or suggests that?

As far as I see, this is not explicitly mentioned in the requirements, but 
midcom has to maintain pinholes and bindings in a middle box and therefore 
the midcom protocol should remember what it is doing with the middle box, 
i.e. states are needed.

Martin

---

From: Scott Brim <[email-masked]>
Date: Fri, 8 Feb 2002 08:48:34 -0500

> the midcom protocol should remember what it is doing with the middle box, 

No.  The requirements are that the interaction between agent and
middlebox be deterministic and that agent and middlebox must be able to
establish a known and stable state.  We say nothing about where state
information related to a particular rule is even held, except that each
side must maintain at least basic state related to its own functioning.
We certainly say nothing about *how* the protocol accomplishes this.