Subject: [core] draft-ietf-core-link-format-11 - discover virtual hosted resources

From: Thomas Fossati <[email-masked]>
Date: Sat, 14 Apr 2012 01:11:35 +0200

We have a couple of points in coap-09 where virtual hosting comes up: one is the (recommended) support of SNI in order to discriminate vhosts on a DTLS secured tunnel, the other is about the use of explicit Uri-Host and Uri-Port to demultiplex different origins on the same endpoint.

So it seems that a CoAP endpoint should be prepared to survive in environments where resources have non trivial authorities (by "trivial authority" I mean resources whose host:port can be inferred from the transport protocol.)

Yet, I could not find any text in link-format-11 explaining how virtual hosted resources can be advertised by the hosting endpoint.

At first one could imagine to just use an absolute URI for the target link, and that would solve all the problems.  But then, how should the querying endpoint interpret an explicit port in the link ?  E.g. if the discovered target URI is <coap://host:123/res>, then 123 is to be taken as the UDP port at which the server must be contacted, or just as the value to fill the Uri-Port option ?

Uri-Port is really nice (if only because it allows virtual hosting with just 2-3 bytes of overhead), but is it actually usable if it does not fit into the discovery mechanism ?

---

From: Klaus Hartke <[email-masked]>
Date: Sat, 14 Apr 2012 01:42:54 +0200

On 14 April 2012 01:11, Thomas Fossati <[email-masked]> wrote:
> how should the querying endpoint interpret an explicit port in the link ?  E.g. if the discovered target URI is <coap://host:123/res>, then 123 is to be taken as the UDP port at which the server must be contacted, or just as the value to fill the Uri-Port option ?

Since it's a coap:// URI, I'd assume that coap-09 Section 6.1 applies:

   The port subcomponent indicates the UDP port at which
   the CoAP server is located.

The Uri-Port option is mostly useful where a server forwards a request
to another node that needs to be able to reconstruct the original
request URI.

Klaus

---

From: Thomas Fossati <[email-masked]>
Date: Sat, 14 Apr 2012 10:25:16 +0200

Hi Klaus,

>   the CoAP server is located.

ok, that's also my understanding.  But then you can't do virtual hosting based on the Uri-Port value because it is 1:1 with the underlying transport protocol port, and as such it can always be inferred by the context.

Basically, if your interpretation (which I share) holds, I don't get the value added by the Uri-Port option.  At least related to the vhosting argument.

> to another node that needs to be able to reconstruct the original

Sorry but I don't get it.  The Proxy-Uri seem to have all the information.
Could you please give me an example ?

Thanks, t.

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 14 Apr 2012 13:20:37 +0200

> Sorry but I don't get it.  The Proxy-Uri seem to have all the  

On the leg that uses a Proxy-Uri option, yes.
Not all legs for all use cases for intermediaries do.

> Could you please give me an example ?

Imagine a reverse proxy that distributes queries to origin servers  
based on some criteria (say, Uri-Path).
Towards to origin server, the fully decoded form of options (Uri-*) is  
used.
On the way between the reverse proxy and the origin server, the  
reverse proxy could use Uri-Port to indicate the port number that was  
originally used for the request (i.e., towards the reverse proxy).

That is a bit of a special use case, but it is good to be able to  
express Uri-Port in the last leg to the origin server.

Now where does the reverse proxy get the configuration that tells it  
what origin server to use, and at what port number?
That is outside the scope here.
One assumption is that the link-format is a good source for such  
configuration information, but we are not completely specifying this  
case.
I think what you are complaining about is that we don't have a good  
way to express configuration for such a case in the form of a URI.
(I think the fact that this becomes visible in link-format is just a  
reflection on us using URIs in link-format.)

The reason the same considerations do not apply to Uri-Host is that we  
assume to have some form of indirection here (such as DNS) mapping reg- 
name style host parts to IP addresses, so it is common to have  
multiple reg-name style host parts map to the same IP address (at  
least in HTTP land).
We don't assume to have that indirection for port numbers, so the main  
use case for Uri-Host does not apply to Uri-Port.

Gruesse, Carsten

---

From: Thomas Fossati <[email-masked]>
Date: Sat, 14 Apr 2012 18:44:34 +0200

Hi Carsten, 

thank you very much for taking the time to provide such detailed explanation.

> Imagine a reverse proxy that distributes queries to origin servers based on some criteria (say, Uri-Path).
> Towards to origin server, the fully decoded form of options (Uri-*) is used.
> On the way between the reverse proxy and the origin server, the reverse proxy could use Uri-Port to indicate the port number that was originally used for the request (i.e., towards the reverse proxy).
> 
> That is a bit of a special use case, but it is good to be able to express Uri-Port in the last leg to the origin server.

If I understand correctly, you are saying that the reverse proxy may use the Uri-Port as a variable to both govern the routing towards the origin servers, and give these origins a parameter that they can use to select the final resource that has to be served.

Anyway, a reverse proxy "owns" the namespace and can organize it at its complete will, and also use an URI-mapping ruleset of choice.

The same granularity that is obtained in your example using the Uri-Port, may be easily achieved using an URI-mapping that moves the port (be it implicit or explicit) from the original URI to e.g. the first segment of the mapped URI:

  (UA) -> http://P:255/a/res -> (P) -> coap://A/255/res -> (A)

So, strictly speaking, the Uri-Port is not crucial to make this scenario work, right ?

> I think what you are complaining about is that we don't have a good way to express configuration for such a case in the form of a URI.

Basically, what I'm complaining about is that virtual hosting based on Uri-Port's works only if the client has a complete a-priori knowledge of the resource it has to access (so that it can contact server S at UDP port 5683 with Uri-Port 255 -- fantastic: 2 bytes vhosting!).

But there is no way for the client to get this information through discovery, which makes the whole cause for Uri-Port a little fragile.

ciao, t.

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 14 Apr 2012 19:28:53 +0200

> So, strictly speaking, the Uri-Port is not crucial to make this scenario work, right ?

Indeed, no. There are many ways to skin a URI.
(With that argument, we might decide to get rid of Uri-Query, too... Well, that would be radical.)

So, are you arguing for removing Uri-Port from the specification?
YAGNI is a strong argument, but it might be hard to put back Uri-Port when we do need it if we want to maintain the order of URI elements in the option sequence.

Grüße, Carsten

---

From: Thomas Fossati <[email-masked]>
Date: Sat, 14 Apr 2012 20:36:25 +0200

> YAGNI is a strong argument, but it might be hard to put back Uri-Port when we do need it if we want to maintain the order of URI elements in the option sequence.

I'm not taking any stance at present, my intent is just to clarify things -- first of all to myself :-)  Let's see what others say.

---

From: Zach Shelby <[email-masked]>
Date: Sun, 15 Apr 2012 20:47:40 +0300

Thomas,

> Uri-Port is really nice (if only because it allows virtual hosting with just 2-3 bytes of overhead), but is it actually usable if it does not fit into the discovery mechanism ?

Actually, as a result of IETF last call, we are actually limiting the scope of the "hosts" relation so that the context and target must be on the same host. See ticket http://trac.tools.ietf.org/wg/core/trac/ticket/193.

What this means in practice is that the target for this relation has to be on the same origin server. The result is that if the target is an absolute URI, then the host part should be interpreted as a virtual host name (and thus should be included in the Uri-Host option). The port of that URI would however be interpreted as an actual port on the origin server, thus this can't be used for identifying a virtual host in the way you are thinking.  There is no reason your virtual host name couldn't be really short though and thus just as efficient as using the Uri-Host field.

Thanks for pointing the need for this out, I will be sure to include clarifying text when closing that ticket. 

Zach

> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core

-- 
Zach Shelby, Chief Nerd, Sensinode Ltd.
http://www.sensinode.com
http://zachshelby.org  - My blog "On the Internet of Things"
http://6lowpan.net - My book "6LoWPAN: The Wireless Embedded Internet"
Mobile: +358 40 7796297

---

From: Thomas Fossati <[email-masked]>
Date: Mon, 16 Apr 2012 08:53:39 +0200

> Thanks for pointing the need for this out, I will be sure to include clarifying text when closing that ticket.

Very good.  Also, contextually adding an example in section 5 would be great.

Thanks.

---

From: Cullen Jennings <[email-masked]>
Date: Fri, 20 Apr 2012 06:42:51 -0600

Virtual hosting could work in coap if we using DNS or something like that because the URI 

coap://server1.example.com 

and 

coap://server2.example.com 

could resolve to the same IP address. But if we think we are not using DNS, it gets harder. Using ports could obviously work but make discover of the server on the non default port hard. 

I don't like how SNMP does this but you could have one server on the default port that all it did was tell you about all the other servers running on different ports. Anyways, I'd like to see us resolve this one way or another.

---

From: Klaus Hartke <[email-masked]>
Date: Sat, 21 Apr 2012 05:25:22 +0200

Hi Zach,

On 15 April 2012 19:47, Zach Shelby <[email-masked]> wrote:
> What this means in practice is that the target for this relation has to be on the same origin server. The result is that if the target is an absolute URI, then the host part should be interpreted as a virtual host name (and thus should be included in the Uri-Host option). [...] There is no reason your virtual host name couldn't be really short though and thus just as efficient as using the Uri-Host field.

Do you mean a server like "sensor1.example.com" could advertise a
virtual host "xy" rather than "temp.sensor1.example.com" in its
well-known core?

What happens when two different servers both advertise a virtual host
"xy"? Can a client with a cache use a stored response that is the
result of retrieving resource "/res" of virtual host "xy" at
[2001:DB8::abcd] to satisfy a request for resource "/res" of virtual
host "xy" at [2001:DB8::9999]? The request URI is <coap://xy/res> in
both cases.

Klaus

---

From: Zach Shelby <[email-masked]>
Date: Sat, 21 Apr 2012 09:39:56 +0300

> virtual host "xy" rather than "temp.sensor1.example.com" in its

There is no "sensor1.example.com" in the first place as you are not using DNS in this case. The origin server in this example would be identified by [2001:DB8::abcd]:5683. If /.well-known/core returns a link  <coap://xy/res> then maybe a cache should identify that as coap://xy.[2001:DB8::abcd]/res or in some other way associated with the IP address:port as this is the only unique identifier it could possibly have for the server. I do agree that is a bit awkward, other ideas?

Zach

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 21 Apr 2012 10:13:10 +0200

On Apr 21, 2012, at 08:39, Zach Shelby wrote:

> coap://xy.[2001:DB8::abcd]/res

Ouch.

One of our problems seems to be that to set up a reverse proxy, you want to be able to both tell the origin servers what the original SNI was as well as specify the actual address of the origin server.  You can't cram both into an URI.  (We had the same problem with Uri-Port before.)

But URIs are the thing we write down in link-format documents.  So link-format is maybe a bit weak to configure reverse-proxy style situations?  We should not try to fix that problem by tweaking the URIs, but by finding ways to represent the desirable information in link-format.

Grüße, Carsten

---

From: Zach Shelby <[email-masked]>
Date: Sat, 21 Apr 2012 15:54:28 +0300

> But URIs are the thing we write down in link-format documents.  So link-format is maybe a bit weak to configure reverse-proxy style situations?  We should not try to fix that problem by tweaking th…

Right, so you would prefer to represent virtual host names in some other way. What about something like this?

</res>;vh="host1",
</res>;vh="host2"

Thus when requesting the first link Uri-Host=host1 and for the second link Uri-Host=host2. This leaves us with the same caching problem that Klaus pointed out, how to differentiate between a cached resource host1/res on this server vs. host1/res on some other server? 

Zach

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 21 Apr 2012 17:14:09 +0200

> Right, so you would prefer to represent virtual host names in some other way. What about something like this?

Well, I'd prefer not to *have* virtual host names :-)

Once we want to have a single endpoint represent multiple endpoint identities, we run into this problem.  It's exacerbated by reverse proxies that need to carry around these identities together with the real endpoint (address/port) information.

I understand why all this was necessary in an IPv4 HTTP world, but I'm not so sure for CoAP.

Grüße, Carsten

---

From: Cullen Jennings <[email-masked]>
Date: Mon, 23 Apr 2012 07:08:12 -0700

up leveling a bit … why do we think we need virtual hosts?

---

From: Zach Shelby <[email-masked]>
Date: Mon, 23 Apr 2012 21:58:57 +0300

> up leveling a bit … why do we think we need virtual hosts?

Well, we have the ability to identify a virtual host using the Uri-Host option in a request. Thomas has a need for using CoAP virtual hosts, which is why this came up. We actually make use of virtual CoAP hosts in our backend servers as well, but we don't have a need to discover them using link-format. Thomas thinks it would be good to have a way to discover the available virtual host names using link-format.

Now, Klaus and Carsten have pointed out that trying to indicate virtual hosts (and then identify the resource in a cache) using the link-format is problematic. So probably we will end up only being able to indicate port numbers of other available servers on a host in the link-format. You can of course use DNS if you really want to use virtual hosts. 

Zach