Subject: [core] How is con supposed to be used to do resolving

From: Jim Schaad <[email-masked]>
Date: Wed, 15 Nov 2017 20:38:25 +0800

I have been having problems with the 'con' attribute.  This is in part
because I had decided that the original way that I understood the attribute
to work was never going to be successful.  I therefore had decided that it
was going to be the problem of the query endpoint since it would most likely
be in a better position to deal with the problem having more detailed
information about what it was looking for and what it wanted.  I find that
this is not the way the document has evolved so I am going to ask my
questions of the way the document is currently setup.

Start by assuming that the following has been execute:

POST coap://rd.example.com/rd?ep=sensor1&con="coap://sensor1.example.net"

  Content-Format: application/link-format

</sensors>;ct=40;title="Sensor Index"

</sensors/temp>;rt="temperature-c";if="sensor"

</sensors/light>;rt="light-lux";if="sensor"

</sensors/light>;rel="describedby";anchor="http://www.example.net/sensors/li
ghts"

<http://www.example.net/sensors/light>;rel="describes";anchor="/sensors/ligh
t"

</sensors/temp>;rt="alternate";anchor="/t"

</t>;rt="temperature-c";if="sensor"

POST coap://rd.example.com/rd?ep=sensor2&con="coap://sensor2.example.net"

  Content-Format: application/link-format

</sensors>;ct=40;title="Sensor Index"

</sensors/temp>;rt="temperature-c";if="sensor"

</sensors/light>;rt="light-lux";if="sensor"

</sensors/light>;rel="describedby";anchor="/information/lights"

<http://www.example.net/sensors/light>;rel="describes";anchor="/sensors/ligh
t"

</sensors/temp>;rt="alternate";anchor="/t"

</t>;rt="temperature-c";if="sensor"

</information/lights>;ct=40;title="Light Description"

</information/lights>;rel=describes;anchor="/sensors/light"

</information/lights>;rel=describes;anchor="coap://sensor1.example.net/senso
rs/light"

I now execute a series of lookups and try to figure out both what is
returned and what it means.

GET coap://rd.example.com/rd/lookup?if="sensor"

Return payload is

</sensors/temp>;rt="temperature-c";if="sensor";anchor="coap://sensor1.exampl
e.net"

</sensors/light>;rt="light-lux";if="sensor";anchor="coap://sensor1.example.n
et"

</t>;rt="temperature-c";if="sensor";anchor="coap://sensor1.example.net"

</sensors/temp>;rt="temperature-c";if="sensor";anchor="coap://sensor2.exampl
e.net"

</sensors/light>;rt="light-lux";if="sensor";anchor="coap://sensor2.example.n
et"

</t>;rt="temperature-c";if="sensor";anchor="coap://sensor2.example.net"

That is reasonable understandable.  One can use the anchor as a root and
resolve the href against it and get the address of the resource without too
many problems.

GET coap://rd.example.com/rd/lookup;?rel="describedby"

Return payload is

</sensors/light>;rel="describedby";anchor="http://www.example.net/sensors/li
ghts"

</sensors/light>;rel="describedby";anchor="coap://sensor2.example.net/inform
ation/lights"

Now this seems to be a slight problem, the resource on the second cannot be
found by resolving the href with the anchor as the answer in that case would
be coap://sensor2.example.net/information/lights/sensors/light  which is
wrong.  One could say that it is resolved just against the authority portion
and get the right answer.  However if the following was registered

</temp>;rt="temperature-c";anchor="/sensor"

The rules would not work correctly as in that case you want to use the
context plus the anchor plus the href to get the right answer.  The RFC
states that the anchor for rel=hosts is a relative URI so this would seem to
be totally legal.

GET coap://rd.examples.com/rd/lookup?rel="describes"

<http://www.example.net/sensors/light>;rel="describes";anchor="coap://sensor
1.example.net/sensors/light"

<http://www.example.net/sensors/light>;rel="describes";anchor="coap://sensor
2.example.net/sensors/light"

</information/lights>;rel=describes;anchor="coap://sensor2.example.net/senso
rs/light"

</information/lights>;rel=describes;anchor="coap://sensor1.example.net/senso
rs/light"

Both of the last two lines give problems.  One of the has the right
authority but if the full path is used then the wrong location.  The other
one has the wrong authority so it would be unfindable.

I would like to see the rules make explicit.  However, my current set of
inferred rules would be:

1.  If there is no anchor - set the anchor to the con value.

2.  If the anchor value is relative, resolve the anchor value to the con
value and replace.

I think this set of rules is wrong and would like to see a corrected set of
rules.

---

From: Christian Amsüss <[email-masked]>
Date: Wed, 15 Nov 2017 14:58:01 +0100

Hello Jim,

On Wed, Nov 15, 2017 at 08:38:25PM +0800, Jim Schaad wrote:
> </sensors/light>;rel="describedby";anchor="http://www.example.net/sensors/lights"
>
> </sensors/light>;rel="describedby";anchor="http://www.example.net/sensors/lights"
> </sensors/light>;rel="describedby";anchor="coap://sensor2.example.net/information/lights"

> Now this seems to be a slight problem, the resource on the second
> cannot be found by resolving the href with the anchor as the answer in
> that case would be
> coap://sensor2.example.net/information/lights/sensors/light  which is
> wrong.

The second example resource causes no problems. Resolving the reference
"/sensors/light" against "coap://sensor2.example.net/information/lights"
gives "coap://sensor2.example.net/sensors/light". The term "Resolve
against" means the process described in RFC3988 Section 5.2. (I just
added that to the terminology section).

(The first example is more critical as it doesn't mean what you probably
intend it to, but as this was not your main point, I'm banning it into a
footnote to not interrupt the flow here[1].)

> However if the following was registered
> 

Sorry, maybe I don't fully understand that part. Why would three
components be needed, and why is the `rel` relevant here?

I don't see anything problematic about this example. The RD does not
simply replace the anchor attribute literlaly with the con, but it
resolves the anchor against the con (as you suggested in the next
paragraph). This yields the con when the anchor is absent/empty, but in
that case would give

| </temp>;rt="temperature-c";anchor="coap://registered-device-address/sensor"

in the resource lookup interface.

> 2.  If the anchor value is relative, resolve the anchor value to the con

These are already the intended rules; however, they are not spellt out
explicitly for the RD but only for the client in the appendix, and are
supposed to follow from the data model always dealing in absolute
references. We'll need to do a better job getting them across.

(What makes that job hard to me is that it conflicts with keeping the
normative part terse and not repeating what other documents already say,
because that's what I think caused the RFC5988/RFC6690 mismatch).

> [...]
> </information/lights>;rel=describes;anchor="coap://sensor2.example.net/sensors/light"
> </information/lights>;rel=describes;anchor="coap://sensor1.example.net/sensos/light"
> 

There, the point I previously only made in a footnote[1] strikes: What the
second device registered was

| </information/lights>;rel=describes;anchor="coap://sensor1.example.net/sensors/light"

and that is a statement about
coap://sensor1.example.net/information/lights (the href resolved against
the anchor) -- a resource that was nowhere else described.

The RD's response just reflects that.

> I think this set of rules is wrong and would like to see a corrected set of

Do you still think that they are broken?

Thanks for your input
Christian

[1] footnote:

The wrong statement here is the link registered at POST. We're
applying the same RFC6690 rules to .well-known/core as to lookup links,
and the registered link does not contain any statement about the
resource coap://sensor1.example.net/sensors/light, as I presume was
intended.

The rule there is that hrefs are resolved via their anchors -- something
I don't like because it differs subtly from 5988, but unless we do a
6690bis before RD, these are the rules. (See
<https://mailarchive.ietf.org/arch/msg/core/TmFHLYZ6G_PAJXw8JAGY0zB06FA>).

That link would be interpreted by both the RD and any other 6690 client
as (fully resolved) <http://www.example.net/sensors/light>;rel="describedby";
anchor="http://www.example.net/sensors/lights". What I think you intend to
express with the link can not be expressed in link-format serialization,
only by explicitly stating one's known address and thus defeating
protocol-negotiation. (If RFC6690 used the mechanism 5988 intended and
5988 clarifies, that would be possible easily).

-- 
To use raw power is to make yourself infinitely vulnerable to greater powers.
  -- Bene Gesserit axiom

---

From: Jim Schaad <[email-masked]>
Date: Thu, 16 Nov 2017 08:58:23 +0800

Ok - Lets go back one step.
RFC 6690 has the algorithm.

To resolve what is in the <>s use the following:

(a) The anchor parameter, when specified.
(b) Origin of the target URI, when specified
(c) Origin of the link format resource's base URI.

That provides two problems.  The first is that if the anchor parameter is
not an origin URI, is the relative path ignored, and if it is a relative
path URI does that mean you then do (c).  Also if it is not 'hosts' then it
makes no sense to try and resolve the target URI with it.  If you are using
a relation of 'describesby' then the target of the anchor has nothing to do
with where the actual resource you are talking about is.  It should ways use
(c) to resolve the target URI.

(a) The anchor parameter, when specified and the relation is 'hosts'

I am unclear what (b) refers to as trying to resolve what is in the angle
brackets by what is in the angle brackets (the target URI [A] does not make
any sense.  On the other hand, if the target URI is an absolute URI, then
trying to resolve it against an anchor value makes no sense either as the
original URI would be returned.

I messed up my memory on (c) as I always thought it was the document URI and
not the origin URI.  This would modify some of what I did below.

When we are talking about a links resource, it always makes sense to use (c)
as the last step, because this is after all a resource on the server.  This
is less true if we are talking about an RD where the last step is always
going to give the wrong answer.

My new algorithm would therefore be:

1.  If there is no anchor and the relation is hosts - add the anchor with
the con value.
2.  Otherwise, resolve the 'href' using 'con' value and replace.

Additional comments can be found below as well.

[A] RFC 6690 - Each link conveys one target URI as a URI-reference inside
angle brackets ("<>").

-----Original Message-----
From: Christian Amsüss [mailto:[email-masked]] 
Sent: Wednesday, November 15, 2017 9:58 PM
To: Jim Schaad <[email-masked]>
Cc: [email-masked]; 'core' <[email-masked]>
Subject: Re: How is con supposed to be used to do resolving

Hello Jim,

The second example resource causes no problems. Resolving the reference
"/sensors/light" against "coap://sensor2.example.net/information/lights"
gives "coap://sensor2.example.net/sensors/light". The term "Resolve against"
means the process described in RFC3988 Section 5.2. (I just added that to
the terminology section).

[JLS] I hope you have the RFC number wrong here - this makes zero sense.  I
assume you mean RFC 3986.   I just read this document (oops should have done
it before) and now my mind is going to get this correct.  I am sure that my
code base already did get it correct, I just never tried it out.
[/JLS]

[JLS] The first example was critical as it was part of my main point - I get
the wrong answers for the target URI.

> context plus the anchor plus the href to get the right answer.  The 
> RFC states that the anchor for rel=hosts is a relative URI so this 
> would seem to be totally legal.

Sorry, maybe I don't fully understand that part. Why would three components
be needed, and why is the `rel` relevant here?

I don't see anything problematic about this example. The RD does not simply
replace the anchor attribute literlaly with the con, but it resolves the
anchor against the con (as you suggested in the next paragraph). This yields
the con when the anchor is absent/empty, but in that case would give

[JLS] Part of my problem was that I resolved using the wrong algorithm.
What I should have registered was 

<temp>;rt="temperature-c";anchor="/sensor"

At which point - RD would return

<temp>;rt="temperature-c";anchor="coap://sensor2.example.com/sensor"

And I would get the correct answer when I resolved the target using the
anchor.
[/JLS]

|
</temp>;rt="temperature-c";anchor="coap://registered-device-address/sensor"

> 2.  If the anchor value is relative, resolve the anchor value to the 
> con value and replace.

s/light"
> authority but if the full path is used then the wrong location.  The 
> other one has the wrong authority so it would be unfindable.

|

and that is a statement about
coap://sensor1.example.net/information/lights (the href resolved against the
anchor) -- a resource that was nowhere else described.

[JLS] I don't understand - the resource is described elsewhere, but the
target URI is not resolvable. A better answer to return would have been

<coap://sensor2.example.net/information/lights>;rel=describes;anchor="coap:/
/sensor1.example.net/sensor/light"

As the target URI is now correctly resolved.  It was not a problem when the
same item w/o the full resolution was returned from /.well-known/core
because the correct target URI could be resolved.
[/JLS]

> I think this set of rules is wrong and would like to see a corrected 
> set of rules.

[JLS] Yes.

Thanks for your input
Christian

[1] footnote:

The wrong statement here is the link registered at POST. We're applying the
same RFC6690 rules to .well-known/core as to lookup links, and the
registered link does not contain any statement about the resource
coap://sensor1.example.net/sensors/light, as I presume was intended.

The rule there is that hrefs are resolved via their anchors -- something I
don't like because it differs subtly from 5988, but unless we do a 6690bis
before RD, these are the rules. (See
<https://mailarchive.ietf.org/arch/msg/core/TmFHLYZ6G_PAJXw8JAGY0zB06FA>).

That link would be interpreted by both the RD and any other 6690 client as
(fully resolved) <http://www.example.net/sensors/light>;rel="describedby";
anchor="http://www.example.net/sensors/lights". What I think you intend to
express with the link can not be expressed in link-format serialization,
only by explicitly stating one's known address and thus defeating
protocol-negotiation. (If RFC6690 used the mechanism 5988 intended and
5988 clarifies, that would be possible easily).

--
To use raw power is to make yourself infinitely vulnerable to greater
powers.
  -- Bene Gesserit axiom

---

From: Christian Amsüss <[email-masked]>
Date: Thu, 16 Nov 2017 12:15:34 +0100

Hello Jim,

hello authors and working group: please especially see last paragraphs,

> (c) Origin of the link format resource's base URI.

That's not the sequence. The authority component stems from

a) the <> if it has an authority component, otherwise from
b) the anchor parameter if it contains an authority component, otherwise from
c) the address in which the document was requested.

(I prefer to think of the process as always generating a resolved URI
and then looking at that URI's authority component, but it can be
brought down to that).

> The first is that if the anchor parameter is
> path URI does that mean you then do (c).

What do you mean by "an origin URI"?

If the anchor is a relative reference (precisely: any of path-absolute,
path-noscheme or path-empty), {
    if the <> starts with a slash,
       the anchor is completely irrlevant to the target URI,
    but at any rate, then is the anchor irrelevant to the resolution
}.

But these are complicated rules that were not invented as such, all
emerge from resolving the URIs.

> Also if it is not 'hosts' then it makes no sense to try and resolve
> the target URI with it.

'hosts' is not the relation where the rule about resolving the target
against the anchor comes from. That rule is only a matter of the
link-format file format that says so.

'hosts' is just a relation that is defined for that file format as a
default rel if no other is given -- and has no particularly well-defined
semantics. Think of the device going "I want to advertise that resource
and need to do that in a link. Somebody said that a link needs to have a
relation, so what can I say about that resource? Well, I 'host' it, that's
for sure".

> If you are using a relation of 'describesby' then the target of the
> anchor has nothing to do with where the actual resource you are
> talking about is.  It should ways use (c) to resolve the target URI.

These would be the rules of RFC5988(bis) -- but in both documents, the
rules applied are independent of the "rel" parameter. And they need to:
otherwise, no client could make sense of target attributes in presence
of an unknown rel, and that's not even considering multiple rel values.

(I'm skipping over a section of your mail because I think it is taking
too many turns after an original turn we already disagree on).

> going to give the wrong answer.

One goal of the RD is to have it usable by clients that don't know it's
an RD (or what an RD is).

That is why the RD does not literally reflect the submitted links as
they were serialized, but resolves them as necessary so that getting the
full URIs never touches step c).

> /sensor1.example.net/sensor/light"

That's the intended statement, but that was not what was registered.

sensor2 registered:

and that meant the nonsentical

| <coap://sensor1.example.net/information/lights>;rel=describes;anchor="coap://sensor1.example.net/sensors/light"

already by the time it was stated in
coap://sensor1.example.net/.well-known/core. The device should rather
have registered it as

| <coap://sensor2.example.net/information/lights>;rel=describes;anchor="coap://sensor1.example.net/sensor/light"

in the first place, and yes that means that it needs to know its own
host name, but as long as RFC6690 stays as it is, that can't be
expressed more easily. (And as much as I'd like to offer the rev=
attribute to the rescue here, that has been declared dead by too many
people for me to resurrect it here).

(to everyone in the WG):

The above example and general confusion that exists around this lead me
to think that it might still be better to do jump ship on RFC6690's
href-relative-to-anchor behavior as long as barely anybody implements it
as-is-written anyway.

The many mails on this topic show that web linking is not a too well
understood concept, and if we're (in resource-directory and general WG
use) teaching it in a way that is in important parts different from web
linking as done on the rest of the web (as I'm doing in half this mail),
we're doing nobody a favor.

Best regards
Christian

---

From: peter van der Stok <[email-masked]>
Date: Sat, 18 Nov 2017 02:15:18 +0100

Hi Christian,

As a co-author of RD, but especially as someone who has gone through the 
web-linking learning process:
see below for my opinion,

> we're doing nobody a favor.

I do understand your frustration.
Part of the problem is indeed the complex structure of the web linking 
algorithms.
They allow the construction of nonsense statements as you point out in 
the e-mails.
That is not a problem of the RD.
A suggestion by me is to extend the appendix with the web linking rules 
that are used in the examples.
Consequence, the examples and the associated rules are consistent but 
cover only a part of what is possible.

Personally, I do not think that leaving RFC 6690 will solve the 
misunderstandings, other misunderstandings will come up.

Peter

---

From: Christian Amsüss <[email-masked]>
Date: Sat, 18 Nov 2017 11:01:30 +0100

> misunderstandings, other misunderstandings will come up.

I'm not saying we should leave RFC6690. I'd rather fix what's broken
there before those parts are implemented too widely (my impression is
that so far, constrained devices implement the server side and don't
ship controversial statements), and follow an RFC6690bis in RD (possibly
keeping the normative reference to 6690+errata in RD).

We're doing penance for what went wrong in 6690 in RD -- the differences
to 5988, the ambiguity about the link source in .well-known/core and the
almost undefined semantics of "hosts" are things that are surfacing in
the RD discussion.

Back in the day, my resident expert on repentance taught me that what
counts is active repantance, and for once I agree with him.

Best regards
Christian