Subject: [Tools-discuss] RFCmarkup v1.28

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 26 Jul 2006 17:46:57 +0200

Hi,

  So I've made a set of new changes to rfcmarkup, which should now
be reflected in any htmlized documents retrieved from /html/... on
the first tools server. (I haven't cleaned out the repository on
the two other servers yet - when you test this, please go to
http://www1.tools.ietf.org/html/rfc4321 etc., not to www2 or www3)

What I've done is to serve a document which doesn't have any <h?/>
tags to start with, but if the browser has javascript, I run a snippet
which adds the <h?/> tags, on the theory that most browsers with java-
script turned on will also have css which preserves the look-and
feel of the result - also with the <h?/> tags added.  (I really should
add an explicit check for css capability, but I'll leave that as a
future refinement).

Frank and Julian, especially:  Will you check that the result works
for both of you - Frank should get a result which is back to what it
was before last Saturday, while Julian should get a document which
will let the Document Map plugin work, *and* preserves the look-and
feel of pre-Saturday (except for the greyed-out page header/footers).

Let me know whether this works for you...

Regards,

	Henrik

_______________________________________________
Tools-discuss mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tools-discuss

---

From: Julian Reschke <[email-masked]>
Date: Wed, 26 Jul 2006 18:14:46 +0200

> Let me know whether this works for you...

Works for me, except that font size in print preview seems to be too small.

Best regards, Julian

---

From: Elwyn Davies <[email-masked]>
Date: Wed, 26 Jul 2006 19:49:34 +0100

> Let me know whether this works for you...
This looks generally good in Firefox, but the fount size for <h1> in IE 
(v6.0, XP SP2)  is very much too large.and is not the same as the body 
test for all the other headers (~50% bigger for <h2> and  ~25% bigger 
for <h3>).

Number of nits:
- the sample (RFC4321) has a title which runs to 2 lines - only the 
first one goes into the <h1> - aesthetic rather than vital
- however an internet draft with a two line title does something 
different  (something to do with the blank line?)- see 
http://www1.tools.ietf.org/html/draft-aoun-midcom-intrarealmcalls-00 - 
here the second line of the title is turned into an <h2> (and is 'sort 
of' left justified rather than centred - the span takes in the spaces on 
the left of the title words.
- This draft doesn't get its headers put into <h*> at all: 
http://www1.tools.ietf.org/html/draft-aoun-middlebox-token-authentication-00
- (an unpleasant corner case) 
http://www1.tools.ietf.org/html/draft-aoun-mgcp-nat-package-02 has a 
reference to a draft file in a section title (s9) and the algorithm 
doesn't quite work on the table of contents  (it incorporates the leader 
and the page number into the hyperlink)..  Need a pattern that allows 
only a single non-trailing period perhaps?
(all this was looking for a draft or RFC with a very long section title 
that spills onto two lines - finally found one...)
- s2.8 of  
http://www1.tools.ietf.org/html/draft-ietf-problem-issue-statement-05 - 
the second line of the title isn't in the <h3>
- A badly formed draft (missing the Expires: on the third line of the 
header) produces some unexpected results: 
http://www1.tools.ietf.org/html/draft-ietf-ipngwg-icmp-v3-07. BTW idnits 
does not complain about this.

I agree that the print preview selects the wrong scaling factor (80% in 
Firefox and 75% in IE6) - 100% would be good but I don't know if you 
have any control over this.

Regards,
Elwyn

---

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 26 Jul 2006 21:37:21 +0200

Hi Elwyn,

Thanks for the feedback.

> test for all the other headers (~50% bigger for <h2> and  ~25% bigger 

Hmm!!  That should not happen, given that IE 6.0 understands css.
I've made a change in the css, could you check this out again?

> the left of the title words.

Actually, the difference you see is because I fixed the 2-line title problem
while you were testing, before you got to this draft ;-)

If you follow the link to the plain-text version of that particular draft,
you'll see that the second line is mis-aligned in the original, too.  I've
fixed the span taking in the spaces - it shouldn't do that.

> - (an unpleasant corner case) 

Right.  Currently I ignore sections which don't start flush-left.  I
may try to fix this later.

> only a single non-trailing period perhaps?

Oh, bugger!  Right, I'll see what I can do about that one.

> the second line of the title isn't in the <h3>

Right.  I think I'm going to give up on this one - I think drafts that
lack a blank line between section title and section text may be just as
likely as having a very long section title...

> does not complain about this.

Bah.  Rfcmarkup takes the "11" in "11 July 2005" to be a section number.
... And how the blip should it know that this isn't a section number?...

> have any control over this.

I've now set the font size to 10.5pt.  If I set it to 11pt the text goes
outside the default margins on A4 paper, at least on my Mac.

Remember that since the settings have to work for both A4 and Letter size
paper there will be more unused paper than if you could adjust for only
one of them...

Regards,

	Henrik

---

From: Elwyn Davies <[email-masked]>
Date: Wed, 26 Jul 2006 21:27:33 +0100

> I've made a change in the css, could you check this out again?
This seems to be fixed (even if I can't get rfc4321 to refresh on IE) - 
rfc4320 is fine now and rfc4321 looks good on Firefox.
> fixed the span taking in the spaces - it shouldn't do that.
What about 3 line titles? rfc4319 ... sorry!
> may try to fix this later.
OK
> likely as having a very long section title...
Yes... I wouldn't complain too hard.  Section titles that long are just 
wrong (he said, as editor of the draft in question :-))

> ... And how the blip should it know that this isn't a section number?...
Cos it comes before the title ;-) .. Maybe we just ignore this one, fix 
idnits and wrk with decent id's.
> paper there will be more unused paper than if you could adjust for only
This looks better now.
I was on A4. I get 100% on Firefox but still nominally 75% on IE - but 
the text is bigger and looks ok (strange).

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 27 Jul 2006 01:10:26 +0200

Hi again,

Ok, so I think I've fixed most of the remaining things in v1.29:

on 2006-07-26 22:27 Elwyn Davies said the following:

[...]
> rfc4320 is fine now and rfc4321 looks good on Firefox.

Ok, good.  I've actually done one more tweak to the CSS.  Let me know
if there is any further problem on IE with the headers not having the
same font-size as the rest of the text.

[...]
> What about 3 line titles? rfc4319 ... sorry!

Ouch!  Ok, should be fixed now.

>>> - (an unpleasant corner case) 

Ok, this should also work now.

[...]
>>> only a single non-trailing period perhaps?

Fixed.

> wrong (he said, as editor of the draft in question :-))

Aha ,;-)

Well, better shorten the line if you want the markup to look good ;-)

> idnits and wrk with decent id's.

Since rfcmarkup doesn't try to parse the document, but applies regular
expressions to do its stuff, 'before the title' is a bit hard to handle.
I'm giving up on this, for the time being.

[...]
> the text is bigger and looks ok (strange).

Ok.  Sounds reasonably good.

Thanks,

	Henrik

---

From: Elwyn Davies <[email-masked]>
Date: Thu, 27 Jul 2006 11:05:03 +0100

IE now looks fine - printing on both Firefox and IE looks good. BTW I 
realized that the 75% on IE is not how it scales the printing but is a 
way of zooming the on screen display of the preview. Doh!

The product of a paranoid's breakfast:

1. http://www1.tools.ietf.org/html/rfc3410 : The second item (2.2) that 
claims to be on page 4 in the ToC doesn't get a link (something to do 
with longish title and only one period in the leader?)

2. 
http://www1.tools.ietf.org/html/draft-aoun-middlebox-token-authentication-00: 
The section headers are now <h2> but the title is still body text. This 
one has 'Expires on'

3. http://www1.tools.ietf.org/html/draft-ietf-ipngwg-icmp-v3-07: (no 
'Expires:' at all) - how about not looking for the title etc until after 
the second group of totally blank lines (or the first group that isn't 
at the start of the document)?

4. http://www1.tools.ietf.org/html/draft-aoun-mgcp-nat-package-02: This 
is a very badly formatted draft.. you fixed the link in the ToC problem 
but it has the same problem as #2 above and thereafter the markup of 
section headers is semi-random. Sections 1, 2 and 3 miss out; the first 
three non-empty body text lines on p3 become a header.  Sections 3.x are 
found but not s4 onwards.  s4.x you would have difficulty with as they 
are indented. Horrible! I think I owe you a beer if you can canonicalize 
this one!

5. 
http://www1.tools.ietf.org/html/draft-ietf-ipngwg-icmp-v3-07#section-3.3:  
Another horrid problem: This section contains a reference of the form 
[IPv6, Section 4.5].  The result is that the IPv6 reference is not 
turned into a link and Section 4.5 is turned into an internal link.

Just joking...
It doesn't deal with RFC280 or RFC331 too well! I was looking to see if 
having 'draft' in the title confused the script (it didn't).  RFC1038 
exercises the upper reaches of the header level algorithm, and 
(slightly) surprisingly the document map isn't confused!

/Elwyn
Bug finder general

---

From: Julian Reschke <[email-masked]>
Date: Thu, 27 Jul 2006 12:10:46 +0200

Elwyn Davies schrieb:
> ...
> ...

I'm currently experimenting with rfc2629.xslt knowing about sections in 
external references (see 
<http://greenbytes.de/tech/webdav/rfc2629xslt/rfc2629xslt.html#rfc.section.10.12>), 
and this is one of the output formats I was considering to add. Adding 
support to RFCmarkup should be simple enough, but I'm wondering whether 
that format is used enough in practice to warrant support in rcf2629.xslt?

Best regards, Julian

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 27 Jul 2006 12:44:43 +0200

Hi Elwyn,

Thanks for more feedback;

> with longish title and only one period in the leader?)

Right. Won't fix.

> The section headers are now <h2> but the title is still body text. This 

Mmm.  Right.  Won't fix now (not trivial), but maybe later.

> at the start of the document)?

Can you put that in a regexp ;-) ?

The boldfaced 11 July is taken to be a section - I tried to require the
section numbers to contain a period, but had to revert that as too many
document have major section numbers without a period.

Currently the title is the first group of lines which are preceded by
a line which begins with "Category:" or ends in a year, then a blank
line.  I'll look at changing that to the pattern you suggest, for the
next version.

> are indented. Horrible! I think I owe you a beer if you can canonicalize 

Actually, it seems you're looking at an old cached copy - after refreshing
here, this one looks pretty good too me, on all 3 servers.

> turned into a link and Section 4.5 is turned into an internal link.

Ah. I see.  Yes, this is one of several variations of combination of
references and section numbers.  There are others I know of which I don't
handle.  I can start going through them, but don't expect full coverage
any time soon - the reason I haven't even started on these are that there
are a lot of variation...

> having 'draft' in the title confused the script (it didn't).

Heh.  I think I'll just accept failure on these two...

>  RFC1038 

I'm setting the level to 6 even if it's above 6, which it is here
(document title + 6 levels of sections.)  This document was the
original reason why I added css styles for h7, h8, h9 -- the first time
I looked at this one, the section headers were in a tiny tiny font.
(It seems Firefox tries to handle at least <h7/>, but I've seen reports
that other browsers even crash on <h7/> or larger, so I'll be a good boy
and draw the line at <h6/>).

Regards,

	Henrik

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 27 Jul 2006 12:58:11 +0200

Hi Julian,

> that format is used enough in practice to warrant support in rcf2629.xslt?

I'd say no.  The two variations I believe I've seen most are your first
and third format; "[reference], Section sec" and "Section sec of [reference]".

I think this could be the first time I see the "[reference, Section sec]"
form, and would suggest not supporting it - it's a bit of a bastard.  The
three first you do support seems good enough -- I assume that format 4 and
5 are for cases where you're mentioning multiple sections of a single
reference in your text, and don't want to boringly repeat the reference
every time.

Regards,

	Henrik

---

From: Julian Reschke <[email-masked]>
Date: Thu, 27 Jul 2006 13:29:46 +0200

> reference in your text, and don't want to boringly repeat the reference

Correct.

BTW: one of the reasons I added this to my RFC2629 extension was that 
RFCmarkup supports these kinds of references, and I wanted to directly 
link to sections (rfc2629.xslt now links to tools.ietf.org/html rather 
than www.ietf.org).

Best regards, Julian

---

From: Elwyn Davies <[email-masked]>
Date: Thu, 27 Jul 2006 12:59:23 +0100

Hi Henrik.

A couple of thoughts:

Without doing major parsing you could use re capabilities to separate 
off the header part and only apply certain rules to the parts.
Thus:
Use findall to find all the blank lines.
Identify the start of the title as the first group of blank lines inside 
the document (i.e., ignoring any blank lines at the beginning) - use 
group and start to get positions.
Chop the data up and apply re's as required. then resplice.

Some more below...

/Elwyn
 
>> with longish title and only one period in the leader?)
One possibility if you did want to try would be to identify the leader 
end/number/end of line pattern from the early part of the ToC and then 
apply it throughout ToC.
Not a big deal.
> Mmm.  Right.  Won't fix now (not trivial), but maybe later.
See above.
> line.  I'll look at changing that to the pattern you suggest, for the
See above.
> here, this one looks pretty good too me, on all 3 servers.
Right.. this is pretty much OK apart from the s4.x which are 
additionally indented.  ... let's not bother too much.

<<snip>>

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 27 Jul 2006 15:01:53 +0200

Hi Julian,

> link to sections (rfc2629.xslt now links to tools.ietf.org/html rather 

Aha.  Cool.  Nice to see how one the general availability of one infrastructure
feature results in other features built on it :-)

I'm playing a bit with the thought of post-processing the output of rfcmarkup,
or alternatively writing a scanning parser (rather than rfcmarkup's almost
stateless regexp approach) and combining that with the output of Jari's
getauthors script in order to produce 2629 xml output, which could then be
used with your rfc2629xslt and related tools to provide a number of different
representations.  But I have too many unfinished tools-related tasks running
already to start this one up right now ;-)

> Best regards, Julian

Likewise, best,

	Henrik

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 27 Jul 2006 15:07:46 +0200

Hi Elwyn,

> Chop the data up and apply re's as required. then resplice.

Yes, that could be a possibility.

> end/number/end of line pattern from the early part of the ToC and then 

Right.  As soon as you move from a straight overall re approach to something
more stateful, such as identifying and splitting off doc header, title, toc
etc, you can do better handling.

I'll consider this for a later version, but it should be a major re-write,
to change the approach, rather than trying to tweak this in here and there,
I think.

>> line.  I'll look at changing that to the pattern you suggest, for the

Agreed.  This might be an easier fix than the rest.

> additionally indented.  ... let's not bother too much.

Right.

Regards,

	Henrik

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 28 Jul 2006 13:21:55 +0200

Henrik Levkowetz wrote:

> please go to http://www1.tools.ietf.org/html/rfc4321 etc.,

The RFC 4321 output is fine.  When I ask for a "generic" draft
(no number, no extension) like draft-ietf-usefor-useage I get
draft-ietf-usefor-useage-01 without extension, and the added
(thanks) text link also goes to draft-ietf-usefor-useage-01.

If that's working as designed it's okay, but maybe you wanted
to create a link to draft-ietf-usefor-useage-01.txt (?)

Checking this document:  In its section 1.2 it has a reference
of the style [USEFOR] at the begin of a paragraph, you missed
that.  You caught other [USEFOR] etc. within paragraphs.

At the moment I get an "document contains no data" error for a
broken link like draft-ietf-usefor-article-format - you created
the link based on draft-ietf-usefor-article-format-*.txt, this
is old stuff, you don't have it in your database.

Checking what's going on, if I try the ID URL for this it's an
ordinary non-empty 404.  My browser didn't get the fine point of
the 303 for its GET HTTP/1.0 with the HTML URL.

Trying RFC 1945:  No 303 in HTTP/1.0.  And also no link to the
"obsoleted by", pushing 303 issue, now checking "obsoleted by":

The RFC link (first link before ID and TXT) is rather huge for
modem users... 2616 was the number I'm looking for, not 2617,
off by one.

Looking at 2616, it has "Obsoletes: 2068" without link.  Please
add links for "Obsoletes: nnnn".  Okay, now I see why 1945 has
no "obsoleted by", they didn't say "obsoletes 1945" in 2068.

Maybe that convention was invented later, or they forgot it.
Pop 2616 and 303 issue:  "See other" is about POST vs. GET, so
why do you need this ?

| Many pre-HTTP/1.1 user agents do not understand the 303
| status. When interoperability with such clients is a concern,
| the 302 status code may be used instead, since most user
| agents react to a 302 response as described here for 303.

It's not consistent, I get the 303 only for some forms of an
URL,  And only until you have it in your cache apparently (?)

I just tested rfc1036:  two "document contains no data" for the
303, third attempt worked, I got 1036.  Maybe just delay your
answer until you know what it's going to be, 200 or 404.

If I try html/1032 (not html/rfc1032) it works without the 303.
Testing several 103x, always the same behaviour, html/103x gets
it at once, html/rfc103x runs into the 303 trap.  Later both
work.

Oops, the output "updated by" for 1035 is ugly, numerous links
all in one long line.  With "errata" at the far right end.  The
"updated by" meta data before the begin of the original text is
not forced to use monospaced, you can format it in any way you
like.  Or fold it after 78 columns if you want it to be part of
the following monospaced text of the original document (?)

> if the browser has javascript, I run a snippet which adds the
> <h?/> tags, on the theory that most browsers with java-script
> turned on will also have css which preserves the look-and
> feel of the result

Makes sense.  If I turn JavaScript 1.1 on with my old browser
I'm not at all surprised if nothing works, therefore I almost
never try this.

Bye, Frank

---

From: Henrik Levkowetz <[email-masked]>
Date: Fri, 28 Jul 2006 15:08:11 +0200

Hi Frank,

> to create a link to draft-ietf-usefor-useage-01.txt (?)

Good point.  Fixed.

> that.  You caught other [USEFOR] etc. within paragraphs.

Ah, yes.  I currently have no way of distinguishing that use of
[USEFOR] from a definition of [USEFOR]; currently it treats this
as a reference definition, rather than a reference.  This is a
flaw.

> the 303 for its GET HTTP/1.0 with the HTML URL.

Oh, I see.  I'll use 302 here.  It's not really correct in this
case, but not harmful.

> no "obsoleted by", they didn't say "obsoletes 1945" in 2068.

Handling of "Obsoletes: nnnn" added to TODO list.

> URL,  And only until you have it in your cache apparently (?)

The difference is that in this case, I don't have a matching
document at all; I provide a page which lists a number of
alternatives... (but see above).

> it at once, html/rfc103x runs into the 303 trap.  Later both

If the document hasn't been cached yet, you probably get a 302
in one case, which works, and a 303 in the other case, which
doesn't.  Once the document has been cached, you get it directly.
This should clear up with using 302 instead of 303.

> the following monospaced text of the original document (?)

I'll look at folding it.

> I'm not at all surprised if nothing works, therefore I almost

Ok, and thanks for checking this :-)

Regards,

	Henrik