Subject: Re: [xml2rfc] #115: xml2rfc 2.4.1 v2 puts two spaces after "i.e."

From: Nico Williams <[email-masked]>
Date: Mon, 1 Apr 2013 10:31:31 -0500

On Sun, Mar 31, 2013 at 1:55 AM, Denis Ovsienko <[email-masked]> wrote:
> 31.03.2013, 00:50, "Nico Williams" <[email-masked]>:
>> The simpler thing to do would be to require the author to insert the extra space (or other sentence end maker) then have xml2rfc remove it as necessary.
>
> The additional explicit markup, if any, ought to stand for the case that is less common, such as &nbsp; and &wj; stand for supposedly rare exceptions and plain text stands for the implicit processing methods.

I type two spaces after sentence-ending periods as a matter of course.
 Tools like LyX will generally do the right thing (e.g., refuse to put
them in, or add a wider space).  Tools like xml2rfc ought to do the
right things too: map the two space characters to either a single
space (possibly wide) when formatting text in a variable-width font,
of leave them alone when formatting text in a fixed-width font.

I may be the exception in this regard, but two spaces after a period
is a very easy to use bit of markup.

Nico
--

---

From: Joe Abley <[email-masked]>
Date: Mon, 1 Apr 2013 11:45:24 -0400

> is a very easy to use bit of markup.

Don't we expect the RFC editor to clean up this kind of thing to conform to the style guide as part of the copy-editing process? Why is this something that authors need to concern themselves with?

For what it's worth (one more low-value data point), I don't type two spaces after anything, so we have proof positive that your expectations are not universal.

Joe

---

From: Nico Williams <[email-masked]>
Date: Mon, 1 Apr 2013 11:45:50 -0500

> Don't we expect the RFC editor to clean up this kind of thing to conform to the style guide as part of the copy-editing process? Why is this something that authors need to concern themselves with?

No, we don't.  I don't anyways because I want my formatted I-Ds to
read well.  That's also the same reason that one should want to
spellcheck as they edit I-Ds rather than say "well, that's the
RFC-Editor's job".

> For what it's worth (one more low-value data point), I don't type two spaces after anything, so we have proof positive that your expectations are not universal.

The point isn't that you should adopt my habit because I claim it's a
good habit.

The point was that we probably can't make a good enough list of
abbreviations to exempt from the "automatically add a space after
sentence-ending periods in .txt rendering" rule, so this sort of bug
will always be around unless we add some sentence end markup.  I'm
proposing one such markup.

Other simple sentence-ending markup are possible (e.g., alternate
period characters from Unicode).  Emphasis on simple though: hitting
the space bar twice is the simplest possible bit of markup I can think
of for this purpose.  An XML element would not be simple by
comparison, though it'd certainly be easier to not have to examine
text nodes in order to determine sentence boundaries, but since
xml2rfc already does just that...

Nico
--

---

From: Joe Abley <[email-masked]>
Date: Mon, 1 Apr 2013 14:00:22 -0400

On 2013-04-01, at 12:45, Nico Williams <[email-masked]> wrote:

> read well.

Beauty is in the eye of the beholder, I guess.

I think the words are more important than the amount of white space between them :-)

Joe

---

From: Nico Williams <[email-masked]>
Date: Mon, 1 Apr 2013 13:06:20 -0500

> I think the words are more important than the amount of white space between them :-)

Did you stop reading there?  This isn't about aesthetics or spacing
after sentence ending periods.  It's about how xml2rfc detect those.

---

From: Marc Petit-Huguenin <[email-masked]>
Date: Mon, 01 Apr 2013 11:07:41 -0700

On 04/01/2013 11:00 AM, Joe Abley wrote:
>>> Don't we expect the RFC editor to clean up this kind of thing to
>>> conform to the style guide as part of the copy-editing process? Why is
>>> this something that authors need to concern themselves with?
>> 
>> No, we don't.  I don't anyways because I want my formatted I-Ds to read
>> well.
> 

Iamnotsureaboutthat.

- -- 
Marc Petit-Huguenin
Email: [email-masked]
Blog: http://blog.marc.petit-huguenin.org
Profile: http://www.linkedin.com/in/petithug

---

From: Joe Abley <[email-masked]>
Date: Mon, 1 Apr 2013 14:20:37 -0400

On 2013-04-01, at 14:06, Nico Williams <[email-masked]> wrote:

> Did you stop reading there?

Nope.

Joe

---

From: Nico Williams <[email-masked]>
Date: Mon, 1 Apr 2013 16:06:33 -0500

>> Did you stop reading there?

So... you're just against xml2rfc producing two spaces after
sentence-ending periods?

Why not just say so? :)

I might not mind xml2rfc losing that functionality given that I'm
already given to typing two spaces after sentence-ending periods.
Then there'd be no question of detecting sentence-ending periods...
but the RFC-Editor would have to do more manual labor, which surely
they'd mind.

I'd rather minimize the work-load we create for the RFC-Editor
[because I want the RFC-Editor to have fewer mechanical details to
worry about [which distracts from more important editing functions and
slows down the editors unnecessarily]].

Nico
--

---

From: Joe Abley <[email-masked]>
Date: Mon, 1 Apr 2013 17:16:26 -0400

> So... you're just against xml2rfc producing two spaces after

Not really.

I think authors have latitude to present their I-Ds for discussion in whatever style they think is important to their audience (latitude within the realms of what the machinery will allow you to submit, anyway). This seems fine and great, I have no objection to it, I can't imagine what objection anybody would have.

I think the RFC editor should require of their tools whatever functionality best helps them copy edit I-Ds into RFCs. That seems like a matter for them, and it's not clear to me what useful input document authors can have into that process.

The style guide that the RFC editors follow is perhaps something that the community has an interest in. I don't, really, but I think those that do should be able to speak their minds. This seems to have nothing to do with tools.

It seems like what we have in this thread is people asserting correct vs. incorrect behaviour regarding many things that are (in reality) issues for the RFC editor or the style guide. My original point was simply that "correct" vs. "incorrect" for document authors is subjective, and chances are good that no single perspective will make everybody happy.

The only objective perspective here is the RFC Editor's, and that has little or nothing to do with the opinions of authors. Any input authors want to give in that direction should surely be directed at the style guide, not the tools.

Joe

---

From: Nico Williams <[email-masked]>
Date: Mon, 1 Apr 2013 16:56:47 -0500

> I think the RFC editor should require of their tools whatever functionality best helps them copy edit I-Ds into RFCs. That seems like a matter for them, and it's not clear to me what useful input d…

As long as we (authors) share tools with them (RFC-Editor) then we
both have useful input to contribute regarding those shared tools.

The RFC-Editor could easily make life harder for us by saying that
xml2rfc is no good for them and it's up to us to meet their
requirements and/or migrate to some other tool.  Is spacing after
sentence-ending periods the hill we'd want xml2rfc to die on?

> The style guide that the RFC editors follow is perhaps something that the community has an interest in. I don't, really, but I think those that do should be able to speak their minds. This seems to…

It has plenty to do with tools, since we don't format either I-Ds nor
RFCs by hand any more (well, some might, but the overwhelming majority
of us don't).  The more the tools can automate, the better.

> It seems like what we have in this thread is people asserting correct vs. incorrect behaviour regarding many things that are (in reality) issues for the RFC editor or the style guide. My original p…

There's a bug in xml2rfc in that a) it tries to ensure there's two
spaces between sentences (at least in the .txt output format) and b)
fails to correctly detect non-sentence-ending periods as such.  That's
what this thread is about.  Not style.

You might argue that the style that xml2rfc tries to emit is
incorrect.  But that style is already baked in and so far no one has
asked that it be changed.  I'm making no assertions about the value of
"two spaces between sentences" either, nor is anyone else that I can
see.

> The only objective perspective here is the RFC Editor's, and that has little or nothing to do with the opinions of authors. Any input authors want to give in that direction should surely be directe…

Please refer to the *bug* that this thread is about.

Nico
--

---

From: Julian Reschke <[email-masked]>
Date: Sun, 07 Apr 2013 10:46:23 +0200

> is a very easy to use bit of markup.

The issue here is that within <t>, whitespace is not significant. Making 
two SPs somewhat special will likely lead to more problems later on.

Best regards, Julian

---

From: "xml2rfc issue tracker" <[email-masked]>
Date: Mon, 08 Apr 2013 15:18:02 -0000

#115: xml2rfc 2.4.1 v2 puts two spaces after "i.e."

Comment (by [email-masked]):

 Another one is '.' in the middle of a sentence.
 For example, v1 output:
 {{{Attributes MUST NOT contain consecutive or trailing '.' or '/'.}}}
 v2 output has an extra space after '.'

 Another one is "etc.)" in the middle of a sentence.
 For example, v1 output:
 {{{
 to ensure there are no conflicts (processing state,
 fault detection, code path, etc.) when both protocols are run
 }}}
 v2 output has an extra space after "etc.)"

-- 
---------------------------------+-----------------
  Reporter:  [email-masked]  |      Owner:
      Type:  defect              |     Status:  new
  Priority:  medium              |  Milestone:
 Component:  Version 2 cli       |    Version:
Resolution:                      |   Keywords:
---------------------------------+-----------------

Ticket URL: <http://trac.tools.ietf.org/tools/xml2rfc/trac/ticket/115#comment:6>
xml2rfc <http://tools.ietf.org/tools/xml2rfc/>

---

From: Nico Williams <[email-masked]>
Date: Mon, 8 Apr 2013 12:15:35 -0500

On Sun, Apr 7, 2013 at 3:46 AM, Julian Reschke <[email-masked]> wrote:
> The issue here is that within <t>, whitespace is not significant. Making two
> SPs somewhat special will likely lead to more problems later on.

Not so.  xml2rfc uses whitespace in <t>s to decide where to break a
line with flowing text.  And when rendering in a format that has
spaces of various widths then xml2rfc clearly wants make use of them,
as evidenced by this bug report.  This bug report is all about how
xml2rfc treats ". " (period followed by one space) specially -- in <t>
content.

I'm more convinced now than ever that xml2rfc should treat ".  "
(period followed by two spaces) specially, rather than ". " (period
followed by one space).

Either than or we need to add a <sentence> (or <s>) element.  But
users would hate us if we did that.  Really, please, let's do as I
propose.

Nico
--

---

From: Julian Reschke <[email-masked]>
Date: Mon, 08 Apr 2013 19:35:45 +0200

> xml2rfc treats ". " (period followed by one space) specially -- in <t>

Hmm, no. Really.

People use indentation all over the place.

Consider

<t>
    This is a sentence. This is a sentence. This is a sentence.
    This is a sentence. This is a sentence. This is a sentence.
    This is a sentence. This is a sentence. This is a sentence.
</t>

How do you know *reliably* which SPs are significant?

> users would hate us if we did that.  Really, please, let's do as I

The proper fix is to simply drop the two-spaces-after-sentence-end rule. 
It's a relic from the times of typewriters. It's not needed with proper 
proportional fonts.

Best regards, Julian

---

From: Brian Rosen <[email-masked]>
Date: Mon, 8 Apr 2013 13:40:24 -0400

> The proper fix is to simply drop the two-spaces-after-sentence-end rule. It's a relic from the times of typewriters. It's not needed with proper proportional fonts.
> 
Proportional fonts?  We don't do no proportional fonts.  We like ASCII.

---

From: Nico Williams <[email-masked]>
Date: Mon, 8 Apr 2013 13:21:38 -0500

> Proportional fonts?  We don't do no proportional fonts.  We like ASCII.

That's a debate that's off-topic here.  See the RFC-Editor list archives.

---

From: Nico Williams <[email-masked]>
Date: Mon, 8 Apr 2013 13:20:54 -0500

> How do you know *reliably* which SPs are significant?

Any ".  " sequence -> end of sentence.

> It's a relic from the times of typewriters. It's not needed with proper

*Something* still needs to determine the end of a sentence in order to
use a wider space instead of two spaces.

xml2rfc already wants to be that something (witness this bloody bug
report).  If you (we) don't like this, then we should remove the
feature instead of keep adding to the list of cases where xml2rfc does
not add a space after ". ", otherwise we should do something else to
greatly improve the end of sentence detection.

Nico
--

---

From: Tony Hansen <[email-masked]>
Date: Mon, 08 Apr 2013 14:39:59 -0400

> greatly improve the end of sentence detection.

The first step is parity: the new xml2rfc should treat periods the same
way that the previous xml2rfc did.

We can then discuss improvements from there, if any more are needed.

    Tony

---

From: Julian Reschke <[email-masked]>
Date: Mon, 08 Apr 2013 20:59:28 +0200

> Any ".  " sequence -> end of sentence.

We know that there are dots-SP sequences that do not indicate a sentence 
ending, and people might have a CR LF SP SP following due to indentation.

Let's please not break a fundamental aspect of the vocabulary without a 
good reason.

> use a wider space instead of two spaces.

No.

> greatly improve the end of sentence detection.

We can just drop the feature, and use single spaces as everybody else in 
the civilized world does :-)

Best regards, Julian

---

From: Tom Taylor <[email-masked]>
Date: Mon, 08 Apr 2013 15:04:59 -0400

Agreed. I learned the two spaces rule from my mother 60 years ago. I 
unlearned it when I was editing a Cablelabs document and was told double 
spaces would mess up their automatic requirement-finding software.

>>> On 2013-04-01 17:31, Nico Williams wrote:
...
> _______________________________________________
> xml2rfc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/xml2rfc
>

---

From: Nico Williams <[email-masked]>
Date: Mon, 8 Apr 2013 14:46:47 -0500

> ending, and people might have a CR LF SP SP following due to indentation.

So don't do that after an abbreviation.

---

From: Nico Williams <[email-masked]>
Date: Mon, 8 Apr 2013 14:48:49 -0500

>> use a wider space instead of two spaces.

Yes.  Lots of renderers of various formats will use a wide space after
a sentence-ending period.  xml2rfc need not be it when it renders
into, say, HTML.  xml2rfc also need not be it when it renders into
text: I can add the spaces on my own (with the gotcha that I have to
be careful re: indentation).

> We can just drop the feature, and use single spaces as everybody else in the
> civilized world does :-)

I agree up to the comma.

Nico
--

---

From: "xml2rfc issue tracker" <[email-masked]>
Date: Wed, 10 Apr 2013 10:03:42 -0000

Comment (by [email-masked]):

 Let me note an idea that didn't make through the respective thread on the
 mailing list.

 The problem is, the same full stop (".") symbol is used for both terminal
 punctuation (end of sentence) and non-terminal one (end of abbreviation),
 and a processing algorithm cannot reliably tell the context to produce
 appropriate fixed-width font text with two spaces after each sentence. One
 of the solutions is maintaining a list of known exceptions (see above).
 Another solution would be some explicit markup.

 In the latter case such markup should require least additional effort from
 the author, that is, only the abbreviational use should have the markup.
 This markup should have semantics of existing {{{&nbsp;}}} and {{{&wj;}}}
 output modifiers, that is, it may be an XML entity going instead of the
 full stop symbol or before it respectively. For example, if the entity was
 {{{&nse;}}} (Not a Sentence End), the input text would look like:
 {{{
 to ensure there are no conflicts (processing state,
 fault detection, code path, etc&nse;.) when both protocols are run
 }}}

 These solutions can be combined, that is, when the built-in hints miss the
 case and produce a "sentence end", an explicit entity can be used to fix
 it back to "abbreviation". In this case the built-in hints must never
 match at sentence end (the entity fixes only a false negative, but not the
 false positive).

Ticket URL: <http://trac.tools.ietf.org/tools/xml2rfc/trac/ticket/115#comment:7>
xml2rfc <http://tools.ietf.org/tools/xml2rfc/>

---

From: Tom Taylor <[email-masked]>
Date: Thu, 11 Apr 2013 10:23:41 -0400

I was editing a draft yesterday and suddenly realized that this whole 
issue is moot from my point of view. I always put a comma after e.g and 
i.e., following on Henning Schulzrinne's direction when I was working on 
a draft jointly with him.

---

From: Nico Williams <[email-masked]>
Date: Thu, 11 Apr 2013 14:45:57 -0500

On Thu, Apr 11, 2013 at 9:23 AM, Tom Taylor <[email-masked]> wrote:
> I was editing a draft yesterday and suddenly realized that this whole issue
> is moot from my point of view. I always put a comma after e.g and i.e.,
> following on Henning Schulzrinne's direction when I was working on a draft
> jointly with him.

I also do this, but it does not render the issue moot because there
are many abbreviations (really, one can invent new ones at any time)
that aren't naturally followed by a comma or other disambiguator.

There's only three solutions: more sentence end detection heuristics,
special markup (say, two spaces, or a non-ASCII Unicode horizontal
space, or an XML element) to mark ends of sentences, or no special
handling of end of sentence spacing.

In practice even the proponents of variable-width fonts want a wider
space (but not as wide as two spaces) to follow sentence-ending
periods, and if xml2rfc is to produce this at all (in any output
format) then it kinda has to know what sentence endings are.

For me it's trivial and even reflexive, as an author, to add two
spaces in the source after each sentence-ending period.  (Some tools,
like LyX, don't allow this or otherwise auto-correct this to one
space, and that's OK because they have sufficiently good heuristics or
whatever).

Nico
--

---

From: Julian Reschke <[email-masked]>
Date: Fri, 12 Apr 2013 09:28:15 +0200

> periods, and if xml2rfc is to produce this at all (in any output

Not me.

Again; we're wasting a lot of time to solve something that is not a problem.

Best regards, Julian

---

From: Nico Williams <[email-masked]>
Date: Fri, 12 Apr 2013 02:55:02 -0500

> Again; we're wasting a lot of time to solve something that is not a problem.

Actually, that reminds me: this is really an issue for the RFC-Editor.
 draft-iab-rfcformatreq-03 says nothing about this.

---

From: "xml2rfc issue tracker" <[email-masked]>
Date: Wed, 17 Apr 2013 20:44:38 -0000

Changes (by [email-masked]):

 * priority:  medium => major

  Priority:  major               |  Milestone:

Ticket URL: <http://trac.tools.ietf.org/tools/xml2rfc/trac/ticket/115#comment:8>
xml2rfc <http://tools.ietf.org/tools/xml2rfc/>