Subject: [Tools-discuss] Looking for suggestions on a Grammar Checker

From: Fred Baker <[email-masked]>
Date: Sun, 9 Jul 2006 11:41:12 -0400

I sat down and reviewed a draft yesterday and found it particularly  
difficult to read - and I am a native speaker of English. I pity  
those of us who aren't.

Now, some history. Some will recall that in late 1994 I was asked to  
take the text of the document that is now RFC 1716, change it to  
reflect CIDR and the decision to make RIP historical, generally edit  
it for readability, and produce what is now RFC 1812. I got a lot of  
very nice comments about the "edit for readability" part. The way I  
went about that was to get a new laptop (I was changing companies, so  
that went with it), install the latest version of Microsoft Word  
(Cisco uses Microsoft Office internally), make some conversions to  
the nroff source to put it in newline-ends-a-paragraph format, and  
use Microsoft's then-new grammar checker. The program complained  
bitterly about "which" vs "that", an argument trained grammarians  
aren't altogether sure is valid. It also complained bitterly about  
passive voice; changing passive voice to active voice turned out to  
be a very good thing to do, and is something I don't do religiously  
but seriously consider doing when it is pointed out. It made a few  
very specific suggestions on how to reword things and found a long  
list of spelling errors. But the very best thing it did was give up.  
It said that it couldn't parse a number of sentences that I had read  
and thought I understood on first reading. When I read them again, I  
couldn't either, and in each case turned the sentence into two or  
three that were more understandable.

I used that trick on the document I read yesterday, and came up with  
a number of specific changes that I think actually improved the text.  
That said, when I recommended the procedure back to the editors of  
the document, they said (surprise!) that they didn't use Microsoft  
Office, partially because their OS's didn't support it.

Which makes me think - well, actually, I'm on MacOSX, and putting XML  
documents or posted I-Ds into Word for a grammar check is a  
particularly painful and somewhat bizarre procedure. There must be  
something better.

I have downloaded trial versions of Grammatica and Grammarian Pro.  
Grammatica has the nice property that it operates within XMLMind, at  
the touch of a hot key. It isn't very helpful with posted internet  
drafts, but I have a hunch that I can find a way around that, using a  
combination of a script that will remove page breaks and convert the  
text to newline-ends-a-paragraph format and then doing the comparison  
in TextWrangler (my GUI text editor - YMMV). A quick check with an  
xml2rfc XML I-D indicates that it will complain about the XML itself  
but seems to mostly work on the text I write, which is a good start.  
I haven't tried Grammarian Pro yet.

Neither one works well with newline-ends-a-line format, which means  
that checking a posted ID requires me to do some unnatural acts with  
the file. I would that this were different.

It would also be nice to have a tool that would additionally run on  
FreeBSD or on Linux in addition to MacOSX (and hey, running on  
Windows wouldn't be a bad idea), and could be recommended to the IETF  
at large. The Open Office web page notes that it doesn't have a  
grammar checking tool and expects that someone will someday add one.  
It also notes an argument in grammatical circles, that there is  
little hope of anyone ever writing a perfect tool. Some in those  
circles say that absent a perfect tool, a partial tool is a bad idea.  
I agree with that argument, but note that I use a spell checker on  
documents - including on this email. I am not asking for a perfect  
tool, just one that will help. I'm not thinking of this as a "final  
check" tool - the purpose here is not "you can't submit this to the  
IESG if you haven't run this tool on it and had it come out clean".  
I'm thinking of this as a tool one runs on a -00 draft, and on any  
version of it if one is the author, and it helps one write a clearer  
document.

Any suggestions/recommendations?

_______________________________________________
Tools-discuss mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tools-discuss

---

From: Lars Eggert <[email-masked]>
Date: Sun, 9 Jul 2006 11:55:21 -0400

On Jul 9, 2006, at 11:41, Fred Baker wrote:
> Which makes me think - well, actually, I'm on MacOSX, and putting  
> XML documents or posted I-Ds into Word for a grammar check is a  

I don't think there is, unfortunately. If you convert the XML to HTML  
instead of text, importing into Word is a bit less painful.

> Windows wouldn't be a bad idea), and could be recommended to the  
> IETF at large.

http://www.danielnaber.de/languagetool/ was recommended when I asked  
about grammar checkers on this list a while ago. It's multiplatform  
(Java), but its checks are rudimentary at best.

Lars
-- 
Lars Eggert                                     NEC Network Laboratories

---

From: Joe Touch <[email-masked]>
Date: Sun, 09 Jul 2006 09:44:30 -0700

Fred Baker wrote:
> difficult to read - and I am a native speaker of English. I pity those
> of us who aren't.
> 
> take the text of the document that is now RFC 1716, change it to reflect
> CIDR and the decision to make RIP historical, generally edit it for
> readability, and produce what is now RFC 1812. I got a lot of very nice
> comments about the "edit for readability" part. The way I went about
> that was to get a new laptop (I was changing companies, so that went
> with it), install the latest version of Microsoft Word (Cisco uses
> Microsoft Office internally), make some conversions to the nroff source
> to put it in newline-ends-a-paragraph format, and use Microsoft's
> then-new grammar checker. The program complained bitterly about "which"
> vs "that", an argument trained grammarians aren't altogether sure is
> valid. 

That's news to me, and to:
http://www.grammarbook.com/grammar/whoVwhVt.asp
http://www.press.uchicago.edu/Misc/Chicago/cmosfaq/cmosfaq.WhichvsThat.html

which introduces nonrestrictive clauses

that introduces restrictive clauses

> It also complained bitterly about passive voice; changing passive
> voice to active voice turned out to be a very good thing to do, and is
> something I don't do religiously but seriously consider doing when it is
> pointed out. It made a few very specific suggestions on how to reword
> things and found a long list of spelling errors. But the very best thing
> it did was give up. It said that it couldn't parse a number of sentences
> that I had read and thought I understood on first reading. When I read
> them again, I couldn't either, and in each case turned the sentence into
> two or three that were more understandable.
>
>I used that trick on the document I read yesterday, and came up with a
> number of specific changes that I think actually improved the text. That
> said, when I recommended the procedure back to the editors of the
> document, they said (surprise!) that they didn't use Microsoft Office,
> partially because their OS's didn't support it.
> 
> documents or posted I-Ds into Word for a grammar check is a particularly
> painful and somewhat bizarre procedure. There must be something better.

You might consider using the word template instead ;-)

> Grammatica has the nice property that it operates within XMLMind, at the
> touch of a hot key. It isn't very helpful with posted internet drafts,
> but I have a hunch that I can find a way around that, using a
> text to newline-ends-a-paragraph format and then doing the comparison in
> TextWrangler (my GUI text editor - YMMV). A quick check with an xml2rfc
> XML I-D indicates that it will complain about the XML itself but seems
> to mostly work on the text I write, which is a good start. I haven't
> tried Grammarian Pro yet.
> 
> Neither one works well with newline-ends-a-line format, which means that
> checking a posted ID requires me to do some unnatural acts with the
> file. I would that this were different.

I thought the whole point of XML was "markup once, render multiple
times". Wouldn't a different 'output spec' (pardon, I don't know the
name of that for XML) achieve this?

(although you're introducing a separate render-process-review-edit loop,
which makes finding the text you want to correct a bit convoluted. but
that's the price of not using a wysiwyg system ;-)

> FreeBSD or on Linux in addition to MacOSX (and hey, running on Windows
> wouldn't be a bad idea), and could be recommended to the IETF at large.
> The Open Office web page notes that it doesn't have a grammar checking
> tool and expects that someone will someday add one. It also notes an
> argument in grammatical circles, that there is little hope of anyone
> ever writing a perfect tool. Some in those circles say that absent a
> perfect tool, a partial tool is a bad idea. I agree with that argument,
> but note that I use a spell checker on documents - including on this
> email. I am not asking for a perfect tool, just one that will help. I'm
> not thinking of this as a "final check" tool - the purpose here is not
> "you can't submit this to the IESG if you haven't run this tool on it
> and had it come out clean". I'm thinking of this as a tool one runs on a
> -00 draft, and on any version of it if one is the author, and it helps
> one write a clearer document.
>

---

From: Fred Baker <[email-masked]>
Date: Sun, 9 Jul 2006 21:58:05 -0400

> name of that for XML) achieve this?

doesn't that kind of imply that posted drafts are in XML format?

In this comment, I'm talking about reviewing posted .txt drafts.

---

From: Joe Touch <[email-masked]>
Date: Sun, 09 Jul 2006 19:08:07 -0700

Fred Baker wrote:
> In this comment, I'm talking about reviewing posted .txt drafts

Converting .txt to reviewable text (e.g., that can be analyzed by word
at least) isn't that complex (I teach emacs to do it when I need to by
example, but a Perl script isn't that hard):

fold the paragraphs down:
replace \n\n with MAGICSTRING
replace \n with ' '
replace MAGICSTRING with \n

use a somewhat more complex pattern to remove trailer/header pairs

It's fairly easy to automate that; you'll still get aliases from the use
of 'o' for bullets and from figures, but you just ignore those.

Joe

---

From: Fred Baker <[email-masked]>
Date: Mon, 10 Jul 2006 06:47:58 -0400

Never mind. I was asking a question about software packages, but I'm  
getting a tutorial on programming. There are, in fact, a few things I  
know how to do.

---

From: Joe Touch <[email-masked]>
Date: Mon, 10 Jul 2006 10:56:44 -0700

> painful and somewhat bizarre procedure. There must be something better.

The pain/bizarrity seems threefold:

1) XML or TXT needs to be converted to a useful form for analysis

2) it's a 2-3 phase process

3) packages are required to do the analysis

(3) was already addressed; many packages are commercial, unfortunately.

(2) is required if a separate conversion step is required (e.g., for TXT
or XML); if an integrated package is used (e.g., Word), the result is
one-phase.

I was addressing (1).

Was there a different issue that made this painful or bizarre?

Joe

---

From: Fred Baker <[email-masked]>
Date: Mon, 10 Jul 2006 14:10:14 -0400

It would be really nice if there was a package (which is what I was  
asking about) that could accept *lines* that ended in a \n and  
*paragraphs* that ended in \n\n and was able to do a grammar check.  
What I do right now is something not unlike what you propose and put  
it into a package that runs on a subset of laptops and has a grammar  
checker.

As I said, I am playing with a couple of grammar checkers, at least  
one of which will run in XMLMind, but that doesn't help much with a  
document I don't have the xml2rfc file, only the text output of nroff- 
or-whatever, for.

And the QUESTION had to do with recommendations of SOFTWARE PACKAGES,  
not programming tips. Read the subject line.

On Jul 10, 2006, at 1:56 PM, Joe Touch wrote:

>> particularly

---

From: Joe Touch <[email-masked]>
Date: Mon, 10 Jul 2006 11:41:27 -0700

> *paragraphs* that ended in \n\n and was able to do a grammar check.

That wouldn't be sufficient - there are breaks across pages that need to
be addressed.

Joe

---

From: Randall Stewart <[email-masked]>
Date: Mon, 10 Jul 2006 10:32:29 -0400

Fred Baker wrote:

> and had it come out clean".  I'm thinking of this as a tool one runs on 
> a -00 draft, and on any  version of it if one is the author, and it 
> helps one write a clearer  document.
> 

Ok, in digging around I find abiword has some plugins...

One of which is a grammer checker... it appears from the web
page:
http://www.abisource.com/twiki/bin/view/Abiword/PluginMatrix

it has:

-   and ++

the - under stable, means there is NO stable one.. its not
being worked on..

But the ++ under devel.. means  The plugin generally works although it
  is not finished, and it may suffer from some instability.

So I guess this means it is being developed.. but no stable
code yet exists ... :-)

R

-- 
Randall Stewart
NSSTG - Cisco Systems Inc.
803-345-0369 <or> 815-342-5222 (cell)