Subject: Re: [dispatch] Internet Draft for NNTP additions submitted

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 2 Apr 2020 12:25:14 -0700

Do you have a link to this draft?  I find no match in the datatracker.

-MSK

On Sat, Mar 21, 2020 at 7:13 AM E Sam <[email-masked]> wrote:

> For some strange reason I dont think my email went through.
> So here it is again.
>
> I know we’re all deep into IETF107 talk but when you have time check it
> out.
>
> Hello all,
>
> I have made the first I-D for the NNTP additions. It has been submitted to
> the IETF datatracker. The file name is draft-sam-nntpupdates-00.txt
>
>
> Some notes:
> Since we have the errata to change, this draft is by no means represent
> all the changes we need to make.
>
> I reintregrated the dynamic feed proposed in 1999 because I still think
> it is a good idea for tempory adjustments to newsfeeds (for example: when a
> certain newsgroups is getting a spam wave etc.) Peers communicating feed
> info out of band (e.g: emails) is still recommended due to the variety of
> information needed and to avoid “command creep”
>
> Maybe we can also work on USEFOR/USAGE in some way if it interestes  the
> participants - however we want to be NNTP focused since it is a NNTP
> working group after all.
>
> I found some errors in some of the RFCs I referenced - I will be
> submitting an errata soon.
>
> Thanks
> _______________________________________________
> dispatch mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dispatch
>

---

From: Julien ÉLIE <[email-masked]>
Date: Sun, 5 Apr 2020 13:38:04 +0200

Hi all,

> I have made the first I-D for the NNTP additions. It has been submitted 
> to the IETF datatracker. The file name is draft-sam-nntpupdates-00.txt

https://www.ietf.org/id/draft-sam-nntpupdates-00.txt

Here are a few comments:

>    Article numbers MUST lie between 1 and 9,223,372,036,854,775,807, 
>    inclusive.  The client and server MAY use leading zeroes in 
>    specifying article numbers but MUST NOT use more than 19 digits.
We'll need to define a capability to advertise that new feature.

Note that Ken Murchison once suggested (in 2005) a LARGEGROUP extension, 
which seems to be a reasonable implementation according to a few 
discussions since that time:
   https://lists.eyrie.org/pipermail/ietf-nntp/2005-July/005720.html

*************************************************
- Servers which support 64-bit (large) article counters advertise a
capability such as 'LARGEGROUP' (or some better named capability).

- Servers always hide (411 response for [LIST]GROUP, ommission from LIST
ACTIVE response) groups which have exceeded the 32-bit article count
from clients until the client tells the server that it can support such
groups.

- A client tells the server that it supports large groups by using the
new keyword 'LARGEGROUP' as an argument to the CAPABILITY command.

- The server can then use the existing response codes for the
[LIST]GROUP and LIST ACTIVE commands.
*************************************************

Also note Clive Feather's BIGNUM proposal:
   https://lists.eyrie.org/pipermail/ietf-nntp/2005-July/005802.html

>    4.1.  XPAT
As for improvements in extended wildmat searches, we'll need 
standardizing how uwildmat works.  Hopefully this is quite well 
explained in INN's documentation:
   https://www.eyrie.org/~eagle/software/inn/docs/uwildmat.html

The major challenge though PAT is how to deal with whitespace and 
MIME-encoded strings in header fields, and of course also 
internationalized header fields.
Examples:

   Subject: 
=?UTF-8?Q?CNFS_sur_INN2_was:_Et_l=c3=a0=2c_=c3=a7a_refonctionne_=3f?=
      =?UTF-8?Q?_-_no_reply_-_ignore?=

   Newsgroups: trigofacile.test.ᾅ

>    4.2.  WHOAMI
A IANA registry for returned information (and future new information) 
will be useful to add in our current NNTP IANA registry.

Note the behaviour of (unstandardized) XUID command in news.individual.net:

XUID
280 User information
     User-ID: 90977
     EID: none
     Access: [90977]
     Client: denver.dinauz.org (37.59.56.11)
     Reading: allowed
     Posting: allowed
     Newnews: allowed
     Posting into readonly groups: not allowed
     Setting Approved header: not allowed
     Cancel-Lock addition: enabled
     Setting individual path header: not allowed
     Sending cancel control messages and supersedes: allowed
     Sending newgroup/rmgroup control messages: not allowed
     LIST ACTIVE does not include control.cancel
     Organization: extern
     Subscription: expires in 361 days (2021-04-01)
     UID check for cancel messages: yes
     UID check for supersedes messages: yes

     Amount of articles posted today: 0 (Maximum: 100)
.

And similar useful information could be given to peers, giving them 
known configuration (max article size, fed newsgroups, path, spam 
policy...).

>    5.  Dynamic Feeds
I see that you re-integrated ideas from old draft for Dynamic Feed 
Adjustment.  That's great!
     http://www.eyrie.org/~eagle/nntp/drafts/draft-court-dynfeed-01.txt

>    6.  Header Related Commands
Besides IHAVEHDR, we should have a look at how Diablo implements its 
"MODE HEADFEED" to correctly standardize header-only feed.
Maybe CHECKHDR and TAKEHDR will also be useful.

> participants - however we want to be NNTP focused since it is a NNTP 

We'll have to prioritize efforts between needed NNTP extensions and 
improvements in article format.

FWIW, other possible work I had in mind:
- DANE/DNSSEC for NNTP (to advertise the use of NNTP/TLS on port 563)
- BATCH command, with rnews format
- update to RFC 3977 (either NNTP Version 3 or integrate all errata somehow)
- full-text search extension (in bodies)
- internationalized Netnews headers (newsgroups, distributions, mails...)
- USEAGE (Usenet Best Practice)
- format of NoCeM articles (for cancels)
- format and use of PGPMoose (signing articles)
- format of signed control articles / pgpverify stuff

Well, that's a great amount of job!

-- 
Julien ÉLIE

« Petite annonce : Sourd rencontrerait sourde pour terrain
   d'entente. »

---

From: Ken Murchison <[email-masked]>
Date: Sat, 18 Apr 2020 08:20:03 -0400

On 4/5/20 7:38 AM, Julien ÉLIE wrote:
> Hi all,
>
>> I have made the first I-D for the NNTP additions. It has been 
>> submitted to the IETF datatracker. The file name is 
>> draft-sam-nntpupdates-00.txt
>
> Note that Ken Murchison once suggested (in 2005) a LARGEGROUP 
> extension, which seems to be a reasonable implementation according to 
> a few discussions since that time:

At this point, why not just define a NNTP VERSION 3 capability which 
indicates that all commands that accept/return article numbers can be 63 
bit?  This requires no changes to existing commands, and clients will 
know right away from the capabilities whether they should proceed or not.

-- 
Kenenth Murchison
Senior Software Developer
Fastmail US LLC

---

From: Julien ÉLIE <[email-masked]>
Date: Sun, 19 Apr 2020 08:55:36 +0200

Hi Ken,
> know right away from the capabilities whether they should proceed or not.

This would mean that the server will send 63-bit article numbers to 
clients that may not be prepared to deal with them.  In answers to LIST 
ACTIVE and GROUP for instance.  It may make them crash (segfault) and 
break inter-operability...

-- 
Julien ÉLIE

« En fait, je n'ai qu'une prétention, c'est de ne pas plaire à tout
   le monde. Plaire à tout le monde, c'est plaire à n'importe qui. »
   (Sacha Guitry)

---

From: Julien ÉLIE <[email-masked]>
Date: Sun, 19 Apr 2020 14:41:19 +0200

Hi,

> Lets be honest, every newsreader that is going to support 64 bit article 
> numbers already has its been more than 10 years. We should do it in a 
> way that allows for new conforming newsreaders while not breaking old ones.

Pretty easy to test.
Just done with a dummy news server, and Thunderbird Version 68.7.0 (64 
bit) released on April 8th, 2020.

By hand:

% telnet localhost 50300
200 Testing server
CAPABILITIES
101 Capability list:
VERSION 2 3
LIST ACTIVE
READER
.
LIST
215 Newsgroups in form "group high low status"
trigofacile.test 9000000000000000000 9000000000000000000 y
.
QUIT

I configure Thunderbird to access this dummy news server.

200 Testing server
MODE READER
200 Reader mode
LIST
215 Newsgroups in form "group high low status"
trigofacile.test 9000000000000000000 9000000000000000000 y
.
GROUP trigofacile.test
211 1 9000000000000000000 9000000000000000000 trigofacile.test

[Thunderbird then just hangs.]

I change article numbers, and retest:

200 Testing server
MODE READER
200 Reader mode
GROUP trigofacile.test
211 1 900 900 trigofacile.test
XOVER 900-900
500 What?
HEAD 900
500 What?

(I have not implemented responses to XOVER and HEAD.  Thunderbird looks 
fine, though, with article number 900.)

Another test:

200 Testing server
MODE READER
200 Reader mode
GROUP trigofacile.test
211 1 2147483648 2147483648 trigofacile.test

[Thunderbird then just hangs.]

And last test:

200 Testing server
MODE READER
200 Reader mode
GROUP trigofacile.test
211 1 2147483647 2147483647 trigofacile.test
XOVER 2147483647-2147483647
500 What?
HEAD 2147483647
500 What?

So, 2^31-1 works fine, but above not.

There is really interoperability problem with current news readers (in 
2020), so defining VERSION 3 of NNTP like this is very dangerous.

-- 
Julien ÉLIE

« C'est la goutte qui fait déborder l'amphore ! »
   (Assurancetourix)

---

From: E Sam <[email-masked]>
Date: Sun, 19 Apr 2020 08:55:59 -0400

Oh so thats why thunderbird broke under my highwinds-omicron-Greg Lyda Inc.
account

Thats true, so yes defining it like this could be very dangerous.

All the big tools to download binaries are open source can easily support
asking for 64 bit article numbers (they download by MID anyway, so woudlnt
be an issue).

I agree with what Julien or someone said about adding a LARGEGROUP
capability instead of assuming it under Version 3.