Subject: [apps-discuss] 2006 proposal to add spam score and spam notification to email

From: Randall Gellens <[email-masked]>
Date: Fri, 3 Aug 2012 16:38:46 -0700

Hi all,

During the apparea/appwg discussion of 
draft-ordogh-spam-reporting-using-imap, I was asked to send 
information regarding a proposal I'd made a number of years back to 
do this.

Attached are slides outlining the proposal from July 27, 2006 (so 
pretty much exactly six years ago).  Page 20, for example, shows spam 
score being included in the message notification to the client, and 
the client sending an indication if a message is or is not spam back 
to the server.

-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly selected tag: ---------------
I think we'd get fewer weird bug reports if we stopped selling our
software off planet.

---

From: Ned Freed <[email-masked]>
Date: Sat, 04 Aug 2012 19:08:54 -0700 (PDT)

Apologies in advance if this ground has already been covered and decided
on at the F2F meeting....

> the client sending an indication if a message is or is not spam back

Yes, this seems like the right overall model to me. Of course there needs
to be consensus on that, and then the devil will be in the details.

I guess my question is how to proceed. I'm not wild about the current draft for
various reasons, but most of all because it already nails down too many of the
specifics that I think need to be open for discussion and consensus. And it
does it without really specifying the overall model, which I think needs to be
clearly laid out.

My idea of an approach is to:

(1) Get agreement on the model (or if you prefer, information flow)
(2) Decide on what sort of information is carried, e.g.,
    spam/notspam, spam/notspam/undetermined, spam score, spam/virus scores,
    something even more general
(3) Decide on mechanism, e.g., headers, flags, annotations, commands

Does this sort of approach make sense? Can it be done in appsawg or do we need
a WG?

				Ned

---

From: Randall Gellens <[email-masked]>
Date: Sat, 4 Aug 2012 21:23:35 -0700

At 7:08 PM -0700 8/4/12, Ned Freed wrote:

>  Does this sort of approach make sense?

I think it makes sense.  I agree that text laying out the model and 
information flow would be helpful, and then discussion on the 
specifics of the information carried.

My (now ancient) proposal was that the server include a spam score 
(if available) in the message metadata, and the client can confirm a 
message as spam or as not spam.  Including the spam score allows the 
client to take this into account when deciding which messages it 
downloads when.  The confirmation as spam or not spam allows the 
server to refine the scoring algorithm, as well as take other action.

>   Can it be done in appsawg or do we need a WG?

I suggest this may depend on how much interest there is in actively 
working on it.  If there are a lot of people who want to be active, 
then we may need a WG.  If there will be a few people who will 
comment and review, then appsawg is fine.

-------------- Randomly selected tag: ---------------
Thoughts, like fleas, jump from man to man.  But they don't bite everybody.
    --Stanislaw Lec

---

From: "John Levine" <[email-masked]>
Date: 5 Aug 2012 04:49:48 -0000

>Does this sort of approach make sense? Can it be done in appsawg or do we need

I doubt that any large mail provider would implement anything like
this.  They definitely want ways for users to report back when they
think the spam filters got it wrong, but not per-user rules and
certainly not reporting spam scores.

Per user rules in a large system are bad news because most users are
remarkably bad at saying what it is they want the rules to do.  I see
this all the time in which users report ordinary mail from lists
they've subscribed to as spam.  Maybe they don't want the list any
more, maybe they just decided to report everything in the inbox as
spam.  There's no way to tell. 

I have also heard from people who should be in a position to know that
in large mail systems, overall systems that can see the whole mail
stream manage filters better than individuals do, and if you're
getting a lot of requests for per-user tuning, that tells you that
your spam filters don't work very well.  Your time is better spent
fixing the overall filters rather than providing knobs.  One cable ISP
told me that they were getting tons of requests for per-user rules,
then they revamped their filters to work better, and the requests just
stopped.

Reporting spam scores is mostly an invitation to spammers to use them
to game the filters.  I know that Spamassassin does it, but large mail
systems don't use Spamassassin.

R's,
John

---

From: Ned Freed <[email-masked]>
Date: Sun, 05 Aug 2012 07:53:10 -0700 (PDT)

> certainly not reporting spam scores.

Actually, I know of several very large providers that have implemented per-user
rules of various sorts, including but not limited to blacklisting. (The
storage requirements for this can be truly impressive, BTW.)

But even if that weren't the case, IETF standards have a broaders audience
than large service providers.

> spam.  There's no way to tell.

OK... but now you seem to be arguing against your own brief. If a user
adds a private white or black list rule, that's, well, private and only
affects them when they get the rules wrong. At most it's a support call
generator, and yes, this has been an issue for large providers that
support per-user rules.

OTOH, you say they want feedback when users say the filters got it wrong, and
if the user is incorrect on that... now you have incorrect feedback informaion.

> then they revamped their filters to work better, and the requests just

The counterargument is that users like to feel that they are in control.

> systems don't use Spamassassin.

As it happens that's also incorrect: Some large providers do in fact use
SpamAssassin variants. I personally think it's a poor choice for large scale
setups, but it's hardly the only or even close to the worst decision decision
I've seen a provider make.

As for reporting scores, any system that's capable of reporting a score
can be used to report a simple binary outcome. But the opposite is
not true. And again, the target audience here needs to cover a lot more
than large ISPs.

At most the takeaway I see here is that we need to document how to use
specific subsets of the feature set that's provided.

				Ned

---

From: "John Levine" <[email-masked]>
Date: 5 Aug 2012 15:29:49 -0000

> OTOH, you say they want feedback when users say the filters got it 
> wrong, and if the user is incorrect on that... now you have incorrect 
> feedback informaion.

The feedback doesn't directly turn into a rule.  Providers treat them with 
considerable scepticsm unless there are either a lot of similar reports, 
or the stuff that's reported has other characteristics of spam.

> The counterargument is that users like to feel that they are in control.

That's why you give them a spam button, but you're vague about what it 
does.

> SpamAssassin variants.

Maybe they do, but I don't recally seeing any handy scores in the
headers you can use to tune your spam.  Yahoo puts a lot of opaque
stuff in the headers which I presume includes filtering results, but I
never heard of anyone reverse engineering it.

> specific subsets of the feature set that's provided.

And make it clear that it's all 100% optional.

Plan B: talk to some large providers and see what they might be interested 
in implementing across the board.  I know there's a lot of interest in the 
IMAP spam button if we can sort out the patent issues.

R's,
John

---

From: Barry Leiba <[email-masked]>
Date: Mon, 6 Aug 2012 11:29:28 -0400

My sense is that we should work closely with MAAWG in developing the model,
and then work on the protocol.  We have a good connection to MAAWG.

Barry

On Sunday, August 5, 2012, John Levine wrote:

> _______________________________________________
> apps-discuss mailing list
> [email-masked] <javascript:;>
> https://www.ietf.org/mailman/listinfo/apps-discuss
>

---

From: "John R Levine" <[email-masked]>
Date: 6 Aug 2012 12:00:05 -0400

> and then work on the protocol.  We have a good connection to MAAWG.

Yup, that was plan B.

>> IMAP spam button if we can sort out the patent issues.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
"I dropped the toothpaste", said Tom, crestfallenly.

---

From: Ned Freed <[email-masked]>
Date: Mon, 06 Aug 2012 11:10:43 -0700 (PDT)

> or the stuff that's reported has other characteristics of spam.

Which is precisely my point. This entire mechanism is about standardizing a
means of passing information around. It is not about what is done with that
information by any party. It provides a way for a provider to pass an
assessment of the message to a user and a way for a user to pass an assessment
of the message (and indirectly an assessment  of that assessment) back to the
provider. 

It's never been about turning things into rules. Randy's proposed information
flow only talks about the provider updating a database with user-provided
information. You're the one that made that conflation when you said:

> That's why you give them a spam button, but you're vague about what it

Exactly.

> And make it clear that it's all 100% optional.

I think you mean that you can use one part without another. If you're referring
to the whole thing, since when is anything the IETF does mandatory?

> IMAP spam button if we can sort out the patent issues.

It's definitely important that we cover all the capabilities the providers say
they want. That said, limiting ourselves to specifically what meets a small
sample of providers say they currently need is a bad idea. It's just as
important not to be to specific as it is not to be too general.

As for patent issues, that's part of the reason for basing this on something
that appeared in a 2006 presentation.

				Ned

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 07 Aug 2012 21:09:21 +0200

On Mon 06/Aug/2012 20:10:43 +0200 Ned Freed wrote:
>>> OTOH, you say they want feedback when users say the filters got
>>> it wrong, and if the user is incorrect on that... now you have
>>> incorrect feedback information.
> 
>> The feedback doesn't directly turn into a rule.  Providers treat
>> them with considerable skepticism unless there are either a lot
>> of similar reports, or the stuff that's reported has other
>> characteristics of spam.
> 
> Which is precisely my point. This entire mechanism is about
> standardizing a means of passing information around. It is not
> about what is done with that information by any party. It provides
> a way for a provider to pass an assessment of the message to a user
> and a way for a user to pass an assessment of the message (and
> indirectly an assessment  of that assessment) back to the 
> provider.

IMHO, a score contains the same amount of information as a random
number unless someone specifies what is done with such number by each
party, or at least how it is generated.  That part of Randy's 2006
proposal differs from the approach taken by the REPUTE WG inasmuch as
the latter considers sender's scores rather than specific message's.

As for Ned's approach to spam reporting, it seems overly deterministic
in attributing a role to each party.  My understanding is that it is
not practical to determine who will want to do what.  The only
certainty is that doing anything implies more work, possibly more than
what an existing abuse team is prepared to tackle...

>> Plan B: talk to some large providers and see what they might be
>> interested in implementing across the board.  I know there's a
>> lot of interest in the IMAP spam button if we can sort out the
>> patent issues.
> 
> It's definitely important that we cover all the capabilities the 
> providers say they want. That said, limiting ourselves to
> specifically what meets a small sample of providers say they
> currently need is a bad idea. It's just as important not to be too
> specific as it is not to be too general.

For the case at hand, Zoltan's I-D stems from the assumption that both
sides of the IMAP connection are part of the same system, which may
not always be the case.  For interoperability, it would be better to
specify only the minimal information necessary to identify the
message, but still allow similar systems to recognize each other and
pass any additional data they may need.

> As for patent issues, that's part of the reason for basing this on
> something that appeared in a 2006 presentation.

The ASRG analysis of the problem, that John summarized in a few lines
http://wiki.asrg.sp.am/wiki/Adding_a_junk_button_to_MUAs#Via_IMAP
is documented to be of February 2010, which is earlier than OMA's
http://datatracker.ietf.org/ipr/1609/.  Moreover, I'm unable to find
anything with APN/163342.  Using (APD/201106$ AND AN/"Research In
Motion Limited") I found a 13/162,703 --resembling IMAP BURL-- that
focuses mainly on the possibility to select a "guilty" attachment in
the message being reported.  That is to say, the covered parts seem to
match the "too specific" cases that Ned suggests we avoid.

---

From: Ned Freed <[email-masked]>
Date: Tue, 07 Aug 2012 15:29:25 -0700 (PDT)

> party, or at least how it is generated.

Er, no. Sure, scoring information may be meaningless to the recipient - in fact
it may actually be encrypted to insure the recipient can't figure out what's in
it - but it's a little silly to assume it's going to be meaningless to the
system that generated it in the first place, don't you think? The point of
attaching it may be to assist the spam filter in understanding where it went
wrong when a user reports misclassification.

Alternately, the scoring information may be the name of the filtering system
that generated the score. People seem to be assuming service providers all use
a single filtering system. I can assure you that's not the case; many employ
several, and may use them in various different combinations or permutations
based on all sorts of factors. 

Yet another example where this information may be useful is when the user
claims missclassification when in fact it was the user's own rules that caused
the misclassification.

I could go on and list many other use cases, but I think this makes the point. 
There is a strong need to support a fairly general model here.

  That part of Randy's 2006

> in attributing a role to each party.

Huh? Since I haven't suggested any sort of approach, this makes no sense to me
at all. What I have suggested is a process by which we might be able to
converge on something that can be codified into a standard. That's it.

Perhaps you're talking about Randy's model. If so, the only thing that is
remotely deterministic about it is the sequence of steps involved, and that
sequence is pretty much necessitated by the order of the underlying operations
that are involved. (It's hard to return feedback on something you don't yet
have, for example.)

Indeed, as I noted above, you're the one who appears to be tying a general
model to very specific and very limited specific cases and then complaining
that it's more general than it needs to be.

> My understanding is that it is

Again - huh? This again doesn't seem in any way connected to what's being
proposed. You now appear to be assuming that there are necessarily people other
than the recipient involved in processing the information flows here. Not only
is that not called, for, it's actually a nongoal, at least in the sense of
anyone being directly involved. This sort of feedback reporting is intended to
do things like assess provider policy in general and spam filter effectiveness,
possibly including use for filter improvement (or possibly not). It definitely
not intended to go to "Team America - Spam Cops" (*) so they can send in the
SWAT team.

> pass any additional data they may need.

I agree on the first part - it's one of the many problems I have with his
proposal. But as for the second, I completely disagree, for at least two
reasons: (1) You are again failing to note the closed-loop nature of the model,
which means that not all parties necessarily have to fully understand all of
the information flowing around, and (2) Just because the two sides may not be
part of the same system is no reason to hamstring the case where they *are*
part of the same system.

				Ned

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 08 Aug 2012 14:02:07 +0200

On Wed 08/Aug/2012 00:29:25 +0200 Ned Freed wrote:
> at all.

I meant the (1), (2), (3) approach described in
http://www.ietf.org/mail-archive/web/apps-discuss/current/msg06783.html

> that it's more general than it needs to be.

I'd have used "comprehensive" rather than "general", but I'm not
complaining about that.  For the specific points of the proposal, to
add spam score to POP/IMAP notifications may be useful for some
bandwidth-constrained cases where downloading only the headers is not
practical, but is tactically inadvisable, IMHO.  The proposal's
concept of spam notifications predates RFCs 5965 and 6650, so I
consider only the part of it from the client to the POP/IMAP box,
which is what we seem to be currently missing --"There is, however, no
standard mechanism for this signaling", in the words of RFC 6650.

>> My understanding is that it is not practical to determine who
>> will want to do what.  The only certainty is that doing anything
>> implies more work, possibly more than what an existing abuse team
>> is prepared to tackle...
> 
> Again - huh? This again doesn't seem in any way connected to what's
> being proposed. You now appear to be assuming that there are
> necessarily people other than the recipient involved in processing
> the information flows here.

Yup, since it is a flow, it is legitimate to assume that it involves
more than one actor.  And, yes, I assumed some sort of involvement of
the senders, which implies assessing their honesty, when possible.

> Not only is that not called, for, it's actually a nongoal, at least
> in the sense of anyone being directly involved. This sort of
> feedback reporting is intended to do things like assess provider
> policy in general and spam filter effectiveness, possibly including
> use for filter improvement (or possibly not).

You mean that should remain within the client-ISP bailiwick?

> It definitely not intended to go to "Team America - Spam Cops" (*)
> so they can send in the SWAT team.

:-)

> part of the same system is no reason to hamstring the case where they *are*

Dunno.  To keep practical, I was thinking of something like SREP UID
123 (KEYWORD BUNCH-OF-DATA)...  where zero or more data sets, each one
formatted according to the relevant keyword, can be sent by the
client.  Servers may respond as appropriate for the keywords they
understand, or may add new keywords+data items in the hope that the
client knows how to interpret them.

---

From: Ned Freed <[email-masked]>
Date: Wed, 08 Aug 2012 13:51:22 -0700 (PDT)

> http://www.ietf.org/mail-archive/web/apps-discuss/current/msg06783.html

Then you need to explain how that *process* proposal has anything to do with
the model that ends up getting selected, because I'm not seeing it.

> standard mechanism for this signaling", in the words of RFC 6650.

Quite the contrary. Right now we have a bunch of pieces, with a few probably
missing, and absolutely no roadmap as to how to put them together to solve
various problems. The result is chaos in the field, as well as no clear
understanding of how to construct the missing pieces, as past discussions
of this area have clearly shown.

> the senders, which implies assessing their honesty, when possible.

Sure, it's legitimate to assume more than one actor is involved, but you're
assuming far more than that: You're assuming a specific sort of actor is
present that is rarely if ever past of these flows in practice.

> You mean that should remain within the client-ISP bailiwick?

No, I never said that.

> client knows how to interpret them.

And now you're discussing specific mechanism details without any clear
agreement on what we're trying to do. Can't you see that this is a *major*
problem?

				Ned

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 09 Aug 2012 09:10:11 +0200

On Wed 08/Aug/2012 22:51:22 +0200 Ned Freed wrote:
> Then you need to explain how that *process* proposal has anything
> to do with the model that ends up getting selected, because I'm not
> seeing it.

Step (1) requires to formalize a model.  AIUI, this would be an
overall model for spam reporting, possibly including a distributed
system to reckon domain reputation.  Its definition implies outlining
a collective process whereby a network-wide understanding of senders
behavior emerges.  An astounding, fascinating exercise.  But that's
not the kind of model you had in mind, is it?

> Right now we have a bunch of pieces, with a few probably missing,
> and absolutely no roadmap as to how to put them together to solve 
> various problems. The result is chaos in the field, as well as no
> clear understanding of how to construct the missing pieces, as past
> discussions of this area have clearly shown.

Nicely said, I agree on that.  However, some of those pieces might
solve some specific problem, and there is a possibility that they gain
some momentum.

>> To keep practical, I was thinking of [... specific mechanism
>> details elided ...]
> 
> And now you're discussing specific mechanism details without any
> clear agreement on what we're trying to do. Can't you see that this
> is a *major* problem?

I see that, but I don't think there are better ways to do it.  Each
"piece" has to be vital by itself, and gain momentum.  Specific
mechanisms may fail irrespectively of the role that we could have
agreed they would have played in the global model.  In that case, they
are forgotten.  The model itself may turn out to be unworkable without
them.  Hence, steps (1), (2), and (3) risk to be carried out in vain.
 OTOH, guessed right, successful mechanisms can form the groundwork
that further pieces will be based upon, in an apparently chaotic,
self-driven becoming.

IMAP SREP is a very small piece.  The smaller the better?

---

From: Ned Freed <[email-masked]>
Date: Thu, 09 Aug 2012 15:07:54 -0700 (PDT)

> system to reckon domain reputation.

Where did I say anything like that? In fact I have repeatedly said the exact
opposite: Any model we come up with needs to be able to support a wide variety
of possible use cases and specific reporting designs. This includes, but
is not limited to, cases where there is no information flowing from the
provider to the client, no information flowing from the client to the provider,
where the information is binary in nature, where the information is
numeric, where the information is labels or tags, where some of the information
is opaque to some of the parties.

> Its definition implies outlining
> behavior emerges. 

It implies no such thing. In fact the main thing it implies is trying to
prevent people with one specific approach in mind from limiting the model to
supporting that approach and nothing else. 

> An astounding, fascinating exercise.  But that's

Of course not.

> solve some specific problem, and there is a possibility that they gain

Exactly! It helps to get people on the same page in regards to mechanism,
if nothing else.

> IMAP SREP is a very small piece.  The smaller the better?

But what if it turns out to the *wrong* piece? (As it happens I think there's a
very high probability of that being the case.) That's why starting with a
specific piece in mind is a bad idea: You're assuming a solution then trying
to find problems that fit it.

				Ned

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 10 Aug 2012 12:08:12 +0200

On Fri 10/Aug/2012 00:07:54 +0200 Ned Freed wrote:
> Where did I say anything like that? In fact I have repeatedly said
> the exact opposite: Any model we come up with needs to be able to
> support a wide variety of possible use cases and specific reporting
> designs.

I guess I may have erroneously inferred something that you didn't
mean.  The OP attached slides about spam control for (mobile) clients,
presenting an overview of spam and anti-spam techniques.  Obviously,
everyone of us has his own version of such an overall model in his
mind.  I thought you referred to such thing.  A minor part of that
model concerns users signaling spam/non-spam labels to the provider
responsible for accepting messages on their behalf.  This latter part
can --and must-- be focused much more sharply.

> This includes, but is not limited to, cases where there is no
> information flowing from the provider to the client, no information
> flowing from the client to the provider, where the information is
> binary in nature, where the information is numeric, where the
> information is labels or tags, where some of the information is
> opaque to some of the parties.

Hm... I'm unable to figure out a real use case for about a half of the
characterizations you exemplify.  For example, "no information flowing
from the client to the provider"...?

> In fact the main thing [modeling] implies is trying to prevent
> people with one specific approach in mind from limiting the model
> to supporting that approach and nothing else.

What I assume is a client tagging a message.  It doesn't seem to be
limiting to me.  What am I missing?

> But what if it turns out to the *wrong* piece? (As it happens I
> think there's a very high probability of that being the case.)

I'd like to know why you think so.

> That's why starting with a specific piece in mind is a bad idea:
> You're assuming a solution then trying to find problems that fit
> it.

Some (all?) IMAP servers have the functionality, either native or by
means of unofficial patches, to track moves or flag settings in order
to infer the user's reckoning of spamminess.  So, problems that fit
this piece can be found easily.

The solution is a mechanism for reporting the (perceived) worthiness
of a message back to who (presumably) accepted it.  Its model
identifies the mailbox provider with the SMTP admin, and requires a
different mechanism for each type of message store.  Here we do IMAP.

Why a new command?  Using the "junk" folder badly overloads its usage.
 Flags are intrinsically boolean, so that one would need to
standardize keyword templates in order to account for extensibility.
In both cases, handlers need to get deeply involved in the internal
mechanics of the IMAP server, while their semantics concerns the SMTP
server.  The new command entails the execution of a site-specific
program/function for all and only invocations, with suitable parameter
passing conventions.  So, it seems to provide for better
implementations, code re-usability, data encapsulation, low coupling,
and so forth...

I realize I'm still having in mind that specific use case, and
discussing specific mechanism details.  I cannot avoid it, because the
wrongness of a piece cannot be ascertained from its model --well,
providing it's self-consistent-- but needs to refer to the overall
model that we talked about upthread.  Or is there some
intermediate-level model that we should refer to?

---

From: Ned Freed <[email-masked]>
Date: Fri, 10 Aug 2012 07:40:17 -0700 (PDT)

> presenting an overview of spam and anti-spam techniques.

Yes, but the reason for using the figure from that document as a starting point
is because it may help avoid some patent issues. Not because we intend to
slavishly support the rest of it to the exclusion of  other use cases.

> Obviously,

> from the client to the provider"...?

It is far and away the most common case implemented today. Surely
the most common case is relevant?

> limiting to me.  What am I missing?

You *really* need to go back and reread the last discussion we had 
on this, which almost immediately bogged down in all sorts of competing
proposals because everyone had a different specific use case in mind.

That's why we need to agree on a general model first. And that model needs to
include cases where the specific piece *you* happen to be interested in isn't
needed.

> I'd like to know why you think so.

Patent issues. Deployment issues - why add commands when it may be possible to
do what is needed using existing capabilities? Poor design of the actual
command.

That's just a start.

But at this point you're starting to get into the actual design of the
mechanism, and that's a game I refuse to play.

				Ned

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 10 Aug 2012 21:35:31 +0200

> the most common case is relevant?

I still can't see it.  How can a signal carry no information?  I'd
call it noise, in that case.

>> What am I missing?
>  on this, which almost immediately bogged down in all sorts of
> competing proposals because everyone had a different specific use
> case in mind.

You must mean the "Call for Adoption" thread that Murray started on
May 12.  The relevant points I found are as follows:

John Levine and Dave Crocker said they think a single flag is enough
to retain the relevant one bit of information on the message status.
http://www.ietf.org/mail-archive/web/apps-discuss/current/msg05742.html
http://www.ietf.org/mail-archive/web/apps-discuss/current/msg05759.html

You said that it's not as flexible as a general reporting mechanism.
For example, it couldn't be used to report things like signature
validation failures.  You also noted there's at least two bits
associated with spamitude, and raised some doubts against using IMAP
as a gateway.
http://www.ietf.org/mail-archive/web/apps-discuss/current/msg05748.html
http://www.ietf.org/mail-archive/web/apps-discuss/current/msg05757.html
http://www.ietf.org/mail-archive/web/apps-discuss/current/msg05737.html

Cyrus said we already have $Junk and $NotJunk.
http://www.ietf.org/mail-archive/web/apps-discuss/current/msg05753.html

A prior thread on this topic, "Spam reporting over IMAP", was started
on January 9, also by Murray.  There are relevant architectural
objections:

Lyndon said IMAP is absolutely the wrong place to put spam reporting.
http://www.ietf.org/mail-archive/web/apps-discuss/current/msg04058.html

John Klensin questioned whether there is a strong enough case to be
made that this belongs in IMAP to justify further clutter in the
protocol, and whether it would be helpful to graft something of this
nature onto a SIEVE server.
http://www.ietf.org/mail-archive/web/apps-discuss/current/msg04107.html
http://www.ietf.org/mail-archive/web/apps-discuss/current/msg04055.html

I have omitted all administrative concerns.  Of the technical ones, I
only reported those that apparently conflict with the solution I have
tried to uphold upthread.

> That's why we need to agree on a general model first. And that
> model needs to include cases where the specific piece *you* happen
> to be interested in isn't needed.

Let me slightly reshape the proposal as a *storage gateway*.  We can
have the same command, with minimal syntax variations, provide a
gateway for IMAP and POP3.  Yes, that's after all what you've said
against gateways.  My only excuse for doing so is that BURL is not so
widely supported in IMAP and not at all in POP3.  The very purpose of
the command is to provide a sort of out-of-band channel from clients
to the server admins:  Reporting a message.  Reporting spam is the use
case at hand.

IMHO, this is simpler and easier than BURL and/or some additional
functionality designed around it.  For POP3, clients have to delay
DELE until they deem the user won't report on a given message, while
server-side development is exactly the same.

For both protocols, the command identifies an existing item in the
message store (by UIDL, UID, or whatever).  The command fails if the
message doesn't exist.  The command accepts an extensible set of
parameters about the message, beside its identification.  The server
does not have to understand any of those parameters, as its task is to
just pass them, along with the reported message, to a system-specific
configured script or executable.  The response that the latter returns
is to be gatewayed back to the client.  Again, neither the server nor
the client have to understand anything about the response, but they
may do so.

Could that be agreed upon?  I think it gets over most of the above
concerns.

>> I'd like to know why you think [there's a very high probability
>> of IMAP SREP being the *wrong* piece].
> 
> Patent issues.

Covered stuff goes in separately specified extensions.  BTW, the
reason I asked to remove the "kleansed" I-D was because of copyright
issues, not patent ones.

> Deployment issues - why add commands when it may be possible to do
> what is needed using existing capabilities? Poor design of the
> actual command.

Does the reshaped proposal address those concerns?

---

From: Ned Freed <[email-masked]>
Date: Fri, 10 Aug 2012 13:22:51 -0700 (PDT)

> call it noise, in that case.

I't called "leaving out one component of the model while leaving other pieces
in place".

Failure to recognize that not everyone will implement (or is interested in
implementing) the whole thing will lead to folks saying, "This is all or
nothing and all doesn't fit our needs, so let's ignore it in toto." And then
things don't interoperate. This was precisely John Levine's point in an earlier
message, and I agree with it.

> May 12.  The relevant points I found are as follows:

Relevant points? None of the content of that thread is relevant in the
slightest. The only relevant point is that different people have different
use cases in mind and there is no agreement on a general model, so consensus
could not be reached.

> on January 9, also by Murray.  There are relevant architectural

And there were several other threads before that. Same result.

Aren't you seeing a pattern here? this is a political issue, not a technnical
issue. You don't solve political issues with piecemeal technical proposals.

> > to be interested in isn't needed.

The rest of your message is al about specifics of the technical proposal.
I've already stated that I have zero interest in discussing this at this 
point.

				Ned

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 13 Aug 2012 12:06:58 +0200

On Fri 10/Aug/2012 22:22:51 +0200 Ned Freed wrote:
> 
>>>> What am I missing?
>>>
>>> You *really* need to go back and reread the last discussion we
>>> had on this, which almost immediately bogged down in all sorts
>>> of competing proposals because everyone had a different
>>> specific use case in mind.
> 
>> You must mean the "Call for Adoption" thread that Murray started
>> on May 12.  The relevant points I found are as follows:
> 
> Relevant points? None of the content of that thread is relevant in
> the slightest. The only relevant point is that different people
> have different use cases in mind and there is no agreement on a
> general model, so consensus could not be reached.
> 
>> A prior thread on this topic, "Spam reporting over IMAP", was
>> started on January 9, also by Murray.  There are relevant
>> architectural objections:
> 
> Aren't you seeing a pattern here? this is a political issue, not a
> technical issue. You don't solve political issues with piecemeal
> technical proposals.

I'm unclear on what you mean by "political".  It may be an inter-SDO
issue, either with this particular liaison[*] or with OMA in general,
or you may mean a generic acceptation of "political" as a component
that makes people unwieldily nervous and irrational, as it often
happens with anti-spam related topics.  In any case, technical
documents are the only practical thing that we may come out with,
however piecemeal.

It'd be nice if this thread concludes saying something workable about
the subject.

[*] https://datatracker.ietf.org/liaison/1090/

---

From: Ned Freed <[email-masked]>
Date: Mon, 13 Aug 2012 08:08:29 -0700 (PDT)

> happens with anti-spam related topics.

Why does it have to be any one of these? Does it help to try and nail this
sort of thing down? I seriously doubt it. What matters is making forward
progress.

> In any case, technical

Of course.

> It'd be nice if this thread concludes saying something workable about

I've already put forward a proposal as to how I think we should proceed.

What I've heard from you are mostly comments that boil down to saying that we
should repeat the same exercise that's failed multiple times in the past, along
with several attempts to start that exercise. I think your proposed approach is
extremely unlikely to produce a useful result, and I have explained why.

John Levine started out with some incorrect assumptions about the result I want
to see from the process, which then morphed into the important point that any
proposal has to be clear that not all use cases will employ all components
the general model supports.

And Barry has stated that we need to "work closely with MAAWG" to develop the
model, which I take as agreement that we need to have a model in place first.

As far as MAAWG goes, that's fine with me as long as it doesn't translate into
waiting for input that never arrives. (Given how the recent media types
exercise unfolded, I won't be particularly happy with that outcome.) I hope
that contact with MAAWG has already been made, but I've heard nothing one way
or the other about that.

				Ned

---

From: "John Levine" <[email-masked]>
Date: 13 Aug 2012 19:39:03 -0000

>that contact with MAAWG has already been made, but I've heard nothing one way

Since Dave Crocker and I are two of MAAWG's senior technical advisors,
I wouldn't worry too much.

MAAWG does relatively more at meetings and less in e-mail than the
IETF, so I expect there won't be much to say until the Baltimore
meeting in October.  I'll see if I can grab a session for ISP
brainstorming about what sorts of user spam reporting hooks they'd be
interested in standardizing.

R's,
John