Subject: [Tools-discuss] IMAP access to large archives

From: Dave Crocker <[email-masked]>
Date: Fri, 4 Mar 2016 06:39:43 -0800

G'day.

The IMAP interface to IETF email archives is wonderful.  Very, very 
large kudos for providing this.

However...

The structure of it creates a challenge that is worth discussing:

      Some of these archives are quite large and IMAP interaction with 
them is cumbersome and bloated.

For example, every time I want to use the interface to look at the ietf 
mailing list archive, my system spins for a very long time, building an 
index for -- and, I believe, downloading -- the 110K+ messages in the 
archive.

It's possible that the nature of IMAP leaves no good choices for better 
interaction, but I thought I'd raise the topic to see if others are 
reacting to this and possibly have suggestions.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: "Simon Perreault" <[email-masked]>
Date: Fri, 4 Mar 2016 11:28:52 -0500

> reacting to this and possibly have suggestions.

Can't you disable local copying? On Thunderbird at least there is such
an option. Anyway, the point is that IMAP does not require the client to
download everything and build a local index, it's an
implementation-specific "feature".

Simon

---

From: Dave Crocker <[email-masked]>
Date: Fri, 4 Mar 2016 08:45:31 -0800

> implementation-specific "feature".

First, a common default is to download an entire copy.

Second is that it at least has to download the index, and IMAP is 
transaction-heavy.

This makes at least startup with a large folder expensive in time and 
wasteful in storage.

d/
-- 

   Dave Crocker

---

From: "Marc Blanchet" <[email-masked]>
Date: Fri, 04 Mar 2016 11:56:45 -0500

> implementation-specific "feature".

good point. While not wanting to start discussion on specific IMAP 
implementations, Apple Mail is not really good at large folders. This is 
a reason that I moved to another client.  Dave, if you have Apple Mail, 
consider testing Thunderbird or another client, you may see a big 
difference.

Marc.

>
> Simon
>
> -- 
> Tools-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tools-discuss
>
> Please report datatracker.ietf.org bugs at 
> http://tools.ietf.org/tools/ietfdb
> Please report tools.ietf.org bugs at 
> http://tools.ietf.org/tools/issues or
> send email to [email-masked]

---

From: Dave Crocker <[email-masked]>
Date: Fri, 4 Mar 2016 09:11:24 -0800

On 3/4/2016 8:56 AM, Marc Blanchet wrote:
> Dave, if you have Apple Mail, consider testing Thunderbird or another
> client, you may see a big difference.

Should have mentioned that I already use Thunderbird.  My concerns are 
based on that usage.

d/
-- 

   Dave Crocker

---

From: Bob Hinden <[email-masked]>
Date: Fri, 4 Mar 2016 09:15:42 -0800

Marc,

>> 
> good point. While not wanting to start discussion on specific IMAP implementations, Apple Mail is not really good at large folders. This is a reason that I moved to another client.  Dave, if you have Apple Mail, consider testing Thunderbird or another client, you may see a big difference.

I think it’s a problem if IMAP access to email archives is only useful on a subset of IMAP clients.

Bob

---

From: Michael Richardson <[email-masked]>
Date: Fri, 04 Mar 2016 13:57:33 -0500

Simon Perreault <[email-masked]> wrote:
    >> For example, every time I want to use the interface to look at the
    >> ietf mailing list archive, my system spins for a very long time,
    >> building an index for -- and, I believe, downloading -- the 110K+
    >> messages in the archive.
    >>
    >> It's possible that the nature of IMAP leaves no good choices for
    >> better interaction, but I thought I'd raise the topic to see if others
    >> are reacting to this and possibly have suggestions.

    > an option. Anyway, the point is that IMAP does not require the client
    > to download everything and build a local index, it's an

I have the same problem as Dave, using Thunderbird (icedove).

I haven't found "disable local copying" can you tell me where that is?

I already noted that one should pick a local folder as the Sent folder.

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Michael Richardson <[email-masked]>
Date: Fri, 04 Mar 2016 13:59:27 -0500

> an option. Anyway, the point is that IMAP does not require the client

Under the account settings/preferences, under Synchronization&Storage,
I found "Keep message for this account on this computer" and it was checked,
and I unchecked it.
Do you think that's it?

There is also an option to sync only some number of days.

---

From: "Simon Perreault" <[email-masked]>
Date: Fri, 4 Mar 2016 14:04:04 -0500

> I found "Keep message for this account on this computer" and it was checked,

Yes, that's exactly it. Unchecking doesn't delete the cache AFAICT, you
have to re-create the account for that to happen.

Simon

---

From: Dave Crocker <[email-masked]>
Date: Fri, 4 Mar 2016 11:22:38 -0800

> I haven't found "disable local copying" can you tell me where that is?

      Tools > Account Settings > [account] >
         Synchronization & Storage > Advanced

Turn off all of the ietf ones.

d/
-- 

   Dave Crocker

---

From: Dave Crocker <[email-masked]>
Date: Fri, 4 Mar 2016 11:25:26 -0800

By way of trying to make a specific suggestion to deal with the specific 
issue of very large archives, what about:

    For any archive that is active and has messages dating back for more 
than a year: Also supply a "Latest 12 months" copy.

d/
-- 

   Dave Crocker