Re: Patch: cleanup body caching
On Friday, 28 April 2006 at 21:22, Rocco Rutte wrote:
> Hi,
>
> it rocks to see the performance on remote IMAP servers with the body
> caching.
>
> However, this IMHO isn't really the way to go for several reasons.
>
> First, mutt still has POP support and the while the code is in imap/
> it's unreachable. Second: the naming conventions are totally different
> from the header caching, which might be confusing or is at least
> inconsistent.
>
> I've started a long-term patch series which, in the end, is supposed to
> contain a totally refactored caching infrastructure (nice buzzword,
> isn't it ;-).
>
> The first unspectacular version is at:
>
> <http://user.cs.tu-berlin.de/~pdmef/mutt/patches/>
>
> as 'pdmef+patch+cache' along with a description, diff against CVS head
> and a dump of git logs (there it is again ;-) excluding those from CVS
> head (that's import since I put most of the reasoning for my local
> chanegs in there).
>
> Here's what I plan to do (roughly in that order).
>
> First, move it from imap/* out to the top level directory to make body
> caching available everywhere. This also needs changing the filename
> generation to be IMAP-independent. This is what the patch does now.
I don't really mind the code moving, but I'm not a fan of the md5 file
names (for the body cache OR the header cache). As you note, they are
(IHMO needlessly) opaque. I'd prefer to keep the names as the
URLs. For one thing, the current cache looks enough like a maildir
that it's possible with a couple of symlinks to open it as one for a
poor man's read-only offline mode. I like doing this on occasion and
it's a lot easier to do when ls returns something meaningful.
I'm not sure your four-level cache isn't overkill, and I doubt message
'packs' will be a win. Unlike your typical git directory, the body
cache isn't append-only. Remember, the headers are already cached in a
single file, so body opens are actually much nicer when they're
already in their own files.
> Try to port POP to it and see if POP can benefit from header caching as
> well. This needs to be done to see if the API makes sense.
ok, I guess.
> Split up header caching into two different interfaces. There should be
> one for raw (but abstract) database storage. Only things like
> open/close, store integer, list, string, etc. should be provided. On top
> of that a header cache can be implemented which uses the storage backend
> to have clean separation of the logic.
I did enough of this to let IMAP store a couple of extra fields in the
cache. Is it not enough?
> Afterwards the body caching needs to be refactored (maybe) to see if it
> can benefit from the DB backend as well.
again, I'm not sure I see where you're going with this.
> For easier maintaince and convenience, we need tools (at least built
> locally for developers) to work with the cache, i.e. list its contents,
> etc. For example, right now it's hard to tell (from user's point of
> view) whether a header cache file is still in use if he happens to
> remove a few mailboxes from the folder list.
IMHO that's because the current naming scheme is silly. If the names
reflected the value of $folder directly we wouldn't need tools.