[Mutt] #3166: mutt: reading header cache on ext3 with dir_index is dead slow with maildir due to missing inode sort

[Mutt <fleas@xxxxxxxx>]

#3166: mutt: reading header cache on ext3 with dir_index is dead slow with 
maildir
due to missing inode sort
------------------------------+---------------------------------------------
 Reporter:  antonio@xxxxxxxx  |       Owner:  mutt-dev
     Type:  defect            |      Status:  new     
 Priority:  minor             |   Milestone:          
Component:  mutt              |     Version:  1.5.19  
 Keywords:                    |  
------------------------------+---------------------------------------------
 Forwarding from http://bugs.debian.org/508988

 ---

 Hi,

 mutt doesn't seem to sort the inode list of a maildir if a header cache
 file
 is found and maildir_header_cache_verify is unset:

 #if USE_HCACHE

     if (option(OPTHCACHEVERIFY))

     {

       DO_SORT();

       ret = stat(fn, &lastchanged);

     }


 This leads to insane long time to open a maildir with a lot of mails (like
 20000) if the header cache is not in the FS cache (like right after boot).
 It reads the header cache with the unsorted inode list, which causes
 pretty
 random seeks in the header cache file.

 To reproduce:

 1. open a large maildir (> 20000 mails) on ext3 with dir_index enabled
    (should be on by default since may years) to make sure that the header
    cache is created

 2. unset maildir_header_cache_verify in ~/.muttrc

 3. echo 7 > /proc/sys/vm/drop_caches

 4. open the maildir again

 It should now take very long to open the maildir. Without step 2, it is
 quick again.

 For reference, here are 2 strace excerpts, one taken with ext3, one taken
 with XFS.  I created them before I discovered the real cause of the
 slowness, and only saw that it is slow on ext3, but fast on XFS.  Just
 look
 at the timestamps.  XFS takes 3 seconds, ext3 takes 18 seconds:

 http://tikei.de/mutt_hcache_ext3.txt

 http://tikei.de/mutt_hcache_xfs.txt

 You can also see that the seek offsets are somewhat linear for XFS, but
 pretty random for ext3. The latter is the real cause: the hard disk is
 seeking to death.

 Regards,
 Tino

-- 
Ticket URL: <http://dev.mutt.org/trac/ticket/3166>
Mutt <http://www.mutt.org/>
The Mutt mail user agent