<<< Date Index >>>     <<< Thread Index >>>

Re: [Mutt] #2981: header_cache: opening a changed Maildir takes way



#2981: header_cache: opening a changed Maildir takes way longer than it

Changes (by pdmef):

 * cc: 434544@xxxxxxxxxxxxxxx (added)

Comment:

 I've now done some more tests with profiling. In the resulting gprof
 output, the only hot spot I see is from sorting the mailbox. As libc
 functions aren't counted, this further supports my guess that readdir() is
 to be blamed.

 Another very simple test is comparing the time of 'cd /huge/folder/cur &&
 /bin/ls > /dev/null' to the 'Scanning...' phase in mutt. Result: these two
 numbers are nearly identical. The rest of the time mutt needs to bring up
 the folder is the time it takes to read headers (>98% from hcache) and
 sort the mailbox. For my 300k folder this takes around 35 seconds, while
 the first-pass and /bin/ls alone require 25 seconds.

 I even made some experiments by not using a linked list but an array
 reducing the malloc() calls to just 7 from 300k (for pathname management)
 for scanning this folder. Result: no difference.

 So, to the best of my knowledge so far I conclude that mutt can't do
 anything to speed up the scanning phase.

 For the reading phase I only roughly went through the code, but mutt
 really only reads the headers of each message it doesn't have in its
 cache. It would be interesting to know how other tools reading the first
 few lines of some messages perform to find out if mutt can be improved.

 For reading the messages, mutt uses an fopen() wrapper and fgets(), so
 maybe it would be interesting to know if e.g. mmap() approaches or the
 like could help, and if so how much.

 Again, I'm sorry and like to see this fixed myself, but I don't any see
 any obvious optimization at the moment.

-- 
Ticket URL: <http://dev.mutt.org/trac/ticket/2981#comment:3>