Re: Why does a maildir resync take 4mn?
On Tue, Mar 27, 2007 at 06:57:58PM +0000, Rocco Rutte wrote:
> Hi,
>
> * Marc MERLIN [07-03-27 08:33:27 -0700] wrote:
>
> >I have
> >set folder="~/Maildir/"
> >set header_cache="~/Maildir/"
> >set maildir_header_cache_verify = no
>
> >The header cache definitely helps, I can open a 40,000 message maildir in
> >seconds instead of several minutes.
> >However, if the parent directory of the folder I'm viewing, gets modified
> >(let's say I'm in ~/Maildir/.snd/ and I send a new mail), mutt takes a
> >silly
> >time to resync the folder, like 4mn for 40,000 messages.
>
> >Is there anything I can do to make mutt more bearable in this case like
> >this?
> >Why is mutt resyncing everything when I do have maildir_header_cache_verify
> >set to no? What else can I do to stop this?
>
> I didn't explicitely check, but maildir_header_cache_verify only
> prevents a stat() call per message since stat() is slow on NFS. But I
> doubt the difference is that huge.
>
> I guess mutt is spending some time on sorting. How about the system load
> during that period? Can you use top(1) to figure out in what state mutt
> is (maybe in addition to running it with -d)?
The part where I'm confused is that changing to my sent folder happens in
just a few seconds. Mutt opens the index, spends a few seconds reading all
of it, and processing it.
Then it takes another second to read additional messages that weren't in the
index, and it's done. Total is < 5sec
I just tried mutt -d9 and it shows that if I add a message to the sent
folder that I already had opened, mutt spends several minutes doing this:
mutt_match_rx_list: marc<at>merlins.org matches
(marc.*|merlin|merlin_.*|mmerlin|marc.merlin|marcmerlin|root|postmaster|merlingandalf)<at>.*(domain1|domain2|domain3|domain4|domain5|....|domain15)
mail_addr_is_user: yes, marc<at>merlins.org matched by alternates.
mail_addr_is_user: no, all failed.
mutt_match_rx_list: marc<at>merlins.org matches
(marc.*|merlin|merlin_.*|mmerlin|marc.merlin|marcmerlin|root|postmaster|merlingandalf)<at>.*(domain1|domain2|domain3|domain4|domain5|....|domain15)
mail_addr_is_user: yes, marc<at>merlins.org matched by alternates.
mail_addr_is_user: no, all failed.
mutt_match_rx_list: marc<at>merlins.org matches
(marc.*|merlin|merlin_.*|mmerlin|marc.merlin|marcmerlin|root|postmaster|merlingandalf)<at>.*(domain1|domain2|domain3|domain4|domain5|....|domain15)
mail_addr_is_user: yes, marc<at>merlins.org matched by alternates.
mail_addr_is_user: no, all failed.
mutt_match_rx_list: marc<at>merlins.org matches
(marc.*|merlin|merlin_.*|mmerlin|marc.merlin|marcmerlin|root|postmaster|merlingandalf)<at>.*(domain1|domain2|domain3|domain4|domain5|....|domain15)
mail_addr_is_user: yes, marc<at>merlins.org matched by alternates.
mail_addr_is_user: no, all failed.
mutt_match_rx_list: marc<at>merlins.org matches
(marc.*|merlin|merlin_.*|mmerlin|marc.merlin|marcmerlin|root|postmaster|merlingandalf)<at>.*(domain1|domain2|domain3|domain4|domain5|....|domain15)
mail_addr_is_user: yes, marc<at>merlins.org matched by alternates.
mail_addr_is_user: no, all failed.
mail_addr_is_user: no, all failed.
mail_addr_is_user: no, all failed.
mail_addr_is_user: no, all failed.
mail_addr_is_user: no, all failed.
I commented out my
folder-hook . alternates ...
config line, but then it would still take up to 10mn and debug then just said:
mail_addr_is_user: no, all failed.
Note that I forgot to mention that even before I used header cache,
geenrating the list of headers from scratch would maybe take 1mn (i.e. time
to open the folder each time), while the auto resync of the folder triggered
by a new maildir message would take 3 to 4 times as long.
For some reason, those 40,000 mail_addr_is_user calls are very slow, only
maybe 100 per second, or less, yielding for 400 seconds of wait for 40,000
messages.
And the puzzling part is that mail_addr_is_user seems to be significantly slower
on a folder resync.
As for mutt in top, it would show as:
4448 merlin 18 0 57528 53m 2140 R 38.0 5.3 0:16.19 mutt
or
4448 merlin 18 0 57528 53m 2140 D 39.1 5.3 0:20.18 mutt
Does that help?
Can I provide other useful information?
Marc
--
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems & security ....
.... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/