On Thu, Jul 22, 2004 at 10:11:41AM EDT, Nicolas Rachinsky wrote:
> * David Yitzchak Cohen <lists+mutt_users@xxxxxxxxxxxxxx> [2004-07-22 05:07
> -0400]:
> > I guess the problem you're seeing is that mbox is already quite efficient,
> > so almost anything you'd do to cache information from it in yet another
> > file would only add up to incremental performance gains. With maildir,
> > you start out far worse with nothing, so you have far more to gain by
> > caching headers. (Try a 300MB maildir with no headercache, and I think
> > you'd sooner die than wait for it to load ... hehe. . .)
>
> No.
>
> The following commands were executed on a mbox and maildir containing
> the same 22959 mails with a total size of 300MB.
You've got fat emails. My inbox is well over 50K emails. If you're
gonna enter an ext2 directory with that many files, I'd suggest at
least grabbing a Snickers bar ("Not going anywhere for a while? Grab a
Snickers :-)").
Even with your own box, though, notice how much time is wasted (a)
in systemcalls and (b) in actual disk I/O. The second open, BTW,
wouldn't be half as fast if you didn't have 1GB of RAM. (Try 192MB,
and then we can talk ... hehe. . .)
$ echo $((100*199/65))
306
$ echo $((100*86/35))
245
Also, notice above that even with all that RAM, the system time doesn't
improve on the second time with a maildir much more than it does with an
mbox (going from 306% to 245%). The reason is obviously the ridiculously
high syscall rate, causing context switches with kernel I/O services, etc.
A second CPU would probably give you significantly better results.
Your benchmarks are quoted below in their entirety, for ease of reference.
- Dave
> /usr/bin/time -h -l mutt -F /dev/null -f big.mbox -e 'push x'
> 10.01s real 2.47s user 0.65s sys
> 2263 block input operations
> 0 block output operations
>
> /usr/bin/time -h -l mutt -F /dev/null -f big.mbox -e 'push x'
> 3.12s real 2.42s user 0.35s sys
> 0 block input operations
> 0 block output operations
>
>
>
> /usr/bin/time -h -l mutt -F /dev/null -f big.maildir -e 'push x'
> 19.51s real 2.42s user 1.99s sys
> 23598 block input operations
> 0 block output operations
>
> /usr/bin/time -h -l mutt -F /dev/null -f big.maildir -e 'push x'
> 3.46s real 2.28s user 0.86s sys
> 0 block input operations
> 0 block output operations
>
>
>
> /usr/bin/time -h -l mutt -F /dev/null -f big.maildir -e 'set
> maildir_cache=cache' -e 'push x'
> 6.54s real 3.16s user 1.49s sys
> 2 block input operations
> 744 block output operations
>
>
>
> /usr/bin/time -h -l mutt -F /dev/null -f big.maildir -e 'set
> maildir_cache=cache' -e 'push x'
> 1.43s real 0.79s user 0.36s sys
> 0 block input operations
> 12 block output operations
>
>
> /usr/bin/time -h -l mutt -F /dev/null -f big.maildir -e 'set
> maildir_cache=cache' -e 'push x'
> 1.40s real 0.79s user 0.37s sys
> 0 block input operations
> 12 block output operations
>
>
>
> /usr/bin/time -h -l mutt -F /dev/null -f big.maildir -e 'set
> maildir_cache=cache' -e 'unset maildir_cache_verify' -e 'push x'
> 1.20s real 0.72s user 0.25s sys
> 0 block input operations
> 12 block output operations
>
>
> I removed all except the interessting lines from the time output.
>
> Nicolas
>
> PS:
> Mutt 1.5.6i (2004-02-01)
> Copyright (C) 1996-2002 Michael R. Elkins and others.
> Mutt comes with ABSOLUTELY NO WARRANTY; for details type `mutt -vv'.
> Mutt is free software, and you are welcome to redistribute it
> under certain conditions; type `mutt -vv' for details.
>
> System: FreeBSD 4.10-RELEASE-p2 (i386) [using ncurses 5.1] [using libiconv
> 1.9] [using libidn 0.5.2 (compiled with 0.5.2)]
> Compile options:
> -DOMAIN
> -DEBUG
> -HOMESPOOL +USE_SETGID +USE_DOTLOCK +DL_STANDALONE
> -USE_FCNTL +USE_FLOCK
> +USE_POP +USE_IMAP +IMAP_EDIT_THREADS -USE_GSS +USE_SSL -USE_SASL
> -USE_SASL2
> +HAVE_REGCOMP -USE_GNU_REGEX +COMPRESSED
> +HAVE_COLOR +HAVE_START_COLOR +HAVE_TYPEAHEAD +HAVE_BKGDSET
> +HAVE_CURS_SET +HAVE_META +HAVE_RESIZETERM
> +CRYPT_BACKEND_CLASSIC_PGP +CRYPT_BACKEND_CLASSIC_SMIME
> -CRYPT_BACKEND_GPGME -BUFFY_SIZE -EXACT_ADDRESS -SUN_ATTACHMENT
> +ENABLE_NLS -LOCALES_HACK -HAVE_WC_FUNCS +HAVE_LANGINFO_CODESET
> +HAVE_LANGINFO_YESEXPR
> +HAVE_ICONV -ICONV_NONTRANS +HAVE_LIBIDN +HAVE_GETSID +HAVE_GETADDRINFO
> +USE_HCACHE
> -ISPELL
> SENDMAIL="/usr/sbin/sendmail"
> MAILPATH="/var/mail"
> PKGDATADIR="/usr/local/share/mutt"
> SYSCONFDIR="/usr/local/etc"
> EXECSHELL="/bin/sh"
> -MIXMASTER
> To contact the developers, please mail to <mutt-dev@xxxxxxxx>.
> To report a bug, please use the flea(1) utility.
>
> patch-1.5.6.dw.multiple-crypt-hook.2
> patch-1.5.6.dw.confirm-crypt-hook.1
> patch-1.5.6.dw.crypt-autoselectkey.1
> patch-1.5.6.dw.pgp-menu-traditional.2
> patch-1.5.6.dw.maildir-mtime.1
> patch-1.5.6.dshaw.extend_timeout
> maildir-header-cache.18
> patch-1.5.6.eh.thread_subject.1
> patch-1.5.5.1.cd.trash_folder.3.4
> patch-1.5.1.nr.indicator_not_bright
> 1.3.26.bj.current_shortcut.1-nr
> rr.compressed
> 1.3.28.nr.threadcomplete
> patch-1.5.0.ats.date_conditional.1
> dgc.deepif.1
>
> -----
>
> FreeBSD pc5.i.0x5.de 4.10-RELEASE-p2 FreeBSD 4.10-RELEASE-p2 #0: Fri Jul 2
> 09:03:30 CEST 2004
> nicolas@xxxxxxxxxxxx:/scratch/obj/usr/exported/src/sys/PC5 i386
>
> -----
>
> CPU: AMD Athlon(TM) XP 2000+ (1666.74-MHz 686-class CPU)
>
> 1GB RAM
>
> -----
>
> FFS with softupdates on a Vinum RAID1 on
> ad0: 152627MB <SAMSUNG SV1604N> [310101/16/63] at ata0-master UDMA100
> ad6: 156334MB <Maxtor 6Y160P0> [317632/16/63] at ata3-master UDMA133
>
--
Uncle Cosmo, why do they call this a word processor?
It's simple, Skyler. You've seen what food processors do to food, right?
Please visit this link:
http://rotter.net/israel
Attachment:
pgpNvrW04dBBK.pgp
Description: PGP signature