Re: Mutt and Berkely DB, the results of the cruisade
Brendan Cully <brendan@xxxxxxxxxx> writes:
> On the other hand, I'm tempted to pull db4 entirely, since it seems
> like the most troublesome of the database engines.
I didn't follow the DB4 troubles, but I've fought some of the battles
myself in bogofilter.
My personal guidelines for Berkeley DB are:
- Autoconfiguring Berkeley DB with the dozens of renames done by
distributors, some compiling with --with-uniquename, some not,
different paths and layouts is a major headache. This can't all be
blamed on BDB; some of the blame is with the a.out morons who
traditionally hacked BDB so that several versions could co-exist. The
ELF SONAME (i. e. the ABI) is right on most systems, libdb-M.N.so
(M and N major and minor version).
- Be very paranoid about matching header files to library versions - a
mismatch is going to cause trouble (code snippet below).
I'm willing to claim that the configure.ac check of mutt's is
susceptible to mixed db installations.
- DB is very fast and robust, as long as its functions can complete - if
you abort it (for whatever reason), the effects of database corruption
can be awful - assuming you're not using transactional mode. (Same
goes for system crashes underneath QDBM. It's only robust against
application crashes.)
- DB in transactional mode is robust against most sorts of crashes
(except hardware cheating, such as reordering writes and losing some
of them), as long as you keep the user away from the database
directory, because several precautions have to be taken when handling
the database. Newer DB library versions create files with some sort of
checksum (used to be SHA1, not sure what it is today) to detect
corruptions more reliably.
- Copying the database must be happen at a page-atomic level (try dd and
small page sizes), and the log files must be copied after the .db
files - bogofilter provides scripts to do that.
- It may help to extract the major/minor version from the db.h file that
the compiler finds (use the preprocessor to obtain DB_VERSION_MAJOR
and ...MINOR) and link against libdb-major.minor explicitly.
- If it crashes at runtime, send me a backtrace (and tell me how to
retrieve the source code belonging to the backtrace) and I'll have a
look as soon as my time permits.
Bogofilter's Berkeley DB interface code contains this "fender" which is
run by several of the DB operations:
(EX_ERROR is bogofilter-specific.)
------------------------------------------------------------------------------------
#include <db.h>
/* If header and library version do not match,
* print an error message on stderr and exit with EX_ERROR. */
static void check_db_version(void)
{
int maj, min;
static bool version_ok = false;
if (!version_ok) {
version_ok = true;
(void)db_version(&maj, &min, NULL);
if (DEBUG_DATABASE(1))
fprintf(dbgout, "db_version: Header version %d.%d, library version
%d.%d\n",
DB_VERSION_MAJOR, DB_VERSION_MINOR, maj, min);
if (!(maj == DB_VERSION_MAJOR && min == DB_VERSION_MINOR)) {
fprintf(stderr, "The DB versions do not match.\n"
"This program was compiled for DB version %d.%d,\n"
"but it is linked against DB version %d.%d.\nAborting.\n",
DB_VERSION_MAJOR, DB_VERSION_MINOR, maj, min);
exit(EX_ERROR);
}
}
}
------------------------------------------------------------------------------------
--
Matthias Andree