[Python-Dev] Windows x64 & bsddb 4.4.20 woes

[Gregory P. Smith]

I haven't built the bsddb stuff on windows myself in a few years and have
never had access to a windows x64 system so I'm no silver bullet.  Making
the BerkeleyDB compile and link options match with those of python is the
first place I'd start.  Also you should be able to make a debug build of
BerkeleyDB (though it sounds like you may have tried that already?).  Next
off in the debugging i'd take a look at the assembly to see what exactly it
was failing to do.

If you're at PyCon right now we should meet up and try to figure it out (I
just arrived).

On 3/13/08, Trent Nelson <tnelson at onresolve.com> wrote:
>
> I've been trying to give the Windows x64 builds a bit of TLC the past few
> evenings.  I managed to get a successful build with all external modules
> last night (Tcl/Tk required about a half a dozen code/configuration changes
> each in order to build in a Windows x64 environment with Visual Studio 9,
> I'll deal with that in a separate thread or roundup issue).
>
> Unfortunately, though, we're back to more bsddb issues.  I got about 15
> tests in without error before test_whichdb ran, which results in the
> following being called in dbhash.py:
>
>         return bsddb.hashopen(file, flag, mode)
>
> I can trace that call to DBEnv_open() in _bsddb.c:
>
> static PyObject*
> DBEnv_open(DBEnvObject* self, PyObject* args)
> {
>     int err, flags=0, mode=0660;
>     char *db_home;
>
>     if (!PyArg_ParseTuple(args, "z|ii:open", &db_home, &flags, &mode))
>         return NULL;
>
>     CHECK_ENV_NOT_CLOSED(self);
>
>     MYDB_BEGIN_ALLOW_THREADS;
>     err = self->db_env->open(self->db_env, db_home, flags, mode);
>     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> Placing a breakpoint at the line above and stepping in results in Visual
> Studio reporting: " A buffer overrun has occurred in python_d.exe which has
> corrupted the program's internal state. Press Break to debug the program or
> Continue to terminate the program.".  FWIW, the exception is being raised as
> part of the /GS buffer overflow checks (implemented in gs_result.c, which is
> provided in my VS9 installation).
>
> This has been annoyingly awkward to debug.  I can't break down that call
> into multiple steps in order to try place breakpoints in the db_static
> module.  The callstack isn't that useful either:
>
> _bsddb_d.pyd!__crt_debugger_hook()
> _bsddb_d.pyd!__report_gsfailure(unsigned __int64 StackCookie=2211040)
> _bsddb_d.pyd!__GSHandlerCheckCommon(void *
> EstablisherFrame=0x000000000021bce0, ...)
> _bsddb_d.pyd!__GSHandlerCheck(_EXCEPTION_RECORD *
> ExceptionRecord=0x000000000021bbc0, ...)
> ntdll.dll!00000000773ae13d()
> [Frames below may be incorrect and/or missing, no symbols loaded for
> ntdll.dll]
> ntdll.dll!00000000773aea57()
> ntdll.dll!00000000773b59f8()
> _bsddb_d.pyd!__os_strdup()  + 0x18 bytes
> _bsddb_d.pyd!__os_tmpdir()  + 0x281 bytes
>
> You'd think placing breakpoints in db 4.4.20's __os_strdup and __os_tmpdir
> methods would do something, but alas, the bufferoverflow exception is raised
> before any breakpoints are set.  This makes me suspect there's something
> funky going on with the entire build and linking of db_static (VS should
> honour those breakpoints if the code is being entered, I even added
> db_static to pcbuild.sln and rebuilt but no dice).  I've noticed that
> they're not using consistent compiler flags by default (we use /GS, they use
> /GS-, we allow function level linking, they don't -- note that I did change
> db_static's options to align with _bsddb's but the bufferoverflow exception
> is still being thrown).
>
> Greg, Jesús, I'm CC'ing you guys as stfw'ing seems to bring back you two
> the most when it comes to bsddb issues.  I've still got a list of things to
> try with regarding to debugging this x64 issue, but I wanted to reach out
> now to see if anyone else had encountered it before.  Has bsddb ever been
> built successfully on Win64 and passed all tests or am I venturing into new
> ground?
>
> Martin, you've changed externals/bsddb-4.4.20 with regards to x64 builds
> recently -- have you been able to get things working in your x64
> environments?
>
> Regards,
>
>
>         Trent.
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20080313/7fdde024/attachment.htm