[Python-Dev] Getting python-bz2 into 2.3

[Tim Peters]

[Bob Kline, tries building a more-or-less complete Windows Python]
> The instructions for building Tcl and Tk worked as written with no
> problems (though I would have no idea where to begin with the XXX
> questions, such as "Should we compile with OPTS=threads?").

That's OK, neither do I <wink>.  We didn't before (8.3.2), and changing
anything is risky.

> The project file for the _tkinter module also worked without any
> hitches, but I don't see a test suite (no surprise for a GUI-based
> module) and I have no clue about how to test it or even get some of
> my simple _tkinter program to run.  I'm so far out of my depth here
> with respect to Tcl/Tk that I won't waste list bandwidth to pursue
> this any further.

I don't know how to test it either.  I install Python after building it,
then try Tkinter._test() from a DOS-box Python shell, and try a bunch of
stuff in IDLE.  That all looked fine.  It was more worrisome to me that some
of Tcl and Tk's own tests didn't pass, but I can't dig into that.

> As for the Berkeley DB package, that too built without any difficulties.
> The instructions conclude:
>
> ==============================  snip  ==============================
>     XXX We're actually linking against Release_static\libdb40s.lib.
>     XXX This yields the following warnings:
> """
> Compiling...
> _bsddb.c
> Linking...
>    Creating library ./_bsddb.lib and object ./_bsddb.exp
> LINK : warning LNK4049: locally defined symbol "_malloc" imported
> LINK : warning LNK4049: locally defined symbol "_free" imported
> LINK : warning LNK4049: locally defined symbol "_fclose" imported
> LINK : warning LNK4049: locally defined symbol "_fopen" imported
> _bsddb.pyd - 0 error(s), 4 warning(s)
> """
>     XXX This isn't encouraging, but I don't know what to do about it.
> ==============================  snip  ==============================
>
> I did a little poking around, and found that the problem is that the
> static library against which the _bsddb package is built is compiled
> with the -MT option, which causes the multi-threaded runtime libraries
> to be statically linked into libdb40s.lib, hence the warnings.  The
> warnings should probably be taken seriously, because I think they mean
> that we would end up with (for example) two sets of static heap control
> information.

Right, but it's not clear that it matters.  For example, it's unlikely
there's a case where memory is allocated by something in libdb40s but freed
outside it, or vice versa.  Then two heaps are just two distinct heaps.  I'm
inclined to ignore it for now, since we're probably going to move to a
current Sleepycat release anyway (but I don't know more about that -- Barry
is straightening that out).

It's quite unclear to me what good libdb40s.lib *is* if anyone using, e.g.,
malloc, can't link it in safely!

> The instructions in PCBuild/readme.txt were actually a little
> confusing, because right before the part I just quoted above tells us
> to "[c]hoose configuration "db_buildall - Win32 Release", and build
> db_buildall.exe."

It just echoes the Sleepycat instructions here.

> That version uses the -MD compilation option,

In part.  It also triggers the building of subprojects that end up creating
libdb40s.lib too.

> which uses the DLL version of the runtime libraries (which is what we
> want), but it creates a DLL for libdb40 instead of a library which can
> be statically linked into _bsddb.pyd.

It creates both the DLL and the static library.

> So it looks as there are two ways to eliminate the warnings
> above:
>
>  1. Use the DLL version of libdb40 and link to its import library.

This works but is clumsier:  you can't run the Python tests then from the
build directory, without first physically copying the Sleepycat DLL into
PCbuild, and you *also* need to build-and-copy a distinct debug-mode
Sleepycat DLL else you can't run the debug-build Python tests.  In addition,
the installer needs to be fiddled then to copy more files.  Like the old
1.85 bsddb Windows port from Sam Rushing, and like the current bz2 and zlib
modules, life is easier all around if 3rd-party code can be sucked directly
into the Python wrapper .pyd file.

>  2. Create a third configuration for the underlying Berkeley DB
>     library, using the -MD compilation option, but building a
>     statically linkable library instead of a DLL.

I'm not sure how to do that, but don't want to muck with Sleepycat's build
process regardless.

> I would have assumed that the second option was needed, because I
> assumed that Python wouldn't want to drag around lots of DLLs for the
> various packages that get installed, but then I noticed that the
> _tkinter package is doing just that, so I guess I'm not sure.

It doesn't want to, but Tcl/Tk are big and do their own kinds of linking
tricks.

> If the second option is used, it should presumably be done with the
> cooperation of the folks behind the Berkeley DB library, so it wouldn't
> be necessary to recreate the third configuration by hand each time a
> new release of the library came out.
>
> Finally, I was reluctant to dig any further into this package because I
> read at the top of the instructions:
>
>     "XXX The Sleepycat release we use will probably change before
>      XXX 2.3a1."
>
> and I didn't know whether that referred to a version upgrade or a
> complete replacement of the underlying DB engine with some other
> library, which would make any further work throwaway.

It refers to the current Sleepycat release; if you followed the build
instructions closely <wink>, they started by pointing you to a page of
obsolete releases.  For some reason Barry understands, we can't yet use the
current release (I think the bsddb C API changed, and our wrapper needs to
change accordingly).

Since I'm having a hard time (as above) imagining what good libdb40s.lib is
in its current state, I'm simply hoping that Sleepycat fixed the linking
glitches in their current release.

Thanks for the effort, Bob!  It helps.