[Python-Dev] buildin vs. shared modules

Guido van Rossum guido at python.org
Fri Oct 17 14:04:53 EDT 2003

> >> So, would a patch be accepted (for 2.4, I assume there is no way for
> >> 2.3.3) which made everything builtin except for the following modules:
> >> 
> >> _testcapi - not used outside the testsuite
> >> _tkinter - needs external stuff anyway
> >> pyexpat - may be replaced by a third party module
> >> _ssl - needs Python to be built
> >
> > I'd rather see an explicit list of the "everything" that you want to
> > bundle into the main DLL.
> Here is the list of Python 2.3 extension modules, in decreasing order of
> my preference to be converted into a builtin.
> Needed to start Python - should be builtin:
>   zlib _sre

+1 for _sre.

I'd be +1 for zlib, but see bz2 below for a quibble.  (How important
is this *really* for bootstrap reasons?)

> Used by myself everyday - would like them to be builtin:
>   _socket _winreg mmap select

+1 on _winreg and mmap (they're small enough).

Long ago, when I first set up the VC5 project, there were still some
target systems out there that didn't have a working winsock DLL, and
"import socket" or "import select" would fail there for that reason.
If this is no longer a problem, I'm +1 on this.

> I'm undecided on these modules, I do not use them now but may in the
> future - so I'm undecided:
>   _csv winsound datetime bz2

I'm -1 on bz2; I think bz2 requires a 3rd party external library; for
developers building their own Python who don't want to bother with
that, it's much easier to ignore a DLL that can't be built than to
have to cut a module out of the core DLL.

The same argument applies to zlib -- but I could be swayed by the
counterargument that zlib is needed for zipimport bootrstrap purposes.
(Though is it? you can create zip files without using compression.)

> These should remain in separate pyd files for varous reasons:
>   _tkinter _bsddb _testcapi pyexpat


> Don't know what these do, so I cannot really comment:
>   _symtable parser unicodedata

_symtable is tiny and can be included.

parser is huge but has no external deps; if MvL's argument is correct,
the DLL size increase doesn't translate into a memory usage increase,
so I'd be +1 on including it; ditto for unicodedata.

> And while we're at it, I have looked at sys.builtin_module_names (again,
> from Python 2.3), and wondered if there arn't too many.
> I have *never* used any of these (xxsubtype is only a source code
> example, isn't it):
>   audioop imageop rgbimg xxsubtype

They could all be moved out, but why bother?  (xxsubtype is just a
source code sample module, there's no need to enable it in
distrubutions, but it doesn't hurt anybody either I think!)

> and I guess some of these could also be moved out of python.dll (rotor
> is even deprecated):
>   _hotshot cmath rotor sha md5 xreadlines

Ditto.  None of these are big.  xreadlines should also be deprecated.
But let it stay in the DLL until we stop distributing it (again,
assuming MvL's argument about memory usage is valid).

> ----
> There may be incompatibilities - that's why I asked about 2.3.3 or 2.4.

I wouldn't mess with 2.3.3.

> The biggest problem would probably be that you would have to download
> additional sources - zlib is one example.


> Who cares about the python.dll file getting larger? As Martin explained,
> this shouldn't increase memory usage, and since zlib and _sre are loaded
> anyway at Python starup, the startup time should decrease IMO.


> Let me conclude that I have no pressing need for changing this, but the
> decision whether an extension module is builtin or in a dll should
> follow a certain pattern.

"Historical precedent" is a pattern too. :-)

> To reduce the number of files py2exe (or installer) produces the best
> way would be to build custom python dlls containing the most popular
> extensions as builtins.  Of course this can be done by everyone owning a
> C compiler and a text editor.  And my own version would certainly
> include _ctypes ;-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

More information about the Python-Dev mailing list