[Python-Dev] buildin vs. shared modules

[Shane Holloway (IEEE)]

Thomas Heller wrote:

> Here is the list of Python 2.3 extension modules, in decreasing order of
> my preference to be converted into a builtin.
> 
> Needed to start Python - should be builtin:
> 
>   zlib _sre

+1 -- would not these speed startup time?

> Used by myself everyday - would like them to be builtin:
> 
>   _socket _winreg mmap select

+0 -- I use them a lot, but the overhead importing them is definitely 
acceptable to me.

> I'm undecided on these modules, I do not use them now but may in the
> future - so I'm undecided:
> 
>   _csv winsound datetime bz2

-0 -- Useful modules, but not on my everyday use list.

> These should remain in separate pyd files for varous reasons:
> 
>   _tkinter _bsddb _testcapi pyexpat

Definitely agreed.  :)

> Don't know what these do, so I cannot really comment:
> 
>   _symtable parser unicodedata

Neither do I.  Although unicodedata is fairly big.

> And while we're at it, I have looked at sys.builtin_module_names (again,
> from Python 2.3), and wondered if there arn't too many.
> 
> I have *never* used any of these (xxsubtype is only a source code
> example, isn't it):
> 
>   audioop imageop rgbimg xxsubtype

+1 -- I agree that these would not suffer too badly from being external 
pyds either.

> and I guess some of these could also be moved out of python.dll (rotor
> is even deprecated):
> 
>   _hotshot cmath rotor sha md5 xreadlines

+1 for _hotshot, rotor, and xreadlines -- External would be good.
-0 for sha, md5 -- I like these the way they are, but I see your point.
-1 for cmath -- complex types are part of the language, and should be 
builtin, IMO.

> ----
> There may be incompatibilities - that's why I asked about 2.3.3 or 2.4.

-1 for 2.3.3 or any point release in 2.3
+1 for 2.4

> The biggest problem would probably be that you would have to download
> additional sources - zlib is one example.
> 
> Who cares about the python.dll file getting larger? As Martin explained,
> this shouldn't increase memory usage, and since zlib and _sre are loaded
> anyway at Python starup, the startup time should decrease IMO.

Small is beautiful.  Fast is good.  I don't like the idea of statically 
linking pyds into python simply because we can.  Nor does reducing the 
number of external files for packagers like py2exe.  I know that pain 
too, but I don't want python to suffer from too much bloat for that reason.

> Let me conclude that I have no pressing need for changing this, but the
> decision whether an extension module is builtin or in a dll should
> follow a certain pattern.
> 
> To reduce the number of files py2exe (or installer) produces the best
> way would be to build custom python dlls containing the most popular
> extensions as builtins.  Of course this can be done by everyone owning a
> C compiler and a text editor.  And my own version would certainly
> include _ctypes ;-)
> 
> Thomas
I love ctypes :)  It saves me from doing hard work ;)

Thanks for reading :)
-Shane Holloway