[Distutils] Pondering multi-package packages
Mark W. Alexander
mwa@gate.net
Wed, 24 May 2000 11:30:02 -0400 (EDT)
I apologize of this is documented somewhere. If it
is, I haven't found it. I've done lots of python
modules, but not any packages, so if I'm just
package ignorant feel free to slap me.....
It all started in the early Zope days when I loaded
Zope only to discover that all my scripts using
Marc Lemburg's mxDateTime bombed because Zope's
DateTime produced a namespace collision. The solution
then was to move Zope, since I didn't 'need' it at
the time.
Now I do. So, I contacted Marc who, of course, knew
about the collision and has already started looking
at packaging all his stuff under an umbrella "mx"
package. When I asked about Distutils, he hadn't
really looked at it so I volunteered to see what
it would take.
Since Distutils includes a sample mxDateTime setup.py,
it seemed fairly easy. I hacked it up to push it down
a level (see attempt below). No problem except, of
course, that it doesn't import because there's no
mx.__init__.py. Ok, so I add an mx.__init__.py with
__all__ defined and appropriate imports. Everything's
happy and Distutils does fine.
Now, I add in mx.TextTools and things get murkier.
How does mx.__init__.py determine what __all__
should be and what to import? How do I tell
Distutils about the other packages? I tried a
single all-purpose Distutils setup.py that searches
the mx package directory and adds in the sub-packages
that it finds. So far so good. Now, what about=20
mx.__init__.py?
Should it:=20
1) Search it's package directory at runtime to
determine what __all__ should be and what
subpackages to import? If it does that, then
mx.__init__.py will exist in all possible
mx packages. Distutils probably won't care and
will happily copy the lastest __init__.py into
place (since they "should" all be the same,=20
that's not a problem). BUT -- what if I make
packages with --bdist-WHATEVER. Package manager
WHATEVER is going to not be pleased with=20
multiple packages that all provide mx.__init__.py.
Some type of package forcing is going to have
to occur for all except the first subpackage=20
installed.
- OR -
2) Be created dynamically at "install" time. The
all-purpose setup.py can scan when Distutils
runs and programatically create an __init__.py
at installation time that meets the needs of
the currently installed subpacakges. BUT --
that doesn't help when I used --bdist-WHATEVER
again, because the resulting package is going
to be installed on a system that probably does
not have the same set of subpackages installed.
I'll have to provide a postinstall script to
WHATEVER to create __init__.py on the install
host. Do-able, but now I'm manually tacking
stuff on to the package that --bdist-WHATEVER=20
can't (unless I missed it) handle. And,=20
mx.__init__.py is not registered to the package
somewhat defeating the purpose.
It looks like --bdist-WHATEVER needs information about
pre- and post-installation processing. This would have
to be included in setup.py somewhere. But what should
it be? Package managers (that support pre/post install)
expect shell scripts. It would be really nice if --bdist
took a python function and wrapped it for us in a shell
wrapper, e.g.
#!/bin/sh
python -i <<EOF
def postinstall():
=09 ...
postinstall
EOF
So, am I missing something or are these issues real?
FWIW, I've included the directory scanning setup.py
below. If you try it, it "almost" works for mxDateTime,=20
mxTextTools, and mxNewBuiltins. There's still some
issues with mx sub-modules that I'm ignorant of, but
they're not related to this discussion.
Mark
mwa@gate.net
############## mx.DateTime setup.py ################
#!/usr/bin/env python
import os
"""=09setup.py for Marc-Andr=E9 Lemburg's mx Extension modules
=09Will scan the 'mx' directory for Marc's packages=20
=09(just DateTime, TextTools, and NewBuiltins so far....)=09
=09and provide the Distutils infor for all found packages.
."""
# created 1999/09/19, Greg Ward
# pushed to mx package by Mark Alexander
__revision__ =3D "$Id: mxdatetime_setup.py,v 1.4 2000/03/02 01:49:46 gward =
Exp $"
from distutils.core import setup
DateTimePackages=3D['mx.DateTime', 'mx.DateTime.Examples', 'mx.DateTime.mxD=
ateTime']
DateTimeExtensions=3D[('mx.DateTime.mxDateTime.mxDateTime',
{ 'sources': ['mx/DateTime/mxDateTime/mxDateTime.c']=
,
'include_dirs': ['mxDateTime'],
'macros': [('HAVE_STRFTIME', None),
('HAVE_STRPTIME', None),
('HAVE_TIMEGM', None)], }
)]
TextToolsPackages=3D['mx.TextTools','mx.TextTools','mx.TextTools.Constants'=
,'mx.TextTools.Examples']
TextToolsExtensions=3D[('mx.TextTools.mxTextTools.mxTextTools',
{ 'sources': ['mx/TextTools/mxTextTools/mxTextTools.=
c'],
'include_dirs': ['mxTextTools'],
'macros': [('HAVE_STRFTIME', None),
('HAVE_STRPTIME', None),
('HAVE_TIMEGM', None)], }
)]
NewBuiltinsPackages=3D['mx.NewBuiltins','mx.NewBuiltins.Examples','mx.NewBu=
iltins.mxTools']
NewBuiltinsExtensions=3D[('mx.NewBuiltins.mxTools.mxTools',
{ 'sources': ['mx/NewBuiltins/mxTools/mxTools.c'],
'include_dirs': ['NewBuiltins'],
'macros': [('HAVE_STRFTIME', None),
('HAVE_STRPTIME', None),
('HAVE_TIMEGM', None)], }
),
('mx.NewBuiltins.mxTools.xmap',
{ 'sources': ['mx/NewBuiltins/mxTools/xmap.c'],
'include_dirs': ['NewBuiltins'],
'macros': [('HAVE_STRFTIME', None),
('HAVE_STRPTIME', None),
('HAVE_TIMEGM', None)], }
)]
mxPackages=3D['mx']
mxExtensions=3D[]
if os.path.isdir('mx/DateTime'):
mxPackages=3DmxPackages+DateTimePackages
mxExtensions=3DmxExtensions+DateTimeExtensions
if os.path.isdir('mx/TextTools'):
mxPackages=3DmxPackages+TextToolsPackages
mxExtensions=3DmxExtensions+TextToolsExtensions
if os.path.isdir('mx/NewBuiltins'):
mxPackages=3DmxPackages+NewBuiltinsPackages
mxExtensions=3DmxExtensions+NewBuiltinsExtensions
setup (name =3D "mxDateTime",
version =3D "1.3.0",
description =3D "",
author =3D "Marc-Andr=E9 Lemburg",
author_email =3D "mal@lemburg.com",
url =3D "http://starship.python.net/~lemburg/mxDateTime.html",
packages =3D mxPackages,
# XXX user might have to edit the macro definitions here: yuck!
# Probably do need to support 'Setup' file or something similar.
ext_modules =3D mxExtensions,
)