Mailman 3 May 1999 - Distutils-SIG

Package versioning
by Timothy Docker April 29, 2018

April 29, 2018

Hi all, I've been aware that the distutils sig has been simmerring away, but until recently it has not been directly relevant to what I do. I like the look of the proposed api, but have one question. Will this support an installed system that has multiple versions of the same package installed simultaneously? If not, then this would seem to be a significant limitation, especially when dependencies between packages are considered. Assuming it does, then how will this be achieved? I am presently managing this with a messy arrangement of symlinks. A package is installed with its version number in it's name, and a separate directory is created for an application with links from the unversioned package name to the versioned one. Then I just set the pythonpath to this directory. A sample of what the directory looks like is shown below. I'm sure there is a better solution that this, and I'm not sure that this would work under windows anyway (does windows have symlinks?). So, has this SIG considered such versioning issues yet? Cheers, Tim -------------------------------------------------------------- Tim Docker timd(a)macquarie.com.au Quantative Applications Division Macquarie Bank -------------------------------------------------------------- qad16:qad $ ls -l lib/python/ total 110 drwxr-xr-x 2 mts mts 512 Nov 11 11:23 1.1 -r--r----- 1 root mts 45172 Sep 1 1998 cdrmodule_0_7_1.so drwxr-xr-x 2 mts mts 512 Sep 1 1998 chart_1_1 drwxr-xr-x 3 mts mts 512 Sep 1 1998 Fnorb_0_7_1 dr-xr-x--- 3 mts mts 512 Nov 11 11:21 Fnorb_0_8 drwxr-xr-x 3 mts mts 1536 Mar 3 12:45 mts_1_1 dr-xr-x--- 7 mts mts 512 Nov 11 11:22 OpenGL_1_5_1 dr-xr-x--- 2 mts mts 1024 Nov 11 11:23 PIL_0_3 drwxr-xr-x 3 mts mts 512 Sep 1 1998 Pmw_0_7 dr-xr-x--- 2 mts mts 512 Nov 11 11:21 v3d_1_1 qad16:qad $ ls -l lib/python/1.1 total 30 lrwxrwxrwx 1 root other 29 Apr 10 10:43 _glumodule.so -> ../OpenGL_1_5_1/_glumodule.so lrwxrwxrwx 1 root other 30 Apr 10 10:43 _glutmodule.so -> ../OpenGL_1_5_1/_glutmodule.so lrwxrwxrwx 1 root other 22 Apr 10 10:43 _imaging.so -> ../PIL_0_3/_imaging.so lrwxrwxrwx 1 root other 36 Apr 10 10:43 _opengl_nummodule.so -> ../OpenGL_1_5_1/_opengl_nummodule.so lrwxrwxrwx 1 root other 27 Apr 10 10:43 _tkinter.so -> ../OpenGL_1_5_1/_tkinter.so lrwxrwxrwx 1 mts mts 21 Apr 10 10:43 cdrmodule.so -> ../cdrmodule_0_7_1.so lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 chart -> ../chart_1_1 lrwxrwxrwx 1 root other 12 Apr 10 10:43 Fnorb -> ../Fnorb_0_8 lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 mts -> ../mts_1_1 lrwxrwxrwx 1 root other 15 Apr 10 10:43 OpenGL -> ../OpenGL_1_5_1 lrwxrwxrwx 1 root other 33 Apr 10 10:43 opengltrmodule.so -> ../OpenGL_1_5_1/opengltrmodule.so lrwxrwxrwx 1 root other 33 Apr 10 10:43 openglutil_num.so -> ../OpenGL_1_5_1/openglutil_num.so lrwxrwxrwx 1 root other 10 Apr 10 10:43 PIL -> ../PIL_0_3 lrwxrwxrwx 1 mts mts 10 Apr 10 10:43 Pmw -> ../Pmw_0_7 lrwxrwxrwx 1 root other 10 Apr 10 10:43 v3d -> ../v3d_1_1

3 2

[Distutils-sig] distutils charter and interscript
by John Skaller April 29, 2018

April 29, 2018

3 2

complete GNU readline support, packaging issues
by Ovidiu Predescu June 2, 1999

June 2, 1999

Hi, I've just started with Python and I discovered that the binding to the GNU readline is really basic. So about a week ago I started working on a binding for it and I have most of the work done. I need to finish the bindings for keymap functions and some of the functions in the completion part before the package could be considered complete. I started to investigate the ways to package the work I've done and I discovered the distutil package. I'm trying to write a setup.py file, but I need the following things and I didn't figure out how to express them: - the main C file is obtained by running the m4 program on a .m4 file. How can I specify this dependency and the rule for generating the C file? - I need to define a C preprocessor macro that contains the version of the readline library. The way the things are setup now is by running a configure script that determines the version of the library. Can I do this with setup.py? Thanks, -- Ovidiu Predescu <ovidiu(a)cup.hp.com> http://www.geocities.com/SiliconValley/Monitor/7464/

2 1

extensions in packages
by hinsen＠dirac.cnrs-orleans.fr June 1, 1999

June 1, 1999

Just van Rossum wrote: > foo.bar is registered as a "builtin" in config.c file as > > {"foo.bar", initbar}, > > (Hm, this is problemetic if the is a distinct global builtin module "bar") Or if any other package has a module "bar"! > find_module() should then first check sys.builtin_module_names with the > full name before doing anything else. (probably only when it is confirmed > that "foo" is a package.) All that would be doable, but the real problem is the name of the init function! Only one module can define a global symbol "initbar". So the one for foo.bar would have to be called "initfoo.bar" (or something similar). On the other hand, when the same module is used dynamically, the init function must be called "initbar" again (unless the current import mechanism is changed). Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen(a)cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais -------------------------------------------------------------------------------

8 20

Re: [Distutils] Packages with C extensions
by Perry A. Stoll May 25, 1999

May 25, 1999

> M.-A. Lemburg <mal(a)lemburg.com> writes: >[Problem with dynamic extensions in packages being platform dependent] > >I haven't followed the thread too closely, but was alarmed by >the recent proposals of splitting .so files out of the "normal" >package distribution under a separate dir tree. Fair enough. There's the the GNU configure view of life, where $prefix and $exec_prefix are separate directories, and there is the perl view of life $PERL_ARCHLIB is usually a subdirectory of the install directory). M.A. prefers the perl-ish approach. Fine with me, as long as we do it explicitly. > (you can't have two top-level packages with the same name >on the path: only the first one on the path will be used). That's only because that's how it's done today. Just a matter of some code...(and the thought and design behind it). > [ snipped scheme for having packages do the platform specific import ] I'd rather not burden the package writer. I think it's better to include the batteries for this one. > [recommendation that you just have a different install dir for each platform ] > Disk space is no argument nowadays Ease of maintenance is the overriding argument here. The .py files are the same for all platforms so why do I want different copies of those files when I have python install for three platforms? > its likely that different platforms need different Setup files anyway. But that's a platform dependent file which goes in the $INSTALL_ARCHLIB. In short, I think we need to get this infrastructure into Python itself to ease the creation of package authors. But then I'm probably preaching to the choir. -Perry

2 1

Re: [Distutils] Re: [PSA MEMBERS] packages in Python
by Perry A. Stoll May 21, 1999

May 21, 1999

> I think in this case the only problem is getting all the right >directories on the package's __path__; am I missing something? It >avoids the need for the "conditional" import, but that's largely >separate. Fred, Good point. Can you recommend a concise place that the import mechanism (in all it's glory) is documented? That should solve the problem, except for when freeze-ing or making a static python binary (as previously mentioned by Konrad). I was poking around in ihooks.py. It looks like it should be possible to cook up something approximating this using ihooks. What do you think? -Perry

2 1

Re: [Distutils] Re: [PSA MEMBERS] packages in Python
by Perry A. Stoll May 21, 1999

May 21, 1999

>> The problem here is that the package's __path__ is not being created >> this way now; if anyone has time to work on a patch for >> Python/import.c, I'd be glad to help test it! ;-) >> >I take care of this in my extension module. If I have a platform dependent >implementation of a module I: > >try: > import extension >Fail: > use common I don't think this is the case that's causing problem. The problem is when a submodule on a package is *always* platform dependent (because, for example, it interfaces to another library). -Perry

2 1

Re: [Distutils] Re: [PSA MEMBERS] packages in Python
by Michel Sanner May 21, 1999

May 21, 1999

On May 21, 11:11am, Fred L. Drake wrote: > Subject: [Distutils] Re: [PSA MEMBERS] packages in Python > > Konrad Hinsen writes: > > Shouldn't that be the other way round? I'd expect to be able to override > > general modules by platform-specific modules. > > Greg Ward and I were talking about this stuff the other day, and I > think we decided that there was no good way to have multiple > implementations of a module installed such that the platform dependent > version was sure to take precedence over a platform independent > version; this relies on the sequence of directories in the relevant > search path (whether it be sys.path or a package's __path__). > The general solution seems to be that two things need to be done: a > package's __path__ needs to include *all* the appropriate directories > found along the search path, not just the one holding __init__.py*, > AND the platform dependent modules should have different names from > the platform independent modules. The platform independent module > should be the public interface, and it can load platform dependent > code the same way that string loads functions from strop. > The problem here is that the package's __path__ is not being created > this way now; if anyone has time to work on a patch for > Python/import.c, I'd be glad to help test it! ;-) > I take care of this in my extension module. If I have a platform dependent implementation of a module I: try: import extension Fail: use common This requires minimal amount of coding. Personally I did not see the need of this being automatic -Michel

2 1

Re: [Distutils] extensions in packages
by Michel Sanner May 21, 1999

May 21, 1999

On May 21, 5:07pm, Konrad Hinsen wrote: > > Which makes me wonder how others develop extension modules: I always > use a debugger at some point, and I haven't yet found one which lets > me set breakpoints in dynamic libraries that haven't been loaded yet! > After you import the .so you can set a break point. I do that all the time on my sgi uner dbx or cvd .. no problem. I also have about 10 extensions modules all of which use .so files ! -Michel

1 0

Re: [PSA MEMBERS] packages in Python
by Michel Sanner May 21, 1999

May 21, 1999

On May 21, 10:24am, M.-A. Lemburg wrote: > Subject: Re: [PSA MEMBERS] packages in Python > [Problem with dynamic extensions in packages being platform dependent] > > I haven't followed the thread too closely, but was alarmed by > the recent proposals of splitting .so files out of the "normal" > package distribution under a separate dir tree. This is really > not such a good idea because it would cause the package information > stored in the extension module to be lost (you can't have two > top-level packages with the same name on the path: only the first one > on the path will be used). > > Here is the scheme I would use: create a subpackage for the > extension and have it take care of importing the correct > shared lib for the platform Python is currently running on. > The libs themselves could be placed in plat-<platform> subdirs > of that subpackage and the __init__.py would then load the > shared lib using either a sys.path+__import__() hack or > thread safe via imp.load_dynamic(). > > An even simpler solution is installing the whole package under > .../python1.5/plat-<platform> separately for each supported > platform rather than putting it under site-packages. [Disk space > is no argument nowadays and its likely that different platforms > need different Setup files anyway.] > As someone who maintains Python for several unix based architectures I am not concerned with disk space but really file duplication with the obvious risc to run out of sync. Also, the plat-<platform> scheme is far from being able to capture the complexity of this world. SGI alone has 3 ABIs o32 n32 n64 multiplied by MIPS1, MIPS3, MIPS4 instruction sets yimes IRIX5.x, IRIX6.2, IRIX6.3, IRIX6.4, IRIX6.5 and many of these combinations are incompatible. ! Finally, why have a $prefix and a $ exec_prefix if it is not used to split plateform dependent stuff from platform independent. And we should really take this dot distutil-sig :) -Michel

1 0