Mailman 3 April 2009 - Distutils-SIG

Distutils and easy_install...
by Douglas Mayle April 16, 2009

April 16, 2009

Hey everyone, I'm having an annoying problem and I was directed here to see if you knew what could be done. I'm using distutils for my package instead of setuptools because it's a command line app, and the half second that setup tools adds to each launch for pkg_resource scanning is unacceptable. I use the scripts parameter, and it happily installs the script I expect and things are running along. If I try to use easy_install to install the package, however, (and more importantly, if a user of mine does) it seems that setuptools is monkeypatching the distutils module and replacing setup. This means that instead of just copying my script to bin, setuptools is creating it's own script that does a pkg_resource scan and then loads my script from the original location. Is there any way to ensure that I'm using the distutils.core.setup that I expect, and not the one that setuptools monkeypatches into place? Thanks, Douglas Mayle

2 2

Building eggs of Numpy for Python 2.4 on Win32
by Fadhley Salim April 16, 2009

April 16, 2009

Has anybody managed to build an egg from the Numpy official release binaries? The Numpy developers recon that it's possible to do so. My first attempt was not succsessfull: D:\workspace\numpy>easy_install -zm numpy-1.2.1-win32-superpack-python2.4.exe Processing numpy-1.2.1-win32-superpack-python2.4.exe error: numpy-1.2.1-win32-superpack-python2.4.exe is not a valid distutils Windows .exe Any ideas how I might be able to get this working? This email does not create a legal relationship between any member of the Cr=E9dit Agricole group and the recipient or constitute investment advice. The content of this email should not be copied or disclosed (in whole or part) to any other person. It may contain information which is confidential, privileged or otherwise protected from disclosure. If you are not the intended recipient, you should notify us and delete it from your system. Emails may be monitored, are not secure and may be amended, destroyed or contain viruses and in communicating with us such conditions are accepted. Any content which does not relate to business matters is not endorsed by us. Calyon is authorised by the Comit=e9 des Etablissements de Cr=e9dit et des Entreprises d'Investissement (CECEI) and supervised by the Commission Bancaire in France and subject to limited regulation by the Financial Services Authority. Details about the extent of our regulation by the Financial Services Authority are available from us on request. Calyon is incorporated in France with limited liability and registered in England & Wales. Registration number: FC008194. Registered office: Broadwalk House, 5 Appold Street, London, EC2A 2DA. This message and/or any attachments is intended for the sole use of its addressee. If you are not the addressee, please immediately notify the sender and then destroy the message. As this message and/or any attachments may have been altered without our knowledge, its content is not legally binding on CALYON Crédit Agricole CIB. All rights reserved. Ce message et ses pièces jointes est destiné à l'usage exclusif de son destinataire. Si vous recevez ce message par erreur, merci d'en aviser immédiatement l'expéditeur et de le détruire ensuite. Le présent message pouvant être altéré à notre insu, CALYON Crédit Agricole CIB ne peut pas être engagé par son contenu. Tous droits réservés.

2 1

Re: [Distutils] How to make easy_install handle platlibs?
by P.J. Eby April 15, 2009

April 15, 2009

At 08:53 PM 4/10/2009 -0700, Buck wrote: >I see the kernel version and architecture, but this is insufficient; >RedHat 4 and RedHat 5 both use a 2.6 kernel, but the difference in >provided libraries are sufficient to make many (most?) "impure" >libraries stop working ( numpy, python-ldap, and hashlib for >examples). > >easy_install apparently knows when packages are platform-dependant, >and the necessary directory is sys.exec_prefix (or maybe better: >distutils.sysconfig.get_python_lib(plat_specific=1) ), so this seems >like an easy change. Would a patch be accepted? It would be a pretty major patch, since the distutils logic that's used for determining the installation directory is applied long before easy_install even knows what it's installing (and therefore, whether it's platform dependent). It would probably be a lot easier to improve the platform string generation and comparison logic, as has been done for OS X.

8 11

Re: [Distutils] Should metadata known about compatibility?
by Tarek Ziadé April 14, 2009

April 14, 2009

Oops, I got off list by accident.. On Tue, Apr 14, 2009 at 9:27 PM, Hanno Schlichting <hannosch(a)hannosch.eu> wrote: > Hi Tarek, > > Tarek Ziadé wrote: >> On Tue, Apr 14, 2009 at 9:07 PM, Hanno Schlichting <hannosch(a)hannosch.eu> wrote: >>> Hi, >>> >>> I'd like to suggest that distutils metadata should gain a way of >>> specifying compatibility with packages in addition to the requirement of >>> packages. >>> >>> What are the use cases? >>> >>> - A package wants to express that it works with both Python 2 and 3. >> >> You may use the classifiers for this use case (Martin has added soem trove >> classifier for python 2/3) > > Right, which restricts this system to Python itself and won't make it > possible to use it for other packages. Adding classifiers for each major > version of each framework seems tedious. > >>> - A package wants to express that it works with TurboGears 2 or Plone 4 >> >> How it would be different from install_requires (that can point to Plone==3.xx) > > Many plug-ins would be compatible with both Plone 3.x and Plone 4.x. So > they only require Plone >= 3. The additional information is "works with > Plone 3" and "Plone 4". I think the classifiers works pretty well there. (If we look at your use cases, and the classifiers browsing features at PyPI) The only problem I can see is the fact that they have to be added "manually" at PyPI. If we put apart this problem, do you see any remaining problem ? Cheers Tarek

2 1

Should metadata known about compatibility?
by Hanno Schlichting April 14, 2009

April 14, 2009

Hi, I'd like to suggest that distutils metadata should gain a way of specifying compatibility with packages in addition to the requirement of packages. What are the use cases? - A package wants to express that it works with both Python 2 and 3. - A package wants to express that it works with TurboGears 2 or Plone 4 >From my point of view this information is most useful for framework-like packages, when those do major backwards incompatible changes to their code base. In those circumstances typical question that people want to have answers to are: - For a press release of Python or a Framework, how many packages in PyPi already support the new version? - Running a particular application consisting of 200 packages, can I upgrade to the next version of Framework X? This type of information would always be optional and won't ever be used for all packages. I think it could be nonetheless helpful in many circumstances. On a side note to express this information for Python itself, it would be a good idea to expose the Python version in a distutils way, so I could also write: install_requires=['Python >= 3.0'] Comments? Hanno P.S. We want this kind of behavior for Plone, to give "plugins" to the application / framework the ability to express this information.

1 0

Re: [Distutils] PEP 376 - install/uninstall script in Distutils ?
by P.J. Eby April 13, 2009

April 13, 2009

At 11:23 PM 4/13/2009 +0200, Tarek Ziadé wrote: >Hello > >I am working on PEP 376 ><http://svn.python.org/projects/peps/trunk/pep-0376.txt>, and there's >a part about adding an install and uninstall script into Distutils. By the way, the PEP would benefit from some clarifications and separation of terminology between "project" names and "package" names. AFAICT, it speaks of "packages" to mean both Python packages and a distribution of a Python project. Also, does pip really create a .pth file per installed package (project?)? (Btw, I'm also not clear on the purpose of including MANIFEST in .egg-info.)

1 0

Re: [Distutils] How to make easy_install handle platlibs?
by P.J. Eby April 13, 2009

April 13, 2009

At 11:16 AM 4/13/2009 -0700, Buck wrote: >On Apr 12, 8:51 am, Tres Seaver <tsea...(a)palladion.com> wrote: > > zooko wrote: > > >> It would probably be a lot easier to improve the platform string > > >> generation and comparison logic, as has been done for OS X. > > > > > However, it (egg naming scheme on Linux) currently doesn't. > > > Eggs built on Linux are named something like py2.5-Linux-x86_64. > > > ... > > > > Better: just don't distribute binary eggs for Linux at all: > > > > - Developers have the tools (or can install them) to build from > > 'sdists'. > > > > - Systems without tools are "locked down", which means that they > > shouldn't be installing from public distributions anyway: > > the folks who run them should be able to build *private* eggs > > from 'sdists' which are known to work for their systems. > > > > Tres. > >I have no clue what you mean by 'sdists'. Is this a widely-known >thing? >A URL to an example would be sufficient. An sdist is a source distribution, usually in .tgz or .zip form. Sdists contain certain well-defined directory layouts and files. In particular: * the entire contents are in a subdirectory named for the project and version * that subdirectory contains a setup.py and a PKG-INFO easy_install and pip can both find and retrieve sdists from PyPI and install them, they just go about the actual installation process a bit differently.

1 0

Re: [Distutils] How to make easy_install handle platlibs?
by P.J. Eby April 13, 2009

April 13, 2009

At 11:11 AM 4/13/2009 -0700, Buck wrote: >On Apr 12, 12:46 pm, "P.J. Eby" <p...(a)telecommunity.com> wrote: > > >On Sun, Apr 12, 2009 at 8:54 AM, Andrew Straw > > ><<mailto:straw...@astraw.com>straw...(a)astraw.com> wrote: > > >On the other hand, sticking the egg into the place that distutils > > >uses when not under easy_install would fix this much more simply, > > >although from what I hear this would be a big change. > > > > For easy_install, yes. For pip, probably not so much. In fact, my > > guess would be that pip already supports this, as long as you can > > install from source, rather than from eggs. > >Are you saying I should dump easy_install in favor of pip? I hadn't >heard of it till now. If it meets your other use cases, I would say yes.

1 0

Re: [Distutils] How to make easy_install handle platlibs?
by P.J. Eby April 13, 2009

April 13, 2009

At 10:43 AM 4/12/2009 -0700, Buck Golemon wrote: >On Sun, Apr 12, 2009 at 8:54 AM, Andrew Straw ><<mailto:strawman@astraw.com>strawman(a)astraw.com> wrote: >zooko wrote: > > However, it currently doesn't. Eggs built on Linux are named something > > like py2.5-Linux-x86_64. To know whether such an egg would actually > > work on your Linux system, you would also need to know whether the > > Python was compiled with UCS-2 or UCS-4 internal unicode representation, > > as well as what version of glibc you have. Is there anything else that > > would need to be added into the egg name? > >Yes, if you used symbols from any shared library in an extension module, >you'd need to know the version of that shared library. So it's not just >libc. This is the same on any OS, not just linux. > >-Andrew > > >Exactly. The platform name/version ( RedHat 3 / Ubuntu 7.07 / Gentoo >X.X ) can serve as a much better proxy for the "version of all >shared libraries" than just the kernel version alone (2.4 / 2.6). > >To add to Andrew's example, python-ldap created for Redhat 4 depends >on ldap_r.so.1.0.4 but for RedHat 5 depends on ldap_r.so.1.0.6. Both >have kernel 2.6, so are indistinguishable under current naming scheme. > >I notice that for Debian/Ubuntu the lsb_release command is >implemented in Python, although it's mostly just reading out >/etc/debian_version. RedHat's lsb_release uses bash, but similarly >just reads in a file. Gentoo also has a lsb_release. On OS X, there's a similar thing done to get the platform name/version, and there's special version comparison code to determine compatibility; if this could be implemented for other platforms, that would be great. >The "completely correct" way to do this would be to run ldd on all >binaries, stick the result in a metadata file somewhere, then load >the version of the egg where the most (hopefully all) libraries exist. > >On the other hand, sticking the egg into the place that distutils >uses when not under easy_install would fix this much more simply, >although from what I hear this would be a big change. For easy_install, yes. For pip, probably not so much. In fact, my guess would be that pip already supports this, as long as you can install from source, rather than from eggs.

2 1

RFC: Updating PEP 345
by Tres Seaver April 13, 2009

April 13, 2009

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I haven't heard back from Richard Jones, who is the PEP owner, but have perpared to update PEP 345[1] in line with what to be the majority view (at PyCon) for adding "distribution-level" dependency metadata (as opposed to importable package / module dependencies) to the PKG-INFO file. I have backed off on the notion of overloading 'Requires:' / 'Provides:' / 'Obsoletes:', following Jim's notion of deprecating them in favor of new fields. I named them 'Requires-Dist:', 'Provides-Dist:', and 'Obsoletes-Dist'. "Stock" distutils should probably spell the arguments to distutils.core.setup predictably: 'requires_dist', 'provides_dist', 'obsoletes_dist'. setuptools can treat 'install_requires' as an undeprecated alias for 'requires_dist'. [1] http://svn.python.org/view/peps/trunk/pep-0345.txt?view=markup Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver(a)palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJ3iiI+gerLs4ltQ4RAtVLAKCZsXVoS9a0f0B0o8eAErsPmZP6OACePvJ7 8nx3i8Z4LEOsj0SsNczDtAk= =l7Kk -----END PGP SIGNATURE-----

9 37