Distutils-SIG May 2010

distutils-sig@python.org

46 participants
72 discussions

Re: [Distutils] Having a Python package install itself under a different name
by P.J. Eby 03 Jun '10

03 Jun '10

At 01:11 AM 5/28/2010 +0200, cool-RR wrote: >The thing I want the most right now is anyone's opinion of what >could possibly go wrong with the scheme that P.J. offered, before I >start moving my codebase to it. It's quite routine. Certainly, few of my PyPI projects have the same project and package name. For example, 'Importing' contains the module 'peak.util.imports'. It's also done by many other packages I'm not involved with. For example, 'AeroCalc' is the PyPI name of a project that ships an 'aerocalc' package, 'AdjectorClient' ships various packages such as adjector.core and adjector.model... and those are just two things I found by randomly clicking on the A's in the giant list of PyPI packages! I'm sure there will be some found in every letter of the alphabet. ;-) The only thing that can go wrong is if somebody installs the Python 2 and Python 3 versions to the same installation location. However, as long as both versions alert the user when used on the wrong Python version, they will quickly discover their mistake. ;-)

2 2

Re: [Distutils] pip vs easy_install vs distutils2
by P.J. Eby 03 Jun '10

03 Jun '10

At 10:19 AM 5/29/2010 +0200, Tarek Ziadé wrote: >I was not thinking about this proposal. If this what Guido proposed at >the summit, then I misunderstood. I don't know what he proposed at the summit - I'm referring to the bootstrap script he wrote that actually does this. It was a couple years ago on Python-Dev, IIRC. >The problem with the feature you describe (bootstrap embed in the >archive itself) is that we imply that the packager chooses himself an >installer and forces the end-user to use it when he installs the >software. This means that the end user might end up with several >installers installed for the same Python. So? It's not like they're going to accidentally run 'easy_install' when they meant to type 'pip', or vice versa. ;-) >It also implies that the >developer provides a smart setup.py script that embeds that bootstrap, >and runs some code. Only if they need something beyond what the distutils provide. >It will also allow distributions to be "dumb" envelopes with static >metadata that are the same all the time, no matter which tool created >them, and eventually remove setup.py in favor of statically described >metadata using PEP 345. For packages with complex build requirements or distutils extensions (e.g. numpy), this is unlikely to happen any time soon. Conversely, for packages where this *is* the case, the current distutils is adequate, and having a bootstrapper that can install them would be a win. >Today, for instance, if an installer wants to install a distribution >based on setuptools, it has to run the "egg_info" command to extract >the metadata, on the target system. It's a good thing Guido loaned me the time machine so I could go back to 2005 and fix that one, then: http://svn.python.org/view/sandbox/trunk/setuptools/setuptools/command/sdis… Five years should be long enough ago that by now all the setuptools-based source distributions on PyPI today will already have the metadata you need... so when you go back and try again to build your new installer today, it will already be there for you to use. Just download any random setuptools-based sdist from PyPI and run "tar -tzf" or "unzip -v" on it to see! For that matter, it's long enough ago that you should also have the fix already retroactively included in Distribute as well, so you should probably even be able to look at one of Distribute's own source distributions and see that it now contains the metadata you require. ;-)

9 31

pip vs easy_install vs distutils2
by Tarek Ziadé 01 Jun '10

01 Jun '10

Hello, Distutils2 is going to be added back in Python (hopefully in 3.2) and without an install script, it's pretty useless as-is. We've discussed during the summit at Pycon to create some kind of bootstrap script in Python, to allow people to set up an installer of their choice, but I think it's a bad idea. = summary of the summit proposal = IIRC: - an "install" script is added in the scripts, for people to install distributions in Python. Could be called "pyinstall" for example. - when first used, it would ask the user to choose a third-party installer (like Pip). Then it would download it and install it with a simple "python setup.py install" - from there, the install script would be linked to that installer. If I recall it correctly, this feature was proposed to be able to have a "modern" installer in Python without including it in the standard library. (so it can have its own shorter release cycles). The bootstrap story would just make it easier for people to get an installer, without having to do extra manual steps. The problem I have with this approach is that we need to manage somewhere at PyPI a list of potential installers, and maybe deal with upgrades and replacements. Plus, I am not sure that a user will really understand what to do when he's asked to chose an installer. Sounds like something we should only ask to power users, and people that know what they are doing with p7g. So a bootstrap script is useless for them. = alternative proposal = Let's add that script but powered by Distutils2. It could be Pip if people from this project think it's a good idea and want to merge, or an easy_install derivation, or a new script from scratch. IOW: you get an installer for free in the stdlib without having to think. Now for the problem of the release cycle (e.g. once in the standard library it has to wait 18 months for a new version), I propose that Distutils2 allow the usage of a third party installer through configuration. IOW, Distutils2 would ship with an installer, but could use through a simple change in distutils.cfg, another one installed by a third party project that is more recent. For this to work, we can define an installer standard interface ala wsgi. Basically, we state that an installer has to implement a simple function that takes a name of a project to install, and an optional version predicate: def install(name, version=None): # if version is None, it means the latest one. ... This needs more work, uninstall is missing in that description, and what about the script options, etc. but you get the idea: make sure people can use the installer of their choice, if it turns out that the one provided by Distutils2 is not good enough anymore for any reason. Any opinion ? Regards, Tarek -- Tarek Ziadé | http://ziade.org

2 1

Re: [Distutils] pip vs easy_install vs distutils2
by P.J. Eby 31 May '10

31 May '10

At 09:12 PM 5/29/2010 +0200, Tarek Ziadé wrote: > > For packages with complex build requirements or distutils extensions (e.g. > > numpy), this is unlikely to happen any time soon. > > > > Conversely, for packages where this *is* the case, the current distutils is > > adequate, and having a bootstrapper that can install them would be a win. > >I don't understand where the win is in this case. That packages with complex build requirements will still work. See above.

2 1

Re: [Distutils] pip vs easy_install vs distutils2
by P.J. Eby 28 May '10

28 May '10

At 01:13 AM 5/29/2010 +0200, Tarek Ziadé wrote: >The problem I have with this approach is that we need to manage >somewhere at PyPI a list of potential installers, >and maybe deal with upgrades and replacements. Plus, I am not sure >that a user will really understand what to >do when he's asked to chose an installer. Sounds like something we >should only ask to power users, and >people that know what they are doing with p7g. So a bootstrap script >is useless for them. Actually, the way it would (presumably) work would be just a script like Guido's that downloads a source archive from PyPI and runs setup.py install. So, it's not really a matter of "choosing" an installer -- it'd just be, "download and install a package from source". If the package itself needs one of those other things in order to build itself or install dependencies, then the next stage of bootstrap is its problem (e.g. via ez_setup), not the stdlib's. So, I don't see the "power users only" objection making sense, at least if we drop back to what I believe was Guido's original proposal for a bootstrap script a few years ago.

1 0

Re: [Distutils] Having a Python package install itself under a different name
by P.J. Eby 28 May '10

28 May '10

At 08:00 PM 5/28/2010 +0200, cool-RR wrote: >On Fri, May 28, 2010 at 7:44 PM, P.J. Eby ><<mailto:pje@telecommunity.com>pje(a)telecommunity.com> wrote: >At 02:53 AM 5/28/2010 +0200, cool-RR wrote: > >On Fri, May 28, 2010 at 2:36 AM, P.J. Eby ><<mailto:pje@telecommunity.com><mailto:pje@telecommunity.com>pje(a)telecommunity.com> >wrote: >However, as long as both versions alert the user when used on the >wrong Python version, they will quickly discover their mistake. ;-) > >Yes, I thought about this too. You suggest doing that on the setup >script, the package itself, or both? > > >The package itself; doing it in the setup script is ridiculously >hard. (Not detecting the Python version, but detecting whether >you're installing both versions to the same location.) > > >I'm not sure we're on the same page here. My intention is that >Python 2.x users will use only `garlicsim`, and Python 3.x users >will use only `garlicsim_py3`. But both projects will contain a package named 'garlicsim' - so if they're installed to the same location, then either one will overwrite the other (distutils/pip install) or override the other (setuptools). In principle, they won't be installed to the same location, unless somebody has a directory that's used for both Python 2 packages and Python 3 packages -- and if they do that, they'll have other problems. (i.e., they shouldn't be doing it) > Why would I want to detect anything beyond the Python version in > the setup script? Even though you detect the version in the setup script, it won't stop the problem of installing both versions to the same location. That's why the package itself needs to give a good error message if imported from the wrong Python version, to handle the case where somebody's sys.path is wrong (e.g. due to a PYTHONPATH that includes both Python 2 and 3 packages). Again, people are going to have other problems if they do that, unrelated to your package. Putting in the check is just a courtesy measure so they don't have to spend a lot of time figuring out what they did wrong.

2 1

Re: [Distutils] Having a Python package install itself under a different name
by P.J. Eby 28 May '10

28 May '10

At 02:53 AM 5/28/2010 +0200, cool-RR wrote: >On Fri, May 28, 2010 at 2:36 AM, P.J. Eby ><<mailto:pje@telecommunity.com>pje(a)telecommunity.com> wrote: >However, as long as both versions alert the user when used on the >wrong Python version, they will quickly discover their mistake. ;-) > >Yes, I thought about this too. You suggest doing that on the setup >script, the package itself, or both? The package itself; doing it in the setup script is ridiculously hard. (Not detecting the Python version, but detecting whether you're installing both versions to the same location.)

3 5

Re: [Distutils] pkginfo python 3 port
by Sridhar Ratnakumar 28 May '10

28 May '10

On 2010-05-27, at 10:06 AM, Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Sridhar Ratnakumar wrote: >> For a start, how about this patch? >> http://gist.github.com/415137 >> >> `email.parser` is available till 2.5; not sure about <=2.4 though. > > Your patch does break with Python 2.4, which I would like to continue > supporting. Maybe we can add some more conditional imports and glue > functions? Something like the attached patch. Sounds good. > BTW, your patch also breaks a unit test, due to differences in behavior > between the rfc822 parser and the email one: > > ====================================================================== > FAIL: test_parse_Description > (pkginfo.tests.test_distribution.DistributionTests) > - ---------------------------------------------------------------------- > Traceback (most recent call last): > File > "/home/tseaver/projects/parcel/src/pkginfo/pkginfo/tests/test_distribution.py", > line 104, in test_parse_Description > 'This package enables integration with\n' > AssertionError: 'This package enables integration with\n foo > servers.' != 'This package enables integration with\n foo servers.' Hmm. RFC 822 (rfc822) and RFC 2822 have different "unfolding" rules for multiple line header. Curiously http://docs.python.org/library/rfc822.html is advertised as "Parse RFC 2822 mail headers," and yet it seems to unfold multiple lines in accordance with RFC 822. I don't know what the solution to this problem is. I see that lines in the `Description` field (in PKG-INFO) are preceded with 8 spaces. So PKG-INFO is generated in accordance with RFC 822. Is there a way to parse a RFC 822 message in Python 3? -srid

3 5

Unexpected error <python setup.py install_data --root /tmp>
by Kent Tenney 28 May '10

28 May '10

In a docutils svn checkout. [docutils/trunk/docutils]$ python setup.py install --root /tmp OK [docutils/trunk/docutils]$ python setup.py install_data --root /tmp distutils.errors.DistutilsFileError: could not delete '/usr/local/lib/python2.6/dist-packages/docutils/parsers/rst/include/README.txt': Permission denied What am I missing here? Thanks, Kent

3 3

Re: [Distutils] Having a Python package install itself under a different name
by P.J. Eby 27 May '10

27 May '10

At 11:16 PM 5/27/2010 +0200, cool-RR wrote: >On Thu, May 27, 2010 at 11:03 PM, P.J. Eby ><<mailto:pje@telecommunity.com>pje(a)telecommunity.com> wrote: >At 10:40 PM 5/27/2010 +0200, cool-RR wrote: >Is there a way that `garlicsim_py3` will install itself under the >alias `garlicsim`? What I want is for a Python 3 user to be able to >`import garlicsim` and refer to the module all the time as >`garlicsim`, but that it will really be `garlicsim_py3`. > > >Why not just develop them both with the same package name? Your >project name (in setup(name='...')) can still be different. > > >Oh, I didn't even know that was possible. That sounds great, and I >wish I would have known about this option 6 months ago. > >If this works well, I'll start converting my entire codebase to this >method. Before I jump in, is there anything important I should keep >in mind, any possible problems? > >Also, this is plain setuptools, right? I mean, not requiring Distribute. It works with plain distutils, too. Just use setup(name='garlicsym_py3', packages=['garlicsym']) in your Python 3 branch of setup.py, and setup(name='garlicsym', packages=['garlicsym']) in your Python 2 branch.

1 0

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

Distutils-SIG May 2010