From ziade.tarek at gmail.com Wed Jun 1 10:27:16 2011 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 1 Jun 2011 10:27:16 +0200 Subject: [Distutils] Distribute 0.6.17 In-Reply-To: References: Message-ID: Aur?lien has fixed the issue: https://bitbucket.org/tarek/distribute/changeset/191f38f47256 Could someone double-check by running the tip, that everything is now working fine ? I'll release 0.6.18 right after Thanks On Tue, May 31, 2011 at 9:40 PM, Jeff Shell wrote: > I've seen the same issue in Mac OS X with namespace packages that are > more than one level deep. 'zope.server' works fine, 'zope.app.server' > (or any other 'zope.app.*') blows up. Works fine with distribute > 0.6.16, blows up with 0.6.17. > > On Tue, May 31, 2011 at 4:04 AM, Tarek Ziad? wrote: >> On Tue, May 31, 2011 at 11:50 AM, Maurits van Rees >> wrote: >> ... >>> ImportError: cannot import name utils >>> >>> When I go in with a pdb in this utils.py everything seems fine but then the >>> import error just happens a bit further on, failing to import >>> plone.app.layout. ?On Plone 4 an import of Shared.DC.ZRDB.Search fails. >>> ?Again, with distribute 0.6.16 it works fine. >>> >>> This is on Mac OS X. ?Possibly I have strange ways of installing python, but >>> it has worked fine so far. ?Can anyone else reproduce this? >> >> Could you dump and compare sys.path in both cases ? thanks > > -- > Jeff Shell > -- Tarek Ziad? | http://ziade.org From m.van.rees at zestsoftware.nl Wed Jun 1 12:01:34 2011 From: m.van.rees at zestsoftware.nl (Maurits van Rees) Date: Wed, 01 Jun 2011 12:01:34 +0200 Subject: [Distutils] Distribute 0.6.17 In-Reply-To: References: Message-ID: Op 01-06-11 10:27, Tarek Ziad? schreef: > Aur?lien has fixed the issue: > https://bitbucket.org/tarek/distribute/changeset/191f38f47256 > > Could someone double-check by running the tip, that everything is now > working fine ? > > I'll release 0.6.18 right after Yes, this works now for me. Thank you both! -- Maurits van Rees Web App Programmer at Zest Software: http://zestsoftware.nl Personal website: http://maurits.vanrees.org/ From ziade.tarek at gmail.com Wed Jun 1 12:22:34 2011 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 1 Jun 2011 12:22:34 +0200 Subject: [Distutils] Distribute 0.6.17 In-Reply-To: References: Message-ID: On Wed, Jun 1, 2011 at 12:01 PM, Maurits van Rees wrote: > Op 01-06-11 10:27, Tarek Ziad? schreef: >> >> Aur?lien has fixed the issue: >> https://bitbucket.org/tarek/distribute/changeset/191f38f47256 >> >> Could someone double-check by running the tip, that everything is now >> working fine ? >> >> I'll release 0.6.18 right after > > Yes, this works now for me. ?Thank you both! 0.6.18 is released. Thanks to all for the tests, and to Aur?lien for the fix. Cheers Tarek -- Tarek Ziad? | http://ziade.org From tseaver at palladion.com Wed Jun 1 15:12:50 2011 From: tseaver at palladion.com (Tres Seaver) Date: Wed, 01 Jun 2011 09:12:50 -0400 Subject: [Distutils] [buildout] [Errno 104] Connection reset by peer In-Reply-To: <45760E6A-E3D0-47AD-B975-EA4D28FCEB9C@gmail.com> References: <45760E6A-E3D0-47AD-B975-EA4D28FCEB9C@gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 05/30/2011 03:13 PM, Anton Panasenko wrote: > Hi guys, > > Who knows how to fix the problem: > > Download error: [Errno 104] Connection reset by peer -- Some packages may not be found! This error indicates either that you have a transient network failure, or that your index server (likely pypi.python.org) is down. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk3mOtIACgkQ+gerLs4ltQ6BpQCfQBUP1wtRYo8nNf0KHyoOHEVN dN8AnAyaefXWkvyjjtffZR+CrqhC+y81 =jJ77 -----END PGP SIGNATURE----- From justrandombytes at gmail.com Tue Jun 7 09:00:38 2011 From: justrandombytes at gmail.com (James) Date: Tue, 7 Jun 2011 00:00:38 -0700 Subject: [Distutils] does easy_install.py need to guess the site-packages path? Message-ID: As an example I'm looking at setuptools-0.6c11-py2.7.egg, can someone tell me why setuptools/command/east_install.py in get_site_dirs() does its own computation of where site-packages is? As of 2.7 there are already 4 separate places in the python where this path is computed. Maybe I'm missing something but I don't see why an egg would have an opinion on this matter. Regards, - James -------------- next part -------------- An HTML attachment was scrubbed... URL: From merwok at netwok.org Tue Jun 7 17:30:23 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Tue, 07 Jun 2011 17:30:23 +0200 Subject: [Distutils] DRAFT PEP 396 - module version number In-Reply-To: <20110412142313.34c79298@neurotica.wooz.org> References: <20110324141747.6d90382e@neurotica.wooz.org> <4D8C035E.4050306@netwok.org> <20110405103108.77f144a2@neurotica.wooz.org> <5c2331c6216e5085bdc02629b9a42986@netwok.org> <20110412142313.34c79298@neurotica.wooz.org> Message-ID: <4DEE440F.6000600@netwok.org> Hi, Just two things I thought about while perusing my archives yesterday. >>> #. For modules which are also packages, the module namespace > SHOULD >>> include the ``__version__`` attribute. >> >> I?m still not sure ?module namespace? will be clear to everyone. > > Really? We know what a module is, and we know what a namespace is, so given > the context, I think it should be clear. FYI, I noticed that PEP 382 (namespace packages) says ?the namespace package itself? to refer to the corp/__init__.py file of the corp.somelib package. In the Python tutorial, we have things like ?the __init__.py code?. So it looks like there?s no agreed term to refer to this module. A note about the setup.cfg field used to get the version from a file: It will definitely need to use another name than ?version?: Tarek wants the setup.cfg field to be a direct translation from PEP 345 (modulo case insensitivity), so the description-file is another field, and version-file should certainly be one too. (I hope it?s short enough to comply with your wish of keeping the common case simple.) Regards From erik.m.bray at gmail.com Tue Jun 7 20:48:38 2011 From: erik.m.bray at gmail.com (Erik Bray) Date: Tue, 7 Jun 2011 14:48:38 -0400 Subject: [Distutils] why does easy_install set +x on all .py files? Message-ID: Hi all, I've used setuptools for a long time, and yet I've only just noticed this behavior. It came up because I wanted nose to run some installed tests, and it was mysterious as to why it wasn't finding them. Debug output from nose revealed the answer: it was skipping searching those modules because they're exectuable. That's fine, because I can just add --exe to nose. But I was wondering why it is that easy_install sets 0755 on all files? Why not preserve the existing executable bit set on the file? Thanks, Erik From barry at python.org Tue Jun 7 23:17:06 2011 From: barry at python.org (Barry Warsaw) Date: Tue, 7 Jun 2011 17:17:06 -0400 Subject: [Distutils] DRAFT PEP 396 - module version number In-Reply-To: <4DEE440F.6000600@netwok.org> References: <20110324141747.6d90382e@neurotica.wooz.org> <4D8C035E.4050306@netwok.org> <20110405103108.77f144a2@neurotica.wooz.org> <5c2331c6216e5085bdc02629b9a42986@netwok.org> <20110412142313.34c79298@neurotica.wooz.org> <4DEE440F.6000600@netwok.org> Message-ID: <20110607171706.6977ffc1@neurotica.wooz.org> Hi ?ric, On Jun 07, 2011, at 05:30 PM, ?ric Araujo wrote: >Just two things I thought about while perusing my archives yesterday. > >>>> #. For modules which are also packages, the module namespace > SHOULD >>>> include the ``__version__`` attribute. >>> >>> I?m still not sure ?module namespace? will be clear to everyone. >> >> Really? We know what a module is, and we know what a namespace is, so given >> the context, I think it should be clear. > >FYI, I noticed that PEP 382 (namespace packages) says ?the namespace >package itself? to refer to the corp/__init__.py file of the >corp.somelib package. In the Python tutorial, we have things like ?the >__init__.py code?. So it looks like there?s no agreed term to refer to >this module. So, what *should* we call this thing? Note that in the PEP, I'm specifically talking about the namespace, not the file. Maybe "module's namespace" would be clearer? >A note about the setup.cfg field used to get the version from a file: It >will definitely need to use another name than ?version?: Tarek wants the >setup.cfg field to be a direct translation from PEP 345 (modulo case >insensitivity), so the description-file is another field, and >version-file should certainly be one too. (I hope it?s short enough to >comply with your wish of keeping the common case simple.) The PEP currently uses `version-from-file` which seems okay to me. Would that work? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From pje at telecommunity.com Wed Jun 8 03:30:23 2011 From: pje at telecommunity.com (P.J. Eby) Date: Tue, 07 Jun 2011 21:30:23 -0400 Subject: [Distutils] does easy_install.py need to guess the site-packages path? In-Reply-To: References: Message-ID: <20110608013029.F36203A4060@sparrow.telecommunity.com> At 12:00 AM 6/7/2011 -0700, James wrote: >As an example I'm looking at setuptools-0.6c11-py2.7.egg, can >someone tell me why setuptools/command/east_install.py in >get_site_dirs() does its own computation of where site-packages is? >As of 2.7 there are already 4 separate places in the python where >this path is computed. Maybe I'm missing something but I don't see >why an egg would have an opinion on this matter. It doesn't do that in order to know where to install things; it does it to know what places are *safe* to install things that need .pth support. (And unfortunately, there is no way to obtain this list from the site module directly.) From ben+python at benfinney.id.au Wed Jun 8 05:48:43 2011 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 08 Jun 2011 13:48:43 +1000 Subject: [Distutils] DRAFT PEP 396 - module version number References: <20110324141747.6d90382e@neurotica.wooz.org> <4D8C035E.4050306@netwok.org> <20110405103108.77f144a2@neurotica.wooz.org> <5c2331c6216e5085bdc02629b9a42986@netwok.org> <20110412142313.34c79298@neurotica.wooz.org> <4DEE440F.6000600@netwok.org> <20110607171706.6977ffc1@neurotica.wooz.org> Message-ID: <87vcwhyodg.fsf@benfinney.id.au> Barry Warsaw writes: > Hi ?ric, > > On Jun 07, 2011, at 05:30 PM, ?ric Araujo wrote: > > >Just two things I thought about while perusing my archives yesterday. > > > >>>> #. For modules which are also packages, the module namespace > >>>> SHOULD include the ``__version__`` attribute. > >>> > >>> I?m still not sure ?module namespace? will be clear to everyone. [?] Agreed. Damn English and its ample opportunities for ambiguity. > So, what *should* we call this thing? Note that in the PEP, I'm > specifically talking about the namespace, not the file. Maybe > "module's namespace" would be clearer? +1. I think the grammar could be even clearer: For modules which are also packages, the ``__version__`` attribute SHOULD be in the module's namespace. -- \ ?The opposite of a correct statement is a false statement. But | `\ the opposite of a profound truth may well be another profound | _o__) truth.? ?Niels Bohr | Ben Finney -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From fdrake at acm.org Wed Jun 8 05:56:36 2011 From: fdrake at acm.org (Fred Drake) Date: Tue, 7 Jun 2011 23:56:36 -0400 Subject: [Distutils] DRAFT PEP 396 - module version number In-Reply-To: <87vcwhyodg.fsf@benfinney.id.au> References: <20110324141747.6d90382e@neurotica.wooz.org> <4D8C035E.4050306@netwok.org> <20110405103108.77f144a2@neurotica.wooz.org> <5c2331c6216e5085bdc02629b9a42986@netwok.org> <20110412142313.34c79298@neurotica.wooz.org> <4DEE440F.6000600@netwok.org> <20110607171706.6977ffc1@neurotica.wooz.org> <87vcwhyodg.fsf@benfinney.id.au> Message-ID: On Tue, Jun 7, 2011 at 11:48 PM, Ben Finney wrote: > ? ?For modules which are also packages, the ``__version__`` attribute > ? ?SHOULD be in the module's namespace. This suggests to me that there's no need to describe it as a special case. A importable directory contains an __init__.py; it's represented as a module at runtime, just like any other. -Fred -- Fred L. Drake, Jr.? ? "Give me the luxuries of life and I will willingly do without the necessities." ?? --Frank Lloyd Wright From justrandombytes at gmail.com Wed Jun 8 06:54:16 2011 From: justrandombytes at gmail.com (James) Date: Tue, 7 Jun 2011 21:54:16 -0700 Subject: [Distutils] does easy_install.py need to guess the site-packages path? In-Reply-To: <20110608013029.F36203A4060@sparrow.telecommunity.com> References: <20110608013029.F36203A4060@sparrow.telecommunity.com> Message-ID: On Tue, Jun 7, 2011 at 6:30 PM, P.J. Eby wrote: > At 12:00 AM 6/7/2011 -0700, James wrote: > >> As an example I'm looking at setuptools-0.6c11-py2.7.egg, can someone tell >> me why setuptools/command/east_install.py in get_site_dirs() does its own >> computation of where site-packages is? As of 2.7 there are already 4 >> separate places in the python where this path is computed. Maybe I'm missing >> something but I don't see why an egg would have an opinion on this matter. >> > > It doesn't do that in order to know where to install things; it does it to > know what places are *safe* to install things that need .pth support. > > I'm seeing it make up the path and then fail in a copytree since the computed dest directory in my case doesn't exist So I'm not exactly sure what you mean when you say it doesn't. > (And unfortunately, there is no way to obtain this list from the site > module directly.) > I guess that is the nub, oh well. cheers, - James -------------- next part -------------- An HTML attachment was scrubbed... URL: From pje at telecommunity.com Wed Jun 8 08:09:30 2011 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 08 Jun 2011 02:09:30 -0400 Subject: [Distutils] does easy_install.py need to guess the site-packages path? In-Reply-To: References: <20110608013029.F36203A4060@sparrow.telecommunity.com> Message-ID: <20110608060947.A8DBA3A4060@sparrow.telecommunity.com> At 09:54 PM 6/7/2011 -0700, James wrote: >On Tue, Jun 7, 2011 at 6:30 PM, P.J. Eby ><pje at telecommunity.com> wrote: >At 12:00 AM 6/7/2011 -0700, James wrote: >As an example I'm looking at setuptools-0.6c11-py2.7.egg, can >someone tell me why setuptools/command/east_install.py in >get_site_dirs() does its own computation of where site-packages is? >As of 2.7 there are already 4 separate places in the python where >this path is computed. Maybe I'm missing something but I don't see >why an egg would have an opinion on this matter. > > >It doesn't do that in order to know where to install things; it does >it to know what places are *safe* to install things that need .pth support. > >I'm seeing it make up the path and then fail in a copytree since the >computed dest directory in my case doesn't exist So I'm not exactly >sure what you mean when you say it doesn't. Unless you override it, easy_install asks the distutils where to install things. So, the "made up path" you're referring to came from the distutils. (Which has yet *another* set of code to generate the path(s)!) If you set "DISTUTILS_DEBUG=yes" in your environment variables, and run the easy_install again, you'll see exactly where every setting comes from. The output of get_site_dirs() is *not* used to determine the installation path, only to *validate* it. >(And unfortunately, there is no way to obtain this list from the >site module directly.) > >I guess that is the nub, oh well. > >cheers, >- James From barry at python.org Thu Jun 9 00:14:03 2011 From: barry at python.org (Barry Warsaw) Date: Wed, 8 Jun 2011 18:14:03 -0400 Subject: [Distutils] DRAFT PEP 396 - module version number In-Reply-To: <87vcwhyodg.fsf@benfinney.id.au> References: <20110324141747.6d90382e@neurotica.wooz.org> <4D8C035E.4050306@netwok.org> <20110405103108.77f144a2@neurotica.wooz.org> <5c2331c6216e5085bdc02629b9a42986@netwok.org> <20110412142313.34c79298@neurotica.wooz.org> <4DEE440F.6000600@netwok.org> <20110607171706.6977ffc1@neurotica.wooz.org> <87vcwhyodg.fsf@benfinney.id.au> Message-ID: <20110608181403.2829d9c4@neurotica.wooz.org> Given Ben's and Fred's feedback, what do you think about this list in the Specification's section: #. In general, modules in the standard library SHOULD NOT have version numbers. They implicitly carry the version number of the Python release they are included in. #. On a case-by-case basis, standard library modules which are also released in standalone form for other Python versions MAY include a module version number when included in the standard library, and SHOULD include a version number when packaged separately. #. When a module (or package) includes a version number, the version SHOULD be available in the ``__version__`` attribute. #. For modules which live inside a namespace package, the sub-package name SHOULD include the ``__version__`` attribute. The namespace module itself SHOULD NOT include its own ``__version__`` attribute. #. The ``__version__`` attribute's value SHOULD be a string. #. Module version numbers SHOULD conform to the normalized version format specified in PEP 386 [6]_. #. Module version numbers SHOULD NOT contain version control system supplied revision numbers, or any other semantically different version numbers (e.g. underlying library version number). #. The ``version`` attribute in a classic distutils ``setup.py`` file, or the PEP 345 [7]_ ``Version`` metadata field SHOULD be derived from the ``__version__`` field, or vice versa. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From ben+python at benfinney.id.au Thu Jun 9 02:42:17 2011 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 09 Jun 2011 10:42:17 +1000 Subject: [Distutils] DRAFT PEP 396 - module version number References: <20110324141747.6d90382e@neurotica.wooz.org> <4D8C035E.4050306@netwok.org> <20110405103108.77f144a2@neurotica.wooz.org> <5c2331c6216e5085bdc02629b9a42986@netwok.org> <20110412142313.34c79298@neurotica.wooz.org> <4DEE440F.6000600@netwok.org> <20110607171706.6977ffc1@neurotica.wooz.org> <87vcwhyodg.fsf@benfinney.id.au> <20110608181403.2829d9c4@neurotica.wooz.org> Message-ID: <87ei33zvh2.fsf@benfinney.id.au> Barry Warsaw writes: > #. The ``__version__`` attribute's value SHOULD be a string. I may be fighting against the tide here; but this screams to me that the PEP should not be talking at all about ?version number? (except to point out that they're strings, not numbers). Instead, the term should be ?version string? throughout. Much damage is done to understanding version strings by calling them ?version numbers?, with all the baggage about parsing and sequencing etc. that entails. We don't have to make the same mistake in this PEP. -- \ ?He may look like an idiot and talk like an idiot but don't let | `\ that fool you. He really is an idiot.? ?Groucho Marx | _o__) | Ben Finney -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From ben+python at benfinney.id.au Thu Jun 9 02:52:31 2011 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 09 Jun 2011 10:52:31 +1000 Subject: [Distutils] DRAFT PEP 396 - module version number References: <20110324141747.6d90382e@neurotica.wooz.org> <4D8C035E.4050306@netwok.org> <20110405103108.77f144a2@neurotica.wooz.org> <5c2331c6216e5085bdc02629b9a42986@netwok.org> <20110412142313.34c79298@neurotica.wooz.org> <4DEE440F.6000600@netwok.org> <20110607171706.6977ffc1@neurotica.wooz.org> <87vcwhyodg.fsf@benfinney.id.au> <20110608181403.2829d9c4@neurotica.wooz.org> Message-ID: <87aadrzv00.fsf@benfinney.id.au> Barry Warsaw writes: > #. When a module (or package) includes a version number, the version > SHOULD be available in the ``__version__`` attribute. > > #. For modules which live inside a namespace package, the sub-package > name SHOULD include the ``__version__`` attribute. The namespace > module itself SHOULD NOT include its own ``__version__`` attribute. I still find a little ambiguity in ?the ``__version__`` attribute?, but it's much improved, thanks. I think those paragraphs are good. -- \ ?It is clear that thought is not free if the profession of | `\ certain opinions makes it impossible to earn a living.? | _o__) ?Bertrand Russell, _Free Thought and Official Propaganda_, 1928 | Ben Finney -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From fdrake at acm.org Thu Jun 9 04:47:50 2011 From: fdrake at acm.org (Fred Drake) Date: Wed, 8 Jun 2011 22:47:50 -0400 Subject: [Distutils] DRAFT PEP 396 - module version number In-Reply-To: <20110608181403.2829d9c4@neurotica.wooz.org> References: <20110324141747.6d90382e@neurotica.wooz.org> <4D8C035E.4050306@netwok.org> <20110405103108.77f144a2@neurotica.wooz.org> <5c2331c6216e5085bdc02629b9a42986@netwok.org> <20110412142313.34c79298@neurotica.wooz.org> <4DEE440F.6000600@netwok.org> <20110607171706.6977ffc1@neurotica.wooz.org> <87vcwhyodg.fsf@benfinney.id.au> <20110608181403.2829d9c4@neurotica.wooz.org> Message-ID: On Wed, Jun 8, 2011 at 6:14 PM, Barry Warsaw wrote: > #. For modules which live inside a namespace package, the sub-package > ? name SHOULD include the ``__version__`` attribute. ?The namespace > ? module itself SHOULD NOT include its own ``__version__`` attribute. I've no idea what you're saying here. What's a sub-package? If you're referring to a package like zope.testing, I'd just call that a package; there's nothing special about that. I'd expect the __version__, if it exists, to be present in the file zope/testing/__init__.py. An namespace package, like zope or zc, should not have a __version__. Ben Finney wrote: > I may be fighting against the tide here; but this screams to me that the > PEP should not be talking at all about ?version number? (except to point > out that they're strings, not numbers). Instead, the term should be > ?version string? throughout. I'd rather we just say 'version' instead of 'version number' or 'version string'. Natural use of natural language is... natural. A separate sentence can state simply that versions are expressed as strings. -Fred -- Fred L. Drake, Jr.? ? "Give me the luxuries of life and I will willingly do without the necessities." ?? --Frank Lloyd Wright From ben+python at benfinney.id.au Thu Jun 9 06:02:26 2011 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 09 Jun 2011 14:02:26 +1000 Subject: [Distutils] DRAFT PEP 396 - module version number References: <20110324141747.6d90382e@neurotica.wooz.org> <4D8C035E.4050306@netwok.org> <20110405103108.77f144a2@neurotica.wooz.org> <5c2331c6216e5085bdc02629b9a42986@netwok.org> <20110412142313.34c79298@neurotica.wooz.org> <4DEE440F.6000600@netwok.org> <20110607171706.6977ffc1@neurotica.wooz.org> <87vcwhyodg.fsf@benfinney.id.au> <20110608181403.2829d9c4@neurotica.wooz.org> Message-ID: <87vcwfy7n1.fsf@benfinney.id.au> Fred Drake writes: > On Wed, Jun 8, 2011 at 6:14 PM, Barry Warsaw wrote: > > #. For modules which live inside a namespace package, the sub-package > > ? name SHOULD include the ``__version__`` attribute. ?The namespace > > ? module itself SHOULD NOT include its own ``__version__`` attribute. > > I've no idea what you're saying here. What's a sub-package? Moreover, how can a ?sub-package name? include an attribute? I agree that needs to be clarified. > If you're referring to a package like zope.testing, I'd just call that a > package; there's nothing special about that. I'd expect the __version__, > if it exists, to be present in the file zope/testing/__init__.py. Yes, but how to specify that? The ?__init__.py? file is a module. What's that, then; a package module? > Ben Finney wrote: > > I may be fighting against the tide here; but this screams to me that the > > PEP should not be talking at all about ?version number? (except to point > > out that they're strings, not numbers). Instead, the term should be > > ?version string? throughout. > > I'd rather we just say 'version' instead of 'version number' or 'version > string'. Natural use of natural language is... natural. A separate > sentence can state simply that versions are expressed as strings. ?1. A version is a state of the code base at a point in its development; what we're talking about here are strings which act as identifiers for versions of the code. I would argue for ?version identifier?, but that has virtually no actual use :-) -- \ ?I was sleeping the other night, alone, thanks to the | `\ exterminator.? ?Emo Philips | _o__) | Ben Finney From fdrake at acm.org Thu Jun 9 06:31:20 2011 From: fdrake at acm.org (Fred Drake) Date: Thu, 9 Jun 2011 00:31:20 -0400 Subject: [Distutils] DRAFT PEP 396 - module version number In-Reply-To: <87vcwfy7n1.fsf@benfinney.id.au> References: <20110324141747.6d90382e@neurotica.wooz.org> <4D8C035E.4050306@netwok.org> <20110405103108.77f144a2@neurotica.wooz.org> <5c2331c6216e5085bdc02629b9a42986@netwok.org> <20110412142313.34c79298@neurotica.wooz.org> <4DEE440F.6000600@netwok.org> <20110607171706.6977ffc1@neurotica.wooz.org> <87vcwhyodg.fsf@benfinney.id.au> <20110608181403.2829d9c4@neurotica.wooz.org> <87vcwfy7n1.fsf@benfinney.id.au> Message-ID: I wrote: > If you're referring to a package like zope.testing, I'd just call that a > package; there's nothing special about that. ?I'd expect the __version__, > if it exists, to be present in the file zope/testing/__init__.py. Ben Finney responded > Yes, but how to specify that? The ?__init__.py? file is a module. What's > that, then; a package module? The package zope.testing includes the module zope.testing (implemented in zope/testing/__init__.py), as well as other modules (zope.testing.cleanup for example). The zope.testing module SHOULD contain any appropriate __version__ attribute. The only real special cases are "namespace" packages like zc, zope, and many others. Since those are *not* tied to a single distribution, they SHOULD NOT have __version__ attributes. (I think we agree on this, but language is tedious, since there are many casual uses of the term 'package' here, and 'namespace packages' aren't as commonly known outside particular segments of the community.) -Fred -- Fred L. Drake, Jr.? ? "Give me the luxuries of life and I will willingly do without the necessities." ?? --Frank Lloyd Wright From merwok at netwok.org Thu Jun 9 13:29:58 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Thu, 09 Jun 2011 13:29:58 +0200 Subject: [Distutils] DRAFT PEP 396 - module version number In-Reply-To: <20110607171706.6977ffc1@neurotica.wooz.org> References: <20110324141747.6d90382e@neurotica.wooz.org> <4D8C035E.4050306@netwok.org> <20110405103108.77f144a2@neurotica.wooz.org> <5c2331c6216e5085bdc02629b9a42986@netwok.org> <20110412142313.34c79298@neurotica.wooz.org> <4DEE440F.6000600@netwok.org> <20110607171706.6977ffc1@neurotica.wooz.org> Message-ID: <4DF0AEB6.9080709@netwok.org> Hi, > The PEP currently uses `version-from-file` which seems okay to me. Would that > work? Yes, the PEP uses that field since I said I thought it could not reuse the version field; my email was meant to change ?I think it should be another field? to ?it must definitely be another field?. version-from-file is a good name if a regex is applied to extract a version from the first lines of a file; if the whole contents of the file are the version number (sorry Ben, I cannot call it something else :), it should be version-file to mirror description-file. Regards From merwok at netwok.org Thu Jun 9 15:48:05 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Thu, 09 Jun 2011 15:48:05 +0200 Subject: [Distutils] [PEP376] RECORD line separator In-Reply-To: References: Message-ID: <4DF0CF15.2030007@netwok.org> Hi, > I read pep276, and I have a question. > > This PEP says that line separator of RECORD file is `os.separator`. > and 'bdist-*' command will also create RECORD file. This should not be a concern: Python can open files in universal newline mode, in other words translate os.sep to \n. Regards From barry at python.org Thu Jun 9 17:36:16 2011 From: barry at python.org (Barry Warsaw) Date: Thu, 9 Jun 2011 11:36:16 -0400 Subject: [Distutils] DRAFT PEP 396 - module version number In-Reply-To: <87ei33zvh2.fsf@benfinney.id.au> References: <20110324141747.6d90382e@neurotica.wooz.org> <4D8C035E.4050306@netwok.org> <20110405103108.77f144a2@neurotica.wooz.org> <5c2331c6216e5085bdc02629b9a42986@netwok.org> <20110412142313.34c79298@neurotica.wooz.org> <4DEE440F.6000600@netwok.org> <20110607171706.6977ffc1@neurotica.wooz.org> <87vcwhyodg.fsf@benfinney.id.au> <20110608181403.2829d9c4@neurotica.wooz.org> <87ei33zvh2.fsf@benfinney.id.au> Message-ID: <20110609113616.4e63f116@neurotica.wooz.org> On Jun 09, 2011, at 10:42 AM, Ben Finney wrote: >Barry Warsaw writes: > >> #. The ``__version__`` attribute's value SHOULD be a string. > >I may be fighting against the tide here; but this screams to me that the >PEP should not be talking at all about ?version number? (except to point >out that they're strings, not numbers). Instead, the term should be >?version string? throughout. As I was doing the last edit, I did think about that. The suggestion is not without merit, but I also think "version number" is a commonly accepted term for what this thing is. I mean, we've used that term for ages to describe things like 2.6.7, which clearly isn't a number. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Thu Jun 9 17:44:31 2011 From: barry at python.org (Barry Warsaw) Date: Thu, 9 Jun 2011 11:44:31 -0400 Subject: [Distutils] DRAFT PEP 396 - module version number In-Reply-To: <4DF0AEB6.9080709@netwok.org> References: <20110324141747.6d90382e@neurotica.wooz.org> <4D8C035E.4050306@netwok.org> <20110405103108.77f144a2@neurotica.wooz.org> <5c2331c6216e5085bdc02629b9a42986@netwok.org> <20110412142313.34c79298@neurotica.wooz.org> <4DEE440F.6000600@netwok.org> <20110607171706.6977ffc1@neurotica.wooz.org> <4DF0AEB6.9080709@netwok.org> Message-ID: <20110609114431.3b400241@neurotica.wooz.org> On Jun 09, 2011, at 01:29 PM, ?ric Araujo wrote: >Yes, the PEP uses that field since I said I thought it could not reuse >the version field; my email was meant to change ?I think it should be >another field? to ?it must definitely be another field?. > >version-from-file is a good name if a regex is applied to extract a >version from the first lines of a file; if the whole contents of the >file are the version number (sorry Ben, I cannot call it something else >:), it should be version-file to mirror description-file. Hi ?ric, I hadn't thought about a file that only contains the version number, but that makes perfect sense. Here's what I propose for the PEP. Distutils2 ---------- Because the distutils2 style ``setup.cfg`` is declarative, we can't run any code to extract the ``__version__`` attribute, either via import or via parsing. In consultation with the distutils-sig [9]_, two options are proposed. Both entail containing the version number in a file, and declaring that file in the ``setup.cfg``. When the entire contents of the file contains the version number, the ``version-file`` key will be used:: [metadata] version-file: version.txt When the version number is contained within a larger file, e.g. of Python code, such that the file must be parsed to extract the version, the key ``version-from-file`` will be used:: [metadata] version-from-file: elle.py A parsing method similar to that described above will be performed on the file named after the colon. The exact recipe for doing this will be discussed in the appropriate distutils2 development forum. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Thu Jun 9 17:50:16 2011 From: barry at python.org (Barry Warsaw) Date: Thu, 9 Jun 2011 11:50:16 -0400 Subject: [Distutils] DRAFT PEP 396 - module version number In-Reply-To: References: <20110324141747.6d90382e@neurotica.wooz.org> <4D8C035E.4050306@netwok.org> <20110405103108.77f144a2@neurotica.wooz.org> <5c2331c6216e5085bdc02629b9a42986@netwok.org> <20110412142313.34c79298@neurotica.wooz.org> <4DEE440F.6000600@netwok.org> <20110607171706.6977ffc1@neurotica.wooz.org> <87vcwhyodg.fsf@benfinney.id.au> <20110608181403.2829d9c4@neurotica.wooz.org> Message-ID: <20110609115016.6ba3aa9d@neurotica.wooz.org> On Jun 08, 2011, at 10:47 PM, Fred Drake wrote: >On Wed, Jun 8, 2011 at 6:14 PM, Barry Warsaw wrote: >> #. For modules which live inside a namespace package, the sub-package >> ? name SHOULD include the ``__version__`` attribute. ?The namespace >> ? module itself SHOULD NOT include its own ``__version__`` attribute. > >I've no idea what you're saying here. What's a sub-package? It's a package that lives within another package. >If you're referring to a package like zope.testing, I'd just call that a >package; there's nothing special about that. I'd expect the __version__, >if it exists, to be present in the file zope/testing/__init__.py. That's where I'd expect __version__ too, and it's what the PEP tries to say. Unfortunately, I think it's ambiguous to change "sub-package" to "package" here; it just isn't clear. We're hampered by a lack of consistent terminology in the Python world about these things. I'm willing to add some definitions to the PEP to help make things clearer, or to rewrite something, but we first need to agree on what to call these things. :) >An namespace package, like zope or zc, should not have a __version__. Agreed. Is this any clearer? #. For modules which live inside a namespace package, the module SHOULD include the ``__version__`` attribute. The namespace package itself SHOULD NOT include its own ``__version__`` attribute. >Ben Finney wrote: >> I may be fighting against the tide here; but this screams to me that the >> PEP should not be talking at all about ?version number? (except to point >> out that they're strings, not numbers). Instead, the term should be >> ?version string? throughout. > >I'd rather we just say 'version' instead of 'version number' or 'version >string'. Natural use of natural language is... natural. A separate >sentence can state simply that versions are expressed as strings. #. The ``__version__`` attribute's value SHOULD be a string. I still don't have a problem with using "version number" to express what these things are. We have oodles of existing practice on this. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Thu Jun 9 17:52:02 2011 From: barry at python.org (Barry Warsaw) Date: Thu, 9 Jun 2011 11:52:02 -0400 Subject: [Distutils] DRAFT PEP 396 - module version number In-Reply-To: References: <20110324141747.6d90382e@neurotica.wooz.org> <4D8C035E.4050306@netwok.org> <20110405103108.77f144a2@neurotica.wooz.org> <5c2331c6216e5085bdc02629b9a42986@netwok.org> <20110412142313.34c79298@neurotica.wooz.org> <4DEE440F.6000600@netwok.org> <20110607171706.6977ffc1@neurotica.wooz.org> <87vcwhyodg.fsf@benfinney.id.au> <20110608181403.2829d9c4@neurotica.wooz.org> <87vcwfy7n1.fsf@benfinney.id.au> Message-ID: <20110609115202.722497ed@neurotica.wooz.org> On Jun 09, 2011, at 12:31 AM, Fred Drake wrote: >(I think we agree on this, but language is tedious, since there are many >casual uses of the term 'package' here, and 'namespace packages' aren't as >commonly known outside particular segments of the community.) I think "namespace package" is a pretty commonly accepted term in the Python community. See PEP 382. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From vinay_sajip at yahoo.co.uk Sat Jun 18 13:31:23 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Sat, 18 Jun 2011 11:31:23 +0000 (UTC) Subject: [Distutils] Python 3.3 / packaging - how to make installation conditional on platform? Message-ID: I've a project which needs to install different data files depending on what platform the project is being installed on - e.g. win32, linux2, darwin. Using packaging setup.cfg, how can I indicate different files for different platforms, as well as some common files for all platforms? It's easy enough to do with setup.py, of course. It doesn't seem possible to use project-specific Python files to do this (via setup_hooks, for example), as when pysetup3 runs, the project directory is not in sys.path, so you can't import anything from e.g. files adjacent to setup.cfg. Although my current need is for data files, there might be situations where different packages need installing for different platforms (e.g. having shim or adapter functionality). What's the official packaging solution for this? Regards, Vinay Sajip From vinay_sajip at yahoo.co.uk Sat Jun 18 13:40:25 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Sat, 18 Jun 2011 11:40:25 +0000 (UTC) Subject: [Distutils] =?utf-8?q?Python_3=2E3_/_packaging_-_how_to_make_inst?= =?utf-8?q?allation=09conditional_on_platform=3F?= References: Message-ID: Vinay Sajip yahoo.co.uk> writes: [snip] Ignore my previous post- I believe environment markers are what I was looking for. Don't know how I missed that :-) Regards, Vinay Sajip From vinay_sajip at yahoo.co.uk Sat Jun 18 14:26:26 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Sat, 18 Jun 2011 12:26:26 +0000 (UTC) Subject: [Distutils] 3.3 / packaging - support for Windows installation locations Message-ID: While packaging's support for path-based installation seems reasonably comprehensive, on recent Windows versions, installation locations aren't always simply based on fixed or declaratively specifiable locations. For example, say that some data needs to be installed in a user's "Documents" folder under Windows 7. You might assume that this would be e.g. "C:\Users\Vinay\Documents", but in fact that might not be the correct location. To do it correctly for all scenarios involves jumping through some hoops: from ctypes import windll, Structure, byref, w_char_p # and other stuff besides class GUID(Structure): # define a GUID structure in terms of ctypes longs, shorts etc. Documents_FolderID = GUID(...) # using constants from Windows KnownFolders.h path = w_char_p() # to hold the resulting path windll.shell32.SHGetKnownFolderPath(byref(Documents_FolderID), ..., byref(path), ...) and then use path.value as the location of the Documents folder. What's the best way of handling these sorts of situations with Python 3.3 packaging? Regards, Vinay Sajip From alexis at notmyidea.org Sat Jun 18 14:31:13 2011 From: alexis at notmyidea.org (=?ISO-8859-1?Q?Alexis_M=E9taireau?=) Date: Sat, 18 Jun 2011 14:31:13 +0200 Subject: [Distutils] 3.3 / packaging - support for Windows installation locations In-Reply-To: References: Message-ID: <4DFC9A91.1060705@notmyidea.org> On 06/18/2011 02:26 PM, Vinay Sajip wrote: > What's the best way of handling these sorts of situations with Python 3.3 > packaging? Isn't it something that should be handled by the sysconfig module? (http://docs.python.org/dev/library/sysconfig.html) -- Alexis From vinay_sajip at yahoo.co.uk Sat Jun 18 16:18:28 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Sat, 18 Jun 2011 14:18:28 +0000 (UTC) Subject: [Distutils] 3.3 / packaging - support for Windows installation locations References: <4DFC9A91.1060705@notmyidea.org> Message-ID: Alexis M?taireau notmyidea.org> writes: > > On 06/18/2011 02:26 PM, Vinay Sajip wrote: > > What's the best way of handling these sorts of situations with Python 3.3 > > packaging? > Isn't it something that should be handled by the sysconfig module? > (http://docs.python.org/dev/library/sysconfig.html) > Perhaps, but perhaps not. For example, the sysconfig.get_paths() returns a path like this for the 'data' key: C:\\Users\\Vinay\\AppData\\Roaming\\Python This might be fine for many applications, but say you were to install some Powershell scripts, those would need to live in C:\\Users\\Vinay\\Documents\\WindowsPowershell for Powershell to load them automatically. There's only one reference to the special location path determining code - in PC\bdist_wininst\install.c - which appears to be for executable installers only. This could in theory be handled by a project-specific post-installation step (which could e.g. be in Python code bundled with the project), which could be declared in setup.cfg and invoked by pysetup3 after the installation actions. Is there such a provision? If so, I couldn't find it, but perhaps I missed it and someone could point it out to me. Regards, Vinay Sajip From merwok at netwok.org Sat Jun 18 22:38:29 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Sat, 18 Jun 2011 22:38:29 +0200 Subject: [Distutils] Python 3.3 / packaging - how to make installation conditional on platform? In-Reply-To: References: Message-ID: <4DFD0CC5.2030508@netwok.org> Hi, > It doesn't seem possible to use project-specific Python files to do this (via > setup_hooks, for example), as when pysetup3 runs, the project directory is not > in sys.path, so you can't import anything from e.g. files adjacent to setup.cfg. This is not a design choice but a bug: http://bugs.python.org/issue11637 We definitely want to support project-local hooks module, as they will certainly be used a lot during the transition years. > Ignore my previous post- I believe environment markers are what I was looking > for. Don't know how I missed that :-) I?m not sure data files support environment markers. Anyway, please report any info missing from the docs, they?re far from complete. Thanks for trying out packaging! Cheers From merwok at netwok.org Sat Jun 18 22:46:28 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Sat, 18 Jun 2011 22:46:28 +0200 Subject: [Distutils] 3.3 / packaging - support for Windows installation locations In-Reply-To: References: <4DFC9A91.1060705@notmyidea.org> Message-ID: <4DFD0EA4.2060501@netwok.org> Le 18/06/2011 16:18, Vinay Sajip a ?crit : > Alexis M?taireau notmyidea.org> writes: >> On 06/18/2011 02:26 PM, Vinay Sajip wrote: >>> What's the best way of handling these sorts of situations with Python 3.3 >>> packaging? >> Isn't it something that should be handled by the sysconfig module? >> (http://docs.python.org/dev/library/sysconfig.html) > Perhaps, but perhaps not. For example, the sysconfig.get_paths() returns a path > like this for the 'data' key: > > C:\\Users\\Vinay\\AppData\\Roaming\\Python > > This might be fine for many applications, but say you were to install some > Powershell scripts, those would need to live in > > C:\\Users\\Vinay\\Documents\\WindowsPowershell > > for Powershell to load them automatically. There's only one reference to the > special location path determining code - in PC\bdist_wininst\install.c - which > appears to be for executable installers only. > > This could in theory be handled by a project-specific post-installation step > (which could e.g. be in Python code bundled with the project), which could be > declared in setup.cfg and invoked by pysetup3 after the installation actions. Is > there such a provision? If so, I couldn't find it, but perhaps I missed it and > someone could point it out to me. I think two features I?d like to add to 3.3 may cover your needs. The first one is standard functions to get the right config, data, cache, etc. directories for an application (http://bugs.python.org/issue7175); the second is a configure command for packaging (http://bugs.python.org/issue8254), which would record everything related to the installation (paths and al.) and make it available at build time and run time to other projects, making it possible for a plugin to find the right dir to install files for an application. See also the threads last summer about PEP 376 and plugins on distutils-sig (and I believe python-dev). Cheers From vinay_sajip at yahoo.co.uk Sat Jun 18 23:23:17 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Sat, 18 Jun 2011 21:23:17 +0000 (UTC) Subject: [Distutils] Python 3.3 / packaging - how to make installation conditional on platform? References: <4DFD0CC5.2030508@netwok.org> Message-ID: ?ric Araujo netwok.org> writes: > This is not a design choice but a bug: http://bugs.python.org/issue11637 > We definitely want to support project-local hooks module, as they will > certainly be used a lot during the transition years. Thanks for the pointer. I may need this to be fixed in my pythonv fork, so I will attempt a patch there. > I?m not sure data files support environment markers. Anyway, please > report any info missing from the docs, they?re far from complete. On further study, I see you're right - environment markers will work only with some of the data. This gap can certainly be handled by setup hooks, once #11637 is fixed, but there's also another problem which I've encountered: on Windows at least, installing into the conventional appdata paths is fine for simple applications, but insufficient for applications that need to interact better with the Windows ecosystem. For example, if you want to install Powershell modules, they live not in conventional appdata locations but rather in subdirectories below a user's Documents folder, which is obtained by making calls into the Windows Shell API, such as SHGetKnownFolderPath or SHGetSpecialFolderPath. I can't see any references to these in packaging or sysconfig, so how are these values being determined? I appreciate that might not be a packaging issue but rather a sysconfig one, but it's certainly of interest to packagers :-) The only references I found to the shell APIs is in PC\bdist_winst\install.c, which (I presume) wouldn't be invoked during a "pysetup3 install" operation. So, it seems to me like some way is needed for a project to be able to inject additional categories like {userdocs} and path mappings for them, for its resources. I appreciate that this is more an issue for Windows than Unix platforms, but is there such a mechanism? Incidentally, I noticed a repeated assignment in packaging/install.py:_run_install_from_dir(): func = install_methods[install_method] is called twice in succession, I couldn't see why. Regards, Vinay Sajip From vinay_sajip at yahoo.co.uk Sat Jun 18 23:37:26 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Sat, 18 Jun 2011 21:37:26 +0000 (UTC) Subject: [Distutils] 3.3 / packaging - support for Windows installation locations References: <4DFC9A91.1060705@notmyidea.org> <4DFD0EA4.2060501@netwok.org> Message-ID: ?ric Araujo netwok.org> writes: > I think two features I?d like to add to 3.3 may cover your needs. The > first one is standard functions to get the right config, data, cache, > etc. directories for an application (http://bugs.python.org/issue7175); > the second is a configure command for packaging > (http://bugs.python.org/issue8254), which would record everything > related to the installation (paths and al.) and make it available at > build time and run time to other projects, making it possible for a > plugin to find the right dir to install files for an application. > > See also the threads last summer about PEP 376 and plugins on > distutils-sig (and I believe python-dev). Okay, I'll look at these - thanks. Is there somewhere online where design notes and ideas are being collected / summarised? It's hard to tell which parts of e.g. notmyidea.org and python-distribute.org are still current. Regards, Vinay Sajip From merwok at netwok.org Sat Jun 18 23:59:51 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Sat, 18 Jun 2011 23:59:51 +0200 Subject: [Distutils] 3.3 / packaging - support for Windows installation locations In-Reply-To: References: <4DFC9A91.1060705@notmyidea.org> <4DFD0EA4.2060501@netwok.org> Message-ID: <4DFD1FD7.9000409@netwok.org> > Okay, I'll look at these - thanks. Is there somewhere online where > design notes and ideas are being collected / summarised? Not really. * The bug report about config/cache/etc. paths API contains most of the discussion and links to python-dev threads. This is not related to packaging. * The bug report about configure links to the wiki I used last summer and my repository with the code; a few things are only in Tarek?s head. * The thinking about sysconfig.py, sysconfig.cfg and _sysconfig.cpython3.3.so is scattered on two or three bug reports and https://bitbucket.org/tarek/distutils2/src/tip/docs/design/wiki.rst . > It's hard to tell which parts of e.g. notmyidea.org and > python-distribute.org are still current. Yeah. Last summer, I pulled from all GSoC students repos and merged the docs so that notmyidea could always have all doc, even though not all the code was in the official repo. Then Alexis changed it to use the doc from the main repo, and then I think switched to readthedocs. python-distribute is/was lead by the distribute group, which overlaps with the packaging (distutils2) fellowship. Its packaging guide was not updated for a distutils2-in-the-stdlib world. I could make a new landing page on the Python to explain the status of distutils2, summarize future directions, tell where to find the docs, link to the existing pages like http://wiki.python.org/moin/Distutils/Contributing and http://wiki.python.org/moin/Distutils/FixingBugs . Do you think that would be better? From merwok at netwok.org Sat Jun 18 23:49:08 2011 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Sat, 18 Jun 2011 23:49:08 +0200 Subject: [Distutils] Python 3.3 / packaging - how to make installation conditional on platform? In-Reply-To: References: <4DFD0CC5.2030508@netwok.org> Message-ID: <4DFD1D54.9060009@netwok.org> Le 18/06/2011 23:23, Vinay Sajip a ?crit : > ?ric Araujo netwok.org> writes: >> This is not a design choice but a bug: http://bugs.python.org/issue11637 >> We definitely want to support project-local hooks module, as they will >> certainly be used a lot during the transition years. > Thanks for the pointer. I may need this to be fixed in my pythonv fork, so I > will attempt a patch there. Thanks, I?ll be here to review the patch. It should be easy, just remember to add or update tests. >> I?m not sure data files support environment markers. Anyway, please >> report any info missing from the docs, they?re far from complete. > On further study, I see you're right - environment markers will work only with > some of the data. This gap can certainly be handled by setup hooks, once #11637 > is fixed, but there's also another problem which I've encountered: on Windows at > least, installing into the conventional appdata paths is fine for simple > applications, but insufficient for applications that need to interact better > with the Windows ecosystem. For example, if you want to install Powershell > modules, they live not in conventional appdata locations but rather in > subdirectories below a user's Documents folder, which is obtained by making > calls into the Windows Shell API, such as SHGetKnownFolderPath or > SHGetSpecialFolderPath. I can't see any references to these in packaging or > sysconfig, so how are these values being determined? Well, I don?t think they are. The paths used for Windows are a transposition of UNIX paths to C:\\PythonX.Y, with Scripts, Lib, etc. subdirs; they?re not very Windows-y. IOW, I think improving the installation schemes on Windows for 3.3 to feel more native would be a good thing. See also http://bugs.python.org/issue9878 : Avoid parsing pyconfig.h and Makefile by autogenerating extension module (which means that using C APIs is wholly in line with future direction for sysconfig). > I appreciate that might not be a packaging issue but rather a sysconfig one, > but it's certainly of interest to packagers :-) This is definitely on-topic: sysconfig was extracted from distutils, is maintained by Tarek and evolved by the packaging fellowship. Installation is part of packaging. > The only references I found to the shell APIs is in PC\bdist_winst\install.c, > which (I presume) wouldn't be invoked during a "pysetup3 install" operation. Yep, only during pysetup run bdist_wininst, IIUC. > So, it seems to me like some way is needed for a project to be able to inject > additional categories like {userdocs} and path mappings for them, for its > resources. I appreciate that this is more an issue for Windows than Unix > platforms, but is there such a mechanism? Categories are extensible, so it?s a way. See also the short description of the packaging configure command in my other message. > Incidentally, I noticed a repeated assignment in > packaging/install.py:_run_install_from_dir(): > func = install_methods[install_method] > is called twice in succession, I couldn't see why. Just a small mistake in ebff46b232ed. From vinay_sajip at yahoo.co.uk Sun Jun 19 00:16:24 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Sat, 18 Jun 2011 22:16:24 +0000 (UTC) Subject: [Distutils] 3.3 / packaging - support for Windows installation locations References: <4DFC9A91.1060705@notmyidea.org> <4DFD0EA4.2060501@netwok.org> <4DFD1FD7.9000409@netwok.org> Message-ID: ?ric Araujo netwok.org> writes: > I could make a new landing page on the Python to explain the status of > distutils2, summarize future directions, tell where to find the docs, > link to the existing pages like > http://wiki.python.org/moin/Distutils/Contributing and > http://wiki.python.org/moin/Distutils/FixingBugs . Do you think that > would be better? I think so - when things are in a state of flux and lots of ideas are being discussed, once they have coalesced into a broad consensus, it's useful to be able to see what that consensus is with open issues pointed out, separated from discussions that perhaps went nowhere. That would make both contributions themselves easier, and also allow people working on related areas to take advantage of a better roadmap. Thanks for the helpful summary. Regards, Vinay Sajip From alexis at notmyidea.org Sun Jun 19 12:56:17 2011 From: alexis at notmyidea.org (=?UTF-8?B?QWxleGlzIE3DqXRhaXJlYXU=?=) Date: Sun, 19 Jun 2011 12:56:17 +0200 Subject: [Distutils] 3.3 / packaging - support for Windows installation locations In-Reply-To: <4DFD1FD7.9000409@netwok.org> References: <4DFC9A91.1060705@notmyidea.org> <4DFD0EA4.2060501@netwok.org> <4DFD1FD7.9000409@netwok.org> Message-ID: <4DFDD5D1.4050703@notmyidea.org> On 06/18/2011 11:59 PM, ?ric Araujo wrote: >> > It's hard to tell which parts of e.g. notmyidea.org and >> > python-distribute.org are still current. Yeah, that's right. I will put a redirection from distutils2.notmyidea.org to the new packaging doc hosted on docs.python.org/dev/ as distutils2 is not currently where we develop packaging those days. -- Alexis From vinay_sajip at yahoo.co.uk Mon Jun 20 11:48:13 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Mon, 20 Jun 2011 09:48:13 +0000 (UTC) Subject: [Distutils] Python 3.3 / packaging - how to make installation conditional on platform? References: <4DFD0CC5.2030508@netwok.org> <4DFD1D54.9060009@netwok.org> Message-ID: ?ric Araujo netwok.org> writes: > Well, I don?t think they are. The paths used for Windows are a > transposition of UNIX paths to C:\\PythonX.Y, with Scripts, Lib, etc. > subdirs; they?re not very Windows-y. IOW, I think improving the > installation schemes on Windows for 3.3 to feel more native would be a > good thing. > > See also http://bugs.python.org/issue9878 : Avoid parsing pyconfig.h and > Makefile by autogenerating extension module (which means that using C > APIs is wholly in line with future direction for sysconfig). > Okay. > Categories are extensible, so it?s a way. See also the short > description of the packaging configure command in my other message. Thinking about it further - I'm not sure it'll be sufficient, in a number of cases, at least under Windows but potentially on other platforms, too. Where software is self-contained, having a fixed set of categories for different types of assets is sensible and sufficient. Where software needs to interoperate with other software, for example, things can get a little more complicated. I've already given an example relating to Powershell. As another example: say I'm developing some software to interoperate with legacy systems such as Microsoft Office (including Outlook) or Sharepoint, which are prevalent in the corporate world. The chances are very high that I'll need to install things in what is, from packaging's perspective, an essentially arbitrary set of locations. It's seems that it's not really possible to codify these using the schemes which we have now - schemes which are perfectly useful in the context of e.g. harmonising how we work with Linux distro packagers. I don't know if this has been brought up before, or discussed and rejected - but I think that something along the lines of Debian's prerm/postrm, preinst/postinst should be available, defined in the same way as setup_hooks in setup.cfg, to be called at the appropriate time in the pysetup3 lifecycle. This would provide some measure of standardisation, allowing projects with simple needs to omit them entirely, but those with more complex needs to do things which we can't foresee. As a bare minimum, prerm and postinst: these can rely on the distribution being installed and so have access to packages inside it. Of course, this is the rationale behind the post-install script option for bdist_wininst. Regards, Vinay Sajip From vinay_sajip at yahoo.co.uk Tue Jun 21 10:34:01 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Tue, 21 Jun 2011 08:34:01 +0000 (UTC) Subject: [Distutils] Distribute without 2to3 Message-ID: I'm testing a branch of Python which provides out-of-the-box ability to create virtual enviroments ? la virtualenv, and as part of that testing I have to install Distribute in newly created environments a lot. Though normally running 2to3 as part of the Distribute installation is not a big deal, for this testing I'm doing, it does slow things down a little. While waiting for some tests to finish, I thought I might as well look into whether Distribute could be made to run on 2.x and 3.x from a single code base, thereby avoiding the 2to3 step. I've made an attempt, and things seem to have gone reasonably smoothly: the conversion is done and all unit tests pass on 2.7, 3.2, 3.3 (my branch). The converted Distribute is at https://bitbucket.org/vinay.sajip/distribute3 In case any of you are interested at taking a look, I'd welcome any feedback you have. Regards, Vinay Sajip From regebro at gmail.com Tue Jun 21 11:21:07 2011 From: regebro at gmail.com (Lennart Regebro) Date: Tue, 21 Jun 2011 11:21:07 +0200 Subject: [Distutils] Distribute without 2to3 In-Reply-To: References: Message-ID: On Tue, Jun 21, 2011 at 10:34, Vinay Sajip wrote: > I'm testing a branch of Python which provides out-of-the-box ability to create > virtual enviroments ? la virtualenv, and as part of that testing I have to > install Distribute in newly created environments a lot. Though normally running > 2to3 as part of the Distribute installation is not a big deal, for this testing > I'm doing, it does slow things down a little. > > While waiting for some tests to finish, I thought I might as well look into > whether Distribute could be made to run on 2.x and 3.x from a single code base, > thereby avoiding the 2to3 step. > > I've made an attempt, and things seem to have gone reasonably smoothly: the > conversion is done and all unit tests pass on 2.7, 3.2, 3.3 (my branch). We still need to support Python 2.4, right? That's a trickier issue. But including six.py might help. //Lennart From wichert at wiggy.net Tue Jun 21 11:23:14 2011 From: wichert at wiggy.net (Wichert Akkerman) Date: Tue, 21 Jun 2011 11:23:14 +0200 Subject: [Distutils] Distribute without 2to3 In-Reply-To: References: Message-ID: <4E006302.1000106@wiggy.net> On 06/21/2011 11:21 AM, Lennart Regebro wrote: > On Tue, Jun 21, 2011 at 10:34, Vinay Sajip wrote: >> I'm testing a branch of Python which provides out-of-the-box ability to create >> virtual enviroments ? la virtualenv, and as part of that testing I have to >> install Distribute in newly created environments a lot. Though normally running >> 2to3 as part of the Distribute installation is not a big deal, for this testing >> I'm doing, it does slow things down a little. >> >> While waiting for some tests to finish, I thought I might as well look into >> whether Distribute could be made to run on 2.x and 3.x from a single code base, >> thereby avoiding the 2to3 step. >> >> I've made an attempt, and things seem to have gone reasonably smoothly: the >> conversion is done and all unit tests pass on 2.7, 3.2, 3.3 (my branch). > We still need to support Python 2.4, right? That's a trickier issue. > But including six.py might help. Afaik you can't catch exceptions in a way that is source-compatible with python 3.x and <2.6 due to the switch from "except , to "except as ". Wichert. From pnasrat at gmail.com Tue Jun 21 12:43:35 2011 From: pnasrat at gmail.com (Paul Nasrat) Date: Tue, 21 Jun 2011 11:43:35 +0100 Subject: [Distutils] Distribute without 2to3 In-Reply-To: <4E006302.1000106@wiggy.net> References: <4E006302.1000106@wiggy.net> Message-ID: On 21 June 2011 10:23, Wichert Akkerman wrote: > > On 06/21/2011 11:21 AM, Lennart Regebro wrote: >> >> On Tue, Jun 21, 2011 at 10:34, Vinay Sajip ?wrote: >>> >>> I'm testing a branch of Python which provides out-of-the-box ability to create >>> virtual enviroments ? la virtualenv, and as part of that testing I have to >>> install Distribute in newly created environments a lot. Though normally running >>> 2to3 as part of the Distribute installation is not a big deal, for this testing >>> I'm doing, it does slow things down a little. >>> >>> While waiting for some tests to finish, I thought I might as well look into >>> whether Distribute could be made to run on 2.x and 3.x from a single code base, >>> thereby avoiding the 2to3 step. >>> >>> I've made an attempt, and things seem to have gone reasonably smoothly: the >>> conversion is done and all unit tests pass on 2.7, 3.2, 3.3 (my branch). >> >> We still need to support Python 2.4, right? That's a trickier issue. >> But including six.py might help. > > Afaik you can't catch exceptions in a way that is source-compatible with python 3.x and <2.6 due to the switch from "except , to "except as ". You can, it just isn't particularly pretty - virtualenv and pip access the variable via? sys.exc_info()[1], eg: except Exception: e = sys.exc_info()[1] Paul From vinay_sajip at yahoo.co.uk Tue Jun 21 13:30:29 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Tue, 21 Jun 2011 11:30:29 +0000 (UTC) Subject: [Distutils] Distribute without 2to3 References: Message-ID: Lennart Regebro gmail.com> writes: > We still need to support Python 2.4, right? That's a trickier issue. > But including six.py might help. I'm not sure why 2.4 is a particular issue: I just tested on 2.4.6 without any failures. See https://gist.github.com/1037662 Of course, I'm assuming that "python setup.py test" gives adequate coverage. Regards, Vinay Sajip From regebro at gmail.com Tue Jun 21 14:42:04 2011 From: regebro at gmail.com (Lennart Regebro) Date: Tue, 21 Jun 2011 14:42:04 +0200 Subject: [Distutils] Distribute without 2to3 In-Reply-To: References: Message-ID: On Tue, Jun 21, 2011 at 13:30, Vinay Sajip wrote: > Lennart Regebro gmail.com> writes: > >> We still need to support Python 2.4, right? That's a trickier issue. >> But including six.py might help. > > I'm not sure why 2.4 is a particular issue. It isn't. The big difference is between 2.5 and 2.6. But each additional version brings it's own incompatibilities. > I just tested on 2.4.6 without any > failures. See > > https://gist.github.com/1037662 That's positive but... > Of course, I'm assuming that "python setup.py test" gives adequate coverage. It doesn't. Far from it, unfortunately. //Lennart From fuzzyman at gmail.com Tue Jun 21 15:03:20 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Tue, 21 Jun 2011 14:03:20 +0100 Subject: [Distutils] Distribute without 2to3 In-Reply-To: References: Message-ID: On 21 June 2011 13:42, Lennart Regebro wrote: > On Tue, Jun 21, 2011 at 13:30, Vinay Sajip > wrote: > > Lennart Regebro gmail.com> writes: > > > >> We still need to support Python 2.4, right? That's a trickier issue. > >> But including six.py might help. > > > > I'm not sure why 2.4 is a particular issue. > > It isn't. The big difference is between 2.5 and 2.6. But each > additional version brings it's own incompatibilities. > > Really? In my experience dropping 2.4 support allows you to use the with statement (just as dropping 2.3 support allows you to use decorators) which is a big change. I've found the 2.5-2.6 changes to be much less dramatic. I have to jump through hoops a little bit to test context managers in a code base that remains 2.4 compatible (you can *write* context managers and remain 2.4 compatible - just not use them). Michael > > I just tested on 2.4.6 without any > > failures. See > > > > https://gist.github.com/1037662 > > That's positive but... > > > Of course, I'm assuming that "python setup.py test" gives adequate > coverage. > > It doesn't. Far from it, unfortunately. > > //Lennart > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinay_sajip at yahoo.co.uk Tue Jun 21 15:13:35 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Tue, 21 Jun 2011 13:13:35 +0000 (UTC) Subject: [Distutils] Distribute without 2to3 References: Message-ID: Lennart Regebro gmail.com> writes: > That's positive but... > > > Of course, I'm assuming that "python setup.py test" gives adequate coverage. > > It doesn't. Far from it, unfortunately. That's not a good place to be, given that we'll be reliant on Distribute for a while. Is there a more comprehensive test suite elsewhere? The stuff in the distribute tests folder (not setuptools/tests) looked out-of-date and, in places, broken. This version was developed by running 2to3 on the Distribute code to see what changes were needed, and then making the changes using a compatibility module using an approach similar to Benjamin Peterson's six library. It is of course possible that I've introduced some bugs during the conversion, but any that turn up should be very easy to fix. I'm using this Distribute version to try installing (via pysetup3) all allegedly Py-3K packages on PyPI. So far, out of around 400 packages where I could find a download URL for a source distribution, smoke testing revealed 190 apparent successes, 70 failures and 140 which were untested because of failure to get the tarball. A quick check of the failures shows that at least some of these are due to failures in the tarballs themselves, e.g. Markdown - setup.py is not Py3K compatible (contains print statements) ast2src - hardcoded to require Python 3.1 Benchmarker - failed to recognize --single-version-externally-managed cbucho - fails to import setuptools correctly in Py3K conditional code Of course some potential pysetup3 issues are showing up too, e.g. "cannot detect install method", which is one of the points of doing these tests. Regards, Vinay Sajip From alexis at notmyidea.org Tue Jun 21 15:57:29 2011 From: alexis at notmyidea.org (=?ISO-8859-1?Q?Alexis_M=E9taireau?=) Date: Tue, 21 Jun 2011 15:57:29 +0200 Subject: [Distutils] Distribute without 2to3 In-Reply-To: References: Message-ID: <4E00A349.5070000@notmyidea.org> On 06/21/2011 03:13 PM, Vinay Sajip wrote: > Benchmarker - failed to recognize --single-version-externally-managed --single-version-externally-managed is a setuptools option IIRC. I've looked at its setup.py and found this: if arg1 == 'egg_info': from ez_setup import use_setuptools use_setuptools() if arg1 == 'bdist_egg': from setuptools import setup else: from distutils.core import setup So it is probably recognized as a setuptools project by pysteup3 (in packaging.util) but then uses bare distutils, which doesn't have the --single-version-managed option. Maybe in this case (when the call to setup.py --single--...) fails, we can fallback onto bare distutils install on packaging.install ? -- Alexis From regebro at gmail.com Tue Jun 21 19:35:36 2011 From: regebro at gmail.com (Lennart Regebro) Date: Tue, 21 Jun 2011 19:35:36 +0200 Subject: [Distutils] Distribute without 2to3 In-Reply-To: References: Message-ID: On Tue, Jun 21, 2011 at 15:03, Michael Foord wrote: > Really? In my experience dropping 2.4 support allows you to use the with > statement (just as dropping 2.3 support allows you to use decorators) which > is a big change. Sure, but since we support 2.4 now, I don't think the code uses it. In my experience when making code run on both python 2 and python 3, the bets path is to run 2to3 on the code and make it run under Python 3, and then backport one version at a time. And in that case you aren't using any with-statements or context managers (because the code was running under 2.4 already before you ran 2to3 on it). > I've found the 2.5-2.6 changes to be much less dramatic. Well, in 2.5 you have neither bytes literals, nor "except as", nor "from __future__ import print_function". All of these require big changes during a backport. Of course, you might have done them already for 2.7 even if they were not needed, but with 2.5 they *are* needed. //Lennart From regebro at gmail.com Tue Jun 21 19:37:59 2011 From: regebro at gmail.com (Lennart Regebro) Date: Tue, 21 Jun 2011 19:37:59 +0200 Subject: [Distutils] Distribute without 2to3 In-Reply-To: References: Message-ID: On Tue, Jun 21, 2011 at 15:13, Vinay Sajip wrote: > I'm using this Distribute version to try installing (via pysetup3) all allegedly > Py-3K packages on PyPI. So far, out of around 400 packages where I could find a > download URL for a source distribution, smoke testing revealed 190 apparent > successes, 70 failures and 140 which were untested because of failure to get the > tarball. A quick check of the failures shows that at least some of these are due > to failures in the tarballs themselves, e.g. That's a good test. Next step is to try make a buildout with it, and then do the same under 2.6 and 2.4. If that all passes, it's in a good usable state, I would say. //Lennart From vinay_sajip at yahoo.co.uk Tue Jun 21 22:05:33 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Tue, 21 Jun 2011 20:05:33 +0000 (UTC) Subject: [Distutils] Distribute without 2to3 References: Message-ID: Lennart Regebro gmail.com> writes: > That's a good test. Next step is to try make a buildout with it, and > then do the same under 2.6 and 2.4. If that all passes, it's in a good > usable state, I would say. Thanks. I'm not familiar with buildout, so if someone more savvy wants to give it a quick try, be my guest :-) Regards, Vinay Sajip From vinay_sajip at yahoo.co.uk Wed Jun 22 01:46:35 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Tue, 21 Jun 2011 23:46:35 +0000 (UTC) Subject: [Distutils] Distribute without 2to3 References: Message-ID: Lennart Regebro gmail.com> writes: > That's a good test. Next step is to try make a buildout with it, and > then do the same under 2.6 and 2.4. If that all passes, it's in a good > usable state, I would say. I haven't had a chance to look at buildout and not sure what recipes need to be tried, but just testing installing the projects on PyPI using pythonv, pysetup3 and this Distribute version has been instructive. I was getting some errors with BitBucket's CDN serving up old versions of files, so I tweaked the version numbers on the Disrribute download archive (and fixed one or two bugs) and re-tested. Out of 398 packages on PyPI which have a Python 3 trove classifier, apparently 310 were installed without errors. The other 88 had errors, some of which are project related (e.g. 14 SyntaxErrors) or have specific version requirements (8 insist on Python 3.1, for example), and others of which are due to missing dependencies or missing README files (7 instances). I'm still looking, but I still haven't found any Distribute-related errors. Full results on the 88 failures are at https://gist.github.com/1037662 - IMO 310 out of 398 is not too shabby for this stage in the proceedings (around 78%). Regards, Vinay Sajip From vinay_sajip at yahoo.co.uk Wed Jun 22 02:19:20 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Wed, 22 Jun 2011 00:19:20 +0000 (UTC) Subject: [Distutils] Distribute without 2to3 References: Message-ID: Vinay Sajip yahoo.co.uk> writes: > Out of 398 packages on PyPI which have a Python 3 trove classifier, apparently > 310 were installed without errors. The other 88 had errors, some of which are > project related (e.g. 14 SyntaxErrors) or have specific version requirements (8 > insist on Python 3.1, for example), and others of which are due to missing > dependencies or missing README files (7 instances). During this testing I came across a couple of pysetup3 issues: 1. It doesn't appear to like archive names containing spaces (example: the "ISO8583 Module" on PyPI) 2. It ("pysetup3 install tarball") leaves temporary folders lying around (the ones that contain the expanded archive to be installed, e.g. /tmp tmpiv3p7u pythonselect-1.2 The contents of pythonselect-1.2.tar.gz Although there are similar issues on the tracker, they don't seem to be quite the same - are these known issues, or should I log them? Regards, Vinay Sajip From vinay_sajip at yahoo.co.uk Wed Jun 22 12:54:34 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Wed, 22 Jun 2011 10:54:34 +0000 (UTC) Subject: [Distutils] =?utf-8?q?Python_3=2E3_/_packaging_-_how_to_make_inst?= =?utf-8?q?allation=09conditional_on_platform=3F?= References: <4DFD0CC5.2030508@netwok.org> <4DFD1D54.9060009@netwok.org> Message-ID: Vinay Sajip yahoo.co.uk> writes: > I think that something along the lines of Debian's prerm/postrm, > preinst/postinst should be available, defined in the same way as setup_hooks > setup.cfg, to be called at the appropriate time in the pysetup3 lifecycle. Okay, found the command hooks documentation :-) Perhaps a mention in the main section on the setup.cfg specification would be useful - near where setup_hooks are mentioned. Regards, Vinay Sajip From regebro at gmail.com Wed Jun 22 14:02:48 2011 From: regebro at gmail.com (Lennart Regebro) Date: Wed, 22 Jun 2011 14:02:48 +0200 Subject: [Distutils] Distribute without 2to3 In-Reply-To: References: Message-ID: Hmm. The "option --single-version-externally-managed not recognized" errors look like they might be a distribute bug. The syntax errors look like they are syntax errors in the setup.py files and not distribute bugs. //Lennart On Wed, Jun 22, 2011 at 01:46, Vinay Sajip wrote: > Lennart Regebro gmail.com> writes: > >> That's a good test. Next step is to try make a buildout with it, and >> then do the same under 2.6 and 2.4. If that all passes, it's in a good >> usable state, I would say. > > I haven't had a chance to look at buildout and not sure what recipes need to be > tried, but just testing installing the projects on PyPI using pythonv, pysetup3 > and this Distribute version has been instructive. I was getting some errors with > BitBucket's CDN serving up old versions of files, so I tweaked the version > numbers on the Disrribute download archive (and fixed one or two bugs) and > re-tested. > > Out of 398 packages on PyPI which have a Python 3 trove classifier, apparently > 310 were installed without errors. The other 88 had errors, some of which are > project related (e.g. 14 SyntaxErrors) or have specific version requirements (8 > insist on Python 3.1, for example), and others of which are due to missing > dependencies or missing README files (7 instances). > > I'm still looking, but I still haven't found any Distribute-related errors. Full > results on the 88 failures are at https://gist.github.com/1037662 - IMO 310 out > of 398 is not too shabby for this stage in the proceedings (around 78%). > > Regards, > > Vinay Sajip > > _______________________________________________ > Distutils-SIG maillist ?- ?Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From vinay_sajip at yahoo.co.uk Wed Jun 22 14:14:21 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Wed, 22 Jun 2011 12:14:21 +0000 (UTC) Subject: [Distutils] Distribute without 2to3 References: Message-ID: Lennart Regebro gmail.com> writes: > The "option --single-version-externally-managed not recognized" errors > look like they might be a distribute bug. Or perhaps a pysetup3 bug - I think Alexis commented on this. > The syntax errors look like they are syntax errors in the setup.py > files and not distribute bugs. Yes, I characterised those as project-related (meaning the PyPI project being installed), as are the errors where setup.py tries to read a missing readme.txt or similar file. Regards, Vinay Sajip From vinay_sajip at yahoo.co.uk Wed Jun 22 20:58:03 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Wed, 22 Jun 2011 18:58:03 +0000 (UTC) Subject: [Distutils] Python 3.3 / packaging - custom installation locations References: <4DFD0CC5.2030508@netwok.org> <4DFD1D54.9060009@netwok.org> Message-ID: ?ric Araujo netwok.org> writes: > Categories are extensible, so it?s a way. See also the short > description of the packaging configure command in my other message. Actually, I'd like to make a suggestion re. project-specific categories. It's easy to enable these, as follows: 1. In the install_data constructor, initialise self.categories to an empty dictionary. 2. In install_data.expand_categories, add a "local_vars.update(self.categories)" after the "local_vars = get_paths()" statement. Just this change is sufficient to allow sufficient control over custom installation locations. For projects that need custom categories, they just need to do the necessary setup in an install_data pre-hook: def pre_install_data(cmd): cmd.categories['customcategory'] = '/path/for/my/custom/category' I've implemented this approach in the pythonv branch, and it works well. Comments welcome. Regards, Vinay Sajip From vlada.peric at gmail.com Sat Jun 25 01:37:55 2011 From: vlada.peric at gmail.com (=?UTF-8?Q?Vladimir_Peri=C4=87?=) Date: Sat, 25 Jun 2011 01:37:55 +0200 Subject: [Distutils] Distribute: Running 2to3 on just some files/directories In-Reply-To: References: Message-ID: Hi, I've got a Distribute question: We bundle a library in our application. The library is already py3k compatible but we need to use 2to3 for our code. Is there a way to tell Distribute to skip a directory? Or, alternatively, to feed it a list of to run 2to3 on directly. I looked at the relevant code (in build_py) but I can't seem to get it. How does Distribute know which files have changed? Perhaps we could "trick" it into thinking that directory hasn't changed so that it wouldn't run 2to3 on it. If this is not possible at all, I'd be willing to code it for Distribute (provided someone gives me some pointers). Thanks! [I might've sent this twice, I'm not sure. Sorry if that's the case!] -- Vladimir Peri? From regebro at gmail.com Sat Jun 25 05:54:58 2011 From: regebro at gmail.com (Lennart Regebro) Date: Sat, 25 Jun 2011 05:54:58 +0200 Subject: [Distutils] Distribute: Running 2to3 on just some files/directories In-Reply-To: References: Message-ID: On Sat, Jun 25, 2011 at 01:37, Vladimir Peri? wrote: > I've got a Distribute question: We bundle a library in our > application. The library is already py3k compatible but we need to use > 2to3 for our code. Is there a way to tell Distribute to skip a > directory? No, but you can extract the library into it's own package and set up your package to require it, at which point it will be downloaded and installed when you install your package. That's the correct way of doing it. > Or, alternatively, to feed it a list of to run 2to3 on > directly. I looked at the relevant code (in build_py) but I can't seem > to get it. How does Distribute know which files have changed? Timestamps compared with the target. -- Lennart Regebro: http://regebro.wordpress.com/ Porting to Python 3: http://python3porting.com/ From chris at simplistix.co.uk Sun Jun 26 13:28:20 2011 From: chris at simplistix.co.uk (Chris Withers) Date: Sun, 26 Jun 2011 12:28:20 +0100 Subject: [Distutils] buildout 2.0 - alternative installation providers Message-ID: <4E0717D4.7060001@simplistix.co.uk> Hi Jim, Have you had any dealings with Enthought's Python Distribution: https://www.enthought.com/products/epd.php It looks like a useful tool, providing pre-compiled and compatible versions of a lot of the numeric and scientific libraries that can be a total PITA to get installed and working with each other... They also have their own installer tool, enpkg, which, from what I can tell behaves a lot like easy_install. I wonder what your thoughts are about getting buildout and EPD to play nicely with each other? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From vlada.peric at gmail.com Fri Jun 24 11:10:42 2011 From: vlada.peric at gmail.com (=?UTF-8?Q?Vladimir_Peri=C4=87?=) Date: Fri, 24 Jun 2011 11:10:42 +0200 Subject: [Distutils] Distribute: Running 2to3 on just some files/directories Message-ID: Hi, I've got a Distribute question: We bundle a library in our application. The library is already py3k compatible but we need to use 2to3 for our code. Is there a way to tell Distribute to skip a directory? Or, alternatively, to feed it a list of to run 2to3 on directly. I looked at the relevant code (in build_py) but I can't seem to get it. How does Distribute know which files have changed? Perhaps we could "trick" it into thinking that directory hasn't changed so that it wouldn't run 2to3 on it. If this is not possible at all, I'd be willing to code it for Distribute (provided someone gives me some pointers). Thanks! -- Vladimir Peri? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jim at zope.com Mon Jun 27 15:07:32 2011 From: jim at zope.com (Jim Fulton) Date: Mon, 27 Jun 2011 09:07:32 -0400 Subject: [Distutils] buildout 2.0 - alternative installation providers In-Reply-To: <4E0717D4.7060001@simplistix.co.uk> References: <4E0717D4.7060001@simplistix.co.uk> Message-ID: On Sun, Jun 26, 2011 at 7:28 AM, Chris Withers wrote: > Hi Jim, > > Have you had any dealings with Enthought's Python Distribution: > > https://www.enthought.com/products/epd.php > > It looks like a useful tool, providing pre-compiled and compatible versions > of a lot of the numeric and scientific libraries that can be a total PITA to > get installed and working with each other... > > They also have their own installer tool, enpkg, which, from what I can tell > behaves a lot like easy_install. > > I wonder what your thoughts are about getting buildout and EPD to play > nicely with each other? I'll integrate with whatever the standard installation utility is. Now that's distribute/setuptools. Eventually, that will be distutils 2 (or packaging or whatever). Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton From vinay_sajip at yahoo.co.uk Mon Jun 27 17:05:17 2011 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Mon, 27 Jun 2011 15:05:17 +0000 (UTC) Subject: [Distutils] packaging hooks for "pysetup3 remove projectXYZ" Message-ID: In general it is possible for an installation to make certain system changes other than the record of files written during an installation (for example, under Windows, there may be distribution-specific Windows registry updates). There seems to be no hook that's called when a distribution is removed, which would allow distribution-specific cleanup code to be called (e.g. to undo Windows registry changes). Such a hook should be provided, and I've created a ticket for this: http://bugs.python.org/issue12416 I believe there should be a pre- and a post- hook, just as there is for commands. ?ric has suggested getting opinions from the mailing list, so what do you all think about this? Regards, Vinay Sajip From chester_lab at fltg.net Mon Jun 27 17:31:11 2011 From: chester_lab at fltg.net (FT) Date: Mon, 27 Jun 2011 11:31:11 -0400 Subject: [Distutils] Reducing Package Size? Message-ID: <090938A22A5E475CB198DD01A928F5A3@1B1B1L1> Hi! I want to know how to keep a compiled package to its smallest size? In other words instead of compiling all Python code, keep it to only what the modules use. BT From regebro at gmail.com Tue Jun 28 05:54:32 2011 From: regebro at gmail.com (Lennart Regebro) Date: Tue, 28 Jun 2011 05:54:32 +0200 Subject: [Distutils] Reducing Package Size? In-Reply-To: <090938A22A5E475CB198DD01A928F5A3@1B1B1L1> References: <090938A22A5E475CB198DD01A928F5A3@1B1B1L1> Message-ID: On Mon, Jun 27, 2011 at 17:31, FT wrote: > > Hi! > > ? ?I want to know how to keep a compiled package to its smallest size? In > other words instead of compiling all Python code, keep it to only what the > modules use. What is the difference? //Lennart