Mailman 3 September 2015 - Distutils-SIG

Can now compile simple extensions
by Greg Ward April 29, 2018

April 29, 2018

Hi all -- at long last, I found the time to hack in the ability to compile extension modules to the Distutils. Mainly, this meant adding a 'build_ext' command which uses a CCompiler instance for all its dirty work. I also had to add a few methods to CCompiler (and, of course, UnixCCompiler) to make this work. And I added a new module, 'spawn', which takes care of running sub-programs more efficiently and robustly (no shell involved) than os.system. That's needed, obviously, so we can run the compiler! If you're in the mood for grubbing over raw source code, then get the latest from CVS or download a current snapshot. See http://www.python.org/sigs/distutils-sig/implementation.html for a link to the code snapshot. I'm still waiting for more subclasses of CCompiler to appear. At the very least, we're going to need MSVCCompiler to build extensions on Windows. Any takers? Also, someone who knows the Mac, and how to run compilers programmatically there, will have to figure out how to write a Mac-specific concrete CCompiler subclass. The spawn module also needs a bit of work to be portable. I suspect that _win32_spawn() (the intended analog to my _posix_spawn()) will be easy to implement, if it even needs to go in a separate function at all. It looks from the Python Library documentation for 1.5.2 that the os.spawnv() function is all we need, but it's a bit hard to figure out just what's needed. Windows wizards, please take a look at the 'spawn()' function and see if you can make it work on Windows. As for actually compiling extensions: well, if you can figure out the build_ext command, go ahead and give it a whirl. It's a bit cryptic right now, since there's no documentation and no example setup.py. (I have a working example at home, but it's not available online.) If you feel up to it, though, see if you can read the code and figure out what's going on. I'm just hoping *I'll* be able to figure out what's going on when I get back from the O'Reilly conference next week... ;-) Enjoy -- Greg -- Greg Ward - software developer gward(a)cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

2 1

Two fixes (at last)
by Greg Ward April 29, 2018

April 29, 2018

Hi all -- at long last, I have fixed two problems that a couple people noticed a while ago: * I folded in Amos Latteier's NT patches almost verbatim -- just changed an `os.path.sep == "/"' to `os.name == "posix"' and added some comments bitching about the inadequacy of the current library installation model (I think this is Python's fault, but for now Distutils is slavishly aping the situation in Python 1.5.x) * I fixed the problem whereby running "setup.py install" without doing anything else caused a crash (because 'build' hadn't yet been run). Now, the 'install' command automatically runs 'build' before doing anything; to make this bearable, I added a 'have_run' dictionary to the Distribution class to keep track of which commands have been run. So now not only are command classes singletons, but their 'run' method can only be invoked once -- both restrictions enforced by Distribution. The code is checked into CVS, or you can download a snapshot at http://www.python.org/sigs/distutils-sig/distutils-19990607.tar.gz Hope someone (Amos?) can try the new version under NT. Any takers for Mac OS? BTW, all parties involved in the Great "Where Do We Install Stuff?" Debate should take a good, hard look at the 'set_final_options()' method of the Install class in distutils/install.py; this is where all the policy decisions about where to install files are made. Currently it apes the Python 1.5 situation as closely as I could figure it out. Obviously, this is subject to change -- I just don't know to *what* it will change! Greg -- Greg Ward - software developer gward(a)cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

2 1

Package versioning
by Timothy Docker April 29, 2018

April 29, 2018

Hi all, I've been aware that the distutils sig has been simmerring away, but until recently it has not been directly relevant to what I do. I like the look of the proposed api, but have one question. Will this support an installed system that has multiple versions of the same package installed simultaneously? If not, then this would seem to be a significant limitation, especially when dependencies between packages are considered. Assuming it does, then how will this be achieved? I am presently managing this with a messy arrangement of symlinks. A package is installed with its version number in it's name, and a separate directory is created for an application with links from the unversioned package name to the versioned one. Then I just set the pythonpath to this directory. A sample of what the directory looks like is shown below. I'm sure there is a better solution that this, and I'm not sure that this would work under windows anyway (does windows have symlinks?). So, has this SIG considered such versioning issues yet? Cheers, Tim -------------------------------------------------------------- Tim Docker timd(a)macquarie.com.au Quantative Applications Division Macquarie Bank -------------------------------------------------------------- qad16:qad $ ls -l lib/python/ total 110 drwxr-xr-x 2 mts mts 512 Nov 11 11:23 1.1 -r--r----- 1 root mts 45172 Sep 1 1998 cdrmodule_0_7_1.so drwxr-xr-x 2 mts mts 512 Sep 1 1998 chart_1_1 drwxr-xr-x 3 mts mts 512 Sep 1 1998 Fnorb_0_7_1 dr-xr-x--- 3 mts mts 512 Nov 11 11:21 Fnorb_0_8 drwxr-xr-x 3 mts mts 1536 Mar 3 12:45 mts_1_1 dr-xr-x--- 7 mts mts 512 Nov 11 11:22 OpenGL_1_5_1 dr-xr-x--- 2 mts mts 1024 Nov 11 11:23 PIL_0_3 drwxr-xr-x 3 mts mts 512 Sep 1 1998 Pmw_0_7 dr-xr-x--- 2 mts mts 512 Nov 11 11:21 v3d_1_1 qad16:qad $ ls -l lib/python/1.1 total 30 lrwxrwxrwx 1 root other 29 Apr 10 10:43 _glumodule.so -> ../OpenGL_1_5_1/_glumodule.so lrwxrwxrwx 1 root other 30 Apr 10 10:43 _glutmodule.so -> ../OpenGL_1_5_1/_glutmodule.so lrwxrwxrwx 1 root other 22 Apr 10 10:43 _imaging.so -> ../PIL_0_3/_imaging.so lrwxrwxrwx 1 root other 36 Apr 10 10:43 _opengl_nummodule.so -> ../OpenGL_1_5_1/_opengl_nummodule.so lrwxrwxrwx 1 root other 27 Apr 10 10:43 _tkinter.so -> ../OpenGL_1_5_1/_tkinter.so lrwxrwxrwx 1 mts mts 21 Apr 10 10:43 cdrmodule.so -> ../cdrmodule_0_7_1.so lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 chart -> ../chart_1_1 lrwxrwxrwx 1 root other 12 Apr 10 10:43 Fnorb -> ../Fnorb_0_8 lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 mts -> ../mts_1_1 lrwxrwxrwx 1 root other 15 Apr 10 10:43 OpenGL -> ../OpenGL_1_5_1 lrwxrwxrwx 1 root other 33 Apr 10 10:43 opengltrmodule.so -> ../OpenGL_1_5_1/opengltrmodule.so lrwxrwxrwx 1 root other 33 Apr 10 10:43 openglutil_num.so -> ../OpenGL_1_5_1/openglutil_num.so lrwxrwxrwx 1 root other 10 Apr 10 10:43 PIL -> ../PIL_0_3 lrwxrwxrwx 1 mts mts 10 Apr 10 10:43 Pmw -> ../Pmw_0_7 lrwxrwxrwx 1 root other 10 Apr 10 10:43 v3d -> ../v3d_1_1

3 2

[Distutils-sig] distutils charter and interscript
by John Skaller April 29, 2018

April 29, 2018

3 2

Versioned trove classifiers for Django
by James Bennett April 6, 2017

April 6, 2017

Following up on some IRC discussion with other folks: There is precedent (Plone) for PyPI trove classifiers corresponding to particular versions of a framework. So I'd like to get feedback on the idea of expanding that, particularly in the case of Django. The rationale here is that the ecosystem of Django-related packages is quite large, but -- as I know all too well from a project I'm working on literally at this moment -- it can be difficult to ensure that all of one's dependencies are compatible with the version of Django one happens to be using. Adding trove classifier support at the level of individual versions of Django would, I think, greatly simplify this: tools could easily analyze which packages are compatible with an end user's chosen version, there'd be far less manual guesswork, etc., and the rate of creation of new classifiers would be relatively low (we tend to have one X.Y release/year or thereabouts, and that's the level of granularity needed). Assuming there's consensus around the idea of doing this, what would be the correct procedure for getting such classifiers set up and maintained?

5 17

BDFL Delegates for distutils-sig PEPs
by Nick Coghlan Nov. 10, 2015

Nov. 10, 2015

We've got to a point where the original standing delegations to myself and Richard Jones to act as BDFL-Delegates for metadata interoperability and pypi.python.org related aren't scaling adequately, so given Paul's recent delegation for PEP 470, and Donald handling PEP 503 directly, it seems like an opportune time to put something in writing about that. For PyPA/distutils-sig specific PEPs, we've effectively adopted the following approach to assigning BDFL-Delegates in resolving PEPs 470 and 503: ================================= Whenever a new PEP is put forward on distutils-sig, any PyPA core reviewer that believes they are suitably experienced to make the final decision on that PEP may offer to serve as the BDFL's delegate (or "PEP czar") for that PEP. If their self-nomination is accepted by the other PyPA core reviewer, the lead PyPI maintainer and the lead CPython representative on distutils-sig, then they will have the authority to approve (or reject) that PEP. ================================= And translating the nominated roles to the folks currently filling them: "lead PyPI maintainer" = Donald Stufft; "lead CPython representative on distutils-sig" = me. "PyPA core reviewer" isn't a term we've previously used, but I'm aiming to capture "has approval rights for pull requests to one or more of the PyPA maintained source code or documentation repos". Some further details for the benefit of folks not aware of the relevant history: * a couple of years ago, we amended PEP 1 to give the "Discussions-To" header some additional force for PEPs which don't directly affect CPython: """PEP review and resolution may also occur on a list other than python-dev (for example, distutils-sig for packaging related PEPs that don't immediately affect the standard library). In this case, the "Discussions-To" heading in the PEP will identify the appropriate alternative list where discussion, review and pronouncement on the PEP will occur.""" * we *didn't* update the section about assignment of BDFL-Delegates. Instead, I received a general delegation for packaging metadata interoperability PEPs, and Richard Jones received one for pypi.python.org related PEPs * Richard subsequently passed the latter delegation on to Donald, since Donald had taken over as the lead maintainer for PyPI The section in PEP 1 for CPython BDFL-Delegates reads as follows: ================================= However, whenever a new PEP is put forward, any core developer that believes they are suitably experienced to make the final decision on that PEP may offer to serve as the BDFL's delegate (or "PEP czar") for that PEP. If their self-nomination is accepted by the other core developers and the BDFL, then they will have the authority to approve (or reject) that PEP. ================================= This process can be appropriately described as "volunteering to be blamed" - if a PEP from a less experienced contributor subsequently proves to be a mistake, that's on the BDFL-Delegate for saying "Yes", not on the PEP author for proposing it. Mostly though, it's so there's someone to have the final say on the fiddly little details that go into getting from a general concept to an actual implementation, without getting mired down in design-by-committee on every incidental detail. As PEP authors, we'll also often ask someone else specifically to volunteer as BDFL-Delegate, because we trust their judgement in relation to the topic at hand (e.g. I asked Martin von Löwis to be BDFL-Delegate for the original ensurepip PEP because I knew he was skeptical of the idea, so a design that passed muster with him was likely to have suitably addressed the ongoing maintainability concerns. Guido did something similar when he asked Mark Shannon to be BDFL-Delegate for PEP 484's gradual typing). Regards, Nick. P.S. It's becoming clear to me that I should probably write a companion PEP to PEP 1 that specifically describes distutils-sig's usage of the PEP process (and how that differs from the normal CPython processes), but hopefully this post provides sufficient clarification for now. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

3 12

Did https://mail.python.org/pipermail/python-list/2008-June/467441.html get an answer?
by Robert Collins Oct. 3, 2015

Oct. 3, 2015

I'd like to be able to robustly deal with PKG-INFO and METADATA in pbr, and right now we depend on locale.getpreferredencoding()... if we could depend on utf8 or something that would be great. -Rob -- Robert Collins <rbtcollins(a)hp.com> Distinguished Technologist HP Converged Cloud

2 3

intelcompiler class in distutils
by Hogan, Christopher Oct. 2, 2015

Oct. 2, 2015

Hello, We noticed there are classes in distutils for specific compilers like Borland, Cygwin, MSVC, etc., and are interested in creating a class for the Intel compiler. Is this something that the python developers/community would like to see, or would such a patch have a low chance for acceptance? Chris Hogan Scripting Tools Engineer Scripting, Analyzers and Tools Intel Corporation

3 2

README.rst vs DESCRIPTION.rst
by Thomas Güttler Oct. 1, 2015

Oct. 1, 2015

Zen of Python: There should be one-- and preferably only one --obvious way to do it. I would like to find a default for the description file of a python package. The sampleproject uses DESCRIPTION.rst https://github.com/pypa/sampleproject/blob/master/setup.py But I guess it is more common to use README.rst. For example django uses this file name. Any good reason to **not** use README.rst but a different name like DESCRIPTION.rst? Of course anyone can use the name he wants. I just want an agreement for the name to make life easier for newcomers. I will check this mail thread in a week or two and write a pull request to https://github.com/pypa/sampleproject/blob/master/setup.py if there is an agreement. If an agreement was found, which other documents should be updated? Regards, Thomas Güttler -- http://www.thomas-guettler.de/

8 12

Unable to login to PyPi
by Mike O'Driscoll Sept. 30, 2015

Sept. 30, 2015

Hello, I have been unable to login to the PyPi site for nearly a month now via OpenID (launchpad). I have the following ticket open but have gotten no traction: https://bitbucket.org/pypa/pypi/issues/333/unable-to-login-via-openid Any support would be appreciated. Thanks, -- Mike O'Driscoll

5 8