Distutils-SIG October 2014

distutils-sig@python.org

42 participants
42 discussions

Two fixes (at last)
by Greg Ward 29 Apr '18

29 Apr '18

Hi all -- at long last, I have fixed two problems that a couple people noticed a while ago: * I folded in Amos Latteier's NT patches almost verbatim -- just changed an `os.path.sep == "/"' to `os.name == "posix"' and added some comments bitching about the inadequacy of the current library installation model (I think this is Python's fault, but for now Distutils is slavishly aping the situation in Python 1.5.x) * I fixed the problem whereby running "setup.py install" without doing anything else caused a crash (because 'build' hadn't yet been run). Now, the 'install' command automatically runs 'build' before doing anything; to make this bearable, I added a 'have_run' dictionary to the Distribution class to keep track of which commands have been run. So now not only are command classes singletons, but their 'run' method can only be invoked once -- both restrictions enforced by Distribution. The code is checked into CVS, or you can download a snapshot at http://www.python.org/sigs/distutils-sig/distutils-19990607.tar.gz Hope someone (Amos?) can try the new version under NT. Any takers for Mac OS? BTW, all parties involved in the Great "Where Do We Install Stuff?" Debate should take a good, hard look at the 'set_final_options()' method of the Install class in distutils/install.py; this is where all the policy decisions about where to install files are made. Currently it apes the Python 1.5 situation as closely as I could figure it out. Obviously, this is subject to change -- I just don't know to *what* it will change! Greg -- Greg Ward - software developer gward(a)cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

2 1

Package versioning
by Timothy Docker 29 Apr '18

29 Apr '18

Hi all, I've been aware that the distutils sig has been simmerring away, but until recently it has not been directly relevant to what I do. I like the look of the proposed api, but have one question. Will this support an installed system that has multiple versions of the same package installed simultaneously? If not, then this would seem to be a significant limitation, especially when dependencies between packages are considered. Assuming it does, then how will this be achieved? I am presently managing this with a messy arrangement of symlinks. A package is installed with its version number in it's name, and a separate directory is created for an application with links from the unversioned package name to the versioned one. Then I just set the pythonpath to this directory. A sample of what the directory looks like is shown below. I'm sure there is a better solution that this, and I'm not sure that this would work under windows anyway (does windows have symlinks?). So, has this SIG considered such versioning issues yet? Cheers, Tim -------------------------------------------------------------- Tim Docker timd(a)macquarie.com.au Quantative Applications Division Macquarie Bank -------------------------------------------------------------- qad16:qad $ ls -l lib/python/ total 110 drwxr-xr-x 2 mts mts 512 Nov 11 11:23 1.1 -r--r----- 1 root mts 45172 Sep 1 1998 cdrmodule_0_7_1.so drwxr-xr-x 2 mts mts 512 Sep 1 1998 chart_1_1 drwxr-xr-x 3 mts mts 512 Sep 1 1998 Fnorb_0_7_1 dr-xr-x--- 3 mts mts 512 Nov 11 11:21 Fnorb_0_8 drwxr-xr-x 3 mts mts 1536 Mar 3 12:45 mts_1_1 dr-xr-x--- 7 mts mts 512 Nov 11 11:22 OpenGL_1_5_1 dr-xr-x--- 2 mts mts 1024 Nov 11 11:23 PIL_0_3 drwxr-xr-x 3 mts mts 512 Sep 1 1998 Pmw_0_7 dr-xr-x--- 2 mts mts 512 Nov 11 11:21 v3d_1_1 qad16:qad $ ls -l lib/python/1.1 total 30 lrwxrwxrwx 1 root other 29 Apr 10 10:43 _glumodule.so -> ../OpenGL_1_5_1/_glumodule.so lrwxrwxrwx 1 root other 30 Apr 10 10:43 _glutmodule.so -> ../OpenGL_1_5_1/_glutmodule.so lrwxrwxrwx 1 root other 22 Apr 10 10:43 _imaging.so -> ../PIL_0_3/_imaging.so lrwxrwxrwx 1 root other 36 Apr 10 10:43 _opengl_nummodule.so -> ../OpenGL_1_5_1/_opengl_nummodule.so lrwxrwxrwx 1 root other 27 Apr 10 10:43 _tkinter.so -> ../OpenGL_1_5_1/_tkinter.so lrwxrwxrwx 1 mts mts 21 Apr 10 10:43 cdrmodule.so -> ../cdrmodule_0_7_1.so lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 chart -> ../chart_1_1 lrwxrwxrwx 1 root other 12 Apr 10 10:43 Fnorb -> ../Fnorb_0_8 lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 mts -> ../mts_1_1 lrwxrwxrwx 1 root other 15 Apr 10 10:43 OpenGL -> ../OpenGL_1_5_1 lrwxrwxrwx 1 root other 33 Apr 10 10:43 opengltrmodule.so -> ../OpenGL_1_5_1/opengltrmodule.so lrwxrwxrwx 1 root other 33 Apr 10 10:43 openglutil_num.so -> ../OpenGL_1_5_1/openglutil_num.so lrwxrwxrwx 1 root other 10 Apr 10 10:43 PIL -> ../PIL_0_3 lrwxrwxrwx 1 mts mts 10 Apr 10 10:43 Pmw -> ../Pmw_0_7 lrwxrwxrwx 1 root other 10 Apr 10 10:43 v3d -> ../v3d_1_1

3 2

[Distutils-sig] distutils charter and interscript
by John Skaller 29 Apr '18

29 Apr '18

3 2

force static linking
by gordon 24 Mar '15

24 Mar '15

Hello, I am attempting to build statically linked distributions. I am using docker containers to ensure the deployment environment matches the build environment so there is no compatibility concern. Is there any way to force static linking so that wheels can be installed into a virtual env without requiring specific packages on the host? Thanks, Gordon

7 7

Efficiently supporting multiple platforms in distribution packages
by Paul Sokolovsky 04 Nov '14

04 Nov '14

Hello, I just saw another topic posted, https://mail.python.org/pipermail/distutils-sig/2014-October/025140.html , which raises question similar to which I had in mind for some time. To not hijack that thread, I open a new one, but comment to a message from that thread, https://mail.python.org/pipermail/distutils-sig/2014-October/025142.html So, my case: support different platforms in one distribution package. To give more concrete example, let it be some module implemented using ctypes, with completely different implementation for Linux, MacOSX, and Windows. I'd also like to avoid installing files unneeded for particular platform (a usecase applies to MicroPython http://micropython.org/ , where there can be simply not enough storage space to install cruft). On Mon, 27 Oct 2014 14:04:38 +0000 Paul Moore <p.f.moore@...> wrote: > For a source distribution, you could play clever games in setup.py to > put the right file in place, with the right name. But that's messy and > it means that if you distribute wheels (not that there's much point in > doing so) you need separate wheels for 2.6-, 2.7 and 3.3+. Ok, so are there guidelines, best practices or at least example(s) how to do that? I pretty much would like to avoid inventing my own "clever games" to achieve that. > Alternatively, you could distribute all 3 files, as > > dbf > \ > - __init__.py > - dbf_26.py > - dbf_27.py > - dbf_3.py > > Then in __init__.py do > > if sys.version_info[0] == 3: > from .dbf_3 import * > elif sys.version_info[:2] == (2, 7): > from .dbf_27 import * > else > from .dbf_26 import * For our MicroPython case, we would like to avoid this due to aforementioned reasons. We could probably post-process installation dir after pip to remove unneeded files, but that sounds like hack - we'd rather do it fully on distribution package level, and target to install just a single source file per module, and avoid dispatcher like above (again, for efficiency reasons). There's also another issue - of supporting "cross-installs". Not all MicroPython targets support running pip natively, so it instead runs on a host computer, but would need be instructed to select source variants for a particular target platform. Any hints/pointers on how to achieve this - preferrably in "standard" way - are appreciated! -- Best regards, Paul mailto:pmiscml@gmail.com

3 3

buildout 2.2.3: not completely right?
by Reinout van Rees 04 Nov '14

04 Nov '14

Hi, Since buildout 2.2.3 is out, I've been getting mails out of quite some jenkins jobs. Those jobs always run "python bootstrap.py" followed by "bin/buildout". We always pin buildout, so most of them are at version 2.2.1 $ python bootstrap.py Setting socket time out to 1 seconds. Generated script '<http://jenkins.3di.lizard.net/job/python-subgrid/ws/bin/buildout'.> $ bin/buildout Setting socket time out to 1 seconds. While: Installing. Loading extensions. Error: There is a version conflict. We already have: zc.buildout 2.2.3 "python bootstrap.py" generates a bin/buildout with a 2.2.3 version in it. And running bin/buildout doesn't want to downgrade to the specified version 2.2.1. The bootstrap.py I'm using is http://downloads.buildout.org/2/bootstrap.py I've seen some other problems with versions when there's a globally installed version already available at a higher version. A colleague installed ansible globally on the jenkins machine (...) so there's a new jinja2 in usr/local/lib/pythonsomething. The jinja2 in buildout is pinned to a lower version, but buildout refuses to downgrade it and fails with an "I already have that new version". So... I haven't totally tracked it down yet, but I'm throwing it out on the list here in case others have seen it too. I'll continue debugging after lunch :-) Reinout -- Reinout van Rees http://reinout.vanrees.org/ reinout(a)vanrees.org http://www.nelen-schuurmans.nl/ "Learning history by destroying artifacts is a time-honored atrocity"

1 5

bandersnatch failing on MovieMacro?
by Robert Collins 31 Oct '14

31 Oct '14

2014-10-31 12:01:05,368 ERROR: Error syncing package: MovieMacro@1288231 Traceback (most recent call last): File "/home/robertc/.virtualenvs/bandersnatch/local/lib/python2.7/site-packages/bandersnatch/pa ckage.py", line 63, in sync self.sync_simple_page() File "/home/robertc/.virtualenvs/bandersnatch/local/lib/python2.7/site-packages/bandersnatch/pa ckage.py", line 106, in sync_simple_page '/simple/{0}/'.format(self.quoted_name), self.serial) File "/home/robertc/.virtualenvs/bandersnatch/local/lib/python2.7/site-packages/bandersnatch/ma ster.py", line 44, in get r.raise_for_status() File "/home/robertc/.virtualenvs/bandersnatch/local/lib/python2.7/site-packages/requests/models .py", line 773, in raise_for_status raise HTTPError(http_error_msg, response=self) HTTPError: 404 Client Error: Not Found 2014-10-31 12:01:05,382 INFO: Generating global index page. This is aborting mirror syncs causing new packages etc not to propogate. Is this a bandersnatch bug (e.g. it should keep trying othe things) or server side issue? -Rob -- Robert Collins <rbtcollins(a)hp.com> Distinguished Technologist HP Converged Cloud

1 0

PEP425 - Wheel Binary Package Compatibility
by Matthias Hafner 30 Oct '14

30 Oct '14

Hi there, Following up on https://bitbucket.org/pypa/wheel/issue/124/glibc-incompatibility. How should we deal with incompatibility of dynamically linked libraries? Doing "pip wheel numpy" on a Linux 64bit machine results in , which is linked dynamically against the GLIBC version installed on the build machine. So should the wheel be shipped with GLIBC, or the GLIBC version be specified in the wheel name (means to build a new wheel for each GLIBC version?). Also, maybe this is not the only library it is dynamically linked against? Why does that work on MacOS, btw? Are all library versions fixed there for one version of OSX? Thanks for putting some light into this issue. Best regards, Matthias

11 17

a package with is a module
by Ethan Furman 28 Oct '14

28 Oct '14

If there is an answer, tutorial, readme, or docs that would help, a link would be greatly appreciated. I have finally converted my dbf library to Python3, but I want to also continue supporting at least 2.7, and I see no reason to remove the existing library which also supports back to 2.4. I've never been very happy with the egg format (cool name, but having the source zipped up is irritating when one wants to look at it) and so I have left this library as a couple modules: dbf.py, and dbf_test.py. So I have multiple problems: - how do I tell PyPI that this file is for Python2.4 - 2.6, this other file is for 2.7, and this other other file is for 3.3+ ? - if I have to stick it all in one archive, how do I tell setup.py which to install? - is it possible to have all three source files as, say, .txt files, and then have some Python code before the setup() call which renames the correct one to dbf.py? How do I know where the .txt files are at to rename them? I have skimmed http://python-packaging-user-guide.readthedocs.org/en/latest/, but didn't find what I was looking for there. Thanks for any help. -- ~Ethan~

5 8

"not a supported wheel on this platform"
by Antoine Pitrou 28 Oct '14

28 Oct '14

Hello, Ok, so until I find a better solution I've tried hosting a wheel on a personal server, but then I get the following error when installing it: $ pyvenv-3.4 t $ source t/bin/activate (t) $ pip install https://ssl.pitrou.net/llvmlite-0.1-py2.py3-none-linux_x86_64.whl llvmlite-0.1-py2.py3-none-linux_x86_64.whl is not a supported wheel on this platform. Storing debug log for failure in /home/antoine/.pip/pip.log Yet: (t) $ python Python 3.4.1 (3.4:d1bf37def4fd, May 19 2014, 19:52:53) [GCC 4.8.1] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import sysconfig >>> sysconfig.get_platform() 'linux-x86_64' The log file doesn't tell anything really interesting: ------------------------------------------------------------ /home/antoine/t/t/bin/pip run on Tue Oct 28 15:18:06 2014 llvmlite-0.1-py2.py3-none-linux_x86_64.whl is not a supported wheel on this platform. Exception information: Traceback (most recent call last): File "/home/antoine/t/t/lib/python3.4/site-packages/pip/basecommand.py", line 122, in main status = self.run(options, args) File "/home/antoine/t/t/lib/python3.4/site-packages/pip/commands/install.py", line 257, in run InstallRequirement.from_line(name, None)) File "/home/antoine/t/t/lib/python3.4/site-packages/pip/req.py", line 167, in from_line raise UnsupportedWheel("%s is not a supported wheel on this platform." % wheel.filename) pip.exceptions.UnsupportedWheel: llvmlite-0.1-py2.py3-none-linux_x86_64.whl is not a supported wheel on this platform. Regards Antoine.

2 5

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

Distutils-SIG October 2014