Mailman 3 May 2012 - Distutils-SIG

`pysetup create`, but `pysetup run install_dist` - wtf?
by anatoly techtonik 29 May '12

29 May '12

Hi, http://docs.python.org/dev/packaging/tutorial.html#running-commands What's the point in using 'pysetup run install_dist' instead of just 'pysetup install' ?? -- anatoly t.

2 2

DU2 1.0a5
by Rob Healey 28 May '12

28 May '12

Dear Eric: Do you know when we might be expecting this version release after this last sprint, you stated that it might be soon... Do you know when the next sprint will be scheduled? -- Sincerely yours, Rob G. Healey

2 1

building a shared lib
by Brown, Christopher 28 May '12

28 May '12

Hi, I am developing a Python package and use distutils. The package relies on a shared lib, for which the build process is slightly more than trivial. So, I have created a shell script to build the lib (lets call that buildlib.sh). Then, I have another script in the package root (buildpackage.sh) that first calls buildlib.sh, and the runs the setup.py script to build and/or install the package. This works fine, but I would like to allow users to just run python setup.py build/install to simplify things for them. I can call buildlib.sh from within setup.py, but I am unsure of the right way to do it. Should I just call buildlib.sh everytime from setup.py, regardless of whether it is with 'build' or 'install'? Can I detect if 'build' is being run (whether from 'setup.py build' or 'setup.py install') and only run it then? Is there a better way to handle this? Thanks! -Chris -- Christopher Brown, Ph.D. Assistant Professor Department of Communication Science and Disorders University of Pittsburgh

2 1

bin/buildout hoses itself when the buildout directory has a long path
by PJ Eby 27 May '12

27 May '12

AFAICT, this problem was only reported once, here: https://mail.zope.org/pipermail/zope-dev/2012-February/044108.html but nobody identified the cause or steps to reproduce. What I've figured out is this: if your buildout's absolute path is sufficiently long (I believe it's just if it's longer than your system site-packages' absolute path), then running bin/buildout (vs. the system buildout) will decide NOT to include eggs/zc.buildout in the buildout_paths list in parts/buildout/site.py. The result of this is that the zc.buildout egg gets left out of pkg_resources.working_set, which means that when buildout tries to find itself in the working set, it fails, resulting in the following error: $ bin/buildout Upgraded: zc.buildout version 1.5.2; restarting. Traceback (most recent call last): File "bin/buildout", line 17, in <module> import zc.buildout.buildout File "/usr/lib/python2.6/site-packages/zc.buildout-1.5.2-py2.6.egg/zc/buildout/buildout.py", line 39, in <module> import zc.buildout.download File "/usr/lib/python2.6/site-packages/zc.buildout-1.5.2-py2.6.egg/zc/buildout/download.py", line 20, in <module> from zc.buildout.easy_install import realpath File "/usr/lib/python2.6/site-packages/zc.buildout-1.5.2-py2.6.egg/zc/buildout/easy_install.py", line 81, in <module> pkg_resources.Requirement.parse('zc.buildout')).location AttributeError: 'NoneType' object has no attribute 'location' Notice that the buildout is using the site-packages zc.buildout egg, instead of the one in the local eggs directory. Steps to reproduce: $ cd really/really/long/directory/name/with/lots/of/stuff/emptydir $ buildout init $ bin/buildout Workaround: use "include-site-packages = false" in the [buildout] section. This strongly suggests to me that some part of the site-packages isolation mechanism is broken, at least for zc.buildout itself. It is successfully allowing zc.buildout to be imported from site-packages, but not letting setuptools see that it is present in site-packages. If it does this for other packages that refer to themselves in the way zc.buildout.easy_install does, they will similarly break. (Likewise anything expecting to find entry points, etc.) On a somewhat unrelated note, I am as of yesterday a new zc.buildout user... and I already have a buildout that compiles Erlang, Spidermonkey, CouchDB, and RabbitMQ, and sets them all up with a Celery daemon under Supervisord, all in about 63 lines of .ini file. To say that I am impressed with buildout is an extreme understatement. ;-)

1 0

entry_points.txt survey
by Daniel Holth 24 May '12

24 May '12

It looks like you can call pkg_resources.register_finder() with a function that also yields distributions with a .dist-info directory (though you would have to override the existing find_on_path). From there I think pkg_resources.load_entry_point() is likely to work. >From a survey of 16984 pypi source distributions (the newest for each package surveyed), 5878 define entry_points.txt but 1399 of those are empty. Some of the most popular sections are: (10, 'babel.extractors'), (10, 'lava_server.extensions'), (10, 'paste.composite_factory'), (10, 'turbogears.extensions'), (10, 'yafowil.plugin'), (10, 'zest.releaser.prereleaser.middle'), (11, 'chandler.parcels'), (11, 'rsl.register'), (12, 'hurry.resource.libraries'), (15, 'paste.global_paster_command'), (16, 'pyramid.scaffold'), (17, 'zc.buildout.uninstall'), (18, 'paste.server_runner'), (18, 'setuptools.installation'), (19, 'python.templating.engines'), (20, 'redsolutioncms'), (20, 'zope2.initialize'), (22, 'pytest11'), (23, 'setuptools.file_finders'), (26, 'paste.paster_command'), (30, 'nose.plugins'), (31, 'paste.filter_factory'), (31, 'zc.buildout.extension'), (35, 'toscawidgets.widgets'), (35, 'turbogears.widgets'), (37, 'tw2.widgets'), (47, 'paste.app_install'), (69, 'nose.plugins.0.10'), (73, 'distutils.commands'), (81, 'trytond.modules'), (82, 'gui_scripts'), (83, 'paste.filter_app_factory'), (94, 'fanstatic.libraries'), (96, 'trac.plugins'), (110, 'egg_info.writers'), (120, 'distutils.setup_keywords'), (144, 'paste.paster_create_template'), (190, 'paste.app_factory'), (354, 'zc.buildout'), (1004, 'z3c.autoinclude.plugin'), (1754, 'console_scripts')

2 1

pypm binary package format
by Daniel Holth 23 May '12

23 May '12

I was curious about the pypm binary package format. They are in the .tar.gz format (but with .pypm extension). They contain two files: info.json: {u'author': u'Zope Foundation and Contributors', u'author_email': u'zope-dev(a)zope.org', u'description': '...', u'home_page': u'http://pypi.python.org/pypi/zope.interface', u'install_requires': {u'': [u'distribute'], u'docs': [u'z3c.recipe.sphinxdoc'], u'test': [u'zope.event']}, u'keywords': None, u'license': u'ZPL 2.1', u'maintainer': None, u'maintainer_email': None, u'name': u'zope.interface', u'osarch': u'win32-x86', u'pkg_version': 1, u'pyver': u'2.7', u'summary': u'Interfaces for Python', u'version': u'3.8.0'} data.tar.gz: Lib/site-packages/zope.interface/... (including .pyd or .so) Lib/site-packages/zope.interface-3.8.0-py27.egg-info/(normal egg-info stuff) When installed, Lib/ is unpacked into %APPDATA%\Python\Python27\site-packages\

1 0

command hooks...
by Oscar Benjamin 23 May '12

23 May '12

On 23 May 2012 07:48, Tarek Ziadé <tarek(a)ziade.org> wrote: > On 5/23/12 8:16 AM, Robert Park wrote: > >> On Mon, May 21, 2012 at 3:42 PM, Tarek Ziadé<tarek(a)ziade.org> wrote: >> >>> If you are a packager for a project, you can describe in details your >>> data >>> files, and add more metadata that are understood by PyPI. >>> >>> If you are a debian packager, you will be able to define where the data >>> files of a python project should be installed without having to patch >>> some >>> python code. >>> >> This right here is the killer feature for me. By my limited >> observations, most people solve the data files problem either by >> dumping their data files inside their python modules (which is an ugly >> abuse of the filesystem), or are simply using the cumbersome autotools >> in order to record the installation prefix for their data files. It >> blows my mind that it is standard practice for many GNOME apps written >> in Python to use a C compilation preprocessor in order to set a python >> variable so their python scripts can find their data files. >> >> Currently I am hacking distutils in order to accomplish this, like so: >> >> https://github.com/robru/**gottengeography/blob/master/**setup.py<https://github.com/robru/gottengeography/blob/master/setup.py> >> >> But this is ugly because it modifies the file in-place, so I always >> have to be careful not to accidentally commit the munged file into my >> git repo. >> >> It's absolutely critical that any replacement for distutils have >> built-in functionality for installed python code being able to query >> at run-time the location that data files were placed at install time. >> >> Please read this section and let us know what you think: > > http://docs.python.org/dev/**packaging/setupcfg.html#**resources<http://docs.python.org/dev/packaging/setupcfg.html#resources> How do gettext translation files fit into this scheme? Should they go under appdata? > > > This works in conjunction with the new sysconfig module, which can be > configured system-wide by the linux distribution, or locally per projet > > Then you can use an API to get the file from your code. > > Gosh the documentation is a mess ... we need to fix this - it has bits > from the previous version that should be removed :( > > Cheers > Tarek > > ______________________________**_________________ > Distutils-SIG maillist - Distutils-SIG(a)python.org > http://mail.python.org/**mailman/listinfo/distutils-sig<http://mail.python.org/mailman/listinfo/distutils-sig> >

1 0

packaging in Python-3.3.0 ???
by Rob Healey 19 May '12

19 May '12

Hello once agai: Is packaging in Python-3.3.0 a more complete software than DU2??? If it is, then I will start using it instead...? -- Sincerely yours, Rob G. Healey

2 1

Distutils2 in Python-3.3.0
by Rob Healey 18 May '12

18 May '12

Good evening to all! I am also willing to help out too... What is the process for submitting patches? But I must, in all respects, patches are USELESS without someone to commit them!!! There needs to be more than one person who can commit the patches... -- Sincerely yours, Rob G. Healey

2 1

Odd problem with distribute 0.6.26 and 2to3
by Vinay Sajip 18 May '12

18 May '12

I'm having an odd problem with distribute and 2to3 when working with the venv branch of Python 3.3 (PEP 405 - http://bugs.python.org/issue14712), and I'm hoping one of the Fellowship of the Packaging can help. The problem: I created a venv with no Distribute installed. Then I downloaded and extracted distribute-0.6.26.tar.gz in a scratch folder, and ran setup.py install using the venv's python. This gave errors because some of the setuptools sources still have 2.x syntax, despite having been processed by 2to3. To investigate further, I added logging to distutils refactoring code to print what was being refactored. I won't list the entire log output [1], but when looking at just one file which caused a SyntaxError - setuptools/tests/test_packageindex.py - I see the following output from the "setup.py install" operation: creating build creating build/src ... copying setuptools/tests/test_packageindex.py -> build/src/setuptools/tests ... copying launcher.c -> build/src Skipping implicit fixer: buffer Skipping implicit fixer: idioms Skipping implicit fixer: set_literal Skipping implicit fixer: ws_comma refactored build/src/setuptools/script template (dev).py ... refactored build/src/setuptools/tests/test_packageindex.py ... refactored build/src/release.py Before install bootstrap. Scanning installed packages No setuptools distribution found running install running build running build_py creating build/lib ... copying setuptools/tests/test_packageindex.py -> build/lib/setuptools/tests ... creating /tmp/venv/lib/python3.3/site-packages/setuptools/tests copying build/lib/setuptools/tests/test_packageindex.py -> /tmp/venv/lib/python3.3/site-packages/setuptools/tests ... running install_lib ... creating /tmp/venv/lib/python3.3/site-packages/setuptools creating /tmp/venv/lib/python3.3/site-packages/setuptools/tests copying build/lib/setuptools/tests/test_packageindex.py -> /tmp/venv/lib/python3.3/site-packages/setuptools/tests ... byte-compiling /tmp/venv/lib/python3.3/site- packages/setuptools/tests/test_packageindex.py to test_packageindex.cpython- 33.pyc ... running install_egg_info Writing /tmp/venv/lib/python3.3/site-packages/distribute-0.6.26-py3.3.egg-info setup.py:139: ResourceWarning: unclosed file <_io.TextIOWrapper name='README.txt' mode='r' encoding='UTF-8'> long_description = open('README.txt').read() + open('CHANGES.txt').read(), setup.py:139: ResourceWarning: unclosed file <_io.TextIOWrapper name='CHANGES.txt' mode='r' encoding='UTF-8'> long_description = open('README.txt').read() + open('CHANGES.txt').read(), /home/vinay/projects/python/sandbox/Lib/distutils/dist.py:257: UserWarning: Unknown distribution option: 'test_suite' warnings.warn(msg) /home/vinay/projects/python/sandbox/Lib/distutils/dist.py:257: UserWarning: Unknown distribution option: 'entry_points' warnings.warn(msg) /home/vinay/projects/python/sandbox/Lib/distutils/dist.py:257: UserWarning: Unknown distribution option: 'zip_safe' warnings.warn(msg) File "/tmp/venv/lib/python3.3/site- packages/setuptools/tests/test_packageindex.py", line 17 except Exception, v: ^ SyntaxError: invalid syntax Now the first part seems straightfoward: files are copied to build/src and then 2to3 is run over them. But then, following the "No setuptools distribution found", files are copied to build/lib not from the 2to3-processed output at build/src, but rather the actual 2.x sources shipped with distribute. That looks wrong; at the point where setup is called, src_root is correctly set to build/src, and moreover, that's the first entry on sys.path. I'm not sure why there's no error immediately following the "byte-compiling ..." line when running install_lib, but the error seems to occur a little later, when running install_egg_info ... Can anyone shed any light on what's happening here? Regards, Vinay Sajip [1] https://gist.github.com/2725747

2 1