Dear Eric:
Do you know when we might be expecting this version release after this last
sprint, you stated that it might be soon...
Do you know when the next sprint will be scheduled?
--
Sincerely yours,
Rob G. Healey
Hi,
I am developing a Python package and use distutils. The package relies
on a shared lib, for which the build process is slightly more than
trivial. So, I have created a shell script to build the lib (lets call
that buildlib.sh). Then, I have another script in the package root
(buildpackage.sh) that first calls buildlib.sh, and the runs the
setup.py script to build and/or install the package.
This works fine, but I would like to allow users to just run python
setup.py build/install to simplify things for them. I can call
buildlib.sh from within setup.py, but I am unsure of the right way to do
it. Should I just call buildlib.sh everytime from setup.py, regardless
of whether it is with 'build' or 'install'? Can I detect if 'build' is
being run (whether from 'setup.py build' or 'setup.py install') and only
run it then? Is there a better way to handle this?
Thanks!
-Chris
--
Christopher Brown, Ph.D.
Assistant Professor
Department of Communication Science and Disorders
University of Pittsburgh
AFAICT, this problem was only reported once, here:
https://mail.zope.org/pipermail/zope-dev/2012-February/044108.html but
nobody identified the cause or steps to reproduce.
What I've figured out is this: if your buildout's absolute path is
sufficiently long (I believe it's just if it's longer than your system
site-packages' absolute path), then running bin/buildout (vs. the system
buildout) will decide NOT to include eggs/zc.buildout in the buildout_paths
list in parts/buildout/site.py. The result of this is that the zc.buildout
egg gets left out of pkg_resources.working_set, which means that when
buildout tries to find itself in the working set, it fails, resulting in
the following error:
$ bin/buildout
Upgraded:
zc.buildout version 1.5.2;
restarting.
Traceback (most recent call last):
File "bin/buildout", line 17, in <module>
import zc.buildout.buildout
File
"/usr/lib/python2.6/site-packages/zc.buildout-1.5.2-py2.6.egg/zc/buildout/buildout.py",
line 39, in <module>
import zc.buildout.download
File
"/usr/lib/python2.6/site-packages/zc.buildout-1.5.2-py2.6.egg/zc/buildout/download.py",
line 20, in <module>
from zc.buildout.easy_install import realpath
File
"/usr/lib/python2.6/site-packages/zc.buildout-1.5.2-py2.6.egg/zc/buildout/easy_install.py",
line 81, in <module>
pkg_resources.Requirement.parse('zc.buildout')).location
AttributeError: 'NoneType' object has no attribute 'location'
Notice that the buildout is using the site-packages zc.buildout egg,
instead of the one in the local eggs directory.
Steps to reproduce:
$ cd really/really/long/directory/name/with/lots/of/stuff/emptydir
$ buildout init
$ bin/buildout
Workaround: use "include-site-packages = false" in the [buildout] section.
This strongly suggests to me that some part of the site-packages isolation
mechanism is broken, at least for zc.buildout itself. It is successfully
allowing zc.buildout to be imported from site-packages, but not letting
setuptools see that it is present in site-packages. If it does this for
other packages that refer to themselves in the way zc.buildout.easy_install
does, they will similarly break. (Likewise anything expecting to find
entry points, etc.)
On a somewhat unrelated note, I am as of yesterday a new zc.buildout
user... and I already have a buildout that compiles Erlang, Spidermonkey,
CouchDB, and RabbitMQ, and sets them all up with a Celery daemon under
Supervisord, all in about 63 lines of .ini file. To say that I am
impressed with buildout is an extreme understatement. ;-)
It looks like you can call pkg_resources.register_finder() with a
function that also yields distributions with a .dist-info directory
(though you would have to override the existing find_on_path). From
there I think pkg_resources.load_entry_point() is likely to work.
>From a survey of 16984 pypi source distributions (the newest for each
package surveyed), 5878 define entry_points.txt but 1399 of those are
empty. Some of the most popular sections are:
(10, 'babel.extractors'),
(10, 'lava_server.extensions'),
(10, 'paste.composite_factory'),
(10, 'turbogears.extensions'),
(10, 'yafowil.plugin'),
(10, 'zest.releaser.prereleaser.middle'),
(11, 'chandler.parcels'),
(11, 'rsl.register'),
(12, 'hurry.resource.libraries'),
(15, 'paste.global_paster_command'),
(16, 'pyramid.scaffold'),
(17, 'zc.buildout.uninstall'),
(18, 'paste.server_runner'),
(18, 'setuptools.installation'),
(19, 'python.templating.engines'),
(20, 'redsolutioncms'),
(20, 'zope2.initialize'),
(22, 'pytest11'),
(23, 'setuptools.file_finders'),
(26, 'paste.paster_command'),
(30, 'nose.plugins'),
(31, 'paste.filter_factory'),
(31, 'zc.buildout.extension'),
(35, 'toscawidgets.widgets'),
(35, 'turbogears.widgets'),
(37, 'tw2.widgets'),
(47, 'paste.app_install'),
(69, 'nose.plugins.0.10'),
(73, 'distutils.commands'),
(81, 'trytond.modules'),
(82, 'gui_scripts'),
(83, 'paste.filter_app_factory'),
(94, 'fanstatic.libraries'),
(96, 'trac.plugins'),
(110, 'egg_info.writers'),
(120, 'distutils.setup_keywords'),
(144, 'paste.paster_create_template'),
(190, 'paste.app_factory'),
(354, 'zc.buildout'),
(1004, 'z3c.autoinclude.plugin'),
(1754, 'console_scripts')
I was curious about the pypm binary package format. They are in the
.tar.gz format (but with .pypm extension). They contain two files:
info.json:
{u'author': u'Zope Foundation and Contributors',
u'author_email': u'zope-dev(a)zope.org',
u'description': '...',
u'home_page': u'http://pypi.python.org/pypi/zope.interface',
u'install_requires': {u'': [u'distribute'],
u'docs': [u'z3c.recipe.sphinxdoc'],
u'test': [u'zope.event']},
u'keywords': None,
u'license': u'ZPL 2.1',
u'maintainer': None,
u'maintainer_email': None,
u'name': u'zope.interface',
u'osarch': u'win32-x86',
u'pkg_version': 1,
u'pyver': u'2.7',
u'summary': u'Interfaces for Python',
u'version': u'3.8.0'}
data.tar.gz:
Lib/site-packages/zope.interface/... (including .pyd or .so)
Lib/site-packages/zope.interface-3.8.0-py27.egg-info/(normal egg-info stuff)
When installed, Lib/ is unpacked into %APPDATA%\Python\Python27\site-packages\
On 23 May 2012 07:48, Tarek Ziadé <tarek(a)ziade.org> wrote:
> On 5/23/12 8:16 AM, Robert Park wrote:
>
>> On Mon, May 21, 2012 at 3:42 PM, Tarek Ziadé<tarek(a)ziade.org> wrote:
>>
>>> If you are a packager for a project, you can describe in details your
>>> data
>>> files, and add more metadata that are understood by PyPI.
>>>
>>> If you are a debian packager, you will be able to define where the data
>>> files of a python project should be installed without having to patch
>>> some
>>> python code.
>>>
>> This right here is the killer feature for me. By my limited
>> observations, most people solve the data files problem either by
>> dumping their data files inside their python modules (which is an ugly
>> abuse of the filesystem), or are simply using the cumbersome autotools
>> in order to record the installation prefix for their data files. It
>> blows my mind that it is standard practice for many GNOME apps written
>> in Python to use a C compilation preprocessor in order to set a python
>> variable so their python scripts can find their data files.
>>
>> Currently I am hacking distutils in order to accomplish this, like so:
>>
>> https://github.com/robru/**gottengeography/blob/master/**setup.py<https://github.com/robru/gottengeography/blob/master/setup.py>
>>
>> But this is ugly because it modifies the file in-place, so I always
>> have to be careful not to accidentally commit the munged file into my
>> git repo.
>>
>> It's absolutely critical that any replacement for distutils have
>> built-in functionality for installed python code being able to query
>> at run-time the location that data files were placed at install time.
>>
>> Please read this section and let us know what you think:
>
> http://docs.python.org/dev/**packaging/setupcfg.html#**resources<http://docs.python.org/dev/packaging/setupcfg.html#resources>
How do gettext translation files fit into this scheme? Should they go under
appdata?
>
>
> This works in conjunction with the new sysconfig module, which can be
> configured system-wide by the linux distribution, or locally per projet
>
> Then you can use an API to get the file from your code.
>
> Gosh the documentation is a mess ... we need to fix this - it has bits
> from the previous version that should be removed :(
>
> Cheers
> Tarek
>
> ______________________________**_________________
> Distutils-SIG maillist - Distutils-SIG(a)python.org
> http://mail.python.org/**mailman/listinfo/distutils-sig<http://mail.python.org/mailman/listinfo/distutils-sig>
>
Hello once agai:
Is packaging in Python-3.3.0 a more complete software than DU2??? If it
is, then I will start using it instead...?
--
Sincerely yours,
Rob G. Healey
Good evening to all!
I am also willing to help out too...
What is the process for submitting patches? But I must, in all respects,
patches are USELESS without someone to commit them!!!
There needs to be more than one person who can commit the patches...
--
Sincerely yours,
Rob G. Healey
I'm having an odd problem with distribute and 2to3 when working with the venv
branch of Python 3.3 (PEP 405 - http://bugs.python.org/issue14712), and I'm
hoping one of the Fellowship of the Packaging can help.
The problem: I created a venv with no Distribute installed. Then I downloaded
and extracted distribute-0.6.26.tar.gz in a scratch folder, and ran setup.py
install using the venv's python. This gave errors because some of the setuptools
sources still have 2.x syntax, despite having been processed by 2to3.
To investigate further, I added logging to distutils refactoring code to print
what was being refactored. I won't list the entire log output [1], but when
looking at just one file which caused a SyntaxError -
setuptools/tests/test_packageindex.py - I see the following output from the
"setup.py install" operation:
creating build
creating build/src
...
copying setuptools/tests/test_packageindex.py -> build/src/setuptools/tests
...
copying launcher.c -> build/src
Skipping implicit fixer: buffer
Skipping implicit fixer: idioms
Skipping implicit fixer: set_literal
Skipping implicit fixer: ws_comma
refactored build/src/setuptools/script template (dev).py
...
refactored build/src/setuptools/tests/test_packageindex.py
...
refactored build/src/release.py
Before install bootstrap.
Scanning installed packages
No setuptools distribution found
running install
running build
running build_py
creating build/lib
...
copying setuptools/tests/test_packageindex.py -> build/lib/setuptools/tests
...
creating /tmp/venv/lib/python3.3/site-packages/setuptools/tests
copying build/lib/setuptools/tests/test_packageindex.py ->
/tmp/venv/lib/python3.3/site-packages/setuptools/tests
...
running install_lib
...
creating /tmp/venv/lib/python3.3/site-packages/setuptools
creating /tmp/venv/lib/python3.3/site-packages/setuptools/tests
copying build/lib/setuptools/tests/test_packageindex.py ->
/tmp/venv/lib/python3.3/site-packages/setuptools/tests
...
byte-compiling /tmp/venv/lib/python3.3/site-
packages/setuptools/tests/test_packageindex.py to test_packageindex.cpython-
33.pyc
...
running install_egg_info
Writing /tmp/venv/lib/python3.3/site-packages/distribute-0.6.26-py3.3.egg-info
setup.py:139: ResourceWarning: unclosed file <_io.TextIOWrapper
name='README.txt' mode='r' encoding='UTF-8'>
long_description = open('README.txt').read() + open('CHANGES.txt').read(),
setup.py:139: ResourceWarning: unclosed file <_io.TextIOWrapper
name='CHANGES.txt' mode='r' encoding='UTF-8'>
long_description = open('README.txt').read() + open('CHANGES.txt').read(),
/home/vinay/projects/python/sandbox/Lib/distutils/dist.py:257: UserWarning:
Unknown distribution option: 'test_suite'
warnings.warn(msg)
/home/vinay/projects/python/sandbox/Lib/distutils/dist.py:257: UserWarning:
Unknown distribution option: 'entry_points'
warnings.warn(msg)
/home/vinay/projects/python/sandbox/Lib/distutils/dist.py:257: UserWarning:
Unknown distribution option: 'zip_safe'
warnings.warn(msg)
File "/tmp/venv/lib/python3.3/site-
packages/setuptools/tests/test_packageindex.py", line 17
except Exception, v:
^
SyntaxError: invalid syntax
Now the first part seems straightfoward: files are copied to build/src and then
2to3 is run over them. But then, following the "No setuptools distribution
found", files are copied to build/lib not from the 2to3-processed output at
build/src, but rather the actual 2.x sources shipped with distribute. That looks
wrong; at the point where setup is called, src_root is correctly set to
build/src, and moreover, that's the first entry on sys.path. I'm not sure why
there's no error immediately following the "byte-compiling ..." line when
running install_lib, but the error seems to occur a little later, when running
install_egg_info ...
Can anyone shed any light on what's happening here?
Regards,
Vinay Sajip
[1] https://gist.github.com/2725747
