Hi all,
I've been aware that the distutils sig has been simmerring away, but
until recently it has not been directly relevant to what I do.
I like the look of the proposed api, but have one question. Will this
support an installed system that has multiple versions of the same
package installed simultaneously? If not, then this would seem to be a
significant limitation, especially when dependencies between packages
are considered.
Assuming it does, then how will this be achieved? I am presently
managing this with a messy arrangement of symlinks. A package is
installed with its version number in it's name, and a separate
directory is created for an application with links from the
unversioned package name to the versioned one. Then I just set the
pythonpath to this directory.
A sample of what the directory looks like is shown below.
I'm sure there is a better solution that this, and I'm not sure that
this would work under windows anyway (does windows have symlinks?).
So, has this SIG considered such versioning issues yet?
Cheers,
Tim
--------------------------------------------------------------
Tim Docker timd(a)macquarie.com.au
Quantative Applications Division
Macquarie Bank
--------------------------------------------------------------
qad16:qad $ ls -l lib/python/
total 110
drwxr-xr-x 2 mts mts 512 Nov 11 11:23 1.1
-r--r----- 1 root mts 45172 Sep 1 1998 cdrmodule_0_7_1.so
drwxr-xr-x 2 mts mts 512 Sep 1 1998 chart_1_1
drwxr-xr-x 3 mts mts 512 Sep 1 1998 Fnorb_0_7_1
dr-xr-x--- 3 mts mts 512 Nov 11 11:21 Fnorb_0_8
drwxr-xr-x 3 mts mts 1536 Mar 3 12:45 mts_1_1
dr-xr-x--- 7 mts mts 512 Nov 11 11:22 OpenGL_1_5_1
dr-xr-x--- 2 mts mts 1024 Nov 11 11:23 PIL_0_3
drwxr-xr-x 3 mts mts 512 Sep 1 1998 Pmw_0_7
dr-xr-x--- 2 mts mts 512 Nov 11 11:21 v3d_1_1
qad16:qad $ ls -l lib/python/1.1
total 30
lrwxrwxrwx 1 root other 29 Apr 10 10:43 _glumodule.so -> ../OpenGL_1_5_1/_glumodule.so
lrwxrwxrwx 1 root other 30 Apr 10 10:43 _glutmodule.so -> ../OpenGL_1_5_1/_glutmodule.so
lrwxrwxrwx 1 root other 22 Apr 10 10:43 _imaging.so -> ../PIL_0_3/_imaging.so
lrwxrwxrwx 1 root other 36 Apr 10 10:43 _opengl_nummodule.so -> ../OpenGL_1_5_1/_opengl_nummodule.so
lrwxrwxrwx 1 root other 27 Apr 10 10:43 _tkinter.so -> ../OpenGL_1_5_1/_tkinter.so
lrwxrwxrwx 1 mts mts 21 Apr 10 10:43 cdrmodule.so -> ../cdrmodule_0_7_1.so
lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 chart -> ../chart_1_1
lrwxrwxrwx 1 root other 12 Apr 10 10:43 Fnorb -> ../Fnorb_0_8
lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 mts -> ../mts_1_1
lrwxrwxrwx 1 root other 15 Apr 10 10:43 OpenGL -> ../OpenGL_1_5_1
lrwxrwxrwx 1 root other 33 Apr 10 10:43 opengltrmodule.so -> ../OpenGL_1_5_1/opengltrmodule.so
lrwxrwxrwx 1 root other 33 Apr 10 10:43 openglutil_num.so -> ../OpenGL_1_5_1/openglutil_num.so
lrwxrwxrwx 1 root other 10 Apr 10 10:43 PIL -> ../PIL_0_3
lrwxrwxrwx 1 mts mts 10 Apr 10 10:43 Pmw -> ../Pmw_0_7
lrwxrwxrwx 1 root other 10 Apr 10 10:43 v3d -> ../v3d_1_1
Hi there,
In the lxml project (http://codespeak.net/lxml), we've just noticed the
following problem with lxml eggs: you can easy_install an egg that won't
work for your Python.
This is because Python can be compiled with either 2 or 4 bytes unicode
as its internal representation. Any egg that contains compiled C code
that uses unicode such as lxml will run into trouble: if it's compiled
with a 4 bytes unicode Python, it won't work on a 2 bytes unicode
Python, and vice versa.
This problem is fairly common in Linux. Many distributions such as
Ubuntu and Fedora compile their python with 4 bytes unicode internal
representation. If you compile a Python interpreter by hand it defaults
to 2 bytes unicode, however. Hand-building a Python interpreter is done
fairly commonly by Linux sysadmins for various reasons.
It would therefore be very nice if it became possible to make eggs for
the different unicode compilation options of Python. This configuration
dimension is a real world issue for any binary Python module that does
anything with unicode text..
In an earlier mail to this list:
http://mail.python.org/pipermail/distutils-sig/2005-October/005222.html
M.-A. Lemburg and Phillip Eby had the following discussion:
[MAL]
>>Please make sure that your eggs catch all possible
>>Python binary build dimensions:
>>
>>* Python version
>>* Python Unicode variant (UCS2, UCS4)
>>* OS name
>>* OS version
>>* Platform architecture (e.g. 32-bit vs. 64-bit)
[PJE]
>As far as I know, all of this except the Unicode variant is captured in
>distutils' get_platform(). And if it's not, it should be, since it
>affects any other kind of bdist mechanism.
I'm not sure whether this means this needs to be escalated from
setuptools to the Python interpreter level itself. With this mail, I've
done the job escalating this lxml problem to what appears to be the
right place, though. :)
Thanks,
Martijn
hello,
i have a very basic pure-python package and script that i would like
to distribute as a noarch rpm using distutils. i am using FC5 as my
development platform and have prepared a setup.py that properly builds
and installs my application.
the first problem i have is that the bdist_rpm command results in an
%install section as follows:
%install
python setup.py install --single-version-externally-managed --root=$RPM_BUILD_ROOT --record=INSTALLED_FILES
the trouble here with that %install section is that the python modules
get bytecompiled and thus turned into native code. i'd prefer for
that not to happen, so i have written my own install.sh script and i
provide that to my python setup.py call as follows:
$ python setup.py bdist_rpm --spec-only --install-script /path/to/install.sh
my install.sh passes the --optimize 0 and --no-compile arguments to
distutils, resulting in no .pyc or .pyo files being installed in my
RPM_BUILD_ROOT. the contents of my install.sh are as follows:
python setup.py install --single-version-externally-managed --root=$RPM_BUILD_ROOT --record=INSTALLED_FILES --optimize 0 --no-compile
i thought that would result in no byte compiled files but
unfortunately i was wrong. i'm not sure what's causing it, but
rpmbuild ends up calling brp-python-bytecompile to compile the .py
files. i noticed in the ChangeLog for rpm the following:
- force *.py->*.pyo byte code compilation with brp-python-bytecompile.
so it appears to me as tho there is no way to assemble a noarch RPM
using distutils. does that sound accurate to you all? or are those
folks out there using distutils and an old version of rpm (prior to
2005) that are able to assemble noarch RPMs?
-c
--
WeatherNet Observations for station: home
Temperature: 54.80F Pressure: 30.02in;
Dew Point: 30.76F (40%) Wind: 248 at 5 mph
Recorded: 17:54:59 10/25/06 (http://wsdl.wxnet.org/inquiry/binding.wsdl)
I revisited setuptools again tonight hoping to learn a bit more about it. Unfortunately I'm still stuck where I was last time I looked.
I'm using Combinator to manage my Python libraries. This means that I have
a fair amount of software installed at ~/.local/lib/pythonX.Y/site-packages, a PYTHONPATH which includes only ~/Projects/Divmod/trunk/Combinator, at which location resides a sitecustomize.py which sets up the rest of my path.
>From the documentation, and various experimentation, it seems like this configuration is still not supported. What needs to happen for to change this?
Jean-Paul
I just read an IBM Developerworks article on setuptools today, its
linked at:
http://www-128.ibm.com/developerworks/linux/library/l-cppeak3.html?
ca=dgr-lnxw07PythonEggWithSetuptools
it refers to the ability to use pkg_resources.require() in a Python
script to specify any particular version of a package that happens to
exist in site-packages, overriding the currently active
package...something i used to think setuptools could do, but then
figured it couldnt (even though it seems like obviously useful
functionality), and it seems that it still cannot. from the
article's example:
# install Gnosis_Utils (latest version is 1.2.1)
easy_install -f http://gnosis.cx/download/Gnosis_Utils.More/
Gnosis_Utils
# install version 1.2.0 of Gnosis_Utils
easy_install -f http://gnosis.cx/download/Gnosis_Utils.More/
"Gnosis_Utils==1.2.0"
# re-activate version 1.2.1
easy_install "Gnosis_Utils==1.2.1"
# run a script that requires version 1.2.0
from pkg_resources import require
require("Gnosis_Utils==1.2.0")
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/Library/Frameworks/Python.framework/Versions/2.4/lib/
python2.4/site-packages/setuptools-0.6c3-py2.4.egg/pkg_resources.py",
line 585, in require
needed = self.resolve(parse_requirements(requirements))
File "/Library/Frameworks/Python.framework/Versions/2.4/lib/
python2.4/site-packages/setuptools-0.6c3-py2.4.egg/pkg_resources.py",
line 487, in resolve
raise VersionConflict(dist,req) # XXX put more info here
pkg_resources.VersionConflict: (Gnosis-Utils 1.2.1 (/Library/
Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/
Gnosis_Utils-1.2.1-py2.4.egg), Requirement.parse('Gnosis-Utils==1.2.0'))
bug ? or feature ?
Would appreciate some help with where to start trying to find out what's
going on here.
We're using eggs as the format for extensions to our app and I'm having the
odd bit of strangeness on one machine. I have another with identical
software setup which behaves as expected.
Here is a snippet of our code:
pkg_resources.working_set.add_entry(fullName)
dist_generator = pkg_resources.find_distributions(fullName)
for dist in dist_generator:
try:
extension_class =
dist.load_entry_point("jokosher.extensions", "extension")
extension = extension_class()
print("...done.")
except Exception, e :
print("...failed.")
print(str(e))
and usually it works.
Now because can run with an experimental version of the gstreamer libraries
we can run a script before we run our program which sets various ENV
variables (including PYTHONPATH) to point us to the development libraries
instead of the stock ones. If I run with the stock setup everything is OK.
On one machine (and one machine only) if I run the setup script in order to
pick up the experimental libraries then I get ImportError.
This is a section of the output by running the program with 'python -vv'.
First - with the stock libs (no pre-run script):
importing extension... AddInstrumentType-0.1-py2.4.egg
# zipimport: found 10 names in
/home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg
# zipimport: zlib available
# trying /home/john/code/jokosher/trunk/Jokosher/AddInstrumentType.so
# trying /home/john/code/jokosher/trunk/Jokosher/AddInstrumentTypemodule.so
# trying /home/john/code/jokosher/trunk/Jokosher/AddInstrumentType.py
# trying /home/john/code/jokosher/trunk/Jokosher/AddInstrumentType.pyc
# trying /usr/lib/python2.4/site-packages/gst-0.10/AddInstrumentType.so
# trying
/usr/lib/python2.4/site-packages/gst-0.10/AddInstrumentTypemodule.so
# trying /usr/lib/python2.4/site-packages/gst-0.10/AddInstrumentType.py
# trying /usr/lib/python2.4/site-packages/gst-0.10/AddInstrumentType.pyc
# trying /home/john/code/jokosher/trunk/Jokosher/AddInstrumentType.so
# trying /home/john/code/jokosher/trunk/Jokosher/AddInstrumentTypemodule.so
# trying /home/john/code/jokosher/trunk/Jokosher/AddInstrumentType.py
# trying /home/john/code/jokosher/trunk/Jokosher/AddInstrumentType.pyc
# trying /usr/lib/python24.zip/AddInstrumentType.so
...
...
...
# trying /home/john/code/jokosher/trunk/AddInstrumentType.so
# trying /home/john/code/jokosher/trunk/AddInstrumentTypemodule.so
# trying /home/john/code/jokosher/trunk/AddInstrumentType.py
# trying /home/john/code/jokosher/trunk/AddInstrumentType.pyc
# trying
/home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/__init
__.pyc
# trying
/home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/AddIns
trumentType/__init__.pyc
# trying
/home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/AddIns
trumentType/__init__.pyo
# trying
/home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/AddIns
trumentType/__init__.py
# trying
/home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/AddIns
trumentType.pyc
# trying
/home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/Jokosh
er.so
# trying
/home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/Jokosh
ermodule.so
# trying
/home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/Jokosh
er.py
# trying
/home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/Jokosh
er.pyc
# trying
/home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/gtk.so
# trying
/home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/gtkmod
ule.so
# trying
/home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/gtk.py
# trying
/home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/gtk.py
c
# trying
/home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/os.so
# trying
/home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/osmodu
le.so
# trying
/home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/os.py
# trying
/home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/os.pyc
# trying
/home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/pkg_re
sources.so
# trying
/home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/pkg_re
sourcesmodule.so
# trying
/home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/pkg_re
sources.py
# trying
/home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg/AddInstrumentType/pkg_re
sources.pyc
import AddInstrumentType.AddInstrumentType # loaded from Zip
/home/john/.jokosher/extensions/AddI
nstrumentType-0.1-py2.4.egg/AddInstrumentType/AddInstrumentType.pyc
import AddInstrumentType # loaded from Zip
/home/john/.jokosher/extensions/AddInstrumentType-0.1-
py2.4.egg/AddInstrumentType/__init__.pyc
...done.
No problems there.
But after running the env-altering script all we get out is:
importing extension... AddInstrumentType-0.1-py2.4.egg
# zipimport: found 10 names in
/home/john/.jokosher/extensions/AddInstrumentType-0.1-py2.4.egg
# zipimport: zlib available
...failed.
No module named AddInstrumentType
It doesn't even seem to begin to look for the module. As I said this only
happens on one machine and I know that it used to work on that one too. If
you could give us a clue on what to look for next then that would be
appreciated.
Thanks
--
John Green
Hello,
I want the list of package requirements, which I hoped would be real
simple, but have not been able to find a good solution.
I could download the egg and look in the EGG_INFO directory, but is
there any way to have setuptools develop the list for me? And
hopefully not have to download the egg.
If the above idea doesn't go anywhere - is it possible that the package
requirements could be added to CheeseShop's XMLRPC?
Kindest regards,
Tim Cera
Hi,
I was somewhat surprised to find that distutils on install does honor the
current umask. If run as root and root's umask is 027 (a common case
nowadays), stuff will be installed as non world readable / executable.
Compare this to install(1)'s behavior on Unix.
Shouldn't the default be to DTRT on Unix? Or shouldn't there be at least an
option to specify the target permissions for data and executable files?
--
Regards,
Georg.
Im a little bit disappointed by setupTools. Not that much the tool itself,
than the learning curve.
Maybe the problem at hand is far from simple, and thats what makes the
solution so intricate.
http://peak.telecommunity.com/DevCenter/setuptools: this document is
100 pages long! (by page I mean: full screen).
easyInstall is 40 pages.
I spent about 3 days to figure out what goes where. And I realize Im
not done.
Yet the documentation is remarkable, great presentation, text is
precise and clear, when U read it. Makes U feel guilty U dont find
what U lookin for, but U dont find it.
I just discovered scripts dont get installed directly (in
/usr/local/bin), instead a wrapper in installed, that calls the
script. So an extra file is involved, and extra code, every time the
script is called. Makes some overhead, and it breaks a simple
mechanism I use: my script is able to perform many actions, based on
its name (much like cp, mv and ln are the same executable). at
install, I need to make symlinks to the script, one per action, and
this was automated: this is broken by the wrapping. No big deal, just
makes things less simple.
Good tools have 2 qualities:
- When U need simple things, its easy to use.
- When U need more sophisticated things, U dont need to give up the
tool for a more powerful one: just have to use extra features.
Python is a wonderful illustration of this.
While setupTools obviously provides the latter, I consider it fails on
the former.
I decided to use setupTools after reading, and failing to understand,
distutils. setupTools seemed both more powerful and simpler.
Probably this choice will save me some trouble later. Im not sure it
helped so far.
