[lxml-dev] Binary egg for Mac OS X
Hi, I'm working on documenting Deliverance / xdv for use with Plone. It has lxml as a dependency, and I have run into a serious issue: On Mac OS X, we can't assume that people have Xcode (ie. gcc and friends) installed, thus we can't really compile lxml on those computers, not even using the staticlxml[1] recipe. I see that there are binary eggs for Windows, is there a special reason why there are no binary eggs for OS X, or is it just a matter of not having the infrastructure to make it available? Happy to help find a solution if it's just a matter of locating a reliable way to get it compiled every time there is a new release. [1] http://pypi.python.org/pypi/z3c.recipe.staticlxml/ -- Alexander Limi · http://limi.net
Hi, Alexander Limi wrote:
I'm working on documenting Deliverance / xdv for use with Plone. It has lxml as a dependency, and I have run into a serious issue:
On Mac OS X, we can't assume that people have Xcode (ie. gcc and friends) installed, thus we can't really compile lxml on those computers, not even using the staticlxml[1] recipe.
I see that there are binary eggs for Windows, is there a special reason why there are no binary eggs for OS X, or is it just a matter of not having the infrastructure to make it available?
The main problem is that many MacOS-X users have some kind of package distribution like macports installed, which usually has some distribution specific setup/dependencies/paths/whatever. OTOH, those users won't be the target for a binary distribution of lxml anyway.
Happy to help find a solution if it's just a matter of locating a reliable way to get it compiled every time there is a new release.
Yes, I'd be happy if we could get a static binary egg for each release. I don't have a Mac myself (and I'm definitely not a Mac user), so contributions are welcome. http://codespeak.net/lxml/build.html#building-lxml-on-macos-x Stefan
Stefan Behnel wrote:
Hi,
Alexander Limi wrote:
I'm working on documenting Deliverance / xdv for use with Plone. It has lxml as a dependency, and I have run into a serious issue:
On Mac OS X, we can't assume that people have Xcode (ie. gcc and friends) installed, thus we can't really compile lxml on those computers, not even using the staticlxml[1] recipe.
I see that there are binary eggs for Windows, is there a special reason why there are no binary eggs for OS X, or is it just a matter of not having the infrastructure to make it available?
The main problem is that many MacOS-X users have some kind of package distribution like macports installed, which usually has some distribution specific setup/dependencies/paths/whatever. OTOH, those users won't be the target for a binary distribution of lxml anyway.
Well, I use MacPorts. And every time I have a package that depends on lxml, I need to jump through hoops to avoid random segfauls (and OS X makes those hard to debug!). Luckily, those hoops aren't so big anymore: I tend to use zc.buildout and I can add this to my buildout: [lxml] recipe = z3c.recipe.staticlxml egg = lxml force = false However, people have a *lot* of problems with this. Every Plone sprint it seems someone is spending half a day or more getting lxml to compile. Which is a big shame, because lxml is so nice so people like me want to use it. :) Therefore, the Plone community, at least, is pretty interested in solving this problem.
Happy to help find a solution if it's just a matter of locating a reliable way to get it compiled every time there is a new release.
Yes, I'd be happy if we could get a static binary egg for each release. I don't have a Mac myself (and I'm definitely not a Mac user), so contributions are welcome.
http://codespeak.net/lxml/build.html#building-lxml-on-macos-x
So, if we followed those steps to build a statically compiled egg for each Python version we support (2.4 and 2.6 for Plone...), and uploaded that to PyPI, we'd be able to just depend on this version of lxml and no-one on OSX should ever get these annoying problems? That'd be really nice. ;) If this is the case, we'll find someone to do this. Are you able to give someone access to the PyPI page and any relevant support to make this happen? Finally - what about Linux? Is it rarely/never a problem, or should we be trying to make binary eggs there too? Is it possible to make binary eggs that work across the most common distributions? Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book
Martin Aspeli wrote:
Stefan Behnel wrote:
Alexander Limi wrote:
I'm working on documenting Deliverance / xdv for use with Plone. It has lxml as a dependency, and I have run into a serious issue:
On Mac OS X, we can't assume that people have Xcode (ie. gcc and friends) installed, thus we can't really compile lxml on those computers, not even using the staticlxml[1] recipe.
I see that there are binary eggs for Windows, is there a special reason why there are no binary eggs for OS X, or is it just a matter of not having the infrastructure to make it available? The main problem is that many MacOS-X users have some kind of package distribution like macports installed, which usually has some distribution specific setup/dependencies/paths/whatever. OTOH, those users won't be the target for a binary distribution of lxml anyway.
Well, I use MacPorts.
The problem is that macports is not the only package distribution. Trying to support them all from lxml's setup.py is futile. Hence the static builds that are independent of the installed libraries.
Happy to help find a solution if it's just a matter of locating a reliable way to get it compiled every time there is a new release. Yes, I'd be happy if we could get a static binary egg for each release. I don't have a Mac myself (and I'm definitely not a Mac user), so contributions are welcome.
http://codespeak.net/lxml/build.html#building-lxml-on-macos-x
So, if we followed those steps to build a statically compiled egg for each Python version we support (2.4 and 2.6 for Plone...), and uploaded that to PyPI, we'd be able to just depend on this version of lxml and no-one on OSX should ever get these annoying problems? That'd be really nice. ;)
If this is the case, we'll find someone to do this. Are you able to give someone access to the PyPI page and any relevant support to make this happen?
Sure. Sidnei da Silva provides the Windows builds, and I'm happy to add another package maintainer for MacOS.
Finally - what about Linux? Is it rarely/never a problem, or should we be trying to make binary eggs there too? Is it possible to make binary eggs that work across the most common distributions?
As long as you have a somewhat recent system, installing lxml is trivial here. Plus, updating the system installations of libxml2 and libxslt isn't hard either, so that's a totally different situation than for MacOS. Stefan
Stefan Behnel wrote:
Martin Aspeli wrote:
Stefan Behnel wrote:
Alexander Limi wrote:
I'm working on documenting Deliverance / xdv for use with Plone. It has lxml as a dependency, and I have run into a serious issue:
On Mac OS X, we can't assume that people have Xcode (ie. gcc and friends) installed, thus we can't really compile lxml on those computers, not even using the staticlxml[1] recipe.
I see that there are binary eggs for Windows, is there a special reason why there are no binary eggs for OS X, or is it just a matter of not having the infrastructure to make it available? The main problem is that many MacOS-X users have some kind of package distribution like macports installed, which usually has some distribution specific setup/dependencies/paths/whatever. OTOH, those users won't be the target for a binary distribution of lxml anyway. Well, I use MacPorts.
The problem is that macports is not the only package distribution. Trying to support them all from lxml's setup.py is futile. Hence the static builds that are independent of the installed libraries.
If a static build works for everyone, I'll be ecstatic (pun intended).
Happy to help find a solution if it's just a matter of locating a reliable way to get it compiled every time there is a new release. Yes, I'd be happy if we could get a static binary egg for each release. I don't have a Mac myself (and I'm definitely not a Mac user), so contributions are welcome.
http://codespeak.net/lxml/build.html#building-lxml-on-macos-x So, if we followed those steps to build a statically compiled egg for each Python version we support (2.4 and 2.6 for Plone...), and uploaded that to PyPI, we'd be able to just depend on this version of lxml and no-one on OSX should ever get these annoying problems? That'd be really nice. ;)
If this is the case, we'll find someone to do this. Are you able to give someone access to the PyPI page and any relevant support to make this happen?
Sure. Sidnei da Silva provides the Windows builds, and I'm happy to add another package maintainer for MacOS.
Great! I'll try to find a volunteer.
Finally - what about Linux? Is it rarely/never a problem, or should we be trying to make binary eggs there too? Is it possible to make binary eggs that work across the most common distributions?
As long as you have a somewhat recent system, installing lxml is trivial here. Plus, updating the system installations of libxml2 and libxslt isn't hard either, so that's a totally different situation than for MacOS.
Right. We haven't had too many complaints yet, at least. It'd be nice to have some binary eggs, though, if they can be done reliably, for people who don't have libxml2/libxslt installed. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book
Martin Aspeli wrote:
Stefan Behnel wrote:
Martin Aspeli wrote:
Finally - what about Linux? Is it rarely/never a problem, or should we be trying to make binary eggs there too? Is it possible to make binary eggs that work across the most common distributions? As long as you have a somewhat recent system, installing lxml is trivial here. Plus, updating the system installations of libxml2 and libxslt isn't hard either, so that's a totally different situation than for MacOS.
Right. We haven't had too many complaints yet, at least. It'd be nice to have some binary eggs, though, if they can be done reliably, for people who don't have libxml2/libxslt installed.
I provided binary eggs for AMD64-GNU/Linux at the beginning, and there were a couple of downloads back then (although no feedback on them). However, the various dialects of Linux form a platform that is very easy to target with source code and much harder to target with binary releases. For example, a static build doesn't make any sense here, since most Linux installations have very good package management by now. They automatically update their system libraries from time to time, even without notifying the user (especially for security updates). A static lxml would not benefit from that, whereas a dynamic build will certainly not run on all platforms that lxml supports as a source distribution. So a binary egg that easy_install fetches by default would actually break more than it could fix. The whole problem with MacOS is that it is *not* easy to update the system libraries. That's why a static binary build makes sense. Stefan
Stefan Behnel <stefan_ml@behnel.de> (SB) wrote:
SB> The whole problem with MacOS is that it is *not* easy to update the system SB> libraries. That's why a static binary build makes sense.
On Mac OS X you *shouldn't* update system libraries. With the next system software update they could easily disappear. -- Piet van Oostrum <piet@cs.uu.nl> URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4] Private email: piet@vanoostrum.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martin Aspeli wrote:
Stefan Behnel wrote:
Stefan Behnel wrote:
Alexander Limi wrote:
I'm working on documenting Deliverance / xdv for use with Plone. It has lxml as a dependency, and I have run into a serious issue:
On Mac OS X, we can't assume that people have Xcode (ie. gcc and friends) installed, thus we can't really compile lxml on those computers, not even using the staticlxml[1] recipe.
I see that there are binary eggs for Windows, is there a special reason why there are no binary eggs for OS X, or is it just a matter of not having the infrastructure to make it available? The main problem is that many MacOS-X users have some kind of package distribution like macports installed, which usually has some distribution specific setup/dependencies/paths/whatever. OTOH, those users won't be the target for a binary distribution of lxml anyway. Well, I use MacPorts. The problem is that macports is not the only package distribution. Trying to support them all from lxml's setup.py is futile. Hence the static builds
Martin Aspeli wrote: that are independent of the installed libraries.
If a static build works for everyone, I'll be ecstatic (pun intended).
Happy to help find a solution if it's just a matter of locating a reliable way to get it compiled every time there is a new release. Yes, I'd be happy if we could get a static binary egg for each release. I don't have a Mac myself (and I'm definitely not a Mac user), so contributions are welcome.
http://codespeak.net/lxml/build.html#building-lxml-on-macos-x So, if we followed those steps to build a statically compiled egg for each Python version we support (2.4 and 2.6 for Plone...), and uploaded that to PyPI, we'd be able to just depend on this version of lxml and no-one on OSX should ever get these annoying problems? That'd be really nice. ;)
If this is the case, we'll find someone to do this. Are you able to give someone access to the PyPI page and any relevant support to make this happen? Sure. Sidnei da Silva provides the Windows builds, and I'm happy to add another package maintainer for MacOS.
Great! I'll try to find a volunteer.
Finally - what about Linux? Is it rarely/never a problem,
The last time I had trouble with it (a CentOS 4 box), building the static version myself was any easy workaround.
or should we be trying to make binary eggs there too? Is it possible to make binary eggs that work across the most common distributions?
No, nor would it be desirable to try.
As long as you have a somewhat recent system, installing lxml is trivial here. Plus, updating the system installations of libxml2 and libxslt isn't hard either, so that's a totally different situation than for MacOS.
Right. We haven't had too many complaints yet, at least. It'd be nice to have some binary eggs, though, if they can be done reliably, for people who don't have libxml2/libxslt installed.
- -1 to uploading any binary eggs for Linux: they will get preferred over the source distributions, which is not what I want at all. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFKLQbR+gerLs4ltQ4RAhaUAJ49em5e/XLAzkiX01Q1yW+P7icI/QCgouPI lC7t0OcJWgiTeOGJollO7Ck= =mtPN -----END PGP SIGNATURE-----
Martin Aspeli wrote:
Finally - what about Linux? Is it rarely/never a problem, or should we be trying to make binary eggs there too? Is it possible to make binary eggs that work across the most common distributions?
-1 to binary eggs on Linux. I don't think problems are common and binary eggs might lead to problems. Regards, Martijn
Martijn Faassen wrote:
Martin Aspeli wrote:
Finally - what about Linux? Is it rarely/never a problem, or should we be trying to make binary eggs there too? Is it possible to make binary eggs that work across the most common distributions?
-1 to binary eggs on Linux. I don't think problems are common and binary eggs might lead to problems.
Right, now that you mention it: Linux distributions use different unicode width settings (16/32 bit), which isn't handled by setuptools' egg search algorithm. So providing binary eggs here is actually harmful because they will not work on all platforms. I assume that we do not have this problem on MacOS, as the only vendor that ships the system Python installation is Apple. Stefan
Hi, Stefan Eletzhofer (CC'ed) mentioned that he'd be willing to help out. I did contact the Snakebite people to see if we can automate the builds — but they aren't quite up and running yet. So, for the time being, could we manually build a binary egg for lxml and upload it along with the others? Plone people would be very happy, and it would save a lot of grey hairs for people on Mac OS X. :) — Alexander On Sat, 30 May 2009 04:04:14 -0700, Stefan Behnel <stefan_ml@behnel.de> wrote:
Hi,
Alexander Limi wrote:
I'm working on documenting Deliverance / xdv for use with Plone. It has lxml as a dependency, and I have run into a serious issue:
On Mac OS X, we can't assume that people have Xcode (ie. gcc and friends) installed, thus we can't really compile lxml on those computers, not even using the staticlxml[1] recipe.
I see that there are binary eggs for Windows, is there a special reason why there are no binary eggs for OS X, or is it just a matter of not having the infrastructure to make it available?
The main problem is that many MacOS-X users have some kind of package distribution like macports installed, which usually has some distribution specific setup/dependencies/paths/whatever. OTOH, those users won't be the target for a binary distribution of lxml anyway.
Happy to help find a solution if it's just a matter of locating a reliable way to get it compiled every time there is a new release.
Yes, I'd be happy if we could get a static binary egg for each release. I don't have a Mac myself (and I'm definitely not a Mac user), so contributions are welcome.
http://codespeak.net/lxml/build.html#building-lxml-on-macos-x
Stefan
-- Alexander Limi · http://limi.net
Alexander Limi wrote:
Stefan Eletzhofer (CC'ed) mentioned that he'd be willing to help out. I did contact the Snakebite people to see if we can automate the builds — but they aren't quite up and running yet.
So, for the time being, could we manually build a binary egg for lxml and upload it along with the others? Plone people would be very happy, and it would save a lot of grey hairs for people on Mac OS X. :)
Sure, it's not like there's an lxml release every week or so, so a manual build is quite doable. What I would like to see is a really fat egg, i.e. one that works on all hardware platforms supported by MacOS-X. I have no idea how to do that, so I can't provide any hints. But since there seem to be some MacOS users on the list by now, I hope we can get away with some web digging plus trial and error. Also, the build must not rely on any macports libraries, as not everyone uses macports. So we'd have to make sure it's only built against system libraries (plus the statically built libxml2/libxslt). What about the "system Python vs. macports Python" question? Would such an egg work on both? Or does it make more sense to support only the system Python installation, and to leave macports specific builds to the macports maintainers? Stefan
participants (6)
-
Alexander Limi -
Martijn Faassen -
Martin Aspeli -
Piet van Oostrum -
Stefan Behnel -
Tres Seaver