executable egg & top-level __main__.py
Hi all, I have the following directory structure: myegg/ pkg1/ __init__.py ... pkg2/ __init__.py ... pkg3/ __init__.py ... setup.py __main__.py If I zip this into myegg.egg and run "python myegg.egg", that runs the top-level __main__.py. If I run "python setup.py bdist_egg" and run the resulting egg, I get "/usr/bin/python: can't find '__main__.py'" in the egg. That is correct: according to "unzip -l" __main__.py is not there. How do I get it included? Or more to the point, how do I make that egg directly executable? -- note that I want it to run a script that isn't in any of the bundled packages (but imports from them). (centos 6 with python 2.6 and python-setuptools-0.6.10 and centos 7 with python 2.7 and python-setuptools-0.9.8) -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu
On 6/18/15 4:46 PM, Dimitri Maziuk wrote:
Hi all,
I have the following directory structure:
myegg/ pkg1/ __init__.py ... pkg2/ __init__.py ... pkg3/ __init__.py ... setup.py __main__.py
If I zip this into myegg.egg and run "python myegg.egg", that runs the top-level __main__.py. If I run "python setup.py bdist_egg" and run the resulting egg, I get "/usr/bin/python: can't find '__main__.py'" in the egg. That is correct: according to "unzip -l" __main__.py is not there.
How do I get it included?
Or more to the point, how do I make that egg directly executable? -- note that I want it to run a script that isn't in any of the bundled packages (but imports from them).
(centos 6 with python 2.6 and python-setuptools-0.6.10 and centos 7 with python 2.7 and python-setuptools-0.9.8)
Probably can't answer that without seeing setup.py where there should at least be ``packages = find_packages()``. Also sounds like you may be looking for console_scripts.
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
-- Alex Clark · http://aclark.net
You are doing the right thing zipping it up yourself instead of relying on
bdist_egg. Eggs are a plugin or package format and are not designed to be
directly runnable.
There are some utilities to help you make executable zip files:
https://docs.python.org/dev/library/zipapp.html
https://pex.readthedocs.org/en/latest/
If you find it helpful to create your executable more manually, you could
unzip the egg and __main__.py into a directory and then zip everything back
up.
On Thu, Jun 18, 2015 at 8:51 PM Alex Clark
On 6/18/15 4:46 PM, Dimitri Maziuk wrote:
Hi all,
I have the following directory structure:
myegg/ pkg1/ __init__.py ... pkg2/ __init__.py ... pkg3/ __init__.py ... setup.py __main__.py
If I zip this into myegg.egg and run "python myegg.egg", that runs the top-level __main__.py. If I run "python setup.py bdist_egg" and run the resulting egg, I get "/usr/bin/python: can't find '__main__.py'" in the egg. That is correct: according to "unzip -l" __main__.py is not there.
How do I get it included?
Or more to the point, how do I make that egg directly executable? -- note that I want it to run a script that isn't in any of the bundled packages (but imports from them).
(centos 6 with python 2.6 and python-setuptools-0.6.10 and centos 7 with python 2.7 and python-setuptools-0.9.8)
Probably can't answer that without seeing setup.py where there should at least be ``packages = find_packages()``. Also sounds like you may be looking for console_scripts.
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
-- Alex Clark · http://aclark.net
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
On 2015-06-18 21:15, Daniel Holth wrote: ... Eggs are a plugin or package format and are not designed
to be directly runnable.
Well, google seems to disagree, but ok.
If you find it helpful to create your executable more manually, you could unzip the egg and __main__.py into a directory and then zip everything back up.
So, since this is all done by a python script, is there a hook in setuptools/setup.py where I can open the egg zipfile and do stuff to it as teh last step of bdist_egg? Or maybe I'll see if I can trick it with file_finders instead... In this case I'm shipping the script for remote execution, I can configure the workflow engine to ship both egg and scriptfile. It would be a touch more convenient to roll the script inside the egg and ship just one file, but not worth the effort. However, I think runnable egg can be handy for dealing with python's version of dll hell. Thanks Dimitri
On 2015-06-18 19:50, Alex Clark wrote:
Probably can't answer that without seeing setup.py where there should at least be ``packages = find_packages()``.
Yes there is. Also version and name, that's it: straight from the fine manual.
Also sounds like you may be looking for console_scripts.
No. I mean, the documentation is sketchy to put it charitably, but it doesn't look like running "python myegg.egg" will call any of those. It looks like it should call "setuptools.installation" entry point(s) but that one's not quite what I'm after. What I'm looking for is exactly what I asked: I have a __main__.py in the root of my directory tree next to setup.py, I want it included in the egg when I run "python setup.py bdist_egg". Cheers, Dimitri
Did you try py_modules=["__main__"] in setup.py
On Fri, Jun 19, 2015, 11:14 AM Dimitri Maziuk
On 2015-06-18 19:50, Alex Clark wrote:
Probably can't answer that without seeing setup.py where there should at least be ``packages = find_packages()``.
Yes there is. Also version and name, that's it: straight from the fine manual.
Also sounds like you may be looking for console_scripts.
No. I mean, the documentation is sketchy to put it charitably, but it doesn't look like running "python myegg.egg" will call any of those. It looks like it should call "setuptools.installation" entry point(s) but that one's not quite what I'm after.
What I'm looking for is exactly what I asked: I have a __main__.py in the root of my directory tree next to setup.py, I want it included in the egg when I run "python setup.py bdist_egg".
Cheers, Dimitri
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
On 06/19/2015 10:44 AM, Daniel Holth wrote:
Did you try py_modules=["__main__"] in setup.py
No, of course not: there is no py_modules (or "py_" - anything) anywhere in the developer's guide webpage so how would I know to try it? That is exactly what I was looking for. Thank you. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu
Sorry. The forgettable py_modules parameter is one reason why distutils is
terrible and we should stop using it. No we don't have a general
replacement to recommend yet.
Just keep in mind that having a top-level __main__, if you install two such
packages, one __main__ will clobber the other.
On Fri, Jun 19, 2015 at 12:51 PM Dimitri Maziuk
On 06/19/2015 10:44 AM, Daniel Holth wrote:
Did you try py_modules=["__main__"] in setup.py
No, of course not: there is no py_modules (or "py_" - anything) anywhere in the developer's guide webpage so how would I know to try it?
That is exactly what I was looking for.
Thank you. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu
On 06/19/2015 11:59 AM, Daniel Holth wrote:
Sorry. The forgettable py_modules parameter is one reason why distutils is terrible and we should stop using it. No we don't have a general replacement to recommend yet.
Well, it could be mentioned in the "eggsecutable scripts" section, esp. since what that has now doesn't really parse anyway: """ entry_points = { 'setuptools.installation': [ 'eggsecutable = my_package.some_module:main_func', ] } """ OK, so is "eggsecutable" a keyword or can we call it "rose" instead? It's list so what happens when I put more than one "eggsecutable =" in? Key name implies this will run during installation, that's a bit counter-intuitive (unless it's an error and that does actually run at install and not when you try to execute the egg). And so on. Whereas when you look elsewhere you'll find that a) .egg is a zip file and b) when there's a __main__.py in a zip file, python 2.6+ will execute it.
Just keep in mind that having a top-level __main__, if you install two such packages, one __main__ will clobber the other.
Right, thanks. No, the goal is specifically to not install these packages: just run the egg -- so if anybody wants to install, they're welcome to keep the pieces. That does prompt another question, though: what happens if you install two packages with conflicting 'setuptools.installation'? Or 'console_scripts'? I mean, is the problem really only limited to conflicting __main__.py? -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu
On 19 June 2015 at 18:19, Dimitri Maziuk
Well, it could be mentioned in the "eggsecutable scripts" section, esp. since what that has now doesn't really parse anyway:
While I understand what you want, and it sounds like you have found a way to achieve it, Daniel is right - this feature of eggs (and indeed the whole egg format in general) is essentially deprecated nowadays. If you want an installable binary, you should be using wheels, and if you want a single-file executable, you should be using zipapp (the zipapp module is new in 3.5, but zipping up the correct directory structure has been supported for ages). The documentation around eggs, and indeed around distutils/setuptools, has never been wonderful. That's a long-standing problem we're looking to fix, but we almost certainly won't be improving the documentation for "eggsecutable scrips" - it's more likely that we'll *remove* that documentation and leave the feature present solely for backward compatibility. (Note: I do *not* speak as the setuptools maintainer - Jason may well have a different view - but what I say above is my understanding as one of the pip maintainers of the general direction the packaging community is heading). Paul
On 06/19/2015 02:57 PM, Paul Moore wrote: ... if
you want a single-file executable, you should be using zipapp (the zipapp module is new in 3.5, but zipping up the correct directory structure has been supported for ages).
You're right, I don't really need the egg format for what I'm doing. However, setuptools + setup.py is a familiar idiom like make + Makefile or ant + build.xml. So add a bdist_zip target and have it output a .zip without egg-info cruft instead of making people like me reinvent the wheel every !@#$ing time. Even if the wheel is as simple as piping glob.glob into a zipfile -- which it never is IRL. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu
On 19 June 2015 at 21:24, Dimitri Maziuk
So add a bdist_zip target and have it output a .zip without egg-info cruft instead of making people like me reinvent the wheel every !@#$ing time. Even if the wheel is as simple as piping glob.glob into a zipfile -- which it never is IRL.
That's not a bad idea. It should be possible to do this as a setuptools extension (much as wheel provides the bdist_wheel command). It should probably be bdist_pyz (and produce a file with the .pyz extension) for PEP 441 compatibility. It could then be distributed as a project on PyPI to save everyone else reinventing the wheel, as you say. Paul
Don't worry about it.
FYI the .egg-info (or .dist-info) cruft is necessary if you want to use
certain setuptools features including console_scripts. Without metadata
your module will be importable, but pkg_resources won't be able to tell
that your distribution is available.
On Fri, Jun 19, 2015 at 4:24 PM Dimitri Maziuk
On 06/19/2015 02:57 PM, Paul Moore wrote:
... if
you want a single-file executable, you should be using zipapp (the zipapp module is new in 3.5, but zipping up the correct directory structure has been supported for ages).
You're right, I don't really need the egg format for what I'm doing. However, setuptools + setup.py is a familiar idiom like make + Makefile or ant + build.xml.
So add a bdist_zip target and have it output a .zip without egg-info cruft instead of making people like me reinvent the wheel every !@#$ing time. Even if the wheel is as simple as piping glob.glob into a zipfile -- which it never is IRL.
-- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu
On 06/19/2015 03:43 PM, Daniel Holth wrote:
Don't worry about it.
FYI the .egg-info (or .dist-info) cruft is necessary if you want to use certain setuptools features including console_scripts. Without metadata your module will be importable, but pkg_resources won't be able to tell that your distribution is available.
Even easier then: a bdist_zip target that automagically adds py_modules=["__main__"] and saves the file with extension .pyz instead of .egg. I'm not worried about a few extra bytes and besides I might want to use pkg_resourced data files inside the .pyz some day so I'd rather keep at least that part of it anyway. -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu
participants (4)
-
Alex Clark -
Daniel Holth -
Dimitri Maziuk -
Paul Moore