Self-contained boostrap scripts [was: Re: A new, experimental packaging tool: distil]
On Tue, Mar 26, 2013 at 11:08 AM, Paul Moore <p.f.moore@gmail.com> wrote:
On 26 March 2013 09:49, Philippe Ombredanne <pombredanne@nexb.com> wrote:
Would anyone know of a better way to package things in a single python-executable bootstrapping script file without obfuscating the source contents in compressed/encoded/obfuscated byte arrays?
Packaging as a zip file is a good way - but on Windows the file needs to be named xxx.py (which is surprising, to say the least :-)) for the relevant file association to be triggered (and on Unix, a #! line needs to be prepended). Paul: I was not talking about this type of zips, but rather the same used in virtualenv, i.e. a string in a .py file that contains an encoded zip. That string is then decoded and unzipped at runtime as in here: https://github.com/pypa/virtualenv/blob/develop/virtualenv.py#L1933
This is not a zip, not an egg, not a wheel but some egg-in-py, zip-in-py or wheel-in-py and is similar to a shar shell archive. My point was that on the one hand, I like the fact that everything is self contained in one single .py file that you can execute right away. On the other hand, I find it somewhat discomforting as an emerging best way to package and distribute self-contained bootstrap scripts. Yet I cannot think of a better way atm: for instance splitting things in non-encoded non-binary plain strings would be quite weird too. Virtualenv does it, distil is doing it now, pip tried some of it here https://github.com/pypa/pip/blob/develop/contrib/get-pip.py In contrast, buildout, distribute and setuptools bootstrap scripts do not embed their dependencies and either try to get them satisfied locally or attempt to download the requirements. Having some support to do self-contained bootstrap scripts (as in requiring no network access and embedding all their dependencies) using this shar style could be something to consider normalizing? -- Philippe Ombredanne +1 650 799 0949 | pombredanne@nexB.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com
PEP XXX Yet Another Python .ZIP File Extension One reason Python ZIP applications have not been more widely adopted is because they have no Windows file association and would have to be confusingly named .py to be invoked by the interpreter. Henceforth, files with the extension .pyz shall be known as Python ZIP files. They consist of a ZIP format archive containing at minimum __main__.py, concatenated to two lines #!python or #!pythonw (or the full path to the interpreter), and an explanation # This is a ZIP format archive executable by the Python interpreter. The launcher will be updated to understand this format and Python will register this filename association when it is installed. On Thu, Mar 28, 2013 at 7:40 AM, Philippe Ombredanne <pombredanne@nexb.com> wrote:
On Tue, Mar 26, 2013 at 11:08 AM, Paul Moore <p.f.moore@gmail.com> wrote:
On 26 March 2013 09:49, Philippe Ombredanne <pombredanne@nexb.com> wrote:
Would anyone know of a better way to package things in a single python-executable bootstrapping script file without obfuscating the source contents in compressed/encoded/obfuscated byte arrays?
Packaging as a zip file is a good way - but on Windows the file needs to be named xxx.py (which is surprising, to say the least :-)) for the relevant file association to be triggered (and on Unix, a #! line needs to be prepended). Paul: I was not talking about this type of zips, but rather the same used in virtualenv, i.e. a string in a .py file that contains an encoded zip. That string is then decoded and unzipped at runtime as in here: https://github.com/pypa/virtualenv/blob/develop/virtualenv.py#L1933
This is not a zip, not an egg, not a wheel but some egg-in-py, zip-in-py or wheel-in-py and is similar to a shar shell archive.
My point was that on the one hand, I like the fact that everything is self contained in one single .py file that you can execute right away. On the other hand, I find it somewhat discomforting as an emerging best way to package and distribute self-contained bootstrap scripts. Yet I cannot think of a better way atm: for instance splitting things in non-encoded non-binary plain strings would be quite weird too.
Virtualenv does it, distil is doing it now, pip tried some of it here https://github.com/pypa/pip/blob/develop/contrib/get-pip.py In contrast, buildout, distribute and setuptools bootstrap scripts do not embed their dependencies and either try to get them satisfied locally or attempt to download the requirements. Having some support to do self-contained bootstrap scripts (as in requiring no network access and embedding all their dependencies) using this shar style could be something to consider normalizing?
-- Philippe Ombredanne
+1 650 799 0949 | pombredanne@nexB.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
On 28 March 2013 12:26, Daniel Holth <dholth@gmail.com> wrote:
The launcher will be updated to understand this format and Python will register this filename association when it is installed.
The launcher should need no changes. The Python msi installer would need a change to register the new extension, though. And *creating* such zips is mildly annoying on Windows, due to a general lack of tool support for manipulating binary files in text editors. Oh, and wouldn't "#!/usr/bin/env python(w)" be a better header? That would work on Unix, and the launcher recognises that format. But +1 on the idea in general. Paul
On Thu, Mar 28, 2013 at 8:43 AM, Paul Moore <p.f.moore@gmail.com> wrote:
On 28 March 2013 12:26, Daniel Holth <dholth@gmail.com> wrote:
The launcher will be updated to understand this format and Python will register this filename association when it is installed.
The launcher should need no changes. The Python msi installer would need a change to register the new extension, though.
And *creating* such zips is mildly annoying on Windows, due to a general lack of tool support for manipulating binary files in text editors.
Oh, and wouldn't "#!/usr/bin/env python(w)" be a better header? That would work on Unix, and the launcher recognises that format.
But +1 on the idea in general. Paul
There is no 'cat header zip > newzip'?
On 28 March 2013 12:45, Daniel Holth <dholth@gmail.com> wrote:
On Thu, Mar 28, 2013 at 8:43 AM, Paul Moore <p.f.moore@gmail.com> wrote:
On 28 March 2013 12:26, Daniel Holth <dholth@gmail.com> wrote:
The launcher will be updated to understand this format and Python will register this filename association when it is installed.
The launcher should need no changes. The Python msi installer would need a change to register the new extension, though.
And *creating* such zips is mildly annoying on Windows, due to a general lack of tool support for manipulating binary files in text editors.
Oh, and wouldn't "#!/usr/bin/env python(w)" be a better header? That would work on Unix, and the launcher recognises that format.
But +1 on the idea in general. Paul
There is no 'cat header zip > newzip'?
There are multiple options. And text file vs binary file issues to cover. CMD.EXE: copy /b header+zip newzip Powershell: get-content header,zip -enc Byte | set-content newzip -enc Byte Powershell: cmd /c copy /b header+zip newzip (because the previous version is so ugly...) Or write a Python script, which is what I did. Yes, I know :-( Paul
On 28 March 2013 11:40, Philippe Ombredanne <pombredanne@nexb.com> wrote:
This is not a zip, not an egg, not a wheel but some egg-in-py, zip-in-py or wheel-in-py and is similar to a shar shell archive.
My point was that on the one hand, I like the fact that everything is self contained in one single .py file that you can execute right away. On the other hand, I find it somewhat discomforting as an emerging best way to package and distribute self-contained bootstrap scripts. Yet I cannot think of a better way atm: for instance splitting things in non-encoded non-binary plain strings would be quite weird too.
Yes, my point was that Vinay's usage could be covered by distributing distil as a zip file. All it is doing is decoding it's blob of data (which is an encoded zip file) and then adding the resulting zip to sys.path. The virtualenv situation is different, as there we are trying to ensure that we remain single-file while embedding things that are *not* modules to add to sys.path. And we don't want to download our dependencies because we need to be able to run with no internet connection. But you are right, the embedded script approach is not ideal. I hope that "embedded binary blobs" does not become a common approach. I'd much rather that "runnable zip files" became the norm. It's certainly possible now, but I don't think it's well enough known (and there are administrative issues like the file extension question on Windows that make it more awkward than it should be). Hence my comments, trying to raise awareness a bit. Thanks for the feedback, and in particular the reminder that virtualenv could do with looking at this... I've added a virtualenv issue to remind me to think some more about it. Paul
If it was distributed as such virtualenv could read blobs out of its own zip file, use the get_data() API to read non-module, or add subdirectories inside its zip file to sys.path On Thu, Mar 28, 2013 at 8:32 AM, Paul Moore <p.f.moore@gmail.com> wrote:
On 28 March 2013 11:40, Philippe Ombredanne <pombredanne@nexb.com> wrote:
This is not a zip, not an egg, not a wheel but some egg-in-py, zip-in-py or wheel-in-py and is similar to a shar shell archive.
My point was that on the one hand, I like the fact that everything is self contained in one single .py file that you can execute right away. On the other hand, I find it somewhat discomforting as an emerging best way to package and distribute self-contained bootstrap scripts. Yet I cannot think of a better way atm: for instance splitting things in non-encoded non-binary plain strings would be quite weird too.
Yes, my point was that Vinay's usage could be covered by distributing distil as a zip file. All it is doing is decoding it's blob of data (which is an encoded zip file) and then adding the resulting zip to sys.path.
The virtualenv situation is different, as there we are trying to ensure that we remain single-file while embedding things that are *not* modules to add to sys.path. And we don't want to download our dependencies because we need to be able to run with no internet connection. But you are right, the embedded script approach is not ideal.
I hope that "embedded binary blobs" does not become a common approach. I'd much rather that "runnable zip files" became the norm. It's certainly possible now, but I don't think it's well enough known (and there are administrative issues like the file extension question on Windows that make it more awkward than it should be). Hence my comments, trying to raise awareness a bit.
Thanks for the feedback, and in particular the reminder that virtualenv could do with looking at this... I've added a virtualenv issue to remind me to think some more about it. Paul _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
From: Paul Moore <p.f.moore@gmail.com>
Yes, my point was that Vinay's usage could be covered by distributing distil as a zip file. All it is doing is decoding it's blob of data (which is an encoded zip file) and then adding the resulting zip to sys.path.
[snip]
I hope that "embedded binary blobs" does not become a common approach. I'd much rather that "runnable zip files" became the norm.
I don't know if it's that important to distinguish between the two. I found the approach I'm using with distil to be a tad more flexible in my case. A runnable zip has the advantage that it's harder to tinker with, but with the way distil.py is at the moment, you can tweak e.g. its logging just by changing distil.py. (At some point soon it will have an optional configuration file to control some aspects of its behaviour, but that's by the by.) It also does a bit of processing to process -e and -p and relaunches with a new Python interpreter if needed - developing this logic was quicker because I didn't have to add my changes to the .zip each time I tweaked something. Regards, Vinay Sajip
On Thu, Mar 28, 2013 at 9:11 AM, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
From: Paul Moore <p.f.moore@gmail.com>
Yes, my point was that Vinay's usage could be covered by distributing distil as a zip file. All it is doing is decoding it's blob of data (which is an encoded zip file) and then adding the resulting zip to sys.path.
[snip]
I hope that "embedded binary blobs" does not become a common approach. I'd much rather that "runnable zip files" became the norm.
I don't know if it's that important to distinguish between the two. I found the approach I'm using with distil to be a tad more flexible in my case. A runnable zip has the advantage that it's harder to tinker with, but with the way distil.py is at the moment, you can tweak e.g. its logging just by changing distil.py. (At some point soon it will have an optional configuration file to control some aspects of its behaviour, but that's by the by.) It also does a bit of processing to process -e and -p and relaunches with a new Python interpreter if needed - developing this logic was quicker because I didn't have to add my changes to the .zip each time I tweaked something.
Regards,
Vinay Sajip
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Also if you are looking for tweakability you can run a directory with the same contents of the .zip exactly the same as if it was a zip.
From: Daniel Holth <dholth@gmail.com>
Also if you are looking for tweakability you can run a directory with the same contents of the .zip exactly the same as if it was a zip.
Sure, but my smoke testing involved copying the tweaked distil.py to a network share, then running that file from other Windows, Linux and OS X machines - of course I could have copied whole directory trees, but doing it the way I've done works well enough for me :-) Regards, Vinay Sajip
On 28 March 2013 13:11, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
I don't know if it's that important to distinguish between the two. I found the approach I'm using with distil to be a tad more flexible in my case. A runnable zip has the advantage that it's harder to tinker with, but with the way distil.py is at the moment, you can tweak e.g. its logging just by changing distil.py. (At some point soon it will have an optional configuration file to control some aspects of its behaviour, but that's by the by.) It also does a bit of processing to process -e and -p and relaunches with a new Python interpreter if needed - developing this logic was quicker because I didn't have to add my changes to the .zip each time I tweaked something.
Good point. Paul
From: Philippe Ombredanne <pombredanne@nexb.com>
On the other hand, I find it somewhat discomforting as an emerging best way to package and distribute self-contained bootstrap scripts.
But what is the root cause of that discomfort? The distil approach is slightly more discoverable than a pure zip would be, but for the security conscious all the code is there and available for inspection (unlike installing a distribution directly from PyPI, which will pull you-know-not-what from the network).
Virtualenv does it, distil is doing it now, pip tried some of it here https://github.com/pypa/pip/blob/develop/contrib/get-pip.py In contrast, buildout, distribute and setuptools bootstrap scripts do not embed their dependencies and either try to get them satisfied locally or attempt to download the requirements.
And all this time, they would have been vulnerable to a MITM attack on PyPI because PyPI didn't support verifiable SSL connections until recently. It's good to be cautious, but Bruce Schneier has plenty of stories about caution directed in the wrong directions.
Having some support to do self-contained bootstrap scripts (as in requiring no network access and embedding all their dependencies) using this shar style could be something to consider normalizing?
It seems like a decision for individual developers or developer teams to make on a case-by-case basis - it doesn't seem like something that needs to be "officially" encouraged or discouraged. Regards, Vinay Sajip
On Thu, Mar 28, 2013 at 2:33 PM, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
From: Philippe Ombredanne <pombredanne@nexb.com> On the other hand, I find it somewhat discomforting as an emerging best way to package and distribute self-contained bootstrap scripts.
Virtualenv does it, distil is doing it now, pip tried some of it here https://github.com/pypa/pip/blob/develop/contrib/get-pip.py In contrast, buildout, distribute and setuptools bootstrap scripts do not embed their dependencies and either try to get them satisfied locally or attempt to download the requirements.
And all this time, they would have been vulnerable to a MITM attack on PyPI because PyPI didn't support verifiable SSL connections until recently. It's good to be cautious, but Bruce Schneier has plenty of stories about caution directed in the wrong directions.
I am not so worried about security... I brought the point here because this is the packaging and distribution list, and I see this as an emerging pattern for the packaging and distribution of bootstrap scripts and this is something that has not been discussed much before. Conceptually I find these no different from setup.py scripts, and these have been mostly normalized (or at the minimum have a conventional name and a conventional if not specified interface.) Yet today, for the all important core package and environment management tools, we have bootstrap scripts each with different interfaces and different approaches to self containment or no containment. I feel this is worth discussing as bootstrapping is where everything begins :) -- Philippe Ombredanne +1 650 799 0949 | pombredanne@nexB.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com
Not really trying to tell Vinay to rewrite his script, but IMHO if you expect it unzip is a lot easier than file.write(module.random_attribute.decode('base64')). The runnable zip feature is awesome, not well enough known, and totally worth promoting over the shar pattern; with some minimal tooling you'd be good to go. On Thu, Mar 28, 2013 at 10:44 AM, Philippe Ombredanne <pombredanne@nexb.com> wrote:
On Thu, Mar 28, 2013 at 2:33 PM, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
From: Philippe Ombredanne <pombredanne@nexb.com> On the other hand, I find it somewhat discomforting as an emerging best way to package and distribute self-contained bootstrap scripts.
Virtualenv does it, distil is doing it now, pip tried some of it here https://github.com/pypa/pip/blob/develop/contrib/get-pip.py In contrast, buildout, distribute and setuptools bootstrap scripts do not embed their dependencies and either try to get them satisfied locally or attempt to download the requirements.
And all this time, they would have been vulnerable to a MITM attack on PyPI because PyPI didn't support verifiable SSL connections until recently. It's good to be cautious, but Bruce Schneier has plenty of stories about caution directed in the wrong directions.
I am not so worried about security... I brought the point here because this is the packaging and distribution list, and I see this as an emerging pattern for the packaging and distribution of bootstrap scripts and this is something that has not been discussed much before.
Conceptually I find these no different from setup.py scripts, and these have been mostly normalized (or at the minimum have a conventional name and a conventional if not specified interface.)
Yet today, for the all important core package and environment management tools, we have bootstrap scripts each with different interfaces and different approaches to self containment or no containment.
I feel this is worth discussing as bootstrapping is where everything begins :)
-- Philippe Ombredanne
+1 650 799 0949 | pombredanne@nexB.com DejaCode Enterprise at http://www.dejacode.com nexB Inc. at http://www.nexb.com _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Daniel Holth <dholth <at> gmail.com> writes:
file.write(module.random_attribute.decode('base64')). The runnable zip feature is awesome, not well enough known, and totally worth promoting over the shar pattern; with some minimal tooling you'd be good to go.
Well, maybe the promoting would be better done by actually shipping something this way that shows how well it works, rather than just talking about it. I don't see that the user experience is any better with runnable zips, though I'm not saying it's any worse. After that, it just comes down to individual developer taste, and there's no accounting for that :-) Regards, Vinay Sajip
Daniel Holth <dholth <at> gmail.com> writes:
file.write(module.random_attribute.decode('base64')). The runnable zip feature is awesome, not well enough known, and totally worth promoting over the shar pattern; with some minimal tooling you'd be good to go.
Runnable zips sound great - I certainly haven't come across them before (or if I have, I didn't see the potential at the time). That said, from a Windows perspective, shebangs and mixed text/binary files worry me. The better approach on Windows would be to take a new extension (.pyz? .pyp[ackage]?) and associate that with the launcher. (File extensions on Windows are the moral equivalent of shebang lines.) Changing .zip in any way will upset anyone who has a utility for opening ZIP files (i.e. everyone) and there's no way to launch files differently based on content without changing that association. And, I'm almost certain that most if not all existing ZIP tools on Windows will fail to open files with a shebang, since they've never had to deal with them. I also think that a runnable zip may be a better package installation option than MSIs, but that's another issue :) Cheers, Steve
On Thu, Mar 28, 2013 at 1:54 PM, Steve Dower <Steve.Dower@microsoft.com> wrote:
And, I'm almost certain that most if not all existing ZIP tools on Windows will fail to open files with a shebang, since they've never had to deal with them.
Actually, the opposite is true, at least for 3rd-party (non-Microsoft) archiving tools: they work even when there's a whole .exe file stuck on the front. ;-) Some of them require you to rename from .exe to .zip first, but some actually detect that an .exe is a stub in front of a zip file and give you extraction options in an Explorer right-click. So, no worries on the prepended data front, even if the extension is .zip. What you probably can't safely do is *modify* a .zip with prepended data... and there I'm just guessing, because I've never actually tried.
On Thu, Mar 28, 2013 at 3:49 PM, PJ Eby <pje@telecommunity.com> wrote:
On Thu, Mar 28, 2013 at 1:54 PM, Steve Dower <Steve.Dower@microsoft.com> wrote:
And, I'm almost certain that most if not all existing ZIP tools on Windows will fail to open files with a shebang, since they've never had to deal with them.
Actually, the opposite is true, at least for 3rd-party (non-Microsoft) archiving tools: they work even when there's a whole .exe file stuck on the front. ;-)
Some of them require you to rename from .exe to .zip first, but some actually detect that an .exe is a stub in front of a zip file and give you extraction options in an Explorer right-click.
So, no worries on the prepended data front, even if the extension is .zip. What you probably can't safely do is *modify* a .zip with prepended data... and there I'm just guessing, because I've never actually tried.
It will all work. ZIP is cool! Every offset is relative from the end of the file. The wheel distribution even has a ZipFile subclass that lets you pop() files off the end by truncating the file and rewriting the index. This will work on any ordinary zip file that is just the concatenation of the compressed files in zip directory order, without data or extra space between the compressed files.
Daniel Holth wrote:
On Thu, Mar 28, 2013 at 3:49 PM, PJ Eby <pje@telecommunity.com> wrote:
On Thu, Mar 28, 2013 at 1:54 PM, Steve Dower <Steve.Dower@microsoft.com> wrote:
And, I'm almost certain that most if not all existing ZIP tools on Windows will fail to open files with a shebang, since they've never had to deal with them.
Actually, the opposite is true, at least for 3rd-party (non-Microsoft) archiving tools: they work even when there's a whole .exe file stuck on the front. ;-)
Some of them require you to rename from .exe to .zip first, but some actually detect that an .exe is a stub in front of a zip file and give you extraction options in an Explorer right-click.
So, no worries on the prepended data front, even if the extension is .zip. What you probably can't safely do is *modify* a .zip with prepended data... and there I'm just guessing, because I've never actually tried.
It will all work. ZIP is cool! Every offset is relative from the end of the file.
The wheel distribution even has a ZipFile subclass that lets you pop() files off the end by truncating the file and rewriting the index. This will work on any ordinary zip file that is just the concatenation of the compressed files in zip directory order, without data or extra space between the compressed files.
Ah of course, I totally forgot that it works from the end of the file. I'd still rather they got a new extension though, since nobody is going to teach WinZip what "#! python" means. That's probably an issue for the handful of Windows devs on python-dev rather than here, though. Cheers, Steve
On Thu, Mar 28, 2013 at 6:19 PM, Steve Dower <Steve.Dower@microsoft.com> wrote:
Daniel Holth wrote:
On Thu, Mar 28, 2013 at 3:49 PM, PJ Eby <pje@telecommunity.com> wrote:
On Thu, Mar 28, 2013 at 1:54 PM, Steve Dower <Steve.Dower@microsoft.com> wrote:
And, I'm almost certain that most if not all existing ZIP tools on Windows will fail to open files with a shebang, since they've never had to deal with them.
Actually, the opposite is true, at least for 3rd-party (non-Microsoft) archiving tools: they work even when there's a whole .exe file stuck on the front. ;-)
Some of them require you to rename from .exe to .zip first, but some actually detect that an .exe is a stub in front of a zip file and give you extraction options in an Explorer right-click.
So, no worries on the prepended data front, even if the extension is .zip. What you probably can't safely do is *modify* a .zip with prepended data... and there I'm just guessing, because I've never actually tried.
It will all work. ZIP is cool! Every offset is relative from the end of the file.
The wheel distribution even has a ZipFile subclass that lets you pop() files off the end by truncating the file and rewriting the index. This will work on any ordinary zip file that is just the concatenation of the compressed files in zip directory order, without data or extra space between the compressed files.
Ah of course, I totally forgot that it works from the end of the file.
I'd still rather they got a new extension though, since nobody is going to teach WinZip what "#! python" means. That's probably an issue for the handful of Windows devs on python-dev rather than here, though.
Cheers, Steve
WinZip will ignore anything in the front of the file since the zip directory doesn't reference it. The #! shebang is for Unix, would point to the correct Python, and the +x flag would make it executable. The mini PEP is for the .pyz registration and for publicity.
On Fri, Mar 29, 2013 at 8:43 AM, Daniel Holth <dholth@gmail.com> wrote:
WinZip will ignore anything in the front of the file since the zip directory doesn't reference it. The #! shebang is for Unix, would point to the correct Python, and the +x flag would make it executable. The mini PEP is for the .pyz registration and for publicity.
The two big reasons almost nobody knows about the executable zip files and directories is we forgot to mention it in the original 2.6 What's New (it's there now, but was added much later), and it was done in a tracker issue [1] (with Guido participating) rather than as a PEP. A new PEP to: * register the .pyz and .pyzw extensions in the 3.4 Windows installer * ship a tool for creating an executable pyz or pyzw file from a directory of pure-Python files (warning if any extension files are noticed, and with the option of bytecode precompilation) Would be great. Cheers, Nick. [1] http://bugs.python.org/issue1739468 -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Would pyzw be much better than reading the shebang line for Windows? On Mar 29, 2013 4:11 PM, "Nick Coghlan" <ncoghlan@gmail.com> wrote:
On Fri, Mar 29, 2013 at 8:43 AM, Daniel Holth <dholth@gmail.com> wrote:
WinZip will ignore anything in the front of the file since the zip directory doesn't reference it. The #! shebang is for Unix, would point to the correct Python, and the +x flag would make it executable. The mini PEP is for the .pyz registration and for publicity.
The two big reasons almost nobody knows about the executable zip files and directories is we forgot to mention it in the original 2.6 What's New (it's there now, but was added much later), and it was done in a tracker issue [1] (with Guido participating) rather than as a PEP.
A new PEP to:
* register the .pyz and .pyzw extensions in the 3.4 Windows installer * ship a tool for creating an executable pyz or pyzw file from a directory of pure-Python files (warning if any extension files are noticed, and with the option of bytecode precompilation)
Would be great.
Cheers, Nick.
[1] http://bugs.python.org/issue1739468
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 29 March 2013 22:15, Daniel Holth <dholth@gmail.com> wrote:
Would pyzw be much better than reading the shebang line for Windows?
Yes. A different executable has to be run (console or windows). A .pyz file with pythonw in the shebang would run py.exe and flash up a console window before starting pythonw.exe. Paul
Nick Coghlan <ncoghlan <at> gmail.com> writes:
* ship a tool for creating an executable pyz or pyzw file from a directory of pure-Python files
This could be a variant of my pyzzer.py tool: https://gist.github.com/vsajip/5276787 It doesn't print warnings for extensions or byte-compile anything, but otherwise does more or less what you mentioned. Regards, Vinay Sajip
On Mar 29, 2013 4:12 PM, "Nick Coghlan" <ncoghlan@gmail.com> wrote:
On Fri, Mar 29, 2013 at 8:43 AM, Daniel Holth <dholth@gmail.com> wrote:
WinZip will ignore anything in the front of the file since the zip directory doesn't reference it. The #! shebang is for Unix, would point to the correct Python, and the +x flag would make it executable. The mini PEP is for the .pyz registration and for publicity.
The two big reasons almost nobody knows about the executable zip files and directories is we forgot to mention it in the original 2.6 What's New (it's there now, but was added much later), and it was done in a tracker issue [1] (with Guido participating) rather than as a PEP.
A new PEP to:
* register the .pyz and .pyzw extensions in the 3.4 Windows installer * ship a tool for creating an executable pyz or pyzw file from a directory of pure-Python files (warning if any extension files are noticed, and with the option of bytecode precompilation)
Would be great.
And that pre-compilation could even do it for multiple versions of Python thanks to __pycache__. I've actually contemplated creating a distutils command to do this exact thing. Been thinking about this since 2010: http://sayspy.blogspot.ca/2010/03/various-ways-of-distributing-python.html
Cheers, Nick.
[1] http://bugs.python.org/issue1739468
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
On 29 Mar, 2013, at 21:11, Nick Coghlan <ncoghlan@gmail.com> wrote:
On Fri, Mar 29, 2013 at 8:43 AM, Daniel Holth <dholth@gmail.com> wrote:
WinZip will ignore anything in the front of the file since the zip directory doesn't reference it. The #! shebang is for Unix, would point to the correct Python, and the +x flag would make it executable. The mini PEP is for the .pyz registration and for publicity.
The two big reasons almost nobody knows about the executable zip files and directories is we forgot to mention it in the original 2.6 What's New (it's there now, but was added much later), and it was done in a tracker issue [1] (with Guido participating) rather than as a PEP.
A new PEP to:
* register the .pyz and .pyzw extensions in the 3.4 Windows installer
Also: * add support for .pyz and .pyzw support to Python Launcher on OSX
* ship a tool for creating an executable pyz or pyzw file from a directory of pure-Python files (warning if any extension files are noticed, and with the option of bytecode precompilation)
Ronald
Philippe Ombredanne <pombredanne <at> nexb.com> writes:
Conceptually I find these no different from setup.py scripts, and these have been mostly normalized (or at the minimum have a conventional name and a conventional if not specified interface.)
Except that you programmatically interface (to distutils or setuptools) with setup.py, which is not the case with virtualenv or distil.
I feel this is worth discussing as bootstrapping is where everything begins :)
Oh, certainly it's worthy of discussion - I wasn't meaning to imply otherwise. Regards, Vinay Sajip
participants (9)
-
Brett Cannon -
Daniel Holth -
Nick Coghlan -
Paul Moore -
Philippe Ombredanne -
PJ Eby -
Ronald Oussoren -
Steve Dower -
Vinay Sajip