On Fri, 20 Nov 2020 at 10:55, Abdur-Rahmaan Janhangeer <arj.python@gmail.com> wrote:
After the above thread I contacted a number of Zipapp-related persons including the author of Shiv ( Loren Carvalho ) who was kind enough to reply. Along the conversation he stated that
In fact, shiv itself sort of breaks a fundamental assumption about zipapps: that they remain zipped! shiv pyz files extract themselves on execution, for a myriad of reasons.
Shiv is indeed a very good solution. I've used it myself a lot (almost entirely for command line utilities, not for "something you can double click on to run it", though). My observations are that: 1. The fact that it builds .pyz files rather than .exe files *is* a problem for me. It's solvable by prepending a small exe header (as per the recipe in the zipapp docs) but making that step easier would be a significant improvement. 2. The way shiv unzips things is relatively seamless, and needed in order to support C extensions. Loading shared libraries direct from a zip file is very definitely *not* simple, and is one of the things that makes a project like pyinstaller complex to maintain and support. Shiv takes a sensible "practicality vs purity" trade-off here, IMO. 3. Even taking the above into account, there are some things that don't work perfectly - look at the -E option to shiv, which was something added based on experiences I had, to deal with tools like tox, which expect their dependencies to be visible in subprocesses.
Considering the 3 enhancements proposed by Mr Moore, i'd say that zipapps needs a tidbit twerking to be a great packaging solution.
I don't think zipapps will ever be a total solution. But they *can* be very good if they suit your particular requirements. And many people have fairly simple needs, which suit the capabilities of zipapps.
A successful zip-based packaged format is the JAR (https://en.wikipedia.org/wiki/JAR_(file_format)). JSmooth is a tool for converting JARs into EXEs https://en.wikipedia.org/wiki/JSmooth
Jar format has similar problems to zipapps regarding native libraries, I believe. However, Java's relentless drive to have "100% pure Java" solutions, along with a lot of corporate backing, mitigates this significantly. I don't think anyone would seriously consider using a Postgres driver written in "Pure Python" over a C extension, for example. And let's not even contemplate what a "100% Pure Python" version of numpy would be like :-)
Might give some leads
Jar files were certainly an example that motivated zipimport and the whole import hook machinery that underpins zipapps. If you're interested in this type of technology, TCL "Starkits" (https://www.tcl.tk/starkits/) are another interesting example, based on having the embedded code in a database (Metakit) rather than an archive format (zip). I've often thought about writing an import hook to handle putting packages in an embedded sqlite database, but it's never looked like it would add sufficient extra value over zipfiles to seem worth doing anything about it :-(
Though this thread suggests adding the natural abilities of turning python projects into executables, doing so via zipapps is an option i haven't considered. And it does not seem like it's for tomorrow.
IMO zipapps are definitely an under-appreciated option (of course, I would say that :-)) I think the problems with cases where they aren't a good solution, as well as past experience with the (similar, but more ambitious in ways that didn't work out) egg format has hindered their adoption somewhat. Paul