[Distutils] PEP 427

[Daniel Holth]

On Tue, Jan 28, 2014 at 10:41 PM, Evgeny Sazhin <eugene at sazhin.us> wrote:
>
> On Jan 28, 2014, at 9:54 PM, Daniel Holth <dholth at gmail.com> wrote:
>
>> On Tue, Jan 28, 2014 at 8:04 PM, Evgeny Sazhin <eugene at sazhin.us> wrote:
>>> Hi,
>>>
>>> I saw that people from this list are responsible for Wheel related PEP.
>>> I'm comparatively new to the python packaging and need some help understanding the recommended way of dealing with python packages.
>>>
>>> I'm trying to create a development infrastructure that would allow for simple and unified ways of sharing, *deploying* and *reusing* the code within private entity. I can see that pip with virtual environments and requirements.txt is very similar to dependency management provided by maven or apache ivy. But there seems to be a disconnect between the egg carrying the possibility to be importable and executable, but in the same time considered to be deprecated format which is not fully supported by pip and virtualenv and wheel not having those basic questions answered...
>>>
>>> So, i would like to ask few questions about it :
>>>
>>> 1. I'm coming from java world, so it is bit hard to understand why new packaging format is not importable? What is the reason behind it? Wouldn't it be way easier to deal with the dependencies and imports?
>>> 2. Why unzipping and installing the wheel is the way to go? There is no need to unzip java jar to import it into your code, neither to run your code with it? Why wheel can't be used the same exact way?
>>>
>>>
>>> I would appreciate any insight about the development infrastructure the wheel designers have in mind.
>>>
>>> Thanks in advance,
>>> Eugene
>>
>
>
>> The main reason the packaging format is not explicitly importable is
>> simply because Python hasn't had "jar-like" deployment for as long or
>> as consistently as Java.
>
> Thanks a lot for taking time to respond!
>
> Does it mean that it actually makes sense to look into that direction and make wheel usage closer to jar?
> If wheel is promoted as the ultimate solution to the packaging problems (at least that's my impression ) aren't the abilities to import packages and classes from it as well as make it runnable a kind of top priority and something that is right on the surface?
>
>> So while Java code universally uses a "get
>> resource" API to get stuff on the classpath, a a lot of Python code
>> will try to open a file. C extensions can't be loaded from inside zip
>> files.
>
> I have no knowledge about c extensions scope, but i feel like it might be of less importance then pure python packaging issues? Am I wrong?
> I'd think wheel should do everything in its power to provide pain free workflow for pure python, and if there is a need to add some C extension then solve that problem separately? Is the possibility to add C extension into the wheel so critical, that it should prohibit useful feature for pure python modules?
>
>> And it's a lot harder to edit .py files once they are zipped
>> up, unlike Java where only the compiled and non-editable .class files
>> are zipped.
>
> Why would i want to edit the zipped .py file from a wheel? I want to import a package or a class from it, reuse a function. Nothing more.
>
>
>>
>> ZIP import is a great feature but wheels aren't really designed for
>> it. It's more reliable that way.
>
> More reliable in what regard?
>
> My impression is that the whole thing about pip, virtualenv and wheels is great - it takes into consideration a lot of end user problems, aka getting proper module and installing it to RUN applications. But for whatever reason it misses some important problems on the DEVELOPMENT side.
>
> I'm trying to find a sane way for developer to work, so if wheel is not an answer - what is? Deprecated Egg? Or may be then egg is not going to go anywhere and pip has to learn to deal with them as well as it does with wheels?

It's true that we have not put a lot of effort into trying to support
jar-style development in this system. If you want that then you will
have to do additional work.

However a lot of people do find developing on the filesystem in
virtualenvs to be quite sane. All the source code for the whole
application remains easily editable for debugging. (By the way, these
days the debugger does know how to load .py source from within zip
files.)

You should also give buildout a try. It installs each dependency into
its own directory as an unzipped egg and does not use virtualenv. Egg
will be around for a while longer, it's just not written down in a
PEP.

Wheel solves a coupling problem between the build system and the
installer, allowing those two steps to be performed in separate
processes or even on separate machines. It does not solve all
problems.