[Distutils] Current Python packaging status (from my point of view)

[Nick Coghlan]

On 1 November 2016 at 17:30, Thomas Güttler
<guettliml at thomas-guettler.de> wrote:
> Am 17.09.2016 um 12:29 schrieb Nick Coghlan:
>> Hi folks,
>>
>> Prompted by a few posts I read recently about the current state of the
>> Python packaging ecosystem, I figured it made sense to put together an
>> article summarising my own perspective on the current state of things:
>> http://www.curiousefficiency.org/posts/2016/09/python-packaging-ecosystem.html
>
>
> Thank you for this summarizing article. Yes, a lot was done during the last months.
>
> I liked the part "My core software ecosystem design philosophy" a lot, since
> it explains that both parties (software consumer and software publisher) want
> it to be simple and easy.
>
>
> About conda: if pip and conda overlap in some point. Why not implement
> this in a reusable library which gets used by conda and pip?

For the parts where they genuinely overlap, conda is already able to
just use pip, or else the same libraries that pip uses. For the
platform management pieces (SAT solving for conda repositories,
converting PyPI packages to conda ones, language independent
environment management), what conda does is outside the scope of what
pip supports anyway.

> About funding: Looking for more income is one way to solve this. Why not
> look into the other direction: How to reduce costs?

Thanks to donated infrastructure, the direct costs to the PSF are
incredibly low already. Donald went into some detail on that in
https://caremad.io/posts/2016/05/powering-pypi/ and that's mostly
still accurate (although his funded role with HPE ended recently)

> Heading "Making the presence of a compiler on end user systems optional". Here
> I just can say: Thank you very much. I guess it was a lot of hard work to
> make this all simple and easy for the software consumers and publishers. Thank you.
>
> I wrote some lines, but I deleted my thoughts about the topic "Automating wheel creation", since
> I am a afraid it could raise bad mood in this list again. That's not my goal.

I currently see 3 main ways that could eventually happen:

- the PSF sorts out its general sustainability concerns to the point
where it believes it can credibly maintain such a service on the
community's behalf
- the conda-forge folks branch out into offering wheel building as
well (so it becomes a matter of "publish your Python projects for the
user level conda platform, get platform independent Python wheels as
well")
- someone builds such a service independently of the current PyPI
infrastructure team, and convinces package publishers to start using
it

There's also a 4th variant, which is getting to a point where someone
figures out a pushbutton solution for a build pipeline in a public
PaaS that offers a decent free tier. This is potentially one of the
more promising options, since it means the sustainability risks
related to future growth in demand accrue to the PaaS providers,
rather than to the PSF. However, it's somewhat gated on the Warehouse
migration, since you really want API token support for that kind of
automation, which is something the current PyPI code base doesn't
have, and isn't going to gain.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia