From robin at reportlab.com  Wed Jan  4 07:47:29 2017
From: robin at reportlab.com (Robin Becker)
Date: Wed, 4 Jan 2017 12:47:29 +0000
Subject: [Distutils] Python 3.6 new warnings
Message-ID: <2da4bd38-c39f-3d60-6c68-730fc382011d@chamonix.reportlab.co.uk>

After the recent 3.6 release I started to build reportlab wheels for the new 
release on windows amd_x64 etc etc.

A new warning which I haven't seen before is

> C:\ux\XB33\py36_x86\lib\site-packages\wheel\pep425tags.py:77: RuntimeWarning: Config variable 'Py_DEBUG' is unset, Python ABI tag may be incorrect
>   warn=(impl == 'cp')):
> C:\ux\XB33\py36_x86\lib\site-packages\wheel\pep425tags.py:81: RuntimeWarning: Config variable 'WITH_PYMALLOC' is unset,
> Python ABI tag may be incorrect
>   warn=(impl == 'cp')):

the windows wheels come out named like this

> reportlab-3.3.26-cp27-none-win32.whl
> reportlab-3.3.26-cp27-none-win_amd64.whl
> reportlab-3.3.26-cp33-none-win32.whl
> reportlab-3.3.26-cp33-none-win_amd64.whl
> reportlab-3.3.26-cp34-none-win32.whl
> reportlab-3.3.26-cp34-none-win_amd64.whl
> reportlab-3.3.26-cp35-none-win32.whl
> reportlab-3.3.26-cp35-none-win_amd64.whl
> reportlab-3.3.26-cp36-cp36m-win32.whl
> reportlab-3.3.26-cp36-cp36m-win_amd64.whl

so

1) should I be attempting to make the none's correct using code in setup.py
2) is the tagging correct for cp36? Looking at pep 425 I don't see any 
definitions of the suffices other than d being debug.
-- 
Robin Becker

From ethan at stoneleaf.us  Thu Jan  5 11:45:01 2017
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 05 Jan 2017 08:45:01 -0800
Subject: [Distutils] Can't upload sdist: "File already exists"
In-Reply-To: <6D5351B4-3122-4748-8322-43FE15099888@stufft.io>
References: <CAHkxivenL=1qYyYocFTVRgOx93uKewykOzzSJphxF5hoLWwWmA@mail.gmail.com>
 <CAP1=2W417oDOLu4CYZj0_GEe=h68y_2i_CPTnfDcE29s0PrY+A@mail.gmail.com>
 <CAN-Kwu09a+G=KwddXYaDjAcmzk7=nGQtKS4daZd_oY_E1zC-KA@mail.gmail.com>
 <7238F292-D507-4C2B-ACA6-8C93B1ACB7A2@twistedmatrix.com>
 <6D5351B4-3122-4748-8322-43FE15099888@stufft.io>
Message-ID: <586E780D.4050807@stoneleaf.us>

On 12/22/2016 12:41 PM, Donald Stufft wrote:

> There is a fairly new restriction that you can only have *one* sdist per
>  release now. That should not apply at all to Wheels and if it is, then it
>  is a bug. I can?t reproduce this issue though, so I?m going to guess
>  there was some bit of confusion about exact errors here. If someone
>  actually cannot upload 1 sdist + N wheels, please leave the state as is
> and tell me.

I am also seeing this problem.

My standard files for upload have been a py2 wheel, a py3 wheel, a zip sdist, and a tar sdist, and now the tar sdist is being denied as a duplicate.

Do I need to choose between zip and tar sdists?

--
~Ethan~

From donald at stufft.io  Thu Jan  5 11:46:19 2017
From: donald at stufft.io (Donald Stufft)
Date: Thu, 5 Jan 2017 11:46:19 -0500
Subject: [Distutils] Can't upload sdist: "File already exists"
In-Reply-To: <586E780D.4050807@stoneleaf.us>
References: <CAHkxivenL=1qYyYocFTVRgOx93uKewykOzzSJphxF5hoLWwWmA@mail.gmail.com>
 <CAP1=2W417oDOLu4CYZj0_GEe=h68y_2i_CPTnfDcE29s0PrY+A@mail.gmail.com>
 <CAN-Kwu09a+G=KwddXYaDjAcmzk7=nGQtKS4daZd_oY_E1zC-KA@mail.gmail.com>
 <7238F292-D507-4C2B-ACA6-8C93B1ACB7A2@twistedmatrix.com>
 <6D5351B4-3122-4748-8322-43FE15099888@stufft.io>
 <586E780D.4050807@stoneleaf.us>
Message-ID: <68EB0F39-A408-46CF-9A9A-AD7252A6A107@stufft.io>


> On Jan 5, 2017, at 11:45 AM, Ethan Furman <ethan at stoneleaf.us> wrote:
> 
> Do I need to choose between zip and tar sdists?

Yes.

?
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170105/033cfe6d/attachment.html>

From ethan at stoneleaf.us  Thu Jan  5 12:55:03 2017
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 05 Jan 2017 09:55:03 -0800
Subject: [Distutils] Can't upload sdist: "File already exists"
In-Reply-To: <68EB0F39-A408-46CF-9A9A-AD7252A6A107@stufft.io>
References: <CAHkxivenL=1qYyYocFTVRgOx93uKewykOzzSJphxF5hoLWwWmA@mail.gmail.com>
 <CAP1=2W417oDOLu4CYZj0_GEe=h68y_2i_CPTnfDcE29s0PrY+A@mail.gmail.com>
 <CAN-Kwu09a+G=KwddXYaDjAcmzk7=nGQtKS4daZd_oY_E1zC-KA@mail.gmail.com>
 <7238F292-D507-4C2B-ACA6-8C93B1ACB7A2@twistedmatrix.com>
 <6D5351B4-3122-4748-8322-43FE15099888@stufft.io>
 <586E780D.4050807@stoneleaf.us>
 <68EB0F39-A408-46CF-9A9A-AD7252A6A107@stufft.io>
Message-ID: <586E8877.4050004@stoneleaf.us>

On 01/05/2017 08:46 AM, Donald Stufft wrote:
>> On Jan 5, 2017, at 11:45 AM, Ethan Furman wrote:

>> Do I need to choose between zip and tar sdists?
>
> Yes.

I haven't followed Windows O/Ses for a while now -- are they able to easily access tar files?  Or do *nix machines come standard with unzip now?

Or is it more along the lines of -- you seem to be a developer, so you should be able to install the one you need?  ;)

--
~Ethan~

From donald at stufft.io  Thu Jan  5 12:58:09 2017
From: donald at stufft.io (Donald Stufft)
Date: Thu, 5 Jan 2017 12:58:09 -0500
Subject: [Distutils] Can't upload sdist: "File already exists"
In-Reply-To: <586E8877.4050004@stoneleaf.us>
References: <CAHkxivenL=1qYyYocFTVRgOx93uKewykOzzSJphxF5hoLWwWmA@mail.gmail.com>
 <CAP1=2W417oDOLu4CYZj0_GEe=h68y_2i_CPTnfDcE29s0PrY+A@mail.gmail.com>
 <CAN-Kwu09a+G=KwddXYaDjAcmzk7=nGQtKS4daZd_oY_E1zC-KA@mail.gmail.com>
 <7238F292-D507-4C2B-ACA6-8C93B1ACB7A2@twistedmatrix.com>
 <6D5351B4-3122-4748-8322-43FE15099888@stufft.io>
 <586E780D.4050807@stoneleaf.us>
 <68EB0F39-A408-46CF-9A9A-AD7252A6A107@stufft.io>
 <586E8877.4050004@stoneleaf.us>
Message-ID: <38BE7BB8-D740-400B-BDF7-3AC8AABEFDF8@stufft.io>


> On Jan 5, 2017, at 12:55 PM, Ethan Furman <ethan at stoneleaf.us> wrote:
> 
> On 01/05/2017 08:46 AM, Donald Stufft wrote:
>>> On Jan 5, 2017, at 11:45 AM, Ethan Furman wrote:
> 
>>> Do I need to choose between zip and tar sdists?
>> 
>> Yes.
> 
> I haven't followed Windows O/Ses for a while now -- are they able to easily access tar files?  Or do *nix machines come standard with unzip now?
> 
> Or is it more along the lines of -- you seem to be a developer, so you should be able to install the one you need?  ;)
> 
> --
> ~Ethan~
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig


.tar.gz is the recommended format but either one is OK. I think that there is still the default tool disparity but getting a tool to open either one is fairly trivial, particularly since you can do ``python3 -m tarfile -e foobar-1.0.tar.gz. That being said, non-automated downloads account for a tiny fraction of the total downloads on PyPI so I wouldn?t worry too much about that use case in general.

?
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170105/7f65e249/attachment.html>

From ethan at stoneleaf.us  Thu Jan  5 13:13:23 2017
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 05 Jan 2017 10:13:23 -0800
Subject: [Distutils] Can't upload sdist: "File already exists"
In-Reply-To: <38BE7BB8-D740-400B-BDF7-3AC8AABEFDF8@stufft.io>
References: <CAHkxivenL=1qYyYocFTVRgOx93uKewykOzzSJphxF5hoLWwWmA@mail.gmail.com>
 <CAP1=2W417oDOLu4CYZj0_GEe=h68y_2i_CPTnfDcE29s0PrY+A@mail.gmail.com>
 <CAN-Kwu09a+G=KwddXYaDjAcmzk7=nGQtKS4daZd_oY_E1zC-KA@mail.gmail.com>
 <7238F292-D507-4C2B-ACA6-8C93B1ACB7A2@twistedmatrix.com>
 <6D5351B4-3122-4748-8322-43FE15099888@stufft.io>
 <586E780D.4050807@stoneleaf.us>
 <68EB0F39-A408-46CF-9A9A-AD7252A6A107@stufft.io>
 <586E8877.4050004@stoneleaf.us>
 <38BE7BB8-D740-400B-BDF7-3AC8AABEFDF8@stufft.io>
Message-ID: <586E8CC3.601@stoneleaf.us>

On 01/05/2017 09:58 AM, Donald Stufft wrote:

> .tar.gz is the recommended format ...

.tar.gz it is!  Thanks!

--
~Ethan~

From prometheus235 at gmail.com  Thu Jan  5 14:37:33 2017
From: prometheus235 at gmail.com (Nick Timkovich)
Date: Thu, 5 Jan 2017 13:37:33 -0600
Subject: [Distutils] Can't upload sdist: "File already exists"
In-Reply-To: <586E8CC3.601@stoneleaf.us>
References: <CAHkxivenL=1qYyYocFTVRgOx93uKewykOzzSJphxF5hoLWwWmA@mail.gmail.com>
 <CAP1=2W417oDOLu4CYZj0_GEe=h68y_2i_CPTnfDcE29s0PrY+A@mail.gmail.com>
 <CAN-Kwu09a+G=KwddXYaDjAcmzk7=nGQtKS4daZd_oY_E1zC-KA@mail.gmail.com>
 <7238F292-D507-4C2B-ACA6-8C93B1ACB7A2@twistedmatrix.com>
 <6D5351B4-3122-4748-8322-43FE15099888@stufft.io>
 <586E780D.4050807@stoneleaf.us>
 <68EB0F39-A408-46CF-9A9A-AD7252A6A107@stufft.io>
 <586E8877.4050004@stoneleaf.us>
 <38BE7BB8-D740-400B-BDF7-3AC8AABEFDF8@stufft.io>
 <586E8CC3.601@stoneleaf.us>
Message-ID: <CAHkxivezky6X-wYDHP-awuU+oRLn5gNahWe3nLNrGgcySGh5Cg@mail.gmail.com>

I never determined what was causing my problem, I couldn't reproduce it on
testpypi so I gave up (it's a small package, and if someone wants the
source, they can look at the GH link I have in the metadata).

Specifically, why does it say that the file exists, but I *can not see it
anywhere*, either via the web interface or download it with pip --no-binary?

Is it possible that there exists a "huffman-0.1.2.tar.gz" on PyPI from
someone else who may have had the package name, uploaded some releases,
then deleted it? I am 99% confident that I did not publish/delete (all I do
is twine upload dist/*), let alone delete a specific .tar.gz dist (is that
even possible to do from the command-line?). The error happened when I
tried to do a 0.1.1, figured I messed up, then rebuilt/republished with a
bigger number.

The information isn't exposed to me (maybe logs somewhere?), so I'll
probably just accept PyPI is strange like that.

On Thu, Jan 5, 2017 at 12:13 PM, Ethan Furman <ethan at stoneleaf.us> wrote:

> On 01/05/2017 09:58 AM, Donald Stufft wrote:
>
> .tar.gz is the recommended format ...
>>
>
> .tar.gz it is!  Thanks!
>
>
> --
> ~Ethan~
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170105/5e217dbc/attachment.html>

From dholth at gmail.com  Thu Jan  5 15:12:42 2017
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 05 Jan 2017 20:12:42 +0000
Subject: [Distutils] Python 3.6 new warnings
In-Reply-To: <2da4bd38-c39f-3d60-6c68-730fc382011d@chamonix.reportlab.co.uk>
References: <2da4bd38-c39f-3d60-6c68-730fc382011d@chamonix.reportlab.co.uk>
Message-ID: <CAG8k2+6k+oTKb+utY9JDAnR02jt+5DHYSwnm+5njE_WqNYv2JA@mail.gmail.com>

https://bitbucket.org/pypa/wheel/src/54ddbcc9cec25e1f4d111a142b8bfaa163130a61/wheel/pep425tags.py?at=default&fileviewer=file-view-default#pep425tags.py-66

sysconfig has a lot less information on Windows, bdist_wheel uses a
different mechanism to detect "debug", "pymalloc", and "4-byte unicode",
and newer versions of bdist_wheel warn about using the fallback mechanism.

You might see fewer 'none' if you make sure bdist_wheel is up to date in
all your build environments.

On Wed, Jan 4, 2017 at 7:47 AM Robin Becker <robin at reportlab.com> wrote:

After the recent 3.6 release I started to build reportlab wheels for the new
release on windows amd_x64 etc etc.

A new warning which I haven't seen before is

> C:\ux\XB33\py36_x86\lib\site-packages\wheel\pep425tags.py:77:
RuntimeWarning: Config variable 'Py_DEBUG' is unset, Python ABI tag may be
incorrect
>   warn=(impl == 'cp')):
> C:\ux\XB33\py36_x86\lib\site-packages\wheel\pep425tags.py:81:
RuntimeWarning: Config variable 'WITH_PYMALLOC' is unset,
> Python ABI tag may be incorrect
>   warn=(impl == 'cp')):

the windows wheels come out named like this

> reportlab-3.3.26-cp27-none-win32.whl
> reportlab-3.3.26-cp27-none-win_amd64.whl
> reportlab-3.3.26-cp33-none-win32.whl
> reportlab-3.3.26-cp33-none-win_amd64.whl
> reportlab-3.3.26-cp34-none-win32.whl
> reportlab-3.3.26-cp34-none-win_amd64.whl
> reportlab-3.3.26-cp35-none-win32.whl
> reportlab-3.3.26-cp35-none-win_amd64.whl
> reportlab-3.3.26-cp36-cp36m-win32.whl
> reportlab-3.3.26-cp36-cp36m-win_amd64.whl

so

1) should I be attempting to make the none's correct using code in setup.py
2) is the tagging correct for cp36? Looking at pep 425 I don't see any
definitions of the suffices other than d being debug.
--
Robin Becker
_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG at python.org
https://mail.python.org/mailman/listinfo/distutils-sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170105/1dd65954/attachment-0001.html>

From ethan at stoneleaf.us  Thu Jan  5 15:13:41 2017
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 05 Jan 2017 12:13:41 -0800
Subject: [Distutils] Can't upload sdist: "File already exists"
In-Reply-To: <CAHkxivezky6X-wYDHP-awuU+oRLn5gNahWe3nLNrGgcySGh5Cg@mail.gmail.com>
References: <CAHkxivenL=1qYyYocFTVRgOx93uKewykOzzSJphxF5hoLWwWmA@mail.gmail.com>
 <CAP1=2W417oDOLu4CYZj0_GEe=h68y_2i_CPTnfDcE29s0PrY+A@mail.gmail.com>
 <CAN-Kwu09a+G=KwddXYaDjAcmzk7=nGQtKS4daZd_oY_E1zC-KA@mail.gmail.com>
 <7238F292-D507-4C2B-ACA6-8C93B1ACB7A2@twistedmatrix.com>
 <6D5351B4-3122-4748-8322-43FE15099888@stufft.io>
 <586E780D.4050807@stoneleaf.us>
 <68EB0F39-A408-46CF-9A9A-AD7252A6A107@stufft.io>
 <586E8877.4050004@stoneleaf.us>
 <38BE7BB8-D740-400B-BDF7-3AC8AABEFDF8@stufft.io> <586E8CC3.601@stoneleaf.us>
 <CAHkxivezky6X-wYDHP-awuU+oRLn5gNahWe3nLNrGgcySGh5Cg@mail.gmail.com>
Message-ID: <586EA8F5.6070401@stoneleaf.us>

On 01/05/2017 11:37 AM, Nick Timkovich wrote:

> I never determined what was causing my problem, I couldn't reproduce it
>  on testpypi so I gave up (it's a small package, and if someone wants the
>  source, they can look at the GH link I have in the metadata).

Do you have both a .zip and a .tar source distribution?

--
~Ethan~

From prometheus235 at gmail.com  Thu Jan  5 16:37:18 2017
From: prometheus235 at gmail.com (Nick Timkovich)
Date: Thu, 5 Jan 2017 15:37:18 -0600
Subject: [Distutils] Can't upload sdist: "File already exists"
In-Reply-To: <586EA8F5.6070401@stoneleaf.us>
References: <CAHkxivenL=1qYyYocFTVRgOx93uKewykOzzSJphxF5hoLWwWmA@mail.gmail.com>
 <CAP1=2W417oDOLu4CYZj0_GEe=h68y_2i_CPTnfDcE29s0PrY+A@mail.gmail.com>
 <CAN-Kwu09a+G=KwddXYaDjAcmzk7=nGQtKS4daZd_oY_E1zC-KA@mail.gmail.com>
 <7238F292-D507-4C2B-ACA6-8C93B1ACB7A2@twistedmatrix.com>
 <6D5351B4-3122-4748-8322-43FE15099888@stufft.io>
 <586E780D.4050807@stoneleaf.us>
 <68EB0F39-A408-46CF-9A9A-AD7252A6A107@stufft.io>
 <586E8877.4050004@stoneleaf.us>
 <38BE7BB8-D740-400B-BDF7-3AC8AABEFDF8@stufft.io>
 <586E8CC3.601@stoneleaf.us>
 <CAHkxivezky6X-wYDHP-awuU+oRLn5gNahWe3nLNrGgcySGh5Cg@mail.gmail.com>
 <586EA8F5.6070401@stoneleaf.us>
Message-ID: <CAHkxivftaCE6J=SF43gjFaJRK37TT8Oxn0rKmjHcnS-axCySjA@mail.gmail.com>

No. My build command was: `python setup.py sdist bdist_wheel` which doesn't
generate a zip.

On Thu, Jan 5, 2017 at 2:13 PM, Ethan Furman <ethan at stoneleaf.us> wrote:

> On 01/05/2017 11:37 AM, Nick Timkovich wrote:
>
> I never determined what was causing my problem, I couldn't reproduce it
>>  on testpypi so I gave up (it's a small package, and if someone wants the
>>  source, they can look at the GH link I have in the metadata).
>>
>
> Do you have both a .zip and a .tar source distribution?
>
>
> --
> ~Ethan~
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170105/38542f8c/attachment.html>

From ncoghlan at gmail.com  Thu Jan  5 21:41:31 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 6 Jan 2017 12:41:31 +1000
Subject: [Distutils] Can't upload sdist: "File already exists"
In-Reply-To: <CAHkxivftaCE6J=SF43gjFaJRK37TT8Oxn0rKmjHcnS-axCySjA@mail.gmail.com>
References: <CAHkxivenL=1qYyYocFTVRgOx93uKewykOzzSJphxF5hoLWwWmA@mail.gmail.com>
 <CAP1=2W417oDOLu4CYZj0_GEe=h68y_2i_CPTnfDcE29s0PrY+A@mail.gmail.com>
 <CAN-Kwu09a+G=KwddXYaDjAcmzk7=nGQtKS4daZd_oY_E1zC-KA@mail.gmail.com>
 <7238F292-D507-4C2B-ACA6-8C93B1ACB7A2@twistedmatrix.com>
 <6D5351B4-3122-4748-8322-43FE15099888@stufft.io>
 <586E780D.4050807@stoneleaf.us>
 <68EB0F39-A408-46CF-9A9A-AD7252A6A107@stufft.io>
 <586E8877.4050004@stoneleaf.us>
 <38BE7BB8-D740-400B-BDF7-3AC8AABEFDF8@stufft.io>
 <586E8CC3.601@stoneleaf.us>
 <CAHkxivezky6X-wYDHP-awuU+oRLn5gNahWe3nLNrGgcySGh5Cg@mail.gmail.com>
 <586EA8F5.6070401@stoneleaf.us>
 <CAHkxivftaCE6J=SF43gjFaJRK37TT8Oxn0rKmjHcnS-axCySjA@mail.gmail.com>
Message-ID: <CADiSq7cVOThKish-NyaNzi3yrH5_zWK=E8E2YW5mUZeAZ808ag@mail.gmail.com>

On 6 January 2017 at 07:37, Nick Timkovich <prometheus235 at gmail.com> wrote:
> No. My build command was: `python setup.py sdist bdist_wheel` which doesn't
> generate a zip.

We've seen behaviour like this before with the old upload server
implementation, where it was possible to get a particular 5xx error on
upload that meant the server had registered the file as uploaded, but
the upload hadn't actually worked (this bug in the old server was
actually the main reason the default client config suggested on
packaging.python.org was changed).

However, your command output indicates that you're already using the
new upload server, so that's unlikely to have been the problem :(

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From robin at reportlab.com  Fri Jan  6 06:34:13 2017
From: robin at reportlab.com (Robin Becker)
Date: Fri, 6 Jan 2017 11:34:13 +0000
Subject: [Distutils] Python 3.6 new warnings
In-Reply-To: <CAG8k2+6k+oTKb+utY9JDAnR02jt+5DHYSwnm+5njE_WqNYv2JA@mail.gmail.com>
References: <2da4bd38-c39f-3d60-6c68-730fc382011d@chamonix.reportlab.co.uk>
 <CAG8k2+6k+oTKb+utY9JDAnR02jt+5DHYSwnm+5njE_WqNYv2JA@mail.gmail.com>
Message-ID: <0a07d2f9-49de-e246-a4a1-a37a986bbea4@chamonix.reportlab.co.uk>

On 05/01/2017 20:12, Daniel Holth wrote:
> https://bitbucket.org/pypa/wheel/src/54ddbcc9cec25e1f4d111a142b8bfaa163130a61/wheel/pep425tags.py?at=default&fileviewer=file-view-default#pep425tags.py-66
>
> sysconfig has a lot less information on Windows, bdist_wheel uses a
> different mechanism to detect "debug", "pymalloc", and "4-byte unicode",
> and newer versions of bdist_wheel warn about using the fallback mechanism.

> You might see fewer 'none' if you make sure bdist_wheel is up to date in
> all your build environments.
>
thanks; I will try updating the build environments.

.......

-- 
Robin Becker

From donald at stufft.io  Tue Jan 10 08:24:36 2017
From: donald at stufft.io (Donald Stufft)
Date: Tue, 10 Jan 2017 08:24:36 -0500
Subject: [Distutils] Announcement: TLSv1.2 will become mandatory in the
 future
Message-ID: <103E266D-F645-4B58-9E22-E45DFD207CD4@stufft.io>

Fastly has announced plans to disable TLSv1.0 and TLSv1.1 on their CDN endpoints
which will include PyPI (as well as other Python properties). You can see their
timeline at https://www.fastly.com/blog/phase-two-our-tls-10-and-11-deprecation-plan.

There are two hard cut off dates to remember:

* April 30, 2017, which is when any Python.org site you see that does *not*
  have an EV certificate that is hosted by Fastly will no longer support
  TLSv1.0 and TLSv1.1 (testpypi.python.org, test.pypi.org,
  files.pythonhosted.org, etc). This will affect Warehouse since that uses
  files.pythonhosted.org to serve files.

* June 30, 2018, which is when any Python.org site you see that has an EV
  certificate that is hosted by Fastly will no longer support TSLv1.0 and
  TLSv1.1 (pypi.python.org, pypi.org, etc).

I am going to see about possibly organizing some scheduled "brown outs" of
TLSv1.0 and TLSv1.1 prior to the cut off dates to try and help folks find places
that will need updates. Any scheduled brownouts will be posted to
status.python.org prior to happening.

Looking at the download numbers, the absolute largest driver of TLSv1.0 and
TLSv1.1 traffic to PyPI are old versions of pip or other clients where I cannot
tell the OS that they are being run on. Past that, macOS is going to be the
largest casualty since their system Python does not support TLSv1.2 yet in any
version of their OS.

If you have a Python and you want to check to see if it supports TLSv1.2 or not,
the easiest way to do that is by running:

    python2 -c "import urllib2,json; print(json.loads(urllib2.urlopen('https://www.howsmyssl.com/a/check').read())['tls_version'])"

    OR

    python3 -c "import urllib.request,json; print(json.loads(urllib.request.urlopen('https://www.howsmyssl.com/a/check').read())['tls_version'])"

If you get something other than TLS 1.2, then I suggest making plans to deal
with the inevitable breakage which may start occurring on or before April 30,
2017.



?
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170110/686fa001/attachment.html>

From barry at python.org  Tue Jan 10 11:17:58 2017
From: barry at python.org (Barry Warsaw)
Date: Tue, 10 Jan 2017 11:17:58 -0500
Subject: [Distutils] Announcement: TLSv1.2 will become mandatory in the
 future
References: <103E266D-F645-4B58-9E22-E45DFD207CD4@stufft.io>
Message-ID: <20170110111758.54848ade@subdivisions.wooz.org>

On Jan 10, 2017, at 08:24 AM, Donald Stufft wrote:

>    python3 -c "import urllib.request,json; print(json.loads(urllib.request.urlopen('https://www.howsmyssl.com/a/check').read())['tls_version'])"

For Python 3.5:

python3 -c "import urllib.request,json; print(json.loads(urllib.request.urlopen('https://www.howsmyssl.com/a/check').read().decode('utf-8'))['tls_version'])"

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 801 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170110/c14fc105/attachment.sig>

From donald at stufft.io  Tue Jan 10 15:02:28 2017
From: donald at stufft.io (Donald Stufft)
Date: Tue, 10 Jan 2017 15:02:28 -0500
Subject: [Distutils] Announcement: TLSv1.2 will become mandatory in the
 future
In-Reply-To: <D43715F6-67F9-40FC-AAC1-8EF7897654F2@mac.com>
References: <103E266D-F645-4B58-9E22-E45DFD207CD4@stufft.io>
 <D43715F6-67F9-40FC-AAC1-8EF7897654F2@mac.com>
Message-ID: <FC4EA91A-943A-4829-97DC-42C21D2A71A2@stufft.io>


> On Jan 10, 2017, at 3:01 PM, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
> 
> 
>> On 10 Jan 2017, at 14:24, Donald Stufft <donald at stufft.io <mailto:donald at stufft.io>> wrote:
>> [?] Past that, macOS is going to be the
>> largest casualty since their system Python does not support TLSv1.2 yet in any
>> version of their OS.
> 
> Not just the system Python on OSX, this also affects all Python.org <http://python.org/> installers for OSX except 3.6. The 3.6 installer is the first one that doesn?t use the system installation of OpenSSL.  Annoyingly with OpenSSL on OSX you have to options: either use an up-to-date release or have OpenSSL use the system CA trust store, but not both.  Sigh?
> 
> I have no idea how may users use the Python.org <http://python.org/> installers on OSX, but this has the potential to affect a largish number of users on OSX including newbies (but far from all users on OSX, there?s also a sizeable population using Homebrew or Anaconda).
> 
> Ronald


Ah yea I forgot those :( 

?
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170110/1abfe350/attachment.html>

From ronaldoussoren at mac.com  Tue Jan 10 15:07:26 2017
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 10 Jan 2017 21:07:26 +0100
Subject: [Distutils] Announcement: TLSv1.2 will become mandatory in the
 future
In-Reply-To: <FC4EA91A-943A-4829-97DC-42C21D2A71A2@stufft.io>
References: <103E266D-F645-4B58-9E22-E45DFD207CD4@stufft.io>
 <D43715F6-67F9-40FC-AAC1-8EF7897654F2@mac.com>
 <FC4EA91A-943A-4829-97DC-42C21D2A71A2@stufft.io>
Message-ID: <B22ACDEA-A873-4E8D-94C8-F42416758308@mac.com>


> On 10 Jan 2017, at 21:02, Donald Stufft <donald at stufft.io> wrote:
> 
> 
>> On Jan 10, 2017, at 3:01 PM, Ronald Oussoren <ronaldoussoren at mac.com <mailto:ronaldoussoren at mac.com>> wrote:
>> 
>> 
>>> On 10 Jan 2017, at 14:24, Donald Stufft <donald at stufft.io <mailto:donald at stufft.io>> wrote:
>>> [?] Past that, macOS is going to be the
>>> largest casualty since their system Python does not support TLSv1.2 yet in any
>>> version of their OS.
>> 
>> Not just the system Python on OSX, this also affects all Python.org <http://python.org/> installers for OSX except 3.6. The 3.6 installer is the first one that doesn?t use the system installation of OpenSSL.  Annoyingly with OpenSSL on OSX you have to options: either use an up-to-date release or have OpenSSL use the system CA trust store, but not both.  Sigh?
>> 
>> I have no idea how may users use the Python.org <http://python.org/> installers on OSX, but this has the potential to affect a largish number of users on OSX including newbies (but far from all users on OSX, there?s also a sizeable population using Homebrew or Anaconda).
>> 
>> Ronald
> 
> 
> Ah yea I forgot those :( 

Could you file an issue on bugs.python.org <http://bugs.python.org/> about this? That way Ned will get notified, and the use of system OpenSSL can be reconsidered for a future 2.7 release.


Ronald

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170110/ba12916f/attachment.html>

From nad at python.org  Tue Jan 10 15:47:48 2017
From: nad at python.org (Ned Deily)
Date: Tue, 10 Jan 2017 15:47:48 -0500
Subject: [Distutils] Announcement: TLSv1.2 will become mandatory in the
 future
In-Reply-To: <B22ACDEA-A873-4E8D-94C8-F42416758308@mac.com>
References: <103E266D-F645-4B58-9E22-E45DFD207CD4@stufft.io>
 <D43715F6-67F9-40FC-AAC1-8EF7897654F2@mac.com>
 <FC4EA91A-943A-4829-97DC-42C21D2A71A2@stufft.io>
 <B22ACDEA-A873-4E8D-94C8-F42416758308@mac.com>
Message-ID: <8839A36B-A676-4AE0-AA94-9DADBC3D65DB@python.org>

On Jan 10, 2017, at 15:07, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>> On 10 Jan 2017, at 21:02, Donald Stufft <donald at stufft.io> wrote:
>>> On Jan 10, 2017, at 3:01 PM, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>>>> On 10 Jan 2017, at 14:24, Donald Stufft <donald at stufft.io> wrote:
>>>> [?] Past that, macOS is going to be the
>>>> largest casualty since their system Python does not support TLSv1.2 yet in any
>>>> version of their OS.
>>> Not just the system Python on OSX, this also affects all Python.org installers for OSX except 3.6. The 3.6 installer is the first one that doesn?t use the system installation of OpenSSL.

That's not quite accurate.  The 32-bit-only macOS python.org installers for recent 2.7.x and 3.x releases are also linked with a private current set of OpenSSL libraries.  For 3.6, we no longer supply the 32-bit-only installer and the 64-bit/32-bit installer is now linked with the private OpenSSL as you note.

>>> Annoyingly with OpenSSL on OSX you have to options: either use an up-to-date release or have OpenSSL use the system CA trust store, but not both.  Sigh?

It would be nice if someone would do the work to figure out whether it is feasible to use Apple's own Crypto and TLS API's as apparently libcurl does.

>>> I have no idea how may users use the Python.org installers on OSX, but this has the potential to affect a largish number of users on OSX including newbies (but far from all users on OSX, there?s also a sizeable population using Homebrew or Anaconda).

And MacPorts.  I don't know about Anaconda but the other two already use their own private versions of OpenSSL AFAIK.

--
  Ned Deily
  nad at python.org -- []


From donald at stufft.io  Tue Jan 10 15:50:32 2017
From: donald at stufft.io (Donald Stufft)
Date: Tue, 10 Jan 2017 15:50:32 -0500
Subject: [Distutils] Announcement: TLSv1.2 will become mandatory in the
 future
In-Reply-To: <8839A36B-A676-4AE0-AA94-9DADBC3D65DB@python.org>
References: <103E266D-F645-4B58-9E22-E45DFD207CD4@stufft.io>
 <D43715F6-67F9-40FC-AAC1-8EF7897654F2@mac.com>
 <FC4EA91A-943A-4829-97DC-42C21D2A71A2@stufft.io>
 <B22ACDEA-A873-4E8D-94C8-F42416758308@mac.com>
 <8839A36B-A676-4AE0-AA94-9DADBC3D65DB@python.org>
Message-ID: <EDB5DA71-8C05-45DC-A23F-6EAEC006C74E@stufft.io>


> On Jan 10, 2017, at 3:47 PM, Ned Deily <nad at python.org> wrote:
> 
> On Jan 10, 2017, at 15:07, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>>> On 10 Jan 2017, at 21:02, Donald Stufft <donald at stufft.io> wrote:
>>>> On Jan 10, 2017, at 3:01 PM, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>>>>> On 10 Jan 2017, at 14:24, Donald Stufft <donald at stufft.io> wrote:
>>>>> [?] Past that, macOS is going to be the
>>>>> largest casualty since their system Python does not support TLSv1.2 yet in any
>>>>> version of their OS.
>>>> Not just the system Python on OSX, this also affects all Python.org installers for OSX except 3.6. The 3.6 installer is the first one that doesn?t use the system installation of OpenSSL.
> 
> That's not quite accurate.  The 32-bit-only macOS python.org installers for recent 2.7.x and 3.x releases are also linked with a private current set of OpenSSL libraries.  For 3.6, we no longer supply the 32-bit-only installer and the 64-bit/32-bit installer is now linked with the private OpenSSL as you note.
> 
>>>> Annoyingly with OpenSSL on OSX you have to options: either use an up-to-date release or have OpenSSL use the system CA trust store, but not both.  Sigh?
> 
> It would be nice if someone would do the work to figure out whether it is feasible to use Apple's own Crypto and TLS API's as apparently libcurl does.

It would be really nice if we could deprecate `ssl` (which has a bunch of OpenSSL specific stuff in it) and add a new `tls` module that served as an implementation agnostic library that would use OpenSSL on *nix, SecureTransport on macOS, and SChannel on Windows. However, in the mean time there are some folks poking to see about making something pip suitable that will enable us to use SecureTransport at least.

> 
>>>> I have no idea how may users use the Python.org installers on OSX, but this has the potential to affect a largish number of users on OSX including newbies (but far from all users on OSX, there?s also a sizeable population using Homebrew or Anaconda).
> 
> And MacPorts.  I don't know about Anaconda but the other two already use their own private versions of OpenSSL AFAIK.
> 
> --
>  Ned Deily
>  nad at python.org -- []
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig


?
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170110/24249ae9/attachment-0001.html>

From ronaldoussoren at mac.com  Tue Jan 10 15:01:37 2017
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 10 Jan 2017 21:01:37 +0100
Subject: [Distutils] Announcement: TLSv1.2 will become mandatory in the
 future
In-Reply-To: <103E266D-F645-4B58-9E22-E45DFD207CD4@stufft.io>
References: <103E266D-F645-4B58-9E22-E45DFD207CD4@stufft.io>
Message-ID: <D43715F6-67F9-40FC-AAC1-8EF7897654F2@mac.com>


> On 10 Jan 2017, at 14:24, Donald Stufft <donald at stufft.io> wrote:
> [?] Past that, macOS is going to be the
> largest casualty since their system Python does not support TLSv1.2 yet in any
> version of their OS.

Not just the system Python on OSX, this also affects all Python.org <http://python.org/> installers for OSX except 3.6. The 3.6 installer is the first one that doesn?t use the system installation of OpenSSL.  Annoyingly with OpenSSL on OSX you have to options: either use an up-to-date release or have OpenSSL use the system CA trust store, but not both.  Sigh?

I have no idea how may users use the Python.org <http://python.org/> installers on OSX, but this has the potential to affect a largish number of users on OSX including newbies (but far from all users on OSX, there?s also a sizeable population using Homebrew or Anaconda).

Ronald
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170110/ea5f2870/attachment.html>

From ncoghlan at gmail.com  Tue Jan 10 22:59:43 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 11 Jan 2017 13:59:43 +1000
Subject: [Distutils] Announcement: TLSv1.2 will become mandatory in the
 future
In-Reply-To: <103E266D-F645-4B58-9E22-E45DFD207CD4@stufft.io>
References: <103E266D-F645-4B58-9E22-E45DFD207CD4@stufft.io>
Message-ID: <CADiSq7dbcYa47zDHqrHNX9qCs1YDk3ceVp8UMeq+jE6xfn6UMg@mail.gmail.com>

On 10 January 2017 at 23:24, Donald Stufft <donald at stufft.io> wrote:
> Looking at the download numbers, the absolute largest driver of TLSv1.0 and
> TLSv1.1 traffic to PyPI are old versions of pip or other clients where I
> cannot
> tell the OS that they are being run on.

Can you tell the Python version they're running even with older clients?

I just checked the exact dates/versions where TLS v1.2 was properly
enabled in the various versions of Python that Red Hat ships, and this
change should be fine for:

* RHEL/CentOS 7.2+ (PEP 466 backport released November 2015)
* Red Hat Software Collections 2.2+ (PEP 466 backport released May 2016)

However, folks currently using the system Python 2.6 installation in
RHEL/CentOS 6 are going to need to upgrade to Python 2.7 somehow,
whether that's by:

- upgrading to RHEL/CentOS 7
- doing a parallel install via RHSCL/softwarecollections.org
- doing a parallel install via ius.io

(The problem with RHEL 6 is that even though the *OS* has supported
TLS v1.2 since RHEL 6.5, *Python 2.6* doesn't properly support
accessing them through the standard library's SSL module, since it's
missing the features backported from 3.x by PEP 466)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From donald at stufft.io  Tue Jan 10 23:04:19 2017
From: donald at stufft.io (Donald Stufft)
Date: Tue, 10 Jan 2017 23:04:19 -0500
Subject: [Distutils] Announcement: TLSv1.2 will become mandatory in the
 future
In-Reply-To: <CADiSq7dbcYa47zDHqrHNX9qCs1YDk3ceVp8UMeq+jE6xfn6UMg@mail.gmail.com>
References: <103E266D-F645-4B58-9E22-E45DFD207CD4@stufft.io>
 <CADiSq7dbcYa47zDHqrHNX9qCs1YDk3ceVp8UMeq+jE6xfn6UMg@mail.gmail.com>
Message-ID: <D0821E10-28B3-4FFC-BA7F-B3BBCE3EB3AC@stufft.io>


> On Jan 10, 2017, at 10:59 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> On 10 January 2017 at 23:24, Donald Stufft <donald at stufft.io> wrote:
>> Looking at the download numbers, the absolute largest driver of TLSv1.0 and
>> TLSv1.1 traffic to PyPI are old versions of pip or other clients where I
>> cannot
>> tell the OS that they are being run on.
> 
> Can you tell the Python version they're running even with older clients?
> 
> I just checked the exact dates/versions where TLS v1.2 was properly
> enabled in the various versions of Python that Red Hat ships, and this
> change should be fine for:
> 
> * RHEL/CentOS 7.2+ (PEP 466 backport released November 2015)
> * Red Hat Software Collections 2.2+ (PEP 466 backport released May 2016)
> 
> However, folks currently using the system Python 2.6 installation in
> RHEL/CentOS 6 are going to need to upgrade to Python 2.7 somehow,
> whether that's by:
> 
> - upgrading to RHEL/CentOS 7
> - doing a parallel install via RHSCL/softwarecollections.org
> - doing a parallel install via ius.io
> 
> (The problem with RHEL 6 is that even though the *OS* has supported
> TLS v1.2 since RHEL 6.5, *Python 2.6* doesn't properly support
> accessing them through the standard library's SSL module, since it's
> missing the features backported from 3.x by PEP 466)
> 
> Cheers,
> Nick.
> 
> -- 
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia


No, but it doesn?t matter, the version of Python doesn?t control it at all since we use PROTOCOL_SSLv23 which will automatically negotiate the highest protocol OpenSSL supports, whether Python has bound the PROTOCOL_TLSv1_X constant and implemented the methods for it or not. So Python 2.6 is perfectly capable of talking to a TLSv1.2 site (it however, is not capable of explicitly saying it *needs*  only TLSv1.2).

See:

$ python2.6 -c "import urllib2,json; print(json.loads(urllib2.urlopen('https://www.howsmyssl.com/a/check').read())['tls_version'])"
TLS 1.2

?
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170110/6f183d6a/attachment.html>

From ncoghlan at gmail.com  Wed Jan 11 00:00:32 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 11 Jan 2017 15:00:32 +1000
Subject: [Distutils] Announcement: TLSv1.2 will become mandatory in the
 future
In-Reply-To: <D0821E10-28B3-4FFC-BA7F-B3BBCE3EB3AC@stufft.io>
References: <103E266D-F645-4B58-9E22-E45DFD207CD4@stufft.io>
 <CADiSq7dbcYa47zDHqrHNX9qCs1YDk3ceVp8UMeq+jE6xfn6UMg@mail.gmail.com>
 <D0821E10-28B3-4FFC-BA7F-B3BBCE3EB3AC@stufft.io>
Message-ID: <CADiSq7fwaEZJm04u5pDJCcJhH_yjsqhZjES1Cv9RKKgWyUsoyg@mail.gmail.com>

On 11 January 2017 at 14:04, Donald Stufft <donald at stufft.io> wrote:
> On Jan 10, 2017, at 10:59 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> (The problem with RHEL 6 is that even though the *OS* has supported
> TLS v1.2 since RHEL 6.5, *Python 2.6* doesn't properly support
> accessing them through the standard library's SSL module, since it's
> missing the features backported from 3.x by PEP 466)
>
> No, but it doesn?t matter, the version of Python doesn?t control it at all
> since we use PROTOCOL_SSLv23 which will automatically negotiate the highest
> protocol OpenSSL supports, whether Python has bound the PROTOCOL_TLSv1_X
> constant and implemented the methods for it or not. So Python 2.6 is
> perfectly capable of talking to a TLSv1.2 site (it however, is not capable
> of explicitly saying it *needs*  only TLSv1.2).
>
> See:
>
> $ python2.6 -c "import urllib2,json;
> print(json.loads(urllib2.urlopen('https://www.howsmyssl.com/a/check').read())['tls_version'])"
> TLS 1.2

Ah, excellent. In that case, RHEL 6 should be fine as well, as 6.5 was
released back in 2013, and the extended update support for 6.4 ended
in March 2015.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ronaldoussoren at mac.com  Wed Jan 11 08:09:29 2017
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 11 Jan 2017 14:09:29 +0100
Subject: [Distutils] Announcement: TLSv1.2 will become mandatory in the
 future
In-Reply-To: <8839A36B-A676-4AE0-AA94-9DADBC3D65DB@python.org>
References: <103E266D-F645-4B58-9E22-E45DFD207CD4@stufft.io>
 <D43715F6-67F9-40FC-AAC1-8EF7897654F2@mac.com>
 <FC4EA91A-943A-4829-97DC-42C21D2A71A2@stufft.io>
 <B22ACDEA-A873-4E8D-94C8-F42416758308@mac.com>
 <8839A36B-A676-4AE0-AA94-9DADBC3D65DB@python.org>
Message-ID: <24F74CFC-8B98-4AAB-8B62-DB9C54858B56@mac.com>


> On 10 Jan 2017, at 21:47, Ned Deily <nad at python.org> wrote:
> 
> 
>>>> Annoyingly with OpenSSL on OSX you have to options: either use an up-to-date release or have OpenSSL use the system CA trust store, but not both.  Sigh?
> 
> It would be nice if someone would do the work to figure out whether it is feasible to use Apple's own Crypto and TLS API's as apparently libcurl does.

This should be possible, although I don?t know if the relevant Apple API?s are safe to use after forking (which has bitten us in the past for other Apple APIs, such as the _scproxy extension that detects proxies on OSX).

Sadly enough I don?t really have time to look into this.

Ronald

From eric at trueblade.com  Wed Jan 11 08:12:21 2017
From: eric at trueblade.com (Eric V. Smith)
Date: Wed, 11 Jan 2017 08:12:21 -0500
Subject: [Distutils] Announcement: TLSv1.2 will become mandatory in the
 future
In-Reply-To: <24F74CFC-8B98-4AAB-8B62-DB9C54858B56@mac.com>
References: <103E266D-F645-4B58-9E22-E45DFD207CD4@stufft.io>
 <D43715F6-67F9-40FC-AAC1-8EF7897654F2@mac.com>
 <FC4EA91A-943A-4829-97DC-42C21D2A71A2@stufft.io>
 <B22ACDEA-A873-4E8D-94C8-F42416758308@mac.com>
 <8839A36B-A676-4AE0-AA94-9DADBC3D65DB@python.org>
 <24F74CFC-8B98-4AAB-8B62-DB9C54858B56@mac.com>
Message-ID: <4d696080-4a3f-4b62-246f-e12617d561c8@trueblade.com>

On 1/11/2017 8:09 AM, Ronald Oussoren wrote:
>
>> On 10 Jan 2017, at 21:47, Ned Deily <nad at python.org> wrote:
>>
>>
>>>>> Annoyingly with OpenSSL on OSX you have to options: either use an up-to-date release or have OpenSSL use the system CA trust store, but not both.  Sigh?
>>
>> It would be nice if someone would do the work to figure out whether it is feasible to use Apple's own Crypto and TLS API's as apparently libcurl does.
>
> This should be possible, although I don?t know if the relevant Apple API?s are safe to use after forking (which has bitten us in the past for other Apple APIs, such as the _scproxy extension that detects proxies on OSX).
>
> Sadly enough I don?t really have time to look into this.

While I'm sure this isn't a newbie "easy" issue, maybe open an issue for 
it on the bug tracker and someone will find it and work on it? I'd open 
it myself, but I don't know enough about it to phrase the problem correctly!

Eric.



From brett at python.org  Wed Jan 11 13:26:41 2017
From: brett at python.org (Brett Cannon)
Date: Wed, 11 Jan 2017 18:26:41 +0000
Subject: [Distutils] Announcement: TLSv1.2 will become mandatory in the
 future
In-Reply-To: <EDB5DA71-8C05-45DC-A23F-6EAEC006C74E@stufft.io>
References: <103E266D-F645-4B58-9E22-E45DFD207CD4@stufft.io>
 <D43715F6-67F9-40FC-AAC1-8EF7897654F2@mac.com>
 <FC4EA91A-943A-4829-97DC-42C21D2A71A2@stufft.io>
 <B22ACDEA-A873-4E8D-94C8-F42416758308@mac.com>
 <8839A36B-A676-4AE0-AA94-9DADBC3D65DB@python.org>
 <EDB5DA71-8C05-45DC-A23F-6EAEC006C74E@stufft.io>
Message-ID: <CAP1=2W5BGAQ9kOMwQChJH=w1nJspeFMh9hk9C08Z6OuooCfYyw@mail.gmail.com>

On Tue, 10 Jan 2017 at 12:51 Donald Stufft <donald at stufft.io> wrote:

> [SNIP]
>
>
> It would be really nice if we could deprecate `ssl` (which has a bunch of
> OpenSSL specific stuff in it) and add a new `tls` module that served as an
> implementation agnostic library that would use OpenSSL on *nix,
> SecureTransport on macOS, and SChannel on Windows. However, in the mean
> time there are some folks poking to see about making something pip suitable
> that will enable us to use SecureTransport at least.
>

I know both Cory Benfield and Christian Heimes brought this up briefly at
the PyCon US 2016 language summit at the end of their SSL discussion, but I
don't think it went anywhere because there was some other discussion that
dominated the end of their talk (I've now tweeted at them about this
discussion).

I know Steve has also said he would love to see a agnostic TLS library so
that Windows' built-in libraries for this stuff could be directly used.
With the predicament this is going to put us in I think it makes it very
prudent to create a tls module for the stdlib.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170111/5fbbf37c/attachment.html>

From ethan at stoneleaf.us  Wed Jan 11 14:38:31 2017
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 11 Jan 2017 11:38:31 -0800
Subject: [Distutils] Announcement: TLSv1.2 will become mandatory in the
 future
In-Reply-To: <CAP1=2W5BGAQ9kOMwQChJH=w1nJspeFMh9hk9C08Z6OuooCfYyw@mail.gmail.com>
References: <103E266D-F645-4B58-9E22-E45DFD207CD4@stufft.io>
 <D43715F6-67F9-40FC-AAC1-8EF7897654F2@mac.com>
 <FC4EA91A-943A-4829-97DC-42C21D2A71A2@stufft.io>
 <B22ACDEA-A873-4E8D-94C8-F42416758308@mac.com>
 <8839A36B-A676-4AE0-AA94-9DADBC3D65DB@python.org>
 <EDB5DA71-8C05-45DC-A23F-6EAEC006C74E@stufft.io>
 <CAP1=2W5BGAQ9kOMwQChJH=w1nJspeFMh9hk9C08Z6OuooCfYyw@mail.gmail.com>
Message-ID: <587689B7.2020103@stoneleaf.us>

On 01/11/2017 10:26 AM, Brett Cannon wrote:

> I know both Cory Benfield and Christian Heimes brought this up briefly
>  at the PyCon US 2016 language summit at the end of their SSL discussion,
> but I don't think it went anywhere because there was some other
>  discussion that dominated the end of their talk (I've now tweeted at
>  them about this discussion).
>
> I know Steve has also said he would love to see a agnostic TLS library so
>  that Windows' built-in libraries for this stuff could be directly used.
>  With the predicament this is going to put us in I think it makes it very
>  prudent to create a tls module for the stdlib.

A thread about a new tls module has been started on the security-sig [1].

--
~Ethan~

[1] security-sig at python.org
     https://mail.python.org/pipermail/security-sig/

From ncoghlan at gmail.com  Wed Jan 11 21:58:23 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 12 Jan 2017 12:58:23 +1000
Subject: [Distutils] Announcement: TLSv1.2 will become mandatory in the
 future
In-Reply-To: <CAP1=2W5BGAQ9kOMwQChJH=w1nJspeFMh9hk9C08Z6OuooCfYyw@mail.gmail.com>
References: <103E266D-F645-4B58-9E22-E45DFD207CD4@stufft.io>
 <D43715F6-67F9-40FC-AAC1-8EF7897654F2@mac.com>
 <FC4EA91A-943A-4829-97DC-42C21D2A71A2@stufft.io>
 <B22ACDEA-A873-4E8D-94C8-F42416758308@mac.com>
 <8839A36B-A676-4AE0-AA94-9DADBC3D65DB@python.org>
 <EDB5DA71-8C05-45DC-A23F-6EAEC006C74E@stufft.io>
 <CAP1=2W5BGAQ9kOMwQChJH=w1nJspeFMh9hk9C08Z6OuooCfYyw@mail.gmail.com>
Message-ID: <CADiSq7cGtqNJXXBcy9-hv6o5qqYM7eCH9cAp-M=aY7_pVRjQqw@mail.gmail.com>

On 12 January 2017 at 04:26, Brett Cannon <brett at python.org> wrote:
>
> On Tue, 10 Jan 2017 at 12:51 Donald Stufft <donald at stufft.io> wrote:
>>
>> [SNIP]
>>
>>
>> It would be really nice if we could deprecate `ssl` (which has a bunch of
>> OpenSSL specific stuff in it) and add a new `tls` module that served as an
>> implementation agnostic library that would use OpenSSL on *nix,
>> SecureTransport on macOS, and SChannel on Windows. However, in the mean time
>> there are some folks poking to see about making something pip suitable that
>> will enable us to use SecureTransport at least.
>
>
> I know both Cory Benfield and Christian Heimes brought this up briefly at
> the PyCon US 2016 language summit at the end of their SSL discussion, but I
> don't think it went anywhere because there was some other discussion that
> dominated the end of their talk (I've now tweeted at them about this
> discussion).
>
> I know Steve has also said he would love to see a agnostic TLS library so
> that Windows' built-in libraries for this stuff could be directly used. With
> the predicament this is going to put us in I think it makes it very prudent
> to create a tls module for the stdlib.

Logistically, something I think we should explore for such a module is
using the same ensuretls/tls split that we did for ensurepip/pip. That
way it can be more readily updated in line with the evolution of
network security standards and operating system crpytographic APIs,
rather than being locked into being updated in line with the evolution
of the Python language definition.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From donald at stufft.io  Wed Jan 11 22:00:33 2017
From: donald at stufft.io (Donald Stufft)
Date: Wed, 11 Jan 2017 22:00:33 -0500
Subject: [Distutils] Announcement: TLSv1.2 will become mandatory in the
 future
In-Reply-To: <CADiSq7cGtqNJXXBcy9-hv6o5qqYM7eCH9cAp-M=aY7_pVRjQqw@mail.gmail.com>
References: <103E266D-F645-4B58-9E22-E45DFD207CD4@stufft.io>
 <D43715F6-67F9-40FC-AAC1-8EF7897654F2@mac.com>
 <FC4EA91A-943A-4829-97DC-42C21D2A71A2@stufft.io>
 <B22ACDEA-A873-4E8D-94C8-F42416758308@mac.com>
 <8839A36B-A676-4AE0-AA94-9DADBC3D65DB@python.org>
 <EDB5DA71-8C05-45DC-A23F-6EAEC006C74E@stufft.io>
 <CAP1=2W5BGAQ9kOMwQChJH=w1nJspeFMh9hk9C08Z6OuooCfYyw@mail.gmail.com>
 <CADiSq7cGtqNJXXBcy9-hv6o5qqYM7eCH9cAp-M=aY7_pVRjQqw@mail.gmail.com>
Message-ID: <75AA307A-C19D-45BC-825A-1CF2CCA90F2E@stufft.io>


> On Jan 11, 2017, at 9:58 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> On 12 January 2017 at 04:26, Brett Cannon <brett at python.org> wrote:
>> 
>> On Tue, 10 Jan 2017 at 12:51 Donald Stufft <donald at stufft.io> wrote:
>>> 
>>> [SNIP]
>>> 
>>> 
>>> It would be really nice if we could deprecate `ssl` (which has a bunch of
>>> OpenSSL specific stuff in it) and add a new `tls` module that served as an
>>> implementation agnostic library that would use OpenSSL on *nix,
>>> SecureTransport on macOS, and SChannel on Windows. However, in the mean time
>>> there are some folks poking to see about making something pip suitable that
>>> will enable us to use SecureTransport at least.
>> 
>> 
>> I know both Cory Benfield and Christian Heimes brought this up briefly at
>> the PyCon US 2016 language summit at the end of their SSL discussion, but I
>> don't think it went anywhere because there was some other discussion that
>> dominated the end of their talk (I've now tweeted at them about this
>> discussion).
>> 
>> I know Steve has also said he would love to see a agnostic TLS library so
>> that Windows' built-in libraries for this stuff could be directly used. With
>> the predicament this is going to put us in I think it makes it very prudent
>> to create a tls module for the stdlib.
> 
> Logistically, something I think we should explore for such a module is
> using the same ensuretls/tls split that we did for ensurepip/pip. That
> way it can be more readily updated in line with the evolution of
> network security standards and operating system crpytographic APIs,
> rather than being locked into being updated in line with the evolution
> of the Python language definition.
> 


This doesn?t work well because it?s not something that pip is going to be able to upgrade on Windows, because the .so will be locked when pip imports it on Windows and we won?t be able to uninstall it to do an upgrade. We had to disable the automatic use of pyOpenSSL for this reason too. The only C stuff that pip can reliably use is the standard library.

?
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170111/c0fd5b2d/attachment-0001.html>

From ncoghlan at gmail.com  Wed Jan 11 22:40:14 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 12 Jan 2017 13:40:14 +1000
Subject: [Distutils] Announcement: TLSv1.2 will become mandatory in the
 future
In-Reply-To: <75AA307A-C19D-45BC-825A-1CF2CCA90F2E@stufft.io>
References: <103E266D-F645-4B58-9E22-E45DFD207CD4@stufft.io>
 <D43715F6-67F9-40FC-AAC1-8EF7897654F2@mac.com>
 <FC4EA91A-943A-4829-97DC-42C21D2A71A2@stufft.io>
 <B22ACDEA-A873-4E8D-94C8-F42416758308@mac.com>
 <8839A36B-A676-4AE0-AA94-9DADBC3D65DB@python.org>
 <EDB5DA71-8C05-45DC-A23F-6EAEC006C74E@stufft.io>
 <CAP1=2W5BGAQ9kOMwQChJH=w1nJspeFMh9hk9C08Z6OuooCfYyw@mail.gmail.com>
 <CADiSq7cGtqNJXXBcy9-hv6o5qqYM7eCH9cAp-M=aY7_pVRjQqw@mail.gmail.com>
 <75AA307A-C19D-45BC-825A-1CF2CCA90F2E@stufft.io>
Message-ID: <CADiSq7che3aW9q8zJ6OZEqfM1ow4QOtxsmZ0QK1=TcKCgu2qAg@mail.gmail.com>

On 12 January 2017 at 13:00, Donald Stufft <donald at stufft.io> wrote:
> This doesn?t work well because it?s not something that pip is going to be
> able to upgrade on Windows, because the .so will be locked when pip imports
> it on Windows and we won?t be able to uninstall it to do an upgrade. We had
> to disable the automatic use of pyOpenSSL for this reason too. The only C
> stuff that pip can reliably use is the standard library.

Ugh, I'd completely forgotten about that limitation of Windows filesystems.

And the main alternatives I can think of involve copying files around
as pip starts up, which would be unacceptably slow for a command line
app :(

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From donald at stufft.io  Wed Jan 11 22:47:48 2017
From: donald at stufft.io (Donald Stufft)
Date: Wed, 11 Jan 2017 22:47:48 -0500
Subject: [Distutils] Announcement: TLSv1.2 will become mandatory in the
 future
In-Reply-To: <CADiSq7che3aW9q8zJ6OZEqfM1ow4QOtxsmZ0QK1=TcKCgu2qAg@mail.gmail.com>
References: <103E266D-F645-4B58-9E22-E45DFD207CD4@stufft.io>
 <D43715F6-67F9-40FC-AAC1-8EF7897654F2@mac.com>
 <FC4EA91A-943A-4829-97DC-42C21D2A71A2@stufft.io>
 <B22ACDEA-A873-4E8D-94C8-F42416758308@mac.com>
 <8839A36B-A676-4AE0-AA94-9DADBC3D65DB@python.org>
 <EDB5DA71-8C05-45DC-A23F-6EAEC006C74E@stufft.io>
 <CAP1=2W5BGAQ9kOMwQChJH=w1nJspeFMh9hk9C08Z6OuooCfYyw@mail.gmail.com>
 <CADiSq7cGtqNJXXBcy9-hv6o5qqYM7eCH9cAp-M=aY7_pVRjQqw@mail.gmail.com>
 <75AA307A-C19D-45BC-825A-1CF2CCA90F2E@stufft.io>
 <CADiSq7che3aW9q8zJ6OZEqfM1ow4QOtxsmZ0QK1=TcKCgu2qAg@mail.gmail.com>
Message-ID: <8EDE0A32-0897-4E81-8841-EE53551B0264@stufft.io>


> On Jan 11, 2017, at 10:40 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> On 12 January 2017 at 13:00, Donald Stufft <donald at stufft.io> wrote:
>> This doesn?t work well because it?s not something that pip is going to be
>> able to upgrade on Windows, because the .so will be locked when pip imports
>> it on Windows and we won?t be able to uninstall it to do an upgrade. We had
>> to disable the automatic use of pyOpenSSL for this reason too. The only C
>> stuff that pip can reliably use is the standard library.
> 
> Ugh, I'd completely forgotten about that limitation of Windows filesystems.
> 
> And the main alternatives I can think of involve copying files around
> as pip starts up, which would be unacceptably slow for a command line
> app :(
> 

I don?t think it?s a particularly big deal to tie the tls module to the Python lifecycle though, we?ve got a precident for PEPs that backport important security critical stuff and most things are presumably going to be things that we don?t really even need a backport or a PEP for (I?m thinking things like ciphers and such). Particularly if this new thing is documented up front clearly what things you can depend on for compatibility (api and such) and what things can change in minor releases (keeping up with the security joneses stuff).

I think the big thing that really killed the ssl module for so long in Python was the 2.x vs 3.x split with 2.7 living for a _very_ long time, and then no culture of back porting security important changes to it.


?
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170111/d8690e4f/attachment.html>

From glyph at twistedmatrix.com  Wed Jan 11 23:12:06 2017
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Wed, 11 Jan 2017 20:12:06 -0800
Subject: [Distutils] Announcement: TLSv1.2 will become mandatory in the
 future
In-Reply-To: <CADiSq7che3aW9q8zJ6OZEqfM1ow4QOtxsmZ0QK1=TcKCgu2qAg@mail.gmail.com>
References: <103E266D-F645-4B58-9E22-E45DFD207CD4@stufft.io>
 <D43715F6-67F9-40FC-AAC1-8EF7897654F2@mac.com>
 <FC4EA91A-943A-4829-97DC-42C21D2A71A2@stufft.io>
 <B22ACDEA-A873-4E8D-94C8-F42416758308@mac.com>
 <8839A36B-A676-4AE0-AA94-9DADBC3D65DB@python.org>
 <EDB5DA71-8C05-45DC-A23F-6EAEC006C74E@stufft.io>
 <CAP1=2W5BGAQ9kOMwQChJH=w1nJspeFMh9hk9C08Z6OuooCfYyw@mail.gmail.com>
 <CADiSq7cGtqNJXXBcy9-hv6o5qqYM7eCH9cAp-M=aY7_pVRjQqw@mail.gmail.com>
 <75AA307A-C19D-45BC-825A-1CF2CCA90F2E@stufft.io>
 <CADiSq7che3aW9q8zJ6OZEqfM1ow4QOtxsmZ0QK1=TcKCgu2qAg@mail.gmail.com>
Message-ID: <1683F312-733F-44BB-B17A-F8F1A7E68D32@twistedmatrix.com>


> On Jan 11, 2017, at 7:40 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> On 12 January 2017 at 13:00, Donald Stufft <donald at stufft.io> wrote:
>> This doesn?t work well because it?s not something that pip is going to be
>> able to upgrade on Windows, because the .so will be locked when pip imports
>> it on Windows and we won?t be able to uninstall it to do an upgrade. We had
>> to disable the automatic use of pyOpenSSL for this reason too. The only C
>> stuff that pip can reliably use is the standard library.
> 
> Ugh, I'd completely forgotten about that limitation of Windows filesystems.
> 
> And the main alternatives I can think of involve copying files around
> as pip starts up, which would be unacceptably slow for a command line
> app :(


It's possible for Pip to notice that it wants to replace a particular file; you can "unlock" it by moving it aside.

https://serverfault.com/a/503769 <https://serverfault.com/a/503769>
-glyph
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170111/7f209b39/attachment.html>

From ncoghlan at gmail.com  Thu Jan 12 01:07:03 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 12 Jan 2017 16:07:03 +1000
Subject: [Distutils] Announcement: TLSv1.2 will become mandatory in the
 future
In-Reply-To: <8EDE0A32-0897-4E81-8841-EE53551B0264@stufft.io>
References: <103E266D-F645-4B58-9E22-E45DFD207CD4@stufft.io>
 <D43715F6-67F9-40FC-AAC1-8EF7897654F2@mac.com>
 <FC4EA91A-943A-4829-97DC-42C21D2A71A2@stufft.io>
 <B22ACDEA-A873-4E8D-94C8-F42416758308@mac.com>
 <8839A36B-A676-4AE0-AA94-9DADBC3D65DB@python.org>
 <EDB5DA71-8C05-45DC-A23F-6EAEC006C74E@stufft.io>
 <CAP1=2W5BGAQ9kOMwQChJH=w1nJspeFMh9hk9C08Z6OuooCfYyw@mail.gmail.com>
 <CADiSq7cGtqNJXXBcy9-hv6o5qqYM7eCH9cAp-M=aY7_pVRjQqw@mail.gmail.com>
 <75AA307A-C19D-45BC-825A-1CF2CCA90F2E@stufft.io>
 <CADiSq7che3aW9q8zJ6OZEqfM1ow4QOtxsmZ0QK1=TcKCgu2qAg@mail.gmail.com>
 <8EDE0A32-0897-4E81-8841-EE53551B0264@stufft.io>
Message-ID: <CADiSq7c9jXHpP8-RO9O6yOnqoOH3qL+tKY4v37_+ojTtyYZd2A@mail.gmail.com>

On 12 January 2017 at 13:47, Donald Stufft <donald at stufft.io> wrote:
> I don?t think it?s a particularly big deal to tie the tls module to the
> Python lifecycle though, we?ve got a precident for PEPs that backport
> important security critical stuff and most things are presumably going to be
> things that we don?t really even need a backport or a PEP for (I?m thinking
> things like ciphers and such). Particularly if this new thing is documented
> up front clearly what things you can depend on for compatibility (api and
> such) and what things can change in minor releases (keeping up with the
> security joneses stuff).
>
> I think the big thing that really killed the ssl module for so long in
> Python was the 2.x vs 3.x split with 2.7 living for a _very_ long time, and
> then no culture of back porting security important changes to it.

True, it took ~4 years for 2.7 to really fall unacceptably far behind
the state of the art, and even then it was as much about the lack of
SNI support as it was anything else.

If a new tls module started out with an API management policy that
allowed for new constants and for changes to the default security
settings in maintenance releases, then it would likely only need two
PEPs to define an effective rollout plan:

- one to add it to 3.7+
- one to backport the initial version to 2.7.x (and maybe the other
actively supported 3.x branches)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Thu Jan 12 01:10:11 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 12 Jan 2017 16:10:11 +1000
Subject: [Distutils] Announcement: TLSv1.2 will become mandatory in the
 future
In-Reply-To: <1683F312-733F-44BB-B17A-F8F1A7E68D32@twistedmatrix.com>
References: <103E266D-F645-4B58-9E22-E45DFD207CD4@stufft.io>
 <D43715F6-67F9-40FC-AAC1-8EF7897654F2@mac.com>
 <FC4EA91A-943A-4829-97DC-42C21D2A71A2@stufft.io>
 <B22ACDEA-A873-4E8D-94C8-F42416758308@mac.com>
 <8839A36B-A676-4AE0-AA94-9DADBC3D65DB@python.org>
 <EDB5DA71-8C05-45DC-A23F-6EAEC006C74E@stufft.io>
 <CAP1=2W5BGAQ9kOMwQChJH=w1nJspeFMh9hk9C08Z6OuooCfYyw@mail.gmail.com>
 <CADiSq7cGtqNJXXBcy9-hv6o5qqYM7eCH9cAp-M=aY7_pVRjQqw@mail.gmail.com>
 <75AA307A-C19D-45BC-825A-1CF2CCA90F2E@stufft.io>
 <CADiSq7che3aW9q8zJ6OZEqfM1ow4QOtxsmZ0QK1=TcKCgu2qAg@mail.gmail.com>
 <1683F312-733F-44BB-B17A-F8F1A7E68D32@twistedmatrix.com>
Message-ID: <CADiSq7fyvPd-EU5zyrqxHC3zgbZ_qvh7TmJWAVVSvZFJRSjrBw@mail.gmail.com>

On 12 January 2017 at 14:12, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
> It's possible for Pip to notice that it wants to replace a particular file;
> you can "unlock" it by moving it aside.
>
> https://serverfault.com/a/503769

Very interesting, especially as the way CPython loads extension
modules means that the renaming trick should be safe (even if you
"reload" an extension module, or delete it from sys.modules and import
it again, CPython doesn't reload the underlying DLL).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From steve.dower at python.org  Thu Jan 12 00:30:46 2017
From: steve.dower at python.org (Steve Dower)
Date: Wed, 11 Jan 2017 21:30:46 -0800
Subject: [Distutils] Announcement: TLSv1.2 will become mandatory in
 thefuture
In-Reply-To: <8EDE0A32-0897-4E81-8841-EE53551B0264@stufft.io>
References: <103E266D-F645-4B58-9E22-E45DFD207CD4@stufft.io>
 <D43715F6-67F9-40FC-AAC1-8EF7897654F2@mac.com>
 <FC4EA91A-943A-4829-97DC-42C21D2A71A2@stufft.io>
 <B22ACDEA-A873-4E8D-94C8-F42416758308@mac.com>
 <8839A36B-A676-4AE0-AA94-9DADBC3D65DB@python.org>
 <EDB5DA71-8C05-45DC-A23F-6EAEC006C74E@stufft.io>
 <CAP1=2W5BGAQ9kOMwQChJH=w1nJspeFMh9hk9C08Z6OuooCfYyw@mail.gmail.com>
 <CADiSq7cGtqNJXXBcy9-hv6o5qqYM7eCH9cAp-M=aY7_pVRjQqw@mail.gmail.com>
 <75AA307A-C19D-45BC-825A-1CF2CCA90F2E@stufft.io>
 <CADiSq7che3aW9q8zJ6OZEqfM1ow4QOtxsmZ0QK1=TcKCgu2qAg@mail.gmail.com>
 <8EDE0A32-0897-4E81-8841-EE53551B0264@stufft.io>
Message-ID: <E1cRYun-00015u-1M@se2-syd.hostedmail.net.au>

"I don?t think it?s a particularly big deal to tie the tls module to the Python lifecycle though"

I'd hope that the API of this module is stable enough and the native part of the implementation is OS-specific enough that we may not even need to update it that often. (I'm advocating very strongly for just using the OS APIs to implement it, and those don't change often enough for us to need to worry.)

The Linux builds can link to OpenSSL, but there shouldn't be anything requiring OpenSSL for this module, so the update timeframe is totally different. But I've now joined security-sig, which is where the discussion seems to be, so I'll stop designing things here :)

Top-posted from my Windows Phone

-----Original Message-----
From: "Donald Stufft" <donald at stufft.io>
Sent: ?1/?11/?2017 19:48
To: "Nick Coghlan" <ncoghlan at gmail.com>
Cc: "DistUtils mailing list" <distutils-sig at python.org>
Subject: Re: [Distutils] Announcement: TLSv1.2 will become mandatory in thefuture



On Jan 11, 2017, at 10:40 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:


On 12 January 2017 at 13:00, Donald Stufft <donald at stufft.io> wrote:

This doesn?t work well because it?s not something that pip is going to be
able to upgrade on Windows, because the .so will be locked when pip imports
it on Windows and we won?t be able to uninstall it to do an upgrade. We had
to disable the automatic use of pyOpenSSL for this reason too. The only C
stuff that pip can reliably use is the standard library.


Ugh, I'd completely forgotten about that limitation of Windows filesystems.

And the main alternatives I can think of involve copying files around
as pip starts up, which would be unacceptably slow for a command line
app :(




I don?t think it?s a particularly big deal to tie the tls module to the Python lifecycle though, we?ve got a precident for PEPs that backport important security critical stuff and most things are presumably going to be things that we don?t really even need a backport or a PEP for (I?m thinking things like ciphers and such). Particularly if this new thing is documented up front clearly what things you can depend on for compatibility (api and such) and what things can change in minor releases (keeping up with the security joneses stuff).


I think the big thing that really killed the ssl module for so long in Python was the 2.x vs 3.x split with 2.7 living for a _very_ long time, and then no culture of back porting security important changes to it.



?
Donald Stufft
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170111/b4fddd9f/attachment.html>

From guettliml at thomas-guettler.de  Thu Jan 12 07:04:30 2017
From: guettliml at thomas-guettler.de (=?UTF-8?Q?Thomas_G=c3=bcttler?=)
Date: Thu, 12 Jan 2017 13:04:30 +0100
Subject: [Distutils] How to specify dependencies in Python
Message-ID: <6f19cb4a-ce35-4d86-ee3f-52596a59a3ec@thomas-guettler.de>

I came across a python library which has docs, which start like this:

{{{

  Quickstart

  Include foolib in your requirements.txt file.

}}}

AFAIK dependencies should be specified via `install_requires` in `setup.py`.

Should I talk to the maintainer of the library and create a pull-request for the docs?

Regards,
   Thomas G?ttler


-- 
Thomas Guettler http://www.thomas-guettler.de/

From ncoghlan at gmail.com  Thu Jan 12 07:43:19 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 12 Jan 2017 22:43:19 +1000
Subject: [Distutils] How to specify dependencies in Python
In-Reply-To: <6f19cb4a-ce35-4d86-ee3f-52596a59a3ec@thomas-guettler.de>
References: <6f19cb4a-ce35-4d86-ee3f-52596a59a3ec@thomas-guettler.de>
Message-ID: <CADiSq7coTcSnA1b7HAmO76hsQ0X=F8rgeQ3JkPMaC_KNYP_Epg@mail.gmail.com>

On 12 January 2017 at 22:04, Thomas G?ttler
<guettliml at thomas-guettler.de> wrote:
> I came across a python library which has docs, which start like this:
>
> {{{
>
>  Quickstart
>
>  Include foolib in your requirements.txt file.
>
> }}}
>
> AFAIK dependencies should be specified via `install_requires` in `setup.py`.

Applications and services don't necessarily have a setup.py file -
setup.py is more for pip-installable libraries and frameworks (see
https://caremad.io/posts/2013/07/setup-vs-requirement/ for more on
that topic).

Since the section is titled "Quickstart", it seems reasonable to gloss
over the fact that there are options other than requirements.txt,
especially as folks that have already gone to the effort of learning
how to make their software pip-installable are also likely to have
worked out how that impacts where you specify your dependencies.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From guettliml at thomas-guettler.de  Fri Jan 13 07:23:43 2017
From: guettliml at thomas-guettler.de (=?UTF-8?Q?Thomas_G=c3=bcttler?=)
Date: Fri, 13 Jan 2017 13:23:43 +0100
Subject: [Distutils] How to specify dependencies in Python
In-Reply-To: <CADiSq7coTcSnA1b7HAmO76hsQ0X=F8rgeQ3JkPMaC_KNYP_Epg@mail.gmail.com>
References: <6f19cb4a-ce35-4d86-ee3f-52596a59a3ec@thomas-guettler.de>
 <CADiSq7coTcSnA1b7HAmO76hsQ0X=F8rgeQ3JkPMaC_KNYP_Epg@mail.gmail.com>
Message-ID: <5301feab-056f-2c39-a71c-cbd2cde91239@thomas-guettler.de>

Am 12.01.2017 um 13:43 schrieb Nick Coghlan:
> On 12 January 2017 at 22:04, Thomas G?ttler
> <guettliml at thomas-guettler.de> wrote:
>> I came across a python library which has docs, which start like this:
>>
>> {{{
>>
>>  Quickstart
>>
>>  Include foolib in your requirements.txt file.
>>
>> }}}
>>
>> AFAIK dependencies should be specified via `install_requires` in `setup.py`.
>
> Applications and services don't necessarily have a setup.py file -
> setup.py is more for pip-installable libraries and frameworks (see
> https://caremad.io/posts/2013/07/setup-vs-requirement/ for more on
> that topic).

What is an application?

The django project uses this definition: https://docs.djangoproject.com/en/1.10/ref/applications/

I guess you mean something else, if you talk about "application".

What is an application for you?

Regards,
   Thomas G?ttler

-- 
Thomas Guettler http://www.thomas-guettler.de/

From p.f.moore at gmail.com  Fri Jan 13 08:07:14 2017
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 13 Jan 2017 13:07:14 +0000
Subject: [Distutils] How to specify dependencies in Python
In-Reply-To: <5301feab-056f-2c39-a71c-cbd2cde91239@thomas-guettler.de>
References: <6f19cb4a-ce35-4d86-ee3f-52596a59a3ec@thomas-guettler.de>
 <CADiSq7coTcSnA1b7HAmO76hsQ0X=F8rgeQ3JkPMaC_KNYP_Epg@mail.gmail.com>
 <5301feab-056f-2c39-a71c-cbd2cde91239@thomas-guettler.de>
Message-ID: <CACac1F-7wnh9jd-fyi8V1o38pzgaOFvFBJO1XfVKXZOvUY9krg@mail.gmail.com>

On 13 January 2017 at 12:23, Thomas G?ttler
<guettliml at thomas-guettler.de> wrote:
> Am 12.01.2017 um 13:43 schrieb Nick Coghlan:
>>
>> On 12 January 2017 at 22:04, Thomas G?ttler
>> <guettliml at thomas-guettler.de> wrote:
>>>
>>> I came across a python library which has docs, which start like this:
>>>
>>> {{{
>>>
>>>  Quickstart
>>>
>>>  Include foolib in your requirements.txt file.
>>>
>>> }}}
>>>
>>> AFAIK dependencies should be specified via `install_requires` in
>>> `setup.py`.
>>
>>
>> Applications and services don't necessarily have a setup.py file -
>> setup.py is more for pip-installable libraries and frameworks (see
>> https://caremad.io/posts/2013/07/setup-vs-requirement/ for more on
>> that topic).
>
>
> What is an application?
>
> The django project uses this definition:
> https://docs.djangoproject.com/en/1.10/ref/applications/
>
> I guess you mean something else, if you talk about "application".
>
> What is an application for you?

IMO, an "application" in this context is a standalone program
(implemented in Python). This is as opposed to a "library" which is
written to be imported in other Python code. Obviously, there's some
overlap - some "applications" provide a "programming API" that can be
imported, and some libraries provide command line entry points.

A library (something that you import) should declare in its metadata
that you need to have its dependencies installed before it will work.
That lets you (the user of the library) be ignorant of the
dependencies of the library (which are implementation details as far
as you are concerned) - pip just sorts it out for you from the
metadata. To get the dependency metadata, the *library* should include
its dependencies in install_requires in its setup.py.

For applications on the other hand, you need to install exactly the
environment you tested, so you *don't* use dependency metadata.
Rather, you create a requirements.txt file that contains the exact
versions of everything (direct dependencies) your application needs,
then deploy your application script to a Python environment where
you've run "pip install -r requirements.txt" to set it up correctly.

As I say, there are lots of grey areas here. But from your
description, the library you found sounds like it should have
dependency metadata, and not need you to use a requirements file. On
the other hand, the "Quickstart" may be trying to tell people how to
use foolib in their *application*, in which case it's correct. (It's
oversimplified, as if you're writing a library that depends on foolib
you'd use install_requires, but simply changing the text to just say
"put foolib in your install_requires" is no better, as then you'd have
ignored the application use case...)

If you can point to the actual library you're referring to, it would
be easier to be specific :-)

Paul

From njs at pobox.com  Fri Jan 13 08:13:39 2017
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 13 Jan 2017 05:13:39 -0800
Subject: [Distutils] How to specify dependencies in Python
In-Reply-To: <CACac1F-7wnh9jd-fyi8V1o38pzgaOFvFBJO1XfVKXZOvUY9krg@mail.gmail.com>
References: <6f19cb4a-ce35-4d86-ee3f-52596a59a3ec@thomas-guettler.de>
 <CADiSq7coTcSnA1b7HAmO76hsQ0X=F8rgeQ3JkPMaC_KNYP_Epg@mail.gmail.com>
 <5301feab-056f-2c39-a71c-cbd2cde91239@thomas-guettler.de>
 <CACac1F-7wnh9jd-fyi8V1o38pzgaOFvFBJO1XfVKXZOvUY9krg@mail.gmail.com>
Message-ID: <CAPJVwB=JLUmXqZ25PnhoWOhoSD=J21UcQUG24m5c=ao77zvZag@mail.gmail.com>

It's probably a good idea here to explicitly distinguish between the kind
of application that's a web app you deploy to sort of managed server
environment, and the kind of application that's a command line or GUI tool
that people download and run. The same word means radically different
things to different people :-). I believe Paul's advice below is
specifically addressing the former case, not the latter.

On Jan 13, 2017 5:07 AM, "Paul Moore" <p.f.moore at gmail.com> wrote:

> On 13 January 2017 at 12:23, Thomas G?ttler
> <guettliml at thomas-guettler.de> wrote:
> > Am 12.01.2017 um 13:43 schrieb Nick Coghlan:
> >>
> >> On 12 January 2017 at 22:04, Thomas G?ttler
> >> <guettliml at thomas-guettler.de> wrote:
> >>>
> >>> I came across a python library which has docs, which start like this:
> >>>
> >>> {{{
> >>>
> >>>  Quickstart
> >>>
> >>>  Include foolib in your requirements.txt file.
> >>>
> >>> }}}
> >>>
> >>> AFAIK dependencies should be specified via `install_requires` in
> >>> `setup.py`.
> >>
> >>
> >> Applications and services don't necessarily have a setup.py file -
> >> setup.py is more for pip-installable libraries and frameworks (see
> >> https://caremad.io/posts/2013/07/setup-vs-requirement/ for more on
> >> that topic).
> >
> >
> > What is an application?
> >
> > The django project uses this definition:
> > https://docs.djangoproject.com/en/1.10/ref/applications/
> >
> > I guess you mean something else, if you talk about "application".
> >
> > What is an application for you?
>
> IMO, an "application" in this context is a standalone program
> (implemented in Python). This is as opposed to a "library" which is
> written to be imported in other Python code. Obviously, there's some
> overlap - some "applications" provide a "programming API" that can be
> imported, and some libraries provide command line entry points.
>
> A library (something that you import) should declare in its metadata
> that you need to have its dependencies installed before it will work.
> That lets you (the user of the library) be ignorant of the
> dependencies of the library (which are implementation details as far
> as you are concerned) - pip just sorts it out for you from the
> metadata. To get the dependency metadata, the *library* should include
> its dependencies in install_requires in its setup.py.
>
> For applications on the other hand, you need to install exactly the
> environment you tested, so you *don't* use dependency metadata.
> Rather, you create a requirements.txt file that contains the exact
> versions of everything (direct dependencies) your application needs,
> then deploy your application script to a Python environment where
> you've run "pip install -r requirements.txt" to set it up correctly.
>
> As I say, there are lots of grey areas here. But from your
> description, the library you found sounds like it should have
> dependency metadata, and not need you to use a requirements file. On
> the other hand, the "Quickstart" may be trying to tell people how to
> use foolib in their *application*, in which case it's correct. (It's
> oversimplified, as if you're writing a library that depends on foolib
> you'd use install_requires, but simply changing the text to just say
> "put foolib in your install_requires" is no better, as then you'd have
> ignored the application use case...)
>
> If you can point to the actual library you're referring to, it would
> be easier to be specific :-)
>
> Paul
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170113/dff810ed/attachment.html>

From jim at jimfulton.info  Fri Jan 13 10:25:46 2017
From: jim at jimfulton.info (Jim Fulton)
Date: Fri, 13 Jan 2017 10:25:46 -0500
Subject: [Distutils] How to specify dependencies in Python
In-Reply-To: <5301feab-056f-2c39-a71c-cbd2cde91239@thomas-guettler.de>
References: <6f19cb4a-ce35-4d86-ee3f-52596a59a3ec@thomas-guettler.de>
 <CADiSq7coTcSnA1b7HAmO76hsQ0X=F8rgeQ3JkPMaC_KNYP_Epg@mail.gmail.com>
 <5301feab-056f-2c39-a71c-cbd2cde91239@thomas-guettler.de>
Message-ID: <CAPDm-Fibo8uBQ=TSXCMYvWztTNwjviz4igyuyS_2q8+T4XiEZA@mail.gmail.com>

On Fri, Jan 13, 2017 at 7:23 AM, Thomas G?ttler <
guettliml at thomas-guettler.de> wrote:

> What is an application for you?


Another way to think about this, FWIW, is to distinguish between the "whole
system" (for which "Application" is often a useful shorthand), as opposed
to components (aka libraries).

It's important for components to be flexible, so they're typically very
flexible about versions of their dependencies.

For whole systems, OTOH, it's important that once a configuration is
tested, that the same configuration is used in production, so systems
typically pin all of their dependencies, ideally extending to their
environments (which is a reason why container technology is attractive).

Jim

-- 
Jim Fulton
http://jimfulton.info
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170113/bb88d34b/attachment-0001.html>

From lukasz at langa.pl  Fri Jan 13 13:08:23 2017
From: lukasz at langa.pl (Lukasz Langa)
Date: Fri, 13 Jan 2017 10:08:23 -0800
Subject: [Distutils] RFC: PEP 541 - Package Index Name Retention
Message-ID: <A143359C-B5AE-483C-A0CD-33BD9BBD2BBC@langa.pl>

Hello distutils-sig,
I'd like to get some comments on this. I was asked by Donald to work on it. It removes ambiguity from some of the situations that crop up increasingly often with regards to package names on the PyPI. Looking forward to hearing what you have to say!
- ?


PEP: 541
Title: Package Index Name Retention
Version: $Revision$
Last-Modified: $Date$
Author: ?ukasz Langa <lukasz at langa.pl>
Status: Draft
Type: Process
Content-Type: text/x-rst
Created: 12-January-2017


Abstract
========

This PEP proposes an extension to the Terms of Use [1]_ of the Package
Index [2]_, clarifying expectations of package owners regarding
ownership of a package name on the Package Index, specifically with
regards to conflict resolution.

Existing package repositories such as CPAN [3]_, NPM [4]_, and
GitHub [5]_ will be investigated as prior art in this field.


Rationale
=========

Given that package names on the Index are sharing a single flat
namespace, a unique name is a finite resource.  The growing age of
the Package Index causes a constant rise of situations of conflict
between the current use of the name and a different suggested use of
the same name.

This document aims to provide general guidelines for solving the
most typical cases of such conflicts.


Specification
=============

The main idea behind this document is that the Package Index serves the
community.  Every user is invited to upload content to the Package Index
under the Terms of Use, understanding that it is at the sole risk of
the user.

While the Package Index is not a backup service, the maintainers of the
Package Index do their best to keep that content accessible indefinitely
in its published form. However, in certain edge cases the greater
community's needs might overweigh the individual's expectation of
ownership of a package name.

The use cases covered by this document are:

* Abandoned projects:

    * continued maintenance by a different set of users; or
    * removal from the Index for use with a different project.

* Active projects:

    * resolving disputes over a name.

* Invalid projects.


Implementation
==============

Reachability
------------

The user of the Package Index is solely responsible for being reachable
by the Package Index maintainers for matters concerning projects that
the user owns.  In every case where contacting the user is necessary,
the maintainers will try to do so at least three times, using the
following means of contact:

* the e-mail address on file in the user's profile on the Package Index;
* the e-mail address listed in the Author field for a given project
  uploaded to the Index; and
* any e-mail addresses found in the given project's documentation
  on the Index or on the listed Home Page.

The maintainers stop trying to reach the user after six weeks.


Abandoned projects
------------------

A project is considered *abandoned* when ALL of the following are met:

* owner not reachable (see Reachability above);
* no releases within the past twelve months; and
* no activity from the owner on the project's home page (or no
  home page listed).

All other projects are considered *active*.


Continued maintenance of an abandoned project
---------------------------------------------

If a candidate appears willing to continue maintenance on an *abandoned*
project, ownership of the name is transferred when ALL of the following
are met:

* the project has been determined *abandoned* by the rules described
  above;
* the candidate is able to demonstrate own failed attempts to contact
  the existing owner;
* the candidate is able to demonstrate skin in the game (improvements
  made on the candidate's own fork of the project);
* the candidate is able to demonstrate why a fork under a different name
  is not an acceptable workaround; and
* the maintainers of the Package Index don't have any additional
  reservations.

Under no circumstances will a name be reassigned against the wishes of
a reachable owner.


Removal of an abandoned project
-------------------------------

Projects are never removed from the Package Index solely on the basis
of abandonment.  Artifacts uploaded to the Package Index hold inherent
historical value.

An *abandoned* project can be transferred to a new owner for purposes
of reusing the name when ALL of the following are met:

* the project has been determined *abandoned* by the rules described
  above;
* the candidate is able to demonstrate own failed attempts to contact
  the existing owner;
* the candidate is able to demonstrate skin in the game (the other
  project suggested to reuse the name already exists and meets
  notability requirements);
* the candidate is able to demonstrate why a fork under a different name
  is not an acceptable workaround;
* download statistics on the Package Index for the existing package
  indicate project is not being used; and
* the maintainers of the Package Index don't have any additional
  reservations.


Name conflict resolution for active projects
--------------------------------------------

The maintainers of the Package Index are not arbiters in disputes
around *active* projects.  There are many possible scenarios here,
a non-exclusive list describing some real-world examples is presented
below. None of the following qualify for package name ownership
transfer:

1. User A and User B share project X. After some time they part ways
   and each of them wants to continue the project under name X.
2. User A owns a project X outside the Package Index.  User B creates
   a package under the name X on the Index.  After some time, User A
   wants to publish project X on the Index but realizes name is taken.
   This is true even if User A's project X gains notability and the
   User B's project X is not notable.
3. User A publishes project X to the Package Index.  After some time
   User B proposes bug fixes to the project but no new release is
   published by User A.  This is true even if User A agrees to publish
   a new version and later doesn't, even if User B's changes are merged
   to the source code repository for project X.

Again, the list above is not exclusive.  The maintainers of the Package
Index recommend users to get in touch with each other and solve the
issue by respectful communication (see the PSF Code of Conduct [6]_).


Invalid projects
----------------

A project published on the Package Index meeting ANY of the following
is considered invalid and will be removed from the Index:

* project does not conform to Terms of Use;
* project is malware (designed to exploit or harm systems or users);
* project contains illegal content;
* project violates copyright or licenses;
* project is name squatting (package has no functionality or is
  empty);
* project name, description, or content violates the Code of Conduct;
  or
* project is abusing the Package Index for purposes it was not
  intended.

If you find a project that might be considered invalid, create
a support request [7]_.


Prior art
=========

NPM contains a separate section linked from the front page called
`Package Name Disputes <https://www.npmjs.com/policies/disputes>`_.
It is described as a "living document", as of January 2017 its
contents might be summarized as follows:

* package name squatting is prohibited;
* users wanting to reuse a project name are required to contact the
  existing author, with cc to support at npmjs.com;
* all contact must conform to the NPM Code of Conduct;
* in case of no resolution after a few weeks, npm inc. holds the right
  to the final decision in the matter.

CPAN lets any user upload modules with the same name. PAUSE, a related
index, only lists modules uploaded by the primary maintainer or listed
co-maintainers.  CPAN documentation doesn't address disputes otherwise.

GitHub's terms of service contain an exhaustive list of behavior
not meeting general conditions of use.  While not codified anywhere,
GitHub does agree for users to reclaim abandoned account names by
archiving the abandoned account and letting the other user or
organization rename their account.  This is done on a case-by-case
basis.


Rejected Proposals
==================

The original approach was to hope for the best and solve issues as they
arise without written policy.  This is not sustainable.  The lack of
generally available guidelines in writing on package name conflict
resolution is causing unnecessary tensions.  From the perspective of
users, decisions made by the Package Index maintainers without written
guidelines may appear arbitrary.  From the perspective of the Package
Index maintainers, solving name conflicts is a stressful task due to
risk of unintentional harm due to lack of defined policy.


References
==========

.. [1] Terms of Use of the Python Package Index
   (https://pypi.org/policy/terms-of-use/)

.. [2] The Python Package Index
   (https://pypi.python.org/)

.. [3] The Comprehensive Perl Archive Network
   (http://www.cpan.org/)

.. [4] Node Package Manager
   (https://www.npmjs.com/package/left-pad)

.. [5] GitHub
   (https://github.com/)

.. [6] Python Community Code of Conduct
   (https://www.python.org/psf/codeofconduct/)

.. [7] PyPI Support Requests
   (https://sourceforge.net/p/pypi/support-requests/)


Copyright
=========

This document has been placed in the public domain.


Acknowledgements
================

The many participants of the Distutils and Catalog SIGs for their
ideas over the years.

..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:



From annaraven at gmail.com  Fri Jan 13 13:27:39 2017
From: annaraven at gmail.com (Anna Ravenscroft)
Date: Fri, 13 Jan 2017 10:27:39 -0800
Subject: [Distutils] RFC: PEP 541 - Package Index Name Retention
In-Reply-To: <A143359C-B5AE-483C-A0CD-33BD9BBD2BBC@langa.pl>
References: <A143359C-B5AE-483C-A0CD-33BD9BBD2BBC@langa.pl>
Message-ID: <CA+nrBM9aCH_114WLF8p-JTSf9oMyC=uV=MoCyLxiwYt7erffwg@mail.gmail.com>

+1 on the proposal.

I would suggest to state where it is posted (somewhere obvious) on pypi,
and possibly some kind of automated notification to pypi uploaders be
provided about the new policy.

On Fri, Jan 13, 2017 at 10:08 AM, Lukasz Langa <lukasz at langa.pl> wrote:

> Hello distutils-sig,
> I'd like to get some comments on this. I was asked by Donald to work on
> it. It removes ambiguity from some of the situations that crop up
> increasingly often with regards to package names on the PyPI. Looking
> forward to hearing what you have to say!
> - ?
>
>
> PEP: 541
> Title: Package Index Name Retention
> Version: $Revision$
> Last-Modified: $Date$
> Author: ?ukasz Langa <lukasz at langa.pl>
> Status: Draft
> Type: Process
> Content-Type: text/x-rst
> Created: 12-January-2017
>
>
> Abstract
> ========
>
> This PEP proposes an extension to the Terms of Use [1]_ of the Package
> Index [2]_, clarifying expectations of package owners regarding
> ownership of a package name on the Package Index, specifically with
> regards to conflict resolution.
>
> Existing package repositories such as CPAN [3]_, NPM [4]_, and
> GitHub [5]_ will be investigated as prior art in this field.
>
>
> Rationale
> =========
>
> Given that package names on the Index are sharing a single flat
> namespace, a unique name is a finite resource.  The growing age of
> the Package Index causes a constant rise of situations of conflict
> between the current use of the name and a different suggested use of
> the same name.
>
> This document aims to provide general guidelines for solving the
> most typical cases of such conflicts.
>
>
> Specification
> =============
>
> The main idea behind this document is that the Package Index serves the
> community.  Every user is invited to upload content to the Package Index
> under the Terms of Use, understanding that it is at the sole risk of
> the user.
>
> While the Package Index is not a backup service, the maintainers of the
> Package Index do their best to keep that content accessible indefinitely
> in its published form. However, in certain edge cases the greater
> community's needs might overweigh the individual's expectation of
> ownership of a package name.
>
> The use cases covered by this document are:
>
> * Abandoned projects:
>
>     * continued maintenance by a different set of users; or
>     * removal from the Index for use with a different project.
>
> * Active projects:
>
>     * resolving disputes over a name.
>
> * Invalid projects.
>
>
> Implementation
> ==============
>
> Reachability
> ------------
>
> The user of the Package Index is solely responsible for being reachable
> by the Package Index maintainers for matters concerning projects that
> the user owns.  In every case where contacting the user is necessary,
> the maintainers will try to do so at least three times, using the
> following means of contact:
>
> * the e-mail address on file in the user's profile on the Package Index;
> * the e-mail address listed in the Author field for a given project
>   uploaded to the Index; and
> * any e-mail addresses found in the given project's documentation
>   on the Index or on the listed Home Page.
>
> The maintainers stop trying to reach the user after six weeks.
>
>
> Abandoned projects
> ------------------
>
> A project is considered *abandoned* when ALL of the following are met:
>
> * owner not reachable (see Reachability above);
> * no releases within the past twelve months; and
> * no activity from the owner on the project's home page (or no
>   home page listed).
>
> All other projects are considered *active*.
>
>
> Continued maintenance of an abandoned project
> ---------------------------------------------
>
> If a candidate appears willing to continue maintenance on an *abandoned*
> project, ownership of the name is transferred when ALL of the following
> are met:
>
> * the project has been determined *abandoned* by the rules described
>   above;
> * the candidate is able to demonstrate own failed attempts to contact
>   the existing owner;
> * the candidate is able to demonstrate skin in the game (improvements
>   made on the candidate's own fork of the project);
> * the candidate is able to demonstrate why a fork under a different name
>   is not an acceptable workaround; and
> * the maintainers of the Package Index don't have any additional
>   reservations.
>
> Under no circumstances will a name be reassigned against the wishes of
> a reachable owner.
>
>
> Removal of an abandoned project
> -------------------------------
>
> Projects are never removed from the Package Index solely on the basis
> of abandonment.  Artifacts uploaded to the Package Index hold inherent
> historical value.
>
> An *abandoned* project can be transferred to a new owner for purposes
> of reusing the name when ALL of the following are met:
>
> * the project has been determined *abandoned* by the rules described
>   above;
> * the candidate is able to demonstrate own failed attempts to contact
>   the existing owner;
> * the candidate is able to demonstrate skin in the game (the other
>   project suggested to reuse the name already exists and meets
>   notability requirements);
> * the candidate is able to demonstrate why a fork under a different name
>   is not an acceptable workaround;
> * download statistics on the Package Index for the existing package
>   indicate project is not being used; and
> * the maintainers of the Package Index don't have any additional
>   reservations.
>
>
> Name conflict resolution for active projects
> --------------------------------------------
>
> The maintainers of the Package Index are not arbiters in disputes
> around *active* projects.  There are many possible scenarios here,
> a non-exclusive list describing some real-world examples is presented
> below. None of the following qualify for package name ownership
> transfer:
>
> 1. User A and User B share project X. After some time they part ways
>    and each of them wants to continue the project under name X.
> 2. User A owns a project X outside the Package Index.  User B creates
>    a package under the name X on the Index.  After some time, User A
>    wants to publish project X on the Index but realizes name is taken.
>    This is true even if User A's project X gains notability and the
>    User B's project X is not notable.
> 3. User A publishes project X to the Package Index.  After some time
>    User B proposes bug fixes to the project but no new release is
>    published by User A.  This is true even if User A agrees to publish
>    a new version and later doesn't, even if User B's changes are merged
>    to the source code repository for project X.
>
> Again, the list above is not exclusive.  The maintainers of the Package
> Index recommend users to get in touch with each other and solve the
> issue by respectful communication (see the PSF Code of Conduct [6]_).
>
>
> Invalid projects
> ----------------
>
> A project published on the Package Index meeting ANY of the following
> is considered invalid and will be removed from the Index:
>
> * project does not conform to Terms of Use;
> * project is malware (designed to exploit or harm systems or users);
> * project contains illegal content;
> * project violates copyright or licenses;
> * project is name squatting (package has no functionality or is
>   empty);
> * project name, description, or content violates the Code of Conduct;
>   or
> * project is abusing the Package Index for purposes it was not
>   intended.
>
> If you find a project that might be considered invalid, create
> a support request [7]_.
>
>
> Prior art
> =========
>
> NPM contains a separate section linked from the front page called
> `Package Name Disputes <https://www.npmjs.com/policies/disputes>`_.
> It is described as a "living document", as of January 2017 its
> contents might be summarized as follows:
>
> * package name squatting is prohibited;
> * users wanting to reuse a project name are required to contact the
>   existing author, with cc to support at npmjs.com;
> * all contact must conform to the NPM Code of Conduct;
> * in case of no resolution after a few weeks, npm inc. holds the right
>   to the final decision in the matter.
>
> CPAN lets any user upload modules with the same name. PAUSE, a related
> index, only lists modules uploaded by the primary maintainer or listed
> co-maintainers.  CPAN documentation doesn't address disputes otherwise.
>
> GitHub's terms of service contain an exhaustive list of behavior
> not meeting general conditions of use.  While not codified anywhere,
> GitHub does agree for users to reclaim abandoned account names by
> archiving the abandoned account and letting the other user or
> organization rename their account.  This is done on a case-by-case
> basis.
>
>
> Rejected Proposals
> ==================
>
> The original approach was to hope for the best and solve issues as they
> arise without written policy.  This is not sustainable.  The lack of
> generally available guidelines in writing on package name conflict
> resolution is causing unnecessary tensions.  From the perspective of
> users, decisions made by the Package Index maintainers without written
> guidelines may appear arbitrary.  From the perspective of the Package
> Index maintainers, solving name conflicts is a stressful task due to
> risk of unintentional harm due to lack of defined policy.
>
>
> References
> ==========
>
> .. [1] Terms of Use of the Python Package Index
>    (https://pypi.org/policy/terms-of-use/)
>
> .. [2] The Python Package Index
>    (https://pypi.python.org/)
>
> .. [3] The Comprehensive Perl Archive Network
>    (http://www.cpan.org/)
>
> .. [4] Node Package Manager
>    (https://www.npmjs.com/package/left-pad)
>
> .. [5] GitHub
>    (https://github.com/)
>
> .. [6] Python Community Code of Conduct
>    (https://www.python.org/psf/codeofconduct/)
>
> .. [7] PyPI Support Requests
>    (https://sourceforge.net/p/pypi/support-requests/)
>
>
> Copyright
> =========
>
> This document has been placed in the public domain.
>
>
> Acknowledgements
> ================
>
> The many participants of the Distutils and Catalog SIGs for their
> ideas over the years.
>
> ..
>    Local Variables:
>    mode: indented-text
>    indent-tabs-mode: nil
>    sentence-end-double-space: t
>    fill-column: 70
>    End:
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
cordially,
Anna
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170113/095114a8/attachment-0001.html>

From ethan at stoneleaf.us  Fri Jan 13 13:33:03 2017
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 13 Jan 2017 10:33:03 -0800
Subject: [Distutils] RFC: PEP 541 - Package Index Name Retention
In-Reply-To: <A143359C-B5AE-483C-A0CD-33BD9BBD2BBC@langa.pl>
References: <A143359C-B5AE-483C-A0CD-33BD9BBD2BBC@langa.pl>
Message-ID: <58791D5F.3030800@stoneleaf.us>

On 01/13/2017 10:08 AM, Lukasz Langa wrote:

> I'd like to get some comments on this. I was asked by Donald to work on it. It removes ambiguity from some of the situations that crop up increasingly often with regards to package names on the PyPI. Looking forward to hearing what you have to say!

Overall looks great.  I would suggest removing slang, such as "skin in the game".

Thanks for working on this!

--
~Ethan~

From steve.dower at python.org  Fri Jan 13 13:35:56 2017
From: steve.dower at python.org (Steve Dower)
Date: Fri, 13 Jan 2017 10:35:56 -0800
Subject: [Distutils] RFC: PEP 541 - Package Index Name Retention
In-Reply-To: <A143359C-B5AE-483C-A0CD-33BD9BBD2BBC@langa.pl>
References: <A143359C-B5AE-483C-A0CD-33BD9BBD2BBC@langa.pl>
Message-ID: <4352fd91-bf5d-e840-b2fb-778f9431b0f4@python.org>

Looks great to me. Just a few comments that may help reduce the burden 
on the index maintainers.

On 13Jan2017 1008, Lukasz Langa wrote:
> In every case where contacting the user is necessary,
> the maintainers will try to do so at least three times, using the
> following means of contact:
>
> * the e-mail address on file in the user's profile on the Package Index;
> * the e-mail address listed in the Author field for a given project
>   uploaded to the Index; and
> * any e-mail addresses found in the given project's documentation
>   on the Index or on the listed Home Page.
>
> The maintainers stop trying to reach the user after six weeks.

I don't see any reason to expect the index maintainers to trawl through 
a project's documentation or home page to find contact details. There 
are more than enough ways to provide it on the index, and as far as I'm 
concerned, no reason for uploaders to not provide one.

> An *abandoned* project can be transferred to a new owner for purposes
> of reusing the name when ALL of the following are met:
> ...

The list here is nearly identical to the previous section, apart from 
the added data point of download count (which is good!), and it's not 
clear on reading why we need to list these twice.

> Invalid projects
> ----------------
>
> A project published on the Package Index meeting ANY of the following
> is considered invalid and will be removed from the Index:
> ...
> * project is name squatting (package has no functionality or is
>   empty);
> ...
> If you find a project that might be considered invalid, create
> a support request [7]_.

I would actually like to be able to name-squat for a period between a 
project being started and being released (particularly in my own 
context, I often need to keep a project private until it has been 
internally tested/reviewed/scanned and the lawyers have signed off, at 
which point it may require a new review if the name has to change).

Presumably for a reachable uploader who can give an explanation, this 
won't result in the immediate loss of the name. But suggesting a time 
limit may help reduce support requests ("project is name squatting for 
at least 6 months" feels okay to me, but not wedded to it).

(As a semi-related aside, I'm currently squatting on the 'microsoft' and 
'windows' packages for trademark protection reasons. They may never get 
any functionality, but that's better than someone else having the name. 
This sort of squatting doesn't necessarily need to be explicitly called 
out in policy, but maybe it's worth a mention?)

Cheers,
Steve


From lukasz at langa.pl  Fri Jan 13 13:50:07 2017
From: lukasz at langa.pl (Lukasz Langa)
Date: Fri, 13 Jan 2017 10:50:07 -0800
Subject: [Distutils] RFC: PEP 541 - Package Index Name Retention
In-Reply-To: <4352fd91-bf5d-e840-b2fb-778f9431b0f4@python.org>
References: <A143359C-B5AE-483C-A0CD-33BD9BBD2BBC@langa.pl>
 <4352fd91-bf5d-e840-b2fb-778f9431b0f4@python.org>
Message-ID: <FF17D751-CEC5-4F37-BEA6-C60EFAA2CAB4@langa.pl>

Thanks for review, Steve!


> On Jan 13, 2017, at 10:35 AM, Steve Dower <steve.dower at python.org> wrote:
> 
> I don't see any reason to expect the index maintainers to trawl through a project's documentation or home page to find contact details. There are more than enough ways to provide it on the index, and as far as I'm concerned, no reason for uploaders to not provide one.

The reason of this is courtesy towards existing package owners who might not have conformed to the Author e-mail requirement because it wasn't explicitly formulated. I think the maintainers would try their best to reach the owner anyway, just to be sure there won't be any harm caused by changes.


>> An *abandoned* project can be transferred to a new owner for purposes
>> of reusing the name when ALL of the following are met:
>> ...
> 
> The list here is nearly identical to the previous section

The "skin in the game" behavior is different.


>  it's not clear on reading why we need to list these twice.

It's a different case and I want to limit back-and-forth required by the readers (which is already necessary to parse the rules for abandoned projects).


> I would actually like to be able to name-squat for a period between a project being started and being released (particularly in my own context, I often need to keep a project private until it has been internally tested/reviewed/scanned and the lawyers have signed off, at which point it may require a new review if the name has to change).
> 
> Presumably for a reachable uploader who can give an explanation, this won't result in the immediate loss of the name. But suggesting a time limit may help reduce support requests ("project is name squatting for at least 6 months" feels okay to me, but not wedded to it).

I don't want to suggest arbitrary limits on acceptable name squatting because this can be abused. As long as you squat and nobody calls you out on it before your first functional release, that's okay. If you squat on a great name and somebody comes along with an existing notable project wanting that name, the case it rather clear though.


> (As a semi-related aside, I'm currently squatting on the 'microsoft' and 'windows' packages for trademark protection reasons. They may never get any functionality, but that's better than someone else having the name. This sort of squatting doesn't necessarily need to be explicitly called out in policy, but maybe it's worth a mention?)

I wanted to avoid touching on trademark issues because IANAL.

- ?

From steve.dower at python.org  Fri Jan 13 14:08:06 2017
From: steve.dower at python.org (Steve Dower)
Date: Fri, 13 Jan 2017 11:08:06 -0800
Subject: [Distutils] RFC: PEP 541 - Package Index Name Retention
In-Reply-To: <FF17D751-CEC5-4F37-BEA6-C60EFAA2CAB4@langa.pl>
References: <A143359C-B5AE-483C-A0CD-33BD9BBD2BBC@langa.pl>
 <4352fd91-bf5d-e840-b2fb-778f9431b0f4@python.org>
 <FF17D751-CEC5-4F37-BEA6-C60EFAA2CAB4@langa.pl>
Message-ID: <bfb4912a-a4c5-9101-ddf1-3956245fba7e@python.org>

On 13Jan2017 1050, Lukasz Langa wrote:
> Thanks for review, Steve!
>
>
>> On Jan 13, 2017, at 10:35 AM, Steve Dower <steve.dower at python.org> wrote:
>>> An *abandoned* project can be transferred to a new owner for purposes
>>> of reusing the name when ALL of the following are met:
>>> ...
>>
>> The list here is nearly identical to the previous section
>
> The "skin in the game" behavior is different.

Fair enough. Perhaps we should avoid using the idiom though (as 
suggested earlier) to avoid any potential loss in translation.

>> I would actually like to be able to name-squat for a period between a project being started and being released (particularly in my own context, I often need to keep a project private until it has been internally tested/reviewed/scanned and the lawyers have signed off, at which point it may require a new review if the name has to change).
>>
>> Presumably for a reachable uploader who can give an explanation, this won't result in the immediate loss of the name. But suggesting a time limit may help reduce support requests ("project is name squatting for at least 6 months" feels okay to me, but not wedded to it).
>
> I don't want to suggest arbitrary limits on acceptable name squatting because this can be abused. As long as you squat and nobody calls you out on it before your first functional release, that's okay. If you squat on a great name and somebody comes along with an existing notable project wanting that name, the case it rather clear though.

So perhaps name-squatting belongs in the "this project is abandoned and 
I want the name" section rather than the "this project is invalid and 
I'm flagging it via support channels" section? (Or maybe I misunderstood 
the intent of the separate sections, which I'm sure is also useful 
feedback :) )

>> (As a semi-related aside, I'm currently squatting on the 'microsoft' and 'windows' packages for trademark protection reasons. They may never get any functionality, but that's better than someone else having the name. This sort of squatting doesn't necessarily need to be explicitly called out in policy, but maybe it's worth a mention?)
>
> I wanted to avoid touching on trademark issues because IANAL.

Very good point. Since nobody directly involved in this policy is a 
lawyer, it might be worth clearly stating what the index maintainers are 
responsible for in the case of a potential legal dispute with an 
unreachable package owner, or one who is deliberately/maliciously making 
themselves unreachable.

Or maybe it's a rare enough case that it doesn't matter? We certainly 
resolved our last issue easily enough, though it did require the index 
maintainers to put us in direct contact with the package owner. Maybe 
stating that "the index maintainers are not responsible for evaluating 
the legal status of intellectual property, and may only establish direct 
contact between the complainant and a reachable package owner with 
mutual consent" is the way to go? (And get VanL to sign off on the 
wording, just in case there's some oddity here I'm not aware of.)

Cheers,
Steve

From mal at egenix.com  Fri Jan 13 14:13:43 2017
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 13 Jan 2017 20:13:43 +0100
Subject: [Distutils] RFC: PEP 541 - Package Index Name Retention
In-Reply-To: <A143359C-B5AE-483C-A0CD-33BD9BBD2BBC@langa.pl>
References: <A143359C-B5AE-483C-A0CD-33BD9BBD2BBC@langa.pl>
Message-ID: <ef11e223-59ce-e20e-e275-bbcce1d8804b@egenix.com>

On 13.01.2017 19:08, Lukasz Langa wrote:
> Invalid projects
> ----------------
> 
> A project published on the Package Index meeting ANY of the following
> is considered invalid and will be removed from the Index:
> 
> * project does not conform to Terms of Use;
> * project is malware (designed to exploit or harm systems or users);
> * project contains illegal content;
> * project violates copyright or licenses;

This probably also needs to list "trademarks" and "patents",
as we've already had some cases where packages were violating
trademarks/patents and had to be removed (not only regarding the
name of the package but also regarding contents of the package or
functionality). This is already mentioned in the current terms,
but better make it more explicit here as well.

Likewise, a trademark owner should be able to reserve project
names with the trademark to avoid any such issues to begin with,
e.g. https://pypi.python.org/pypi/Python is such a project :-)

> * project is name squatting (package has no functionality or is
>   empty);
> * project name, description, or content violates the Code of Conduct;
>   or
> * project is abusing the Package Index for purposes it was not
>   intended.
> 
> If you find a project that might be considered invalid, create
> a support request [7]_.

It would also be good to add some wording which makes it clear
that the PSF Board has the final say in any disputes and can
have a project removed/reassigned after careful consideration
even when not meeting all the requirements listed in the PEP.

As an example, the last two bullets you mention above will
often be subject to additional judgement. The board would then have
to decide these on a case-by-case basis.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Jan 13 2017)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...           http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...           http://zope.egenix.com/
________________________________________________________________________

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
                      http://www.malemburg.com/


From prometheus235 at gmail.com  Fri Jan 13 15:34:48 2017
From: prometheus235 at gmail.com (Nick Timkovich)
Date: Fri, 13 Jan 2017 14:34:48 -0600
Subject: [Distutils] RFC: PEP 541 - Package Index Name Retention
In-Reply-To: <ef11e223-59ce-e20e-e275-bbcce1d8804b@egenix.com>
References: <A143359C-B5AE-483C-A0CD-33BD9BBD2BBC@langa.pl>
 <ef11e223-59ce-e20e-e275-bbcce1d8804b@egenix.com>
Message-ID: <CAHkxivfheW7OQxUJpUVy=9J_7_jagjPcrS-pjepZQDpXFRBVCA@mail.gmail.com>

This is a great PEP, glad to see an official policy being worked on!

The "reachability" criteria I think should define how promptly the
responses are expected and to what email(s) they will be sent (if there are
multiple maintainers, owners, authors, etc.). For example, "the first
contact will be sent to the email on record for all owners, maintainers,
and the most-recent release author, in order to notify the user(s) that
another party has requested classification of a PyPI project owned or
maintained by the user as abandoned, and that if no response is received by
(date of email + 6 weeks), it will be deemed abandoned. Reminder emails
will be sent 2 and 4 weeks later."

Maybe outside the scope of the PEP, but where will the tracker for these
things reside? How would I, for example, start the process of flagging a
project as abandoned?

On Fri, Jan 13, 2017 at 1:13 PM, M.-A. Lemburg <mal at egenix.com> wrote:

> On 13.01.2017 19:08, Lukasz Langa wrote:
> > Invalid projects
> > ----------------
> >
> > A project published on the Package Index meeting ANY of the following
> > is considered invalid and will be removed from the Index:
> >
> > * project does not conform to Terms of Use;
> > * project is malware (designed to exploit or harm systems or users);
> > * project contains illegal content;
> > * project violates copyright or licenses;
>
> This probably also needs to list "trademarks" and "patents",
> as we've already had some cases where packages were violating
> trademarks/patents and had to be removed (not only regarding the
> name of the package but also regarding contents of the package or
> functionality). This is already mentioned in the current terms,
> but better make it more explicit here as well.
>
> Likewise, a trademark owner should be able to reserve project
> names with the trademark to avoid any such issues to begin with,
> e.g. https://pypi.python.org/pypi/Python is such a project :-)
>
> > * project is name squatting (package has no functionality or is
> >   empty);
> > * project name, description, or content violates the Code of Conduct;
> >   or
> > * project is abusing the Package Index for purposes it was not
> >   intended.
> >
> > If you find a project that might be considered invalid, create
> > a support request [7]_.
>
> It would also be good to add some wording which makes it clear
> that the PSF Board has the final say in any disputes and can
> have a project removed/reassigned after careful consideration
> even when not meeting all the requirements listed in the PEP.
>
> As an example, the last two bullets you mention above will
> often be subject to additional judgement. The board would then have
> to decide these on a case-by-case basis.
>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Experts (#1, Jan 13 2017)
> >>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
> >>> Python Database Interfaces ...           http://products.egenix.com/
> >>> Plone/Zope Database Interfaces ...           http://zope.egenix.com/
> ________________________________________________________________________
>
> ::: We implement business ideas - efficiently in both time and costs :::
>
>    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>            Registered at Amtsgericht Duesseldorf: HRB 46611
>                http://www.egenix.com/company/contact/
>                       http://www.malemburg.com/
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170113/8b7b2497/attachment.html>

From prometheus235 at gmail.com  Fri Jan 13 15:43:01 2017
From: prometheus235 at gmail.com (Nick Timkovich)
Date: Fri, 13 Jan 2017 14:43:01 -0600
Subject: [Distutils] RFC: PEP 541 - Package Index Name Retention
In-Reply-To: <CAHkxivfheW7OQxUJpUVy=9J_7_jagjPcrS-pjepZQDpXFRBVCA@mail.gmail.com>
References: <A143359C-B5AE-483C-A0CD-33BD9BBD2BBC@langa.pl>
 <ef11e223-59ce-e20e-e275-bbcce1d8804b@egenix.com>
 <CAHkxivfheW7OQxUJpUVy=9J_7_jagjPcrS-pjepZQDpXFRBVCA@mail.gmail.com>
Message-ID: <CAHkxivdGZp-47-CF1wiHMj=-3-bE3wx6t2E4wpOjD2adsO_ohQ@mail.gmail.com>

Regarding name-squatting (I also see htc, ios, apple, android, angular,
debian and more registered on PyPI), would a better solution be a way to
"salt" names on PyPI making them unable to be registered unless someone
files an issue to claim such hot-button trademarks?

On Fri, Jan 13, 2017 at 2:34 PM, Nick Timkovich <prometheus235 at gmail.com>
wrote:

> This is a great PEP, glad to see an official policy being worked on!
>
> The "reachability" criteria I think should define how promptly the
> responses are expected and to what email(s) they will be sent (if there are
> multiple maintainers, owners, authors, etc.). For example, "the first
> contact will be sent to the email on record for all owners, maintainers,
> and the most-recent release author, in order to notify the user(s) that
> another party has requested classification of a PyPI project owned or
> maintained by the user as abandoned, and that if no response is received by
> (date of email + 6 weeks), it will be deemed abandoned. Reminder emails
> will be sent 2 and 4 weeks later."
>
> Maybe outside the scope of the PEP, but where will the tracker for these
> things reside? How would I, for example, start the process of flagging a
> project as abandoned?
>
> On Fri, Jan 13, 2017 at 1:13 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>
>> On 13.01.2017 19:08, Lukasz Langa wrote:
>> > Invalid projects
>> > ----------------
>> >
>> > A project published on the Package Index meeting ANY of the following
>> > is considered invalid and will be removed from the Index:
>> >
>> > * project does not conform to Terms of Use;
>> > * project is malware (designed to exploit or harm systems or users);
>> > * project contains illegal content;
>> > * project violates copyright or licenses;
>>
>> This probably also needs to list "trademarks" and "patents",
>> as we've already had some cases where packages were violating
>> trademarks/patents and had to be removed (not only regarding the
>> name of the package but also regarding contents of the package or
>> functionality). This is already mentioned in the current terms,
>> but better make it more explicit here as well.
>>
>> Likewise, a trademark owner should be able to reserve project
>> names with the trademark to avoid any such issues to begin with,
>> e.g. https://pypi.python.org/pypi/Python is such a project :-)
>>
>> > * project is name squatting (package has no functionality or is
>> >   empty);
>> > * project name, description, or content violates the Code of Conduct;
>> >   or
>> > * project is abusing the Package Index for purposes it was not
>> >   intended.
>> >
>> > If you find a project that might be considered invalid, create
>> > a support request [7]_.
>>
>> It would also be good to add some wording which makes it clear
>> that the PSF Board has the final say in any disputes and can
>> have a project removed/reassigned after careful consideration
>> even when not meeting all the requirements listed in the PEP.
>>
>> As an example, the last two bullets you mention above will
>> often be subject to additional judgement. The board would then have
>> to decide these on a case-by-case basis.
>>
>> --
>> Marc-Andre Lemburg
>> eGenix.com
>>
>> Professional Python Services directly from the Experts (#1, Jan 13 2017)
>> >>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>> >>> Python Database Interfaces ...           http://products.egenix.com/
>> >>> Plone/Zope Database Interfaces ...           http://zope.egenix.com/
>> ________________________________________________________________________
>>
>> ::: We implement business ideas - efficiently in both time and costs :::
>>
>>    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>>     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>>            Registered at Amtsgericht Duesseldorf: HRB 46611
>>                http://www.egenix.com/company/contact/
>>                       http://www.malemburg.com/
>>
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170113/4c29a662/attachment-0001.html>

From barry at python.org  Fri Jan 13 16:44:09 2017
From: barry at python.org (Barry Warsaw)
Date: Fri, 13 Jan 2017 16:44:09 -0500
Subject: [Distutils] RFC: PEP 541 - Package Index Name Retention
References: <A143359C-B5AE-483C-A0CD-33BD9BBD2BBC@langa.pl>
Message-ID: <20170113164409.5b5f8139@subdivisions.wooz.org>

On Jan 13, 2017, at 10:08 AM, Lukasz Langa wrote:

>PEP: 541
>Title: Package Index Name Retention

Overall, +1

I agree with Steve that some short term name squatting may be appropriate.
I'm not sure how you would word that in the PEP, but it will probably
effectively work itself out anyway.  When I come up with a name for a new
project, one of the first things I do is check PyPI and reserve it before the
first upload.  But that first upload will usually happen pretty quickly.

>Name conflict resolution for active projects
>--------------------------------------------
>
>The maintainers of the Package Index are not arbiters in disputes
>around *active* projects.  There are many possible scenarios here,
>a non-exclusive list describing some real-world examples is presented
>below. None of the following qualify for package name ownership
>transfer:

This is another opportunity to encourage cooperation among the parties to
resolve naming conflicts.  I think that's what's called for by the CoC, and
the hope is that the vast majority of disputes (in practice which I think are
pretty rare anyway) can be amicably resolved.

It does bring to mind the ability or lack thereof for renaming projects.  IIRC
that's not possible in current PyPI software but it may be possible in
Warehouse.

As for trademark owners having priority over names, I suppose there's no way
to prevent that, I'm mostly glad that this PEP doesn't speak to that.   I'm
always edgy about the imbalance of power inherent in the issue, allowing a
trademark owner to abuse their power to take over long-established names.
OTOH, trademark owners do often have legitimate concerns against squatters.
So, I say leave all that out of the PEP.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 801 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170113/d3c6a294/attachment.sig>

From ncoghlan at gmail.com  Sat Jan 14 03:04:45 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 14 Jan 2017 18:04:45 +1000
Subject: [Distutils] RFC: PEP 541 - Package Index Name Retention
In-Reply-To: <ef11e223-59ce-e20e-e275-bbcce1d8804b@egenix.com>
References: <A143359C-B5AE-483C-A0CD-33BD9BBD2BBC@langa.pl>
 <ef11e223-59ce-e20e-e275-bbcce1d8804b@egenix.com>
Message-ID: <CADiSq7fgKjq+8dtSeatxSWUR6iFrdh-Tw5x90WcRv8wEnWLt1g@mail.gmail.com>

Regarding reachability: contact attempts should also include the
relevant project's issue tracker if attempts at private contact have
failed.

This step is important as it allows a project's *user* community to
respond, even if the person that actually pushes the buttons to upload
new releases to PyPI is out of contact for some reason.

On 14 January 2017 at 05:13, M.-A. Lemburg <mal at egenix.com> wrote:
> Likewise, a trademark owner should be able to reserve project
> names with the trademark to avoid any such issues to begin with,
> e.g. https://pypi.python.org/pypi/Python is such a project :-)

We also have at least one case where we've reserved a name
indefinitely because it's shipped as part of another popular project.

Specifically, Donald is the owner of pkg_resources so we can ensure
that "pip install pkg_resources" is an error, rather than silently
installing the wrong thing.

The only projects we'd release that name for are either setuptools
splitting out pkg_resources as a separately installable component, or
else a shim endorsed by Jason as the lead setuptools maintainer that
installed setuptools as a dependency (see
https://github.com/ncoghlan/pkg_resources_shim/issues/1 for
discussion).

>From a policy perspective, I think the kind of phrasing I'd be looking
for would be to say that the PyPI maintainers may pre-emptively
declare particular project names invalid for security reasons.

That would not only cover pkg_resources specifically, but also things
like the name normalisation scheme that prevents the use of names that
are deemed "the same as" (as defined by the regex in PEP 508), or
"confusable with" (as defined by the Unicode Consortium), existing
registered names.

>> * project is name squatting (package has no functionality or is
>>   empty);
>> * project name, description, or content violates the Code of Conduct;
>>   or
>> * project is abusing the Package Index for purposes it was not
>>   intended.
>>
>> If you find a project that might be considered invalid, create
>> a support request [7]_.
>
> It would also be good to add some wording which makes it clear
> that the PSF Board has the final say in any disputes and can
> have a project removed/reassigned after careful consideration
> even when not meeting all the requirements listed in the PEP.
>
> As an example, the last two bullets you mention above will
> often be subject to additional judgement. The board would then have
> to decide these on a case-by-case basis.

Right, explicitly mentioning the role of the PSF in the process was
going to be my suggestion as well.

I think there are a few key aspects of that which should be mentioned:

- the Python Software Foundation is the legal entity ultimately
responsible for providing the Python Packaging Index as a community
service
- the PyPI maintainers can always escalate name retention and transfer
questions to the PSF Board for discussion and resolution if they're
not clear on an appropriate course of action
- third parties may escalate name retention and transfer concerns to
the PSF Board if they have a sincere belief that an issue's resolution
is contrary to the documented policy
- issues involving legal claims (copyright disputes, trademark
disputes, patent claims, unauthorized disclosure claims) MUST be
escalated to the Board for review by the PSF's General Counsel

And as with any other changes to the PyPI terms of service, changes to
the project name retention and transfer policy must be approved by the
Foundation (whether that's through the General Counsel's sign-off, or
through a Board resolution would be for the Board to determine, it
doesn't need to be part of the policy)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From lukasz at langa.pl  Sat Jan 14 15:03:07 2017
From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=)
Date: Sat, 14 Jan 2017 12:03:07 -0800
Subject: [Distutils] RFC 2: PEP 541 - Package Index Name Retention
Message-ID: <B7AFED62-EA92-4122-852A-C6B753D5C6F7@langa.pl>

Hello distutils-sig,
Second version for review. As per Nick?s comments, I imagine after this PEP gets accepted, the Board will have to publish a Resolution to finalize it. I don?t think this part of the process needs to be mentioned. Simply put, the PSF is the implementor of the accepted change. As with any other implementations, that might trigger some minor changes to an otherwise accepted PEP.

Changes from RFC 1:
- specified where the document goes on the PyPI website
- removed slang
- covered legal matters, specifically the role of the PSF, pre-emptive invalidation of certain names, and mentioned trademark and patent violations

Full change log: https://github.com/python/peps/commits/master/pep-0541.txt

I decided to not otherwise touch on:
- name squatting. The maintainers review every case reported by a user and are careful not to make any sudden moves. This makes me confident I won?t be stripped from a name registered yesterday on the basis of somebody reporting it tomorrow. On the other hand, legitimizing certain forms of squatting is asking for trouble as it can be abused.

- suggesting anybody can escalate to the PSF Board if they are unhappy with a decision. Yes, this is always an option but we should not advertise it as effectively a ?second chance?. I feel this is asking for trouble and undermines the rest of the document.

- reachability by a bug tracker. The maintainers may decide to reach out on the project?s bug tracker to get attention. But ultimately they are seeking contact with the owner, not with the users.

- reachability attempts schedule. The maintainers should be free to decide on the most reasonable schedule that works with their calendar.

- somebody suggested a part of this PEP should be sending out an announcement about this policy to existing package owners. It?s funny because it?s a chicken and egg problem. The ones that we won?t be able to reach with an announcement might be the ones hit by the policy change ;-)

Alright, I think that?s it. Full body below. Thanks for thoughtful and timely review!
- ?


PEP: 541
Title: Package Index Name Retention
Version: $Revision$
Last-Modified: $Date$
Author: ?ukasz Langa <lukasz at langa.pl>
Status: Draft
Type: Process
Content-Type: text/x-rst
Created: 12-January-2017


Abstract
========

This PEP proposes an extension to the Terms of Use [1]_ of the Package
Index [2]_, clarifying expectations of package owners regarding
ownership of a package name on the Package Index, specifically with
regards to conflict resolution.

Existing package repositories such as CPAN [3]_, NPM [4]_, and
GitHub [5]_ will be investigated as prior art in this field.


Rationale
=========

Given that package names on the Index are sharing a single flat
namespace, a unique name is a finite resource.  The growing age of
the Package Index causes a constant rise of situations of conflict
between the current use of the name and a different suggested use of
the same name.

This document aims to provide general guidelines for solving the
most typical cases of such conflicts.


Specification
=============

The main idea behind this document is that the Package Index serves the
community.  Every user is invited to upload content to the Package Index
under the Terms of Use, understanding that it is at the sole risk of
the user.

While the Package Index is not a backup service, the maintainers of the
Package Index do their best to keep that content accessible indefinitely
in its published form.  However, in certain edge cases the greater
community's needs might overweigh the individual's expectation of
ownership of a package name.

The use cases covered by this document are:

* Abandoned projects:

    * continued maintenance by a different set of users; or
    * removal from the Index for use with a different project.

* Active projects:

    * resolving disputes over a name.

* Invalid projects.

The proposed extension to the Terms of Use, as expressed in the
Implementation section, will be published as a separate document on the
Package Index, linked next to existing Terms of Use in the front page
footer.


Implementation
==============

Reachability
------------

The user of the Package Index is solely responsible for being reachable
by the Package Index maintainers for matters concerning projects that
the user owns.  In every case where contacting the user is necessary,
the maintainers will try to do so at least three times, using the
following means of contact:

* the e-mail address on file in the user's profile on the Package Index;
* the e-mail address listed in the Author field for a given project
  uploaded to the Index; and
* any e-mail addresses found in the given project's documentation
  on the Index or on the listed Home Page.

The maintainers stop trying to reach the user after six weeks.


Abandoned projects
------------------

A project is considered *abandoned* when ALL of the following are met:

* owner not reachable (see Reachability above);
* no releases within the past twelve months; and
* no activity from the owner on the project's home page (or no
  home page listed).

All other projects are considered *active*.


Continued maintenance of an abandoned project
---------------------------------------------

If a candidate appears willing to continue maintenance on an *abandoned*
project, ownership of the name is transferred when ALL of the following
are met:

* the project has been determined *abandoned* by the rules described
  above;
* the candidate is able to demonstrate own failed attempts to contact
  the existing owner;
* the candidate is able to demonstrate improvements made on the
  candidate's own fork of the project;
* the candidate is able to demonstrate why a fork under a different name
  is not an acceptable workaround; and
* the maintainers of the Package Index don't have any additional
  reservations.

Under no circumstances will a name be reassigned against the wishes of
a reachable owner.


Removal of an abandoned project
-------------------------------

Projects are never removed from the Package Index solely on the basis
of abandonment.  Artifacts uploaded to the Package Index hold inherent
historical value.

An *abandoned* project can be transferred to a new owner for purposes
of reusing the name when ALL of the following are met:

* the project has been determined *abandoned* by the rules described
  above;
* the candidate is able to demonstrate own failed attempts to contact
  the existing owner;
* the candidate is able to demonstrate that the project suggested to
  reuse the name already exists and meets notability requirements;
* the candidate is able to demonstrate why a fork under a different name
  is not an acceptable workaround;
* download statistics on the Package Index for the existing package
  indicate project is not being used; and
* the maintainers of the Package Index don't have any additional
  reservations.


Name conflict resolution for active projects
--------------------------------------------

The maintainers of the Package Index are not arbiters in disputes
around *active* projects.  There are many possible scenarios here,
a non-exclusive list describing some real-world examples is presented
below.  None of the following qualify for package name ownership
transfer:

1. User A and User B share project X.  After some time they part ways
   and each of them wants to continue the project under name X.
2. User A owns a project X outside the Package Index.  User B creates
   a package under the name X on the Index.  After some time, User A
   wants to publish project X on the Index but realizes name is taken.
   This is true even if User A's project X gains notability and the
   User B's project X is not notable.
3. User A publishes project X to the Package Index.  After some time
   User B proposes bug fixes to the project but no new release is
   published by User A.  This is true even if User A agrees to publish
   a new version and later doesn't, even if User B's changes are merged
   to the source code repository for project X.

Again, the list above is not exclusive.  The maintainers of the Package
Index recommend users to get in touch with each other and solve the
issue by respectful communication (see the PSF Code of Conduct [6]_).


Invalid projects
----------------

A project published on the Package Index meeting ANY of the following
is considered invalid and will be removed from the Index:

* project does not conform to Terms of Use;
* project is malware (designed to exploit or harm systems or users);
* project contains illegal content;
* project violates copyright, trademarks, patents, or licenses;
* project is name squatting (package has no functionality or is
  empty);
* project name, description, or content violates the Code of Conduct;
  or
* project is abusing the Package Index for purposes it was not
  intended.

The Package Index maintainers pre-emptively declare certain package
names as unavailable for security reasons.

If you find a project that you think might be considered invalid, create
a support request [7]_.  Maintainers of the Package Index will review
the case.


The role of the Python Software Foundation
------------------------------------------

The Python Software Foundation [8]_ is the non-profit legal entity that
provides the Package Index as a community service.

The Package Index maintainers can escalate issues covered by this
document for resolution by the PSF Board if the matter is not clear
enough.  Some decisions *require* additional judgement by the Board,
especially in cases of Code of Conduct violations or legal claims.
Decisions made by the Board are published as Resolutions [9]_.

The Board has the final say in any disputes covered by this document and
can decide to reassign or remove a project from the Package Index after
careful consideration even when not all requirements listed
here are met.


Prior art
=========

NPM contains a separate section linked from the front page called
`Package Name Disputes <https://www.npmjs.com/policies/disputes>`_.
It is described as a "living document", as of January 2017 its
contents might be summarized as follows:

* package name squatting is prohibited;
* users wanting to reuse a project name are required to contact the
  existing author, with cc to support at npmjs.com;
* all contact must conform to the NPM Code of Conduct;
* in case of no resolution after a few weeks, npm inc. holds the right
  to the final decision in the matter.

CPAN lets any user upload modules with the same name.  PAUSE, a related
index, only lists modules uploaded by the primary maintainer or listed
co-maintainers.  CPAN documentation doesn't address disputes otherwise.

GitHub's terms of service contain an exhaustive list of behavior
not meeting general conditions of use.  While not codified anywhere,
GitHub does agree for users to reclaim abandoned account names by
archiving the abandoned account and letting the other user or
organization rename their account.  This is done on a case-by-case
basis.


Rejected Proposals
==================

The original approach was to hope for the best and solve issues as they
arise without written policy.  This is not sustainable.  The lack of
generally available guidelines in writing on package name conflict
resolution is causing unnecessary tensions.  From the perspective of
users, decisions made by the Package Index maintainers without written
guidelines may appear arbitrary.  From the perspective of the Package
Index maintainers, solving name conflicts is a stressful task due to
risk of unintentional harm due to lack of defined policy.


References
==========

.. [1] Terms of Use of the Python Package Index
   (https://pypi.org/policy/terms-of-use/)

.. [2] The Python Package Index
   (https://pypi.python.org/)

.. [3] The Comprehensive Perl Archive Network
   (http://www.cpan.org/)

.. [4] Node Package Manager
   (https://www.npmjs.com/package/left-pad)

.. [5] GitHub
   (https://github.com/)

.. [6] Python Community Code of Conduct
   (https://www.python.org/psf/codeofconduct/)

.. [7] PyPI Support Requests
   (https://sourceforge.net/p/pypi/support-requests/)

.. [8] Python Software Foundation
   (https://www.python.org/psf/)

.. [9] PSF Board Resolutions
   (https://www.python.org/psf/records/board/resolutions/)


Copyright
=========

This document has been placed in the public domain.


Acknowledgements
================

The many participants of the Distutils and Catalog SIGs for their
ideas over the years.

..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   End:


From guettliml at thomas-guettler.de  Mon Jan 16 10:59:27 2017
From: guettliml at thomas-guettler.de (=?UTF-8?Q?Thomas_G=c3=bcttler?=)
Date: Mon, 16 Jan 2017 16:59:27 +0100
Subject: [Distutils] How to specify dependencies in Python
In-Reply-To: <CACac1F-7wnh9jd-fyi8V1o38pzgaOFvFBJO1XfVKXZOvUY9krg@mail.gmail.com>
References: <6f19cb4a-ce35-4d86-ee3f-52596a59a3ec@thomas-guettler.de>
 <CADiSq7coTcSnA1b7HAmO76hsQ0X=F8rgeQ3JkPMaC_KNYP_Epg@mail.gmail.com>
 <5301feab-056f-2c39-a71c-cbd2cde91239@thomas-guettler.de>
 <CACac1F-7wnh9jd-fyi8V1o38pzgaOFvFBJO1XfVKXZOvUY9krg@mail.gmail.com>
Message-ID: <1a13998b-a09e-8d0b-c669-1c769944f015@thomas-guettler.de>



Am 13.01.2017 um 14:07 schrieb Paul Moore:
> On 13 January 2017 at 12:23, Thomas G?ttler
> <guettliml at thomas-guettler.de> wrote:
>> Am 12.01.2017 um 13:43 schrieb Nick Coghlan:
>>>
>>> On 12 January 2017 at 22:04, Thomas G?ttler
>>> <guettliml at thomas-guettler.de> wrote:
>>>>
>>>> I came across a python library which has docs, which start like this:
>>>>
>>>> {{{
>>>>
>>>>  Quickstart
>>>>
>>>>  Include foolib in your requirements.txt file.
>>>>
>>>> }}}
>>>>
>>>> AFAIK dependencies should be specified via `install_requires` in
>>>> `setup.py`.
>>>
>>>
>>> Applications and services don't necessarily have a setup.py file -
>>> setup.py is more for pip-installable libraries and frameworks (see
>>> https://caremad.io/posts/2013/07/setup-vs-requirement/ for more on
>>> that topic).
>>
>>
>> What is an application?
>>
>> The django project uses this definition:
>> https://docs.djangoproject.com/en/1.10/ref/applications/
>>
>> I guess you mean something else, if you talk about "application".
>>
>> What is an application for you?
>
> IMO, an "application" in this context is a standalone program
> (implemented in Python). This is as opposed to a "library" which is
> written to be imported in other Python code. Obviously, there's some
> overlap - some "applications" provide a "programming API" that can be
> imported, and some libraries provide command line entry points.

Hi Paul,

yes, I think the same way. There is some overlap.

> A library (something that you import) should declare in its metadata
> that you need to have its dependencies installed before it will work.
> That lets you (the user of the library) be ignorant of the
> dependencies of the library (which are implementation details as far
> as you are concerned) - pip just sorts it out for you from the
> metadata. To get the dependency metadata, the *library* should include
> its dependencies in install_requires in its setup.py.

Thank you for your explanation. For me this means, I should tell the maintainer
that a library should specify it's dependencies via install_requires.


> For applications on the other hand, you need to install exactly the
> environment you tested, so you *don't* use dependency metadata.
> Rather, you create a requirements.txt file that contains the exact
> versions of everything (direct dependencies) your application needs,
> then deploy your application script to a Python environment where
> you've run "pip install -r requirements.txt" to set it up correctly.


I think requirements.txt should be the result of some kind of Continous-Integration run.
If all tests are successful, then requirements.txt should be created with "pip freeze".

This means, that the Continous-Integration run does not use requirements.txt to
build its environment.

Next question: Where is the best place to store requirements.txt?

I think it should not be in the repo of a library. It should be somehow outside.

Up to now I a missing the right english term for it.

>
> As I say, there are lots of grey areas here. But from your
> description, the library you found sounds like it should have
> dependency metadata, and not need you to use a requirements file. On
> the other hand, the "Quickstart" may be trying to tell people how to
> use foolib in their *application*, in which case it's correct. (It's
> oversimplified, as if you're writing a library that depends on foolib
> you'd use install_requires, but simply changing the text to just say
> "put foolib in your install_requires" is no better, as then you'd have
> ignored the application use case...)
>
> If you can point to the actual library you're referring to, it would
> be easier to be specific :-)

I did not tell the name here, since I don't want to offend the library maintainer.
I think he does not like it, when people talk behind his back about his code.

I will do so after we found a solid solution here.

Regards,
   Thomas G?ttler




-- 
Thomas Guettler http://www.thomas-guettler.de/

From guettliml at thomas-guettler.de  Mon Jan 16 11:03:26 2017
From: guettliml at thomas-guettler.de (=?UTF-8?Q?Thomas_G=c3=bcttler?=)
Date: Mon, 16 Jan 2017 17:03:26 +0100
Subject: [Distutils] How to specify dependencies in Python
In-Reply-To: <CAPDm-Fibo8uBQ=TSXCMYvWztTNwjviz4igyuyS_2q8+T4XiEZA@mail.gmail.com>
References: <6f19cb4a-ce35-4d86-ee3f-52596a59a3ec@thomas-guettler.de>
 <CADiSq7coTcSnA1b7HAmO76hsQ0X=F8rgeQ3JkPMaC_KNYP_Epg@mail.gmail.com>
 <5301feab-056f-2c39-a71c-cbd2cde91239@thomas-guettler.de>
 <CAPDm-Fibo8uBQ=TSXCMYvWztTNwjviz4igyuyS_2q8+T4XiEZA@mail.gmail.com>
Message-ID: <06e6be0b-1625-8bcf-64fe-ef5de231d26b@thomas-guettler.de>



Am 13.01.2017 um 16:25 schrieb Jim Fulton:
>
>
> On Fri, Jan 13, 2017 at 7:23 AM, Thomas G?ttler <guettliml at thomas-guettler.de <mailto:guettliml at thomas-guettler.de>> wrote:
>
>     What is an application for you?
>
>
> Another way to think about this, FWIW, is to distinguish between the "whole system" (for which "Application" is often a
> useful shorthand), as opposed to components (aka libraries).
>
> It's important for components to be flexible, so they're typically very flexible about versions of their dependencies.
>
> For whole systems, OTOH, it's important that once a configuration is tested, that the same configuration is used in
> production, so systems typically pin all of their dependencies, ideally extending to their environments (which is a
> reason why container technology is attractive).

Yes, install_requires in setup.py should define flexible dependencies, but requirements.txt should define fixed 
dependencies via fixed version.

Do you agree with my sentence from above?



-- 
Thomas Guettler http://www.thomas-guettler.de/

From p.f.moore at gmail.com  Mon Jan 16 11:11:34 2017
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 16 Jan 2017 16:11:34 +0000
Subject: [Distutils] How to specify dependencies in Python
In-Reply-To: <1a13998b-a09e-8d0b-c669-1c769944f015@thomas-guettler.de>
References: <6f19cb4a-ce35-4d86-ee3f-52596a59a3ec@thomas-guettler.de>
 <CADiSq7coTcSnA1b7HAmO76hsQ0X=F8rgeQ3JkPMaC_KNYP_Epg@mail.gmail.com>
 <5301feab-056f-2c39-a71c-cbd2cde91239@thomas-guettler.de>
 <CACac1F-7wnh9jd-fyi8V1o38pzgaOFvFBJO1XfVKXZOvUY9krg@mail.gmail.com>
 <1a13998b-a09e-8d0b-c669-1c769944f015@thomas-guettler.de>
Message-ID: <CACac1F87uNFyz8+45mJ59VydRxNQ0O6q3Dc-4txyN2tsE4kiKQ@mail.gmail.com>

On 16 January 2017 at 15:59, Thomas G?ttler
<guettliml at thomas-guettler.de> wrote:
> Thank you for your explanation. For me this means, I should tell the
> maintainer
> that a library should specify it's dependencies via install_requires.

Well yes, but that doesn't mean the quoted comment from Quickstart is
wrong. It's not clear without context whether the comment is intended
to refer to applications using this library, or other libraries using
this library. It's even possible that the project author doesn't fully
understand the subtleties here.

> I think requirements.txt should be the result of some kind of
> Continous-Integration run.
> If all tests are successful, then requirements.txt should be created with
> "pip freeze".
>
> This means, that the Continous-Integration run does not use requirements.txt
> to
> build its environment.
>
> Next question: Where is the best place to store requirements.txt?
>
> I think it should not be in the repo of a library. It should be somehow
> outside.
>
> Up to now I a missing the right english term for it.

I'm not at all sure how true that is, but as I don't write this sort
of application, and certainly don't publish such things, I can't
really comment.

>> If you can point to the actual library you're referring to, it would
>> be easier to be specific :-)
>
>
> I did not tell the name here, since I don't want to offend the library
> maintainer.
> I think he does not like it, when people talk behind his back about his
> code.
>
> I will do so after we found a solid solution here.

OK, but I don't think we can find anything more concrete without more
information, and at some point you're going to end up having to give
enough information that we might as well just look at the project.
After all, it's (presumably) open source, so we can always look at the
code and submit PRs/issues anyway. It seems unlikely that the author
is going to be offended - if there's a risk, why not simply invite
him/her to participate in this thread?

Paul

From jim at jimfulton.info  Mon Jan 16 12:06:40 2017
From: jim at jimfulton.info (Jim Fulton)
Date: Mon, 16 Jan 2017 12:06:40 -0500
Subject: [Distutils] How to specify dependencies in Python
In-Reply-To: <06e6be0b-1625-8bcf-64fe-ef5de231d26b@thomas-guettler.de>
References: <6f19cb4a-ce35-4d86-ee3f-52596a59a3ec@thomas-guettler.de>
 <CADiSq7coTcSnA1b7HAmO76hsQ0X=F8rgeQ3JkPMaC_KNYP_Epg@mail.gmail.com>
 <5301feab-056f-2c39-a71c-cbd2cde91239@thomas-guettler.de>
 <CAPDm-Fibo8uBQ=TSXCMYvWztTNwjviz4igyuyS_2q8+T4XiEZA@mail.gmail.com>
 <06e6be0b-1625-8bcf-64fe-ef5de231d26b@thomas-guettler.de>
Message-ID: <CAPDm-Fg1wtpSz4EYwE_mDLvSwQPX25dw6moihGWA-uH7WxYx2g@mail.gmail.com>

On Mon, Jan 16, 2017 at 11:03 AM, Thomas G?ttler <
guettliml at thomas-guettler.de> wrote:

>
>
> Am 13.01.2017 um 16:25 schrieb Jim Fulton:
>
>>
>>
>> On Fri, Jan 13, 2017 at 7:23 AM, Thomas G?ttler <
>> guettliml at thomas-guettler.de <mailto:guettliml at thomas-guettler.de>>
>> wrote:
>>
>>     What is an application for you?
>>
>>
>> Another way to think about this, FWIW, is to distinguish between the
>> "whole system" (for which "Application" is often a
>> useful shorthand), as opposed to components (aka libraries).
>>
>> It's important for components to be flexible, so they're typically very
>> flexible about versions of their dependencies.
>>
>> For whole systems, OTOH, it's important that once a configuration is
>> tested, that the same configuration is used in
>> production, so systems typically pin all of their dependencies, ideally
>> extending to their environments (which is a
>> reason why container technology is attractive).
>>
>
> Yes, install_requires in setup.py should define flexible dependencies, but
> requirements.txt should define fixed dependencies via fixed version.
>
> Do you agree with my sentence from above?


Are you speaking of a component/library or whole system?

Jim

-- 
Jim Fulton
http://jimfulton.info
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170116/79de95d9/attachment.html>

From offline at offby1.net  Mon Jan 16 14:59:23 2017
From: offline at offby1.net (Chris Rose)
Date: Mon, 16 Jan 2017 11:59:23 -0800
Subject: [Distutils] RFC 2: PEP 541 - Package Index Name Retention
Message-ID: <CAAMdy8sHrye0r2kHY+j8byTZ2N6G-7ePNc9f621fk7toZUk+Zw@mail.gmail.com>

(copied from an email I erroneously sent to python-ideas@)

I want to address one gap in the PEP regarding reclaiming abandoned names:
Version reuse. The problem with reusing names is that existing applications
or installations that reference the old one, unless they pin the version
name precisely. Even in that case, I foresee issues with version collision,
especially if the abandoned project was well-versioned in the same model
(semver or otherwise) that the new project uses.

I'm deeply concerned by the idea of installer code suddenly picking up a
new project... with possibly different dependencies on its own, either with
old or clashing versions. I recognize it's going to be rare, but these
incidents will definitely impact the repeatability of builds depending on
PyPi.

I think the criteria for reuse of a name must include usage limits; if the
package is being downloaded on a steady basis by accounts that can't be
shown to belong to known integration systems, reuse should not be allowed.

-- 
Chris R.
======
Not to be taken literally, internally, or seriously.
Twitter: http://twitter.com/offby1
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170116/33246ae2/attachment.html>

From dsuch at zato.io  Mon Jan 16 16:02:38 2017
From: dsuch at zato.io (Dariusz Suchojad)
Date: Mon, 16 Jan 2017 22:02:38 +0100
Subject: [Distutils] RFC: PEP 541 - Package Index Name Retention
In-Reply-To: <A143359C-B5AE-483C-A0CD-33BD9BBD2BBC@langa.pl>
References: <A143359C-B5AE-483C-A0CD-33BD9BBD2BBC@langa.pl>
Message-ID: <o5jcd8$vu0$1@blaine.gmane.org>

On 13/01/17 19:08, Lukasz Langa wrote:

> Invalid projects
> ----------------
> 
> A project published on the Package Index meeting ANY of the following
> is considered invalid and will be removed from the Index:

[...]

> * project is name squatting (package has no functionality or is
>   empty);

[...]

Greetings,

I'd like to clarify a certain aspect that I reckon is not covered by the
PEP yet.

There are several packages on PyPI in the 'zato' namespace, such as:

https://pypi.python.org/pypi/zato
https://pypi.python.org/pypi/zato-enclog
https://pypi.python.org/pypi/zato-apitest

Naturally, this is a namespace by convention only and on top of that,
one will note that the first link is a 404. The PyPI package 'zato' does
exist but it does not have any release. This is on purpose.

The reason is that although Zato is written mostly in Python, we are not
planning to make it available on PyPI instead opting to provide binary
system packages, including installers for Docker or AWS Elastic
Beanstalk, simply because the installers perform a lot of tasks that are
outside of pip's scope:

https://zato.io/docs/admin/guide/install/index.html

However, there was a case when a third party registered the 'zato'
package in PyPI simply because they thought it a cool idea. This caused
confusion among prospective Zato users who expected to find software
that had never been uploaded to PyPI by its developers. In the end the
third party handed the PyPI package off and everything was resolved
amicably but I'm now worried this can happen again.

In particular, I worry that an eager contributor will eventually author
a script that will find all the packages considered invalid per PEP 541,
they will be deleted and someone else will register 'zato' again and
unfortunately this will cause commotion on our end again. It happened
before thus it's not a hypothetical scenario. And perhaps this time the
third party will be less inclined to cooperate so even more time will be
wasted until the situation is resolved.

Short of adding namespaces to PyPI/Warehouse, I'm wondering how this can
be prevented. Can there be added a clause to the PEP that only packages
whose existence cannot be explain away in email by their maintainers be
considered invalid in case of packages with no functionality nor
contents? I realize that this adds to the PyPI's maintainers workload
which was to be lessened thanks to this PEP but I'm honestly worried
that as it stands now, the PEP does not cover this particular use-case
that I'm concerned about.

Essentially, this is preventive squatting for the greater good, so to
speak, by people who are actually entitled to do it and who would be
doing it anyway if namespaces were available.

kind regards,

-- 
Dariusz Suchojad

https://zato.io
ESB, SOA, REST, APIs and Cloud Integrations in Python



From lukasz at langa.pl  Mon Jan 16 16:15:48 2017
From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=)
Date: Mon, 16 Jan 2017 13:15:48 -0800
Subject: [Distutils] RFC 2: PEP 541 - Package Index Name Retention
In-Reply-To: <CAAMdy8sHrye0r2kHY+j8byTZ2N6G-7ePNc9f621fk7toZUk+Zw@mail.gmail.com>
References: <CAAMdy8sHrye0r2kHY+j8byTZ2N6G-7ePNc9f621fk7toZUk+Zw@mail.gmail.com>
Message-ID: <F64B8B74-2E25-4614-B19F-461F865DEAB3@langa.pl>


> On Jan 16, 2017, at 11:59 AM, Chris Rose <offline at offby1.net> wrote:
> 
> (copied from an email I erroneously sent to python-ideas@)
> 
> I think the criteria for reuse of a name must include usage limits; if the package is being downloaded on a steady basis by accounts that can't be shown to belong to known integration systems, reuse should not be allowed.

I agree, which is why the rules for removal of an abandoned project include the following:

* download statistics on the Package Index for the existing package
 indicate project is not being used;

- ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170116/a7689e78/attachment.html>

From offline at offby1.net  Mon Jan 16 16:18:15 2017
From: offline at offby1.net (Chris Rose)
Date: Mon, 16 Jan 2017 21:18:15 +0000
Subject: [Distutils] RFC 2: PEP 541 - Package Index Name Retention
In-Reply-To: <F64B8B74-2E25-4614-B19F-461F865DEAB3@langa.pl>
References: <CAAMdy8sHrye0r2kHY+j8byTZ2N6G-7ePNc9f621fk7toZUk+Zw@mail.gmail.com>
 <F64B8B74-2E25-4614-B19F-461F865DEAB3@langa.pl>
Message-ID: <CAAMdy8uZt087V6Lq+2Rrq-AGbD3LmRjBwz9rkADUtg2hdQqnBw@mail.gmail.com>

The tricky part there is that "being used" is a tough concept to define.
Over what time period? What amount of downloading counts as "used"?

I believe these concepts need to be made very clear, because the impact of
exploitative replacement is pretty severe if it is made to happen.
On Mon, Jan 16, 2017 at 1:15 PM ?ukasz Langa <lukasz at langa.pl> wrote:

> On Jan 16, 2017, at 11:59 AM, Chris Rose <offline at offby1.net> wrote:
>
> (copied from an email I erroneously sent to python-ideas@)
>
> I think the criteria for reuse of a name must include usage limits; if the
> package is being downloaded on a steady basis by accounts that can't be
> shown to belong to known integration systems, reuse should not be allowed.
>
>
> I agree, which is why the rules for removal of an abandoned project
> include the following:
>
> * download statistics on the Package Index for the existing package
>  indicate project is not being used;
>
> - ?
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170116/431f8bba/attachment-0001.html>

From prometheus235 at gmail.com  Mon Jan 16 16:19:53 2017
From: prometheus235 at gmail.com (Nick Timkovich)
Date: Mon, 16 Jan 2017 15:19:53 -0600
Subject: [Distutils] RFC: PEP 541 - Package Index Name Retention
In-Reply-To: <o5jcd8$vu0$1@blaine.gmane.org>
References: <A143359C-B5AE-483C-A0CD-33BD9BBD2BBC@langa.pl>
 <o5jcd8$vu0$1@blaine.gmane.org>
Message-ID: <CAHkxivd1iNz=Z6eZhN0AAuTU4Rm8gCETHqf+rHuHJ5+OJOXbXw@mail.gmail.com>

If you have a non-release release with some description text and a
home-page that points to where active development is going on (that could
constitute "functionality" in a non-code way), I think that should preempt
a reasonable person (which is hopefully a superset of maintainers) from
deleting it.

On Mon, Jan 16, 2017 at 3:02 PM, Dariusz Suchojad <dsuch at zato.io> wrote:

> On 13/01/17 19:08, Lukasz Langa wrote:
>
> > Invalid projects
> > ----------------
> >
> > A project published on the Package Index meeting ANY of the following
> > is considered invalid and will be removed from the Index:
>
> [...]
>
> > * project is name squatting (package has no functionality or is
> >   empty);
>
> [...]
>
> Greetings,
>
> I'd like to clarify a certain aspect that I reckon is not covered by the
> PEP yet.
>
> There are several packages on PyPI in the 'zato' namespace, such as:
>
> https://pypi.python.org/pypi/zato
> https://pypi.python.org/pypi/zato-enclog
> https://pypi.python.org/pypi/zato-apitest
>
> Naturally, this is a namespace by convention only and on top of that,
> one will note that the first link is a 404. The PyPI package 'zato' does
> exist but it does not have any release. This is on purpose.
>
> The reason is that although Zato is written mostly in Python, we are not
> planning to make it available on PyPI instead opting to provide binary
> system packages, including installers for Docker or AWS Elastic
> Beanstalk, simply because the installers perform a lot of tasks that are
> outside of pip's scope:
>
> https://zato.io/docs/admin/guide/install/index.html
>
> However, there was a case when a third party registered the 'zato'
> package in PyPI simply because they thought it a cool idea. This caused
> confusion among prospective Zato users who expected to find software
> that had never been uploaded to PyPI by its developers. In the end the
> third party handed the PyPI package off and everything was resolved
> amicably but I'm now worried this can happen again.
>
> In particular, I worry that an eager contributor will eventually author
> a script that will find all the packages considered invalid per PEP 541,
> they will be deleted and someone else will register 'zato' again and
> unfortunately this will cause commotion on our end again. It happened
> before thus it's not a hypothetical scenario. And perhaps this time the
> third party will be less inclined to cooperate so even more time will be
> wasted until the situation is resolved.
>
> Short of adding namespaces to PyPI/Warehouse, I'm wondering how this can
> be prevented. Can there be added a clause to the PEP that only packages
> whose existence cannot be explain away in email by their maintainers be
> considered invalid in case of packages with no functionality nor
> contents? I realize that this adds to the PyPI's maintainers workload
> which was to be lessened thanks to this PEP but I'm honestly worried
> that as it stands now, the PEP does not cover this particular use-case
> that I'm concerned about.
>
> Essentially, this is preventive squatting for the greater good, so to
> speak, by people who are actually entitled to do it and who would be
> doing it anyway if namespaces were available.
>
> kind regards,
>
> --
> Dariusz Suchojad
>
> https://zato.io
> ESB, SOA, REST, APIs and Cloud Integrations in Python
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170116/cae583d2/attachment.html>

From dsuch at zato.io  Mon Jan 16 16:28:32 2017
From: dsuch at zato.io (Dariusz Suchojad)
Date: Mon, 16 Jan 2017 22:28:32 +0100
Subject: [Distutils] RFC: PEP 541 - Package Index Name Retention
In-Reply-To: <CAHkxivd1iNz=Z6eZhN0AAuTU4Rm8gCETHqf+rHuHJ5+OJOXbXw@mail.gmail.com>
References: <A143359C-B5AE-483C-A0CD-33BD9BBD2BBC@langa.pl>
 <o5jcd8$vu0$1@blaine.gmane.org>
 <CAHkxivd1iNz=Z6eZhN0AAuTU4Rm8gCETHqf+rHuHJ5+OJOXbXw@mail.gmail.com>
Message-ID: <o5jdtr$pup$1@blaine.gmane.org>

On 16/01/17 22:19, Nick Timkovich wrote:
> If you have a non-release release with some description text and a
> home-page that points to where active development is going on (that could
> constitute "functionality" in a non-code way), I think that should preempt
> a reasonable person (which is hopefully a superset of maintainers) from
> deleting it.

Yes, indeed, this is what I'd like to clarify. Someone is simply bound
to clean up all the packages at one point and this PEP likely will be
the basis for deciding what to delete or not so I'd very much like to
ensure ours will not be removed.

regards,

-- 
Dariusz Suchojad

https://zato.io
ESB, SOA, REST, APIs and Cloud Integrations in Python	


From bussonniermatthias at gmail.com  Mon Jan 16 16:58:43 2017
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Mon, 16 Jan 2017 13:58:43 -0800
Subject: [Distutils] RFC 2: PEP 541 - Package Index Name Retention
In-Reply-To: <CAAMdy8uZt087V6Lq+2Rrq-AGbD3LmRjBwz9rkADUtg2hdQqnBw@mail.gmail.com>
References: <CAAMdy8sHrye0r2kHY+j8byTZ2N6G-7ePNc9f621fk7toZUk+Zw@mail.gmail.com>
 <F64B8B74-2E25-4614-B19F-461F865DEAB3@langa.pl>
 <CAAMdy8uZt087V6Lq+2Rrq-AGbD3LmRjBwz9rkADUtg2hdQqnBw@mail.gmail.com>
Message-ID: <CANJQusXMqUA29MpwsEhmC4Q6ORin3aAiV3fEHUcTObB33KSbsw@mail.gmail.com>

On Mon, Jan 16, 2017 at 1:18 PM, Chris Rose <offline at offby1.net> wrote:
> The tricky part there is that "being used" is a tough concept to define.
> Over what time period? What amount of downloading counts as "used"?
>
> I believe these concepts need to be made very clear, because the impact of
> exploitative replacement is pretty severe if it is made to happen.
>

Would a month  where the old package is made unavailable, but the new
owner is not given access yet be a good compromise ?

It most likely let time the old owner (or old users) to manifest a
decide to "revive" the package if necessary, otherwise give a really
strong signal that if there is still a couple of download, then it
really does not breaks a lot.
-- 
M

From offline at offby1.net  Mon Jan 16 17:02:38 2017
From: offline at offby1.net (Chris Rose)
Date: Mon, 16 Jan 2017 14:02:38 -0800
Subject: [Distutils] RFC 2: PEP 541 - Package Index Name Retention
In-Reply-To: <CANJQusXMqUA29MpwsEhmC4Q6ORin3aAiV3fEHUcTObB33KSbsw@mail.gmail.com>
References: <CAAMdy8sHrye0r2kHY+j8byTZ2N6G-7ePNc9f621fk7toZUk+Zw@mail.gmail.com>
 <F64B8B74-2E25-4614-B19F-461F865DEAB3@langa.pl>
 <CAAMdy8uZt087V6Lq+2Rrq-AGbD3LmRjBwz9rkADUtg2hdQqnBw@mail.gmail.com>
 <CANJQusXMqUA29MpwsEhmC4Q6ORin3aAiV3fEHUcTObB33KSbsw@mail.gmail.com>
Message-ID: <CAAMdy8smmNtzbhbWDCzU7Hfe7Q-ENUrYJpqS5cw36UfeZcuPbQ@mail.gmail.com>

That depends on policy. I don't want to go too far down the trap of
privileging my specific use case, but as a company that vendors
*everything* we depend on, our accesses to PyPi for dependencies are pretty
rare, which means we might run afoul of these changes when ingesting
packages.

I'm going to ask the pointed question: is there actually any serious value
to allowing the replacement of a name for anything that was ever in wide
usage? Trademark violations notwithstanding -- legal stuff requires some
degree of exception to the process -- why should abandonment result in
replacement, as long as the existing code has ever been in use?

On Mon, Jan 16, 2017 at 1:58 PM, Matthias Bussonnier <
bussonniermatthias at gmail.com> wrote:

> On Mon, Jan 16, 2017 at 1:18 PM, Chris Rose <offline at offby1.net> wrote:
> > The tricky part there is that "being used" is a tough concept to define.
> > Over what time period? What amount of downloading counts as "used"?
> >
> > I believe these concepts need to be made very clear, because the impact
> of
> > exploitative replacement is pretty severe if it is made to happen.
> >
>
> Would a month  where the old package is made unavailable, but the new
> owner is not given access yet be a good compromise ?
>
> It most likely let time the old owner (or old users) to manifest a
> decide to "revive" the package if necessary, otherwise give a really
> strong signal that if there is still a couple of download, then it
> really does not breaks a lot.
> --
> M
>



-- 
Chris R.
======
Not to be taken literally, internally, or seriously.
Twitter: http://twitter.com/offby1
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170116/22ea0b2f/attachment-0001.html>

From dsuch at zato.io  Mon Jan 16 17:19:29 2017
From: dsuch at zato.io (Dariusz Suchojad)
Date: Mon, 16 Jan 2017 23:19:29 +0100
Subject: [Distutils] RFC: PEP 541 - Package Index Name Retention
In-Reply-To: <CAJ3HoZ1MohxKefFymKvKyxSdOHcNbVrZbGyiUfzTEo=L5nUhmA@mail.gmail.com>
References: <A143359C-B5AE-483C-A0CD-33BD9BBD2BBC@langa.pl>
 <o5jcd8$vu0$1@blaine.gmane.org>
 <CAHkxivd1iNz=Z6eZhN0AAuTU4Rm8gCETHqf+rHuHJ5+OJOXbXw@mail.gmail.com>
 <o5jdtr$pup$1@blaine.gmane.org>
 <CAJ3HoZ1MohxKefFymKvKyxSdOHcNbVrZbGyiUfzTEo=L5nUhmA@mail.gmail.com>
Message-ID: <30fce2d7-10f5-5d57-5f08-fd3211315a9d@zato.io>

On 16/01/17 23:10, Robert Collins wrote:
> So, this proposition didn't really make sense to me. Folk like Linux
> distros will want the source, and you don't need to upload wheels :-
> setup.py could quite reasonably limit itself to software installation, vs
> configuration. Plenty of pip installable packages are not entirely ready to
> use after pip installation.

Hi Robert,

I'm not clear if this was meant in reply to my email? I'm genuinely
perplexed.

I just don't know how to relate it to the suggestion I made in this message:

https://mail.python.org/pipermail/distutils-sig/2017-January/030015.html

There are several sub-threads in this discussion and I'm not quite sure
what you mean.

regards,

-- 
Dariusz Suchojad

https://zato.io
ESB, SOA, REST, APIs and Cloud Integrations in Python


From robertc at robertcollins.net  Mon Jan 16 17:10:58 2017
From: robertc at robertcollins.net (Robert Collins)
Date: Tue, 17 Jan 2017 11:10:58 +1300
Subject: [Distutils] RFC: PEP 541 - Package Index Name Retention
In-Reply-To: <o5jdtr$pup$1@blaine.gmane.org>
References: <A143359C-B5AE-483C-A0CD-33BD9BBD2BBC@langa.pl>
 <o5jcd8$vu0$1@blaine.gmane.org>
 <CAHkxivd1iNz=Z6eZhN0AAuTU4Rm8gCETHqf+rHuHJ5+OJOXbXw@mail.gmail.com>
 <o5jdtr$pup$1@blaine.gmane.org>
Message-ID: <CAJ3HoZ1MohxKefFymKvKyxSdOHcNbVrZbGyiUfzTEo=L5nUhmA@mail.gmail.com>

So, this proposition didn't really make sense to me. Folk like Linux
distros will want the source, and you don't need to upload wheels :-
setup.py could quite reasonably limit itself to software installation, vs
configuration. Plenty of pip installable packages are not entirely ready to
use after pip installation.

-Rob

On 17 Jan 2017 08:29, "Dariusz Suchojad" <dsuch at zato.io> wrote:

> On 16/01/17 22:19, Nick Timkovich wrote:
> > If you have a non-release release with some description text and a
> > home-page that points to where active development is going on (that could
> > constitute "functionality" in a non-code way), I think that should
> preempt
> > a reasonable person (which is hopefully a superset of maintainers) from
> > deleting it.
>
> Yes, indeed, this is what I'd like to clarify. Someone is simply bound
> to clean up all the packages at one point and this PEP likely will be
> the basis for deciding what to delete or not so I'd very much like to
> ensure ours will not be removed.
>
> regards,
>
> --
> Dariusz Suchojad
>
> https://zato.io
> ESB, SOA, REST, APIs and Cloud Integrations in Python
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170117/f04836d6/attachment.html>

From ncoghlan at gmail.com  Mon Jan 16 18:22:37 2017
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 17 Jan 2017 10:22:37 +1100
Subject: [Distutils] RFC: PEP 541 - Package Index Name Retention
In-Reply-To: <CADiSq7cD0fKMS1bVLR2_POX8U0CH3UYtJDbkG+zckZRqgwR2wQ@mail.gmail.com>
References: <A143359C-B5AE-483C-A0CD-33BD9BBD2BBC@langa.pl>
 <o5jcd8$vu0$1@blaine.gmane.org>
 <CAHkxivd1iNz=Z6eZhN0AAuTU4Rm8gCETHqf+rHuHJ5+OJOXbXw@mail.gmail.com>
 <o5jdtr$pup$1@blaine.gmane.org>
 <CAJ3HoZ1MohxKefFymKvKyxSdOHcNbVrZbGyiUfzTEo=L5nUhmA@mail.gmail.com>
 <30fce2d7-10f5-5d57-5f08-fd3211315a9d@zato.io>
 <CADiSq7ezFmy4JieDx718iDPCv+aoWvJENqW-_0VEjYXxc2BKtQ@mail.gmail.com>
 <CADiSq7ddLZj3wjYyJH-HtU+kVGaQeRmqnZpH2ttWWu+EWPGDAA@mail.gmail.com>
 <CADiSq7d2qT8=pYfQEjegh48ZLLL5zQWkLo-Q120qMqLA1M7xtA@mail.gmail.com>
 <CADiSq7e8hakh+bhy+Sg5v=-DXqEkxCPb2ri5w9B9BkOvT-RCfQ@mail.gmail.com>
 <CADiSq7cXwG_AdcEap+YQg3pmsrysXW9-Pif=n4eyWzMWr3-whA@mail.gmail.com>
 <CADiSq7cD0fKMS1bVLR2_POX8U0CH3UYtJDbkG+zckZRqgwR2wQ@mail.gmail.com>
Message-ID: <CADiSq7ePPa3V=xW3Q2JP3585BGA3Zyz4siECqj3epfvBEmnCaw@mail.gmail.com>

On 17 Jan 2017 09:20, "Dariusz Suchojad" <dsuch at zato.io> wrote:

On 16/01/17 23:10, Robert Collins wrote:
> So, this proposition didn't really make sense to me. Folk like Linux
> distros will want the source, and you don't need to upload wheels :-
> setup.py could quite reasonably limit itself to software installation, vs
> configuration. Plenty of pip installable packages are not entirely ready
to
> use after pip installation.

Hi Robert,

I'm not clear if this was meant in reply to my email? I'm genuinely
perplexed.

I just don't know how to relate it to the suggestion I made in this message:

https://mail.python.org/pipermail/distutils-sig/2017-January/030015.html

There are several sub-threads in this discussion and I'm not quite sure
what you mean.


Robert's referring to the fact that publishing a project sdist on PyPI can
be quite helpful to redistributors, even if "pip install zato" just
installs some helper libraries (or nothing at all) rather than a full Zato
instance.

Publishing at least a stub package with a README and setup.py that says
"Zato is not pip installable, see <url> for details" would also provide a
better experience for prospective users than a plain 404, provide the PyPI
maintainers with a clear  explanation of how the namespace entry is being
used, and the project with download metadata that gives an indication of
how often this mistake is being made, and hence whether or not it's an
installation method that might be worth supporting (even if it's just a
shim around a "docker pull" command).

Cheers,
Nick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170117/ecef54cc/attachment.html>

From steve.dower at python.org  Mon Jan 16 18:29:31 2017
From: steve.dower at python.org (Steve Dower)
Date: Mon, 16 Jan 2017 15:29:31 -0800
Subject: [Distutils] RFC: PEP 541 - Package Index Name Retention
In-Reply-To: <o5jdtr$pup$1@blaine.gmane.org>
References: <A143359C-B5AE-483C-A0CD-33BD9BBD2BBC@langa.pl>
 <o5jcd8$vu0$1@blaine.gmane.org>
 <CAHkxivd1iNz=Z6eZhN0AAuTU4Rm8gCETHqf+rHuHJ5+OJOXbXw@mail.gmail.com>
 <o5jdtr$pup$1@blaine.gmane.org>
Message-ID: <E1cTGjQ-0006rS-VQ@se2-syd.hostedmail.net.au>

Yep, this is what I chose to do for our packages like https://pypi.org/project/microsoft/ (updated since I first posted about it).

The intent of this PEP is to be able to clean up *unmaintained* packages, so if you have an "invalid" package but also a justification and you are reachable, I expect there should be no issues.

Top-posted from my Windows Phone

-----Original Message-----
From: "Dariusz Suchojad" <dsuch at zato.io>
Sent: ?1/?16/?2017 13:29
To: "Distutils-Sig at Python.Org" <Distutils-Sig at Python.Org>
Subject: Re: [Distutils] RFC: PEP 541 - Package Index Name Retention

On 16/01/17 22:19, Nick Timkovich wrote:
> If you have a non-release release with some description text and a
> home-page that points to where active development is going on (that could
> constitute "functionality" in a non-code way), I think that should preempt
> a reasonable person (which is hopefully a superset of maintainers) from
> deleting it.

Yes, indeed, this is what I'd like to clarify. Someone is simply bound
to clean up all the packages at one point and this PEP likely will be
the basis for deciding what to delete or not so I'd very much like to
ensure ours will not be removed.

regards,

-- 
Dariusz Suchojad

https://zato.io
ESB, SOA, REST, APIs and Cloud Integrations in Python	

_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG at python.org
https://mail.python.org/mailman/listinfo/distutils-sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170116/d5781306/attachment-0001.html>

From dsuch at zato.io  Mon Jan 16 18:31:50 2017
From: dsuch at zato.io (Dariusz Suchojad)
Date: Tue, 17 Jan 2017 00:31:50 +0100
Subject: [Distutils] RFC: PEP 541 - Package Index Name Retention
In-Reply-To: <CADiSq7ePPa3V=xW3Q2JP3585BGA3Zyz4siECqj3epfvBEmnCaw@mail.gmail.com>
References: <A143359C-B5AE-483C-A0CD-33BD9BBD2BBC@langa.pl>
 <o5jcd8$vu0$1@blaine.gmane.org>
 <CAHkxivd1iNz=Z6eZhN0AAuTU4Rm8gCETHqf+rHuHJ5+OJOXbXw@mail.gmail.com>
 <o5jdtr$pup$1@blaine.gmane.org>
 <CAJ3HoZ1MohxKefFymKvKyxSdOHcNbVrZbGyiUfzTEo=L5nUhmA@mail.gmail.com>
 <30fce2d7-10f5-5d57-5f08-fd3211315a9d@zato.io>
 <CADiSq7ezFmy4JieDx718iDPCv+aoWvJENqW-_0VEjYXxc2BKtQ@mail.gmail.com>
 <CADiSq7ddLZj3wjYyJH-HtU+kVGaQeRmqnZpH2ttWWu+EWPGDAA@mail.gmail.com>
 <CADiSq7d2qT8=pYfQEjegh48ZLLL5zQWkLo-Q120qMqLA1M7xtA@mail.gmail.com>
 <CADiSq7e8hakh+bhy+Sg5v=-DXqEkxCPb2ri5w9B9BkOvT-RCfQ@mail.gmail.com>
 <CADiSq7cXwG_AdcEap+YQg3pmsrysXW9-Pif=n4eyWzMWr3-whA@mail.gmail.com>
 <CADiSq7cD0fKMS1bVLR2_POX8U0CH3UYtJDbkG+zckZRqgwR2wQ@mail.gmail.com>
 <CADiSq7ePPa3V=xW3Q2JP3585BGA3Zyz4siECqj3epfvBEmnCaw@mail.gmail.com>
Message-ID: <81572dda-700d-e4fb-5709-beeccf8979f3@zato.io>

On 17/01/17 00:22, Nick Coghlan wrote:
> Robert's referring to the fact that publishing a project sdist on PyPI can
> be quite helpful to redistributors, even if "pip install zato" just
> installs some helper libraries (or nothing at all) rather than a full Zato
> instance.
> 
> Publishing at least a stub package with a README and setup.py that says
> "Zato is not pip installable, see <url> for details" would also provide a
> better experience for prospective users than a plain 404, provide the PyPI
> maintainers with a clear  explanation of how the namespace entry is being
> used, and the project with download metadata that gives an indication of
> how often this mistake is being made, and hence whether or not it's an
> installation method that might be worth supporting (even if it's just a
> shim around a "docker pull" command).

Thanks Nick and Robert, yes, that definitely makes sense. This is a nice
suggestion and resolution to my worry.

regards,

-- 
Dariusz Suchojad

https://zato.io
ESB, SOA, REST, APIs and Cloud Integrations in Python



From ethan at stoneleaf.us  Mon Jan 16 20:16:29 2017
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 16 Jan 2017 17:16:29 -0800
Subject: [Distutils] RFC 2: PEP 541 - Package Index Name Retention
In-Reply-To: <CAAMdy8smmNtzbhbWDCzU7Hfe7Q-ENUrYJpqS5cw36UfeZcuPbQ@mail.gmail.com>
References: <CAAMdy8sHrye0r2kHY+j8byTZ2N6G-7ePNc9f621fk7toZUk+Zw@mail.gmail.com>
 <F64B8B74-2E25-4614-B19F-461F865DEAB3@langa.pl>
 <CAAMdy8uZt087V6Lq+2Rrq-AGbD3LmRjBwz9rkADUtg2hdQqnBw@mail.gmail.com>
 <CANJQusXMqUA29MpwsEhmC4Q6ORin3aAiV3fEHUcTObB33KSbsw@mail.gmail.com>
 <CAAMdy8smmNtzbhbWDCzU7Hfe7Q-ENUrYJpqS5cw36UfeZcuPbQ@mail.gmail.com>
Message-ID: <587D706D.1090606@stoneleaf.us>

On 01/16/2017 02:02 PM, Chris Rose wrote:

> That depends on policy. I don't want to go too far down the trap of
>  privileging my specific use case, but as a company that vendors
>  *everything* we depend on, our accesses to PyPi for dependencies are
>  pretty rare, which means we might run afoul of these changes when
>  ingesting packages.

If you have everything vendored then you should be able to easily fall back to older versions that you already have available.

Maybe run your own PyPI server internally?

> I'm going to ask the pointed question: is there actually any serious
>  value to allowing the replacement of a name for anything that was
>  ever in wide usage?

Possibly not, but with automated downloads to various distributions I suspect it becomes very difficult to tell if packages are actually "being used".


>  [...] -- why should abandonment result in replacement, as long as
>  the existing code has ever been in use?

Because PyPI is not an archaeological site?  Although, having said that, perhaps there could be a PyPI/archaeological page for packages that have been replaced.

--
~Ethan~

From jaraco at jaraco.com  Mon Jan 16 15:27:46 2017
From: jaraco at jaraco.com (Jason R. Coombs)
Date: Mon, 16 Jan 2017 20:27:46 +0000
Subject: [Distutils] Setuptools moving away from self install - testing
 needed
Message-ID: <625E08A8-789E-4A11-B828-A968966762D5@jaraco.com>

In [setuptools 581](https://github.com/pypa/setuptools/issues/581), I?ve drafted a version of setuptools that essentially drops support for self installation, relying on pip install as the only supported mechanism for installing setuptools, both in the ez_setup.py bootstrap (for which there?s a minimally compatible shim for replacement) and in the codebase itself.

I invite everyone to install the pre-release linked in the ticket and provide feedback there.  I?ll give it at least a week before I cut an official release. If you need more time, just let me know.

Regards,
Jason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170116/de96fd33/attachment.html>

From glyph at twistedmatrix.com  Tue Jan 17 01:27:45 2017
From: glyph at twistedmatrix.com (Glyph Lefkowitz)
Date: Mon, 16 Jan 2017 22:27:45 -0800
Subject: [Distutils] How to specify dependencies in Python
In-Reply-To: <1a13998b-a09e-8d0b-c669-1c769944f015@thomas-guettler.de>
References: <6f19cb4a-ce35-4d86-ee3f-52596a59a3ec@thomas-guettler.de>
 <CADiSq7coTcSnA1b7HAmO76hsQ0X=F8rgeQ3JkPMaC_KNYP_Epg@mail.gmail.com>
 <5301feab-056f-2c39-a71c-cbd2cde91239@thomas-guettler.de>
 <CACac1F-7wnh9jd-fyi8V1o38pzgaOFvFBJO1XfVKXZOvUY9krg@mail.gmail.com>
 <1a13998b-a09e-8d0b-c669-1c769944f015@thomas-guettler.de>
Message-ID: <C85A1665-2685-41C0-9A53-9B10EC780342@twistedmatrix.com>


> On Jan 16, 2017, at 7:59 AM, Thomas G?ttler <guettliml at thomas-guettler.de> wrote:
> 
> I think requirements.txt should be the result of some kind of Continous-Integration run.
> If all tests are successful, then requirements.txt should be created with "pip freeze".
> 
> This means, that the Continous-Integration run does not use requirements.txt to
> build its environment.
> 
> Next question: Where is the best place to store requirements.txt?
> 
> I think it should not be in the repo of a library. It should be somehow outside.

I think I understand what you're trying to say here, but I think you have it backwards.

The best example I have made of the "right" way to do this is in this project: https://github.com/rackerlabs/mimic/ <https://github.com/rackerlabs/mimic/>

The project has both a setup.py and (several versions of) requirements.txt.

setup.py gives the abstract requirements for the project.  This is what should work, but has not necessarily been exactly tested to.

requirements.txt is the concrete requirements.  This is generated (more or less) by freezing an environment where one does `pip install .` to trigger the setup.py.

However, all continuous integration tests are run against the versions listed in requirements.txt (the details are here: https://github.com/rackerlabs/mimic/blob/5fae30d9e9a45c15f3f0a51fa436a7f25502b742/.travis/install.sh#L101-L105 <https://github.com/rackerlabs/mimic/blob/5fae30d9e9a45c15f3f0a51fa436a7f25502b742/.travis/install.sh#L101-L105> and here https://github.com/rackerlabs/mimic/blob/5fae30d9e9a45c15f3f0a51fa436a7f25502b742/.travis/run.sh#L18 <https://github.com/rackerlabs/mimic/blob/5fae30d9e9a45c15f3f0a51fa436a7f25502b742/.travis/run.sh#L18>) to ensure that new contributors always have a stable set of versions to test against, and their PR won't end up randomly having to fix some upgrade issue.

Finally, we use https://requires.io <https://requires.io/> to submit PRs every time one of the dependencies upgrades.  On a regular basis (every 6 months or so), these do cause errors; when they do, development unrelated to the version upgrade can continue on unimpeded, and the version-compatibility fix lives neatly in its own PR that solves just that problem.

It's a bit subtle to understand the distinction between setup.py and requirements.txt (https://caremad.io/posts/2013/07/setup-vs-requirement/ <https://caremad.io/posts/2013/07/setup-vs-requirement/> is a good attempt, but I think stumbles over explaining some nuances that are obvious to dstufft and not to anyone else), but if you get them lined up correctly then a bunch of edge-cases work very nicely, and reasoning about deployment is much easier.

-glyph
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170116/d08c4f6c/attachment-0001.html>

From guettliml at thomas-guettler.de  Tue Jan 17 11:34:49 2017
From: guettliml at thomas-guettler.de (=?UTF-8?Q?Thomas_G=c3=bcttler?=)
Date: Tue, 17 Jan 2017 17:34:49 +0100
Subject: [Distutils] How to specify dependencies in Python
In-Reply-To: <CAPDm-Fg1wtpSz4EYwE_mDLvSwQPX25dw6moihGWA-uH7WxYx2g@mail.gmail.com>
References: <6f19cb4a-ce35-4d86-ee3f-52596a59a3ec@thomas-guettler.de>
 <CADiSq7coTcSnA1b7HAmO76hsQ0X=F8rgeQ3JkPMaC_KNYP_Epg@mail.gmail.com>
 <5301feab-056f-2c39-a71c-cbd2cde91239@thomas-guettler.de>
 <CAPDm-Fibo8uBQ=TSXCMYvWztTNwjviz4igyuyS_2q8+T4XiEZA@mail.gmail.com>
 <06e6be0b-1625-8bcf-64fe-ef5de231d26b@thomas-guettler.de>
 <CAPDm-Fg1wtpSz4EYwE_mDLvSwQPX25dw6moihGWA-uH7WxYx2g@mail.gmail.com>
Message-ID: <7628a092-580a-1466-5ab6-926024b86e90@thomas-guettler.de>



Am 16.01.2017 um 18:06 schrieb Jim Fulton:
>
>
> On Mon, Jan 16, 2017 at 11:03 AM, Thomas G?ttler <guettliml at thomas-guettler.de <mailto:guettliml at thomas-guettler.de>> wrote:
>
>
>
>     Am 13.01.2017 um 16:25 schrieb Jim Fulton:
>
>
>
>         On Fri, Jan 13, 2017 at 7:23 AM, Thomas G?ttler <guettliml at thomas-guettler.de
>         <mailto:guettliml at thomas-guettler.de> <mailto:guettliml at thomas-guettler.de
>         <mailto:guettliml at thomas-guettler.de>>> wrote:
>
>             What is an application for you?
>
>
>         Another way to think about this, FWIW, is to distinguish between the "whole system" (for which "Application" is
>         often a
>         useful shorthand), as opposed to components (aka libraries).
>
>         It's important for components to be flexible, so they're typically very flexible about versions of their
>         dependencies.
>
>         For whole systems, OTOH, it's important that once a configuration is tested, that the same configuration is used in
>         production, so systems typically pin all of their dependencies, ideally extending to their environments (which is a
>         reason why container technology is attractive).
>
>
>     Yes, install_requires in setup.py should define flexible dependencies, but requirements.txt should define fixed
>     dependencies via fixed version.
>
>     Do you agree with my sentence from above?
>
>
> Are you speaking of a component/library or whole system?

I am speaking of both. And: I think requirements.txt is optional.


-- 
Thomas Guettler http://www.thomas-guettler.de/

From offline at offby1.net  Tue Jan 17 12:25:10 2017
From: offline at offby1.net (Chris Rose)
Date: Tue, 17 Jan 2017 09:25:10 -0800
Subject: [Distutils] RFC 2: PEP 541 - Package Index Name Retention
In-Reply-To: <587D706D.1090606@stoneleaf.us>
References: <CAAMdy8sHrye0r2kHY+j8byTZ2N6G-7ePNc9f621fk7toZUk+Zw@mail.gmail.com>
 <F64B8B74-2E25-4614-B19F-461F865DEAB3@langa.pl>
 <CAAMdy8uZt087V6Lq+2Rrq-AGbD3LmRjBwz9rkADUtg2hdQqnBw@mail.gmail.com>
 <CANJQusXMqUA29MpwsEhmC4Q6ORin3aAiV3fEHUcTObB33KSbsw@mail.gmail.com>
 <CAAMdy8smmNtzbhbWDCzU7Hfe7Q-ENUrYJpqS5cw36UfeZcuPbQ@mail.gmail.com>
 <587D706D.1090606@stoneleaf.us>
Message-ID: <CAAMdy8ucza6isW7dqWaqb5xXohs5rYoUar0G_1QNQfO04G73vQ@mail.gmail.com>

PyPi might not be an archaeological site, but like it or not it *is* a key
part of deployment processes, including those that run headless. I'm
referencing vendoring processes, but the same idea applies when your code
is deployed by any process that includes `pip install` in its steps. While
in an ideal world every user of these packages would host an internal
mirror of the packages they need and rigorously vet them, that's not the
world we live in.

I raise the issue because I believe the bar for taking over an abandoned
name should be nigh-insurmountably high; the risks are in my view severe,
given the way software is built today.

On Mon, Jan 16, 2017 at 5:16 PM, Ethan Furman <ethan at stoneleaf.us> wrote:

> On 01/16/2017 02:02 PM, Chris Rose wrote:
>
> That depends on policy. I don't want to go too far down the trap of
>>  privileging my specific use case, but as a company that vendors
>>  *everything* we depend on, our accesses to PyPi for dependencies are
>>  pretty rare, which means we might run afoul of these changes when
>>  ingesting packages.
>>
>
> If you have everything vendored then you should be able to easily fall
> back to older versions that you already have available.
>
> Maybe run your own PyPI server internally?
>
> I'm going to ask the pointed question: is there actually any serious
>>  value to allowing the replacement of a name for anything that was
>>  ever in wide usage?
>>
>
> Possibly not, but with automated downloads to various distributions I
> suspect it becomes very difficult to tell if packages are actually "being
> used".
>
>
>  [...] -- why should abandonment result in replacement, as long as
>>  the existing code has ever been in use?
>>
>
> Because PyPI is not an archaeological site?  Although, having said that,
> perhaps there could be a PyPI/archaeological page for packages that have
> been replaced.
>
> --
> ~Ethan~
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>



-- 
Chris R.
======
Not to be taken literally, internally, or seriously.
Twitter: http://twitter.com/offby1
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170117/3e9ec1dc/attachment.html>

From jim at jimfulton.info  Tue Jan 17 12:40:32 2017
From: jim at jimfulton.info (Jim Fulton)
Date: Tue, 17 Jan 2017 12:40:32 -0500
Subject: [Distutils] How to specify dependencies in Python
In-Reply-To: <7628a092-580a-1466-5ab6-926024b86e90@thomas-guettler.de>
References: <6f19cb4a-ce35-4d86-ee3f-52596a59a3ec@thomas-guettler.de>
 <CADiSq7coTcSnA1b7HAmO76hsQ0X=F8rgeQ3JkPMaC_KNYP_Epg@mail.gmail.com>
 <5301feab-056f-2c39-a71c-cbd2cde91239@thomas-guettler.de>
 <CAPDm-Fibo8uBQ=TSXCMYvWztTNwjviz4igyuyS_2q8+T4XiEZA@mail.gmail.com>
 <06e6be0b-1625-8bcf-64fe-ef5de231d26b@thomas-guettler.de>
 <CAPDm-Fg1wtpSz4EYwE_mDLvSwQPX25dw6moihGWA-uH7WxYx2g@mail.gmail.com>
 <7628a092-580a-1466-5ab6-926024b86e90@thomas-guettler.de>
Message-ID: <CAPDm-Fjm884meLdLEeunUC8JtbFZbvA=7SQsjd7MXD8+HC=Y_A@mail.gmail.com>

On Tue, Jan 17, 2017 at 11:34 AM, Thomas G?ttler <
guettliml at thomas-guettler.de> wrote:

>
>
> Am 16.01.2017 um 18:06 schrieb Jim Fulton:
>
>>
>>
>> On Mon, Jan 16, 2017 at 11:03 AM, Thomas G?ttler <
>> guettliml at thomas-guettler.de <mailto:guettliml at thomas-guettler.de>>
>> wrote:
>>
>>
>>
>>     Am 13.01.2017 um 16:25 schrieb Jim Fulton:
>>
>>
>>
>>         On Fri, Jan 13, 2017 at 7:23 AM, Thomas G?ttler <
>> guettliml at thomas-guettler.de
>>         <mailto:guettliml at thomas-guettler.de> <mailto:
>> guettliml at thomas-guettler.de
>>         <mailto:guettliml at thomas-guettler.de>>> wrote:
>>
>>             What is an application for you?
>>
>>
>>         Another way to think about this, FWIW, is to distinguish between
>> the "whole system" (for which "Application" is
>>         often a
>>         useful shorthand), as opposed to components (aka libraries).
>>
>>         It's important for components to be flexible, so they're
>> typically very flexible about versions of their
>>         dependencies.
>>
>>         For whole systems, OTOH, it's important that once a configuration
>> is tested, that the same configuration is used in
>>         production, so systems typically pin all of their dependencies,
>> ideally extending to their environments (which is a
>>         reason why container technology is attractive).
>>
>>
>>     Yes, install_requires in setup.py should define flexible
>> dependencies, but requirements.txt should define fixed
>>     dependencies via fixed version.
>>
>>     Do you agree with my sentence from above?
>>
>>
>> Are you speaking of a component/library or whole system?
>>
>
> I am speaking of both. And: I think requirements.txt is optional.


Then I disagree with your statement. :)

I should stop, but I'll take one more stab at this.

It matters whether you're talking about components(/libraries) or whole
systems (/applications).

For components:

Consumers of a component need to be able to to determine the component's
dependencies.  The component uses install_required (and extras_require) for
this.  The version specifications in these dependencies should be as
flexible as possible, to allow reuse in as many whole systems as possible.

Developers of a component will use tools like pip and buildout to automate
their development activities.  For pip, that will usually entail one or
more requirements.txt files. For buildout, that will  entail one or more
(for different development activities) buildout configs and a single
versions.cfg.

For whole systems:

Many whole systems only assemble components.  For these systems, there is
no setup.py file (no python project).

If a whole system includes a Python project (that isn't distributed
separately), it's a matter of taste how much information is included in
setup.py. Personally, I would treat the Python project like a component
project and include its direct dependencies and minimal version constraints.

Typically a whole system managed with pip will use a requirements.txt file
(or possibly multiple) and a system  developed with buildout will have a
buildout config and a versions config.

A whole system could fix all of its dependent versions in install_requires
in a setup script, but that would be cumbersome. By using requirements.txt
or a buildout versions config, a developer can avail themselves of
automation to help maintain the files.

Jim

-- 
Jim Fulton
http://jimfulton.info
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170117/d4366290/attachment.html>

From p.f.moore at gmail.com  Tue Jan 17 12:47:30 2017
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 17 Jan 2017 17:47:30 +0000
Subject: [Distutils] How to specify dependencies in Python
In-Reply-To: <CAPDm-Fjm884meLdLEeunUC8JtbFZbvA=7SQsjd7MXD8+HC=Y_A@mail.gmail.com>
References: <6f19cb4a-ce35-4d86-ee3f-52596a59a3ec@thomas-guettler.de>
 <CADiSq7coTcSnA1b7HAmO76hsQ0X=F8rgeQ3JkPMaC_KNYP_Epg@mail.gmail.com>
 <5301feab-056f-2c39-a71c-cbd2cde91239@thomas-guettler.de>
 <CAPDm-Fibo8uBQ=TSXCMYvWztTNwjviz4igyuyS_2q8+T4XiEZA@mail.gmail.com>
 <06e6be0b-1625-8bcf-64fe-ef5de231d26b@thomas-guettler.de>
 <CAPDm-Fg1wtpSz4EYwE_mDLvSwQPX25dw6moihGWA-uH7WxYx2g@mail.gmail.com>
 <7628a092-580a-1466-5ab6-926024b86e90@thomas-guettler.de>
 <CAPDm-Fjm884meLdLEeunUC8JtbFZbvA=7SQsjd7MXD8+HC=Y_A@mail.gmail.com>
Message-ID: <CACac1F9DzFntGc+mc4F3jS7PyKUHVBdeyFPTSc9BP0u0H1txVg@mail.gmail.com>

On 17 January 2017 at 17:40, Jim Fulton <jim at jimfulton.info> wrote:
>> I am speaking of both. And: I think requirements.txt is optional.
>
> Then I disagree with your statement. :)
>
> I should stop, but I'll take one more stab at this.
>
> It matters whether you're talking about components(/libraries) or whole
> systems (/applications).

FWIW, that was my point as well. Thomas, if you're not understanding
that instructions need to be different depending on whether you're
talking about components/libraries or systems/applications, then we're
not going to reach an agreement.

IMO, the "Quickstart" you quoted that triggered this is pretty much OK
as long as it's talking about developing applications. It's not clear
without context whether that's the case, but unless *you* understand
the distinction Jim and I are trying to make, then I doubt we'd agree
with any suggested rewording of that section that you might propose.

Paul

From donald at stufft.io  Tue Jan 17 13:01:46 2017
From: donald at stufft.io (Donald Stufft)
Date: Tue, 17 Jan 2017 13:01:46 -0500
Subject: [Distutils] RFC 2: PEP 541 - Package Index Name Retention
In-Reply-To: <CAAMdy8ucza6isW7dqWaqb5xXohs5rYoUar0G_1QNQfO04G73vQ@mail.gmail.com>
References: <CAAMdy8sHrye0r2kHY+j8byTZ2N6G-7ePNc9f621fk7toZUk+Zw@mail.gmail.com>
 <F64B8B74-2E25-4614-B19F-461F865DEAB3@langa.pl>
 <CAAMdy8uZt087V6Lq+2Rrq-AGbD3LmRjBwz9rkADUtg2hdQqnBw@mail.gmail.com>
 <CANJQusXMqUA29MpwsEhmC4Q6ORin3aAiV3fEHUcTObB33KSbsw@mail.gmail.com>
 <CAAMdy8smmNtzbhbWDCzU7Hfe7Q-ENUrYJpqS5cw36UfeZcuPbQ@mail.gmail.com>
 <587D706D.1090606@stoneleaf.us>
 <CAAMdy8ucza6isW7dqWaqb5xXohs5rYoUar0G_1QNQfO04G73vQ@mail.gmail.com>
Message-ID: <A75C755D-C1D4-4D47-A248-8067A3FFEB22@stufft.io>


> On Jan 17, 2017, at 12:25 PM, Chris Rose <offline at offby1.net> wrote:
> 
> PyPi might not be an archaeological site, but like it or not it *is* a key part of deployment processes, including those that run headless. I'm referencing vendoring processes, but the same idea applies when your code is deployed by any process that includes `pip install` in its steps. While in an ideal world every user of these packages would host an internal mirror of the packages they need and rigorously vet them, that's not the world we live in.
> 
> I raise the issue because I believe the bar for taking over an abandoned name should be nigh-insurmountably high; the risks are in my view severe, given the way software is built today.
> 


If ~nobody is downloading it from PyPI today and ~nobody is releasing new versions to PyPI then re-using the name should have very little effect, even if in the past people had been using it. The only real use case I can think of where this might not be true is it could break someone?s ability to reproduce a deployment from many years ago but if you need to reproduce your build from years ago, depending on PyPI for that is not really the smartest bet. Otherwise it feels a lot like we?re in a ?if a tree falls in the woods, but nobody is around to hear it does it make a sound?? territory.

One thing we could possibly do is provide the ability for, as part of the relqunishing process, ?lock? the old versions that were uploaded so that the new owner can neither delete them or upload new files for them AND set a ?minimum version? for new uploads for that project. This could mean that one could say that foobar < 4.0 is the old project and foobar >= 4.0 is the new project and existing == continue to work. I?m not sure I feel about that though.

Ultimately, consumers need to either live with these sorts of problems or they need to develop their own solutions for counteracting them (like vendoring) because while this proposed policy *could* cause them these issues, it?s not the only way for them to occur. Specifically, this only deals with cases that the original author is no longer responsive in some way, but fit hey are it?s typically not very hard in my experience to convince someone to give up a name they once used and are no longer interested in maintaining. I?ve acted as the middleman for this very arrangement on a number of occasions.

?
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170117/b466a086/attachment.html>

From bussonniermatthias at gmail.com  Tue Jan 17 13:53:44 2017
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Tue, 17 Jan 2017 10:53:44 -0800
Subject: [Distutils] RFC 2: PEP 541 - Package Index Name Retention
In-Reply-To: <A75C755D-C1D4-4D47-A248-8067A3FFEB22@stufft.io>
References: <CAAMdy8sHrye0r2kHY+j8byTZ2N6G-7ePNc9f621fk7toZUk+Zw@mail.gmail.com>
 <F64B8B74-2E25-4614-B19F-461F865DEAB3@langa.pl>
 <CAAMdy8uZt087V6Lq+2Rrq-AGbD3LmRjBwz9rkADUtg2hdQqnBw@mail.gmail.com>
 <CANJQusXMqUA29MpwsEhmC4Q6ORin3aAiV3fEHUcTObB33KSbsw@mail.gmail.com>
 <CAAMdy8smmNtzbhbWDCzU7Hfe7Q-ENUrYJpqS5cw36UfeZcuPbQ@mail.gmail.com>
 <587D706D.1090606@stoneleaf.us>
 <CAAMdy8ucza6isW7dqWaqb5xXohs5rYoUar0G_1QNQfO04G73vQ@mail.gmail.com>
 <A75C755D-C1D4-4D47-A248-8067A3FFEB22@stufft.io>
Message-ID: <CANJQusV9G3oMHfHaTC2qpvfZ+SsKAJUnbPF+RJR2+d21+Zx4cw@mail.gmail.com>

> One thing we could possibly do is provide the ability for, as part of the
> relqunishing process, ?lock? the old versions that were uploaded so that the
> new owner can neither delete them or upload new files for them AND set a
> ?minimum version? for new uploads for that project. This could mean that one
> could say that foobar < 4.0 is the old project and foobar >= 4.0 is the new
> project and existing == continue to work. I?m not sure I feel about that
> though.


Wouldn't that be a case where the version epoch[1] could (should?) be used ?

> If included in a version identifier, the epoch appears before all other components, separated from the release segment by an exclamation mark:

> E!X.Y  # Version identifier with epoch
> If no explicit epoch is given, the implicit epoch is 0 .

> Most version identifiers will not include an epoch, as an explicit epoch is only needed if a project changes the way it handles version numbering in a way that means the normal version ordering rules will give the wrong answer.

-- 
M

1:https://www.python.org/dev/peps/pep-0440/#version-epochs

From elvis.stansvik at orexplore.com  Thu Jan 19 02:20:41 2017
From: elvis.stansvik at orexplore.com (Elvis Stansvik)
Date: Thu, 19 Jan 2017 08:20:41 +0100
Subject: [Distutils] Has anyone looked at bug 29225?
Message-ID: <CAE-7VJmpqpTNzJAVbOXqit3VT7H98rmBvUETXTCuUV-DhDsNLw@mail.gmail.com>

Hi distutils folks,

Just wanted to ask if anyone has had time to look at my

    http://bugs.python.org/issue29225

? It's my first bug report _and_ first patch for Python, so it's quite
possible I screwed something up. Or people are just busy :)

Cheers,
Elvis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170119/af90a80b/attachment.html>

From elvis.stansvik at orexplore.com  Thu Jan 19 13:00:55 2017
From: elvis.stansvik at orexplore.com (Elvis Stansvik)
Date: Thu, 19 Jan 2017 19:00:55 +0100
Subject: [Distutils] Has anyone looked at bug 29225?
In-Reply-To: <CAP1=2W6Gig-TpmjAmeHfpYhjL6gfHbmVw2xs8CuAFGesxOTEGQ@mail.gmail.com>
References: <CAE-7VJmpqpTNzJAVbOXqit3VT7H98rmBvUETXTCuUV-DhDsNLw@mail.gmail.com>
 <CAP1=2W6Gig-TpmjAmeHfpYhjL6gfHbmVw2xs8CuAFGesxOTEGQ@mail.gmail.com>
Message-ID: <CAE-7VJkVFYQb1hDHO=vyU8nFt=rPas9JzY+LNgVtxrZGpyocig@mail.gmail.com>

2017-01-19 18:52 GMT+01:00 Brett Cannon <brett at python.org>:

> It's probably a combination of people being busy and the fact that it
> involves distutils which not very many people feel comfortable reviewing
> patches for and changing its semantics.
>

Alright, that's understandable and sort of what I suspected.

I hope (and believe) my patch preserves the semantics for all cases except
the specific one it fixes. The wrong behavior which it fixes is so
obviously wrong that I can't imagine someone relying on it somehow :)

Elvis



>
> Regardless of what happens to your patch, though, thanks for taking the
> time to report the problem you ran into and trying to fix it!
>
> On Wed, 18 Jan 2017 at 23:21 Elvis Stansvik <elvis.stansvik at orexplore.com>
> wrote:
>
>> Hi distutils folks,
>>
>> Just wanted to ask if anyone has had time to look at my
>>
>>     http://bugs.python.org/issue29225
>>
>> ? It's my first bug report _and_ first patch for Python, so it's quite
>> possible I screwed something up. Or people are just busy :)
>>
>> Cheers,
>> Elvis
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170119/848e80c1/attachment.html>

From brett at python.org  Thu Jan 19 12:52:54 2017
From: brett at python.org (Brett Cannon)
Date: Thu, 19 Jan 2017 17:52:54 +0000
Subject: [Distutils] Has anyone looked at bug 29225?
In-Reply-To: <CAE-7VJmpqpTNzJAVbOXqit3VT7H98rmBvUETXTCuUV-DhDsNLw@mail.gmail.com>
References: <CAE-7VJmpqpTNzJAVbOXqit3VT7H98rmBvUETXTCuUV-DhDsNLw@mail.gmail.com>
Message-ID: <CAP1=2W6Gig-TpmjAmeHfpYhjL6gfHbmVw2xs8CuAFGesxOTEGQ@mail.gmail.com>

It's probably a combination of people being busy and the fact that it
involves distutils which not very many people feel comfortable reviewing
patches for and changing its semantics.

Regardless of what happens to your patch, though, thanks for taking the
time to report the problem you ran into and trying to fix it!

On Wed, 18 Jan 2017 at 23:21 Elvis Stansvik <elvis.stansvik at orexplore.com>
wrote:

> Hi distutils folks,
>
> Just wanted to ask if anyone has had time to look at my
>
>     http://bugs.python.org/issue29225
>
> ? It's my first bug report _and_ first patch for Python, so it's quite
> possible I screwed something up. Or people are just busy :)
>
> Cheers,
> Elvis
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170119/3ce4ecbf/attachment.html>

From brett at python.org  Fri Jan 20 12:32:37 2017
From: brett at python.org (Brett Cannon)
Date: Fri, 20 Jan 2017 17:32:37 +0000
Subject: [Distutils] Has anyone looked at bug 29225?
In-Reply-To: <CAE-7VJkVFYQb1hDHO=vyU8nFt=rPas9JzY+LNgVtxrZGpyocig@mail.gmail.com>
References: <CAE-7VJmpqpTNzJAVbOXqit3VT7H98rmBvUETXTCuUV-DhDsNLw@mail.gmail.com>
 <CAP1=2W6Gig-TpmjAmeHfpYhjL6gfHbmVw2xs8CuAFGesxOTEGQ@mail.gmail.com>
 <CAE-7VJkVFYQb1hDHO=vyU8nFt=rPas9JzY+LNgVtxrZGpyocig@mail.gmail.com>
Message-ID: <CAP1=2W76_ta3uT0T1vN91fo89uxww5dmoR0H0pC=M3KZ+H=L1g@mail.gmail.com>

Oh, over the 26 years of Python's lifetime you would be surprised what
people come to rely on (including broken semantics which they have written
fixes for themselves which then break when a proper fix is introduced).

On Thu, 19 Jan 2017 at 10:01 Elvis Stansvik <elvis.stansvik at orexplore.com>
wrote:

> 2017-01-19 18:52 GMT+01:00 Brett Cannon <brett at python.org>:
>
> It's probably a combination of people being busy and the fact that it
> involves distutils which not very many people feel comfortable reviewing
> patches for and changing its semantics.
>
>
> Alright, that's understandable and sort of what I suspected.
>
> I hope (and believe) my patch preserves the semantics for all cases except
> the specific one it fixes. The wrong behavior which it fixes is so
> obviously wrong that I can't imagine someone relying on it somehow :)
>
> Elvis
>
>
>
>
> Regardless of what happens to your patch, though, thanks for taking the
> time to report the problem you ran into and trying to fix it!
>
> On Wed, 18 Jan 2017 at 23:21 Elvis Stansvik <elvis.stansvik at orexplore.com>
> wrote:
>
> Hi distutils folks,
>
> Just wanted to ask if anyone has had time to look at my
>
>     http://bugs.python.org/issue29225
>
> ? It's my first bug report _and_ first patch for Python, so it's quite
> possible I screwed something up. Or people are just busy :)
>
> Cheers,
> Elvis
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170120/5d7d73c5/attachment.html>

From elvis.stansvik at orexplore.com  Fri Jan 20 13:53:16 2017
From: elvis.stansvik at orexplore.com (Elvis Stansvik)
Date: Fri, 20 Jan 2017 19:53:16 +0100
Subject: [Distutils] Has anyone looked at bug 29225?
In-Reply-To: <CAP1=2W76_ta3uT0T1vN91fo89uxww5dmoR0H0pC=M3KZ+H=L1g@mail.gmail.com>
References: <CAE-7VJmpqpTNzJAVbOXqit3VT7H98rmBvUETXTCuUV-DhDsNLw@mail.gmail.com>
 <CAP1=2W6Gig-TpmjAmeHfpYhjL6gfHbmVw2xs8CuAFGesxOTEGQ@mail.gmail.com>
 <CAE-7VJkVFYQb1hDHO=vyU8nFt=rPas9JzY+LNgVtxrZGpyocig@mail.gmail.com>
 <CAP1=2W76_ta3uT0T1vN91fo89uxww5dmoR0H0pC=M3KZ+H=L1g@mail.gmail.com>
Message-ID: <CAE-7VJmHGWioOZ22QVdkaxuK6WW_XM1PNOewM32WGZ_-tz08pw@mail.gmail.com>

Den 20 jan. 2017 6:32 em skrev "Brett Cannon" <brett at python.org>:
>
> Oh, over the 26 years of Python's lifetime you would be surprised what
people come to rely on (including broken semantics which they have written
fixes for themselves which then break when a proper fix is introduced).

I can imagine, perhaps I shouldn't have been so confident :) Hopefully if
people have put in their own workarounds, they'll be happy to be able to
drop them, but who knows, heh.

Elvis

>
> On Thu, 19 Jan 2017 at 10:01 Elvis Stansvik <elvis.stansvik at orexplore.com>
wrote:
>>
>> 2017-01-19 18:52 GMT+01:00 Brett Cannon <brett at python.org>:
>>>
>>> It's probably a combination of people being busy and the fact that it
involves distutils which not very many people feel comfortable reviewing
patches for and changing its semantics.
>>
>>
>> Alright, that's understandable and sort of what I suspected.
>>
>> I hope (and believe) my patch preserves the semantics for all cases
except the specific one it fixes. The wrong behavior which it fixes is so
obviously wrong that I can't imagine someone relying on it somehow :)
>>
>> Elvis
>>
>>
>>>
>>>
>>> Regardless of what happens to your patch, though, thanks for taking the
time to report the problem you ran into and trying to fix it!
>>>
>>> On Wed, 18 Jan 2017 at 23:21 Elvis Stansvik <
elvis.stansvik at orexplore.com> wrote:
>>>>
>>>> Hi distutils folks,
>>>>
>>>> Just wanted to ask if anyone has had time to look at my
>>>>
>>>>     http://bugs.python.org/issue29225
>>>>
>>>> ? It's my first bug report _and_ first patch for Python, so it's quite
possible I screwed something up. Or people are just busy :)
>>>>
>>>> Cheers,
>>>> Elvis
>>>> _______________________________________________
>>>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>>>> https://mail.python.org/mailman/listinfo/distutils-sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170120/8f2406e0/attachment.html>

From lele at metapensiero.it  Fri Jan 20 16:56:45 2017
From: lele at metapensiero.it (Lele Gaifax)
Date: Fri, 20 Jan 2017 22:56:45 +0100
Subject: [Distutils] Limit package installation to a specific Python version
Message-ID: <87fukde6oy.fsf@metapensiero.it>

Hi all,

do installers like pip and conda consult trove classifiers, or more generally
is there a way to "mark" a package published on PyPI as installable only in a
Python 3 environment?

An user proposed[1] to change the classifiers of a package to get that result,
so I tried looking at pip and setuptools sources, but didn't find evidence of
that. I see both use python_requires[2] in their setup that seem closer to the
goal, but again, is that actually honored by pip and/or other installers?

Thanks for any hint,
ciao, lele.

[1] https://github.com/python-rapidjson/python-rapidjson/pull/47
[2] https://www.python.org/dev/peps/pep-0345/#requires-python
-- 
nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri
real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia.
lele at metapensiero.it  |                 -- Fortunato Depero, 1929.


From njs at pobox.com  Fri Jan 20 17:56:47 2017
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 20 Jan 2017 14:56:47 -0800
Subject: [Distutils] Limit package installation to a specific Python
 version
In-Reply-To: <87fukde6oy.fsf@metapensiero.it>
References: <87fukde6oy.fsf@metapensiero.it>
Message-ID: <CAPJVwBnWzEQWbtSrf4Jhbj+6Ktrono+wOADQYeLPDipPCE-K2A@mail.gmail.com>

On Fri, Jan 20, 2017 at 1:56 PM, Lele Gaifax <lele at metapensiero.it> wrote:
> Hi all,
>
> do installers like pip and conda consult trove classifiers, or more generally
> is there a way to "mark" a package published on PyPI as installable only in a
> Python 3 environment?
>
> An user proposed[1] to change the classifiers of a package to get that result,
> so I tried looking at pip and setuptools sources, but didn't find evidence of
> that. I see both use python_requires[2] in their setup that seem closer to the
> goal, but again, is that actually honored by pip and/or other installers?
>
> Thanks for any hint,
> ciao, lele.
>
> [1] https://github.com/python-rapidjson/python-rapidjson/pull/47
> [2] https://www.python.org/dev/peps/pep-0345/#requires-python

Neither pip nor conda do anything automatically with trove
classifiers. Conda uses its own metadata added when the conda package
is built; whoever runs your conda-build can probably figure something
out. Wheels have always had this metadata and pip has always honored
it. For sdists, historically there was no way to do this, but as of a
few months ago this was fixed and pip now *does* honor
python_requires, and PyPI has been updated to expose this information
so pip can see it before downloading the sdists:

    https://github.com/pypa/pip/pull/3846
    https://github.com/python/peps/pull/56
    https://github.com/pypa/pip/pull/3877

Since this is a pretty recent change, you probably want to put a
runtime check inside your setup.py for old Python versions, so if
someone uses an old version of pip then they will at least get a nice
explanation.

IPython's an example of a python-3-only project that uses this
belt-and-suspenders approach:

https://github.com/ipython/ipython/blob/f060158665189fd456e54a5a864080efe28b4fdf/setup.py#L29
https://github.com/ipython/ipython/blob/f060158665189fd456e54a5a864080efe28b4fdf/setup.py#L248

-n

-- 
Nathaniel J. Smith -- https://vorpus.org

From lele at metapensiero.it  Fri Jan 20 18:16:49 2017
From: lele at metapensiero.it (Lele Gaifax)
Date: Sat, 21 Jan 2017 00:16:49 +0100
Subject: [Distutils] Limit package installation to a specific Python
 version
References: <87fukde6oy.fsf@metapensiero.it>
 <CAPJVwBnWzEQWbtSrf4Jhbj+6Ktrono+wOADQYeLPDipPCE-K2A@mail.gmail.com>
Message-ID: <87bmv1e2zi.fsf@metapensiero.it>

Great, thank you Nathaniel!
-- 
nickname: Lele Gaifax | Quando vivr? di quello che ho pensato ieri
real: Emanuele Gaifas | comincer? ad aver paura di chi mi copia.
lele at metapensiero.it  |                 -- Fortunato Depero, 1929.


From bussonniermatthias at gmail.com  Fri Jan 20 18:21:09 2017
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Fri, 20 Jan 2017 15:21:09 -0800
Subject: [Distutils] Limit package installation to a specific Python
 version
In-Reply-To: <CAPJVwBnWzEQWbtSrf4Jhbj+6Ktrono+wOADQYeLPDipPCE-K2A@mail.gmail.com>
References: <87fukde6oy.fsf@metapensiero.it>
 <CAPJVwBnWzEQWbtSrf4Jhbj+6Ktrono+wOADQYeLPDipPCE-K2A@mail.gmail.com>
Message-ID: <CANJQusWcQYsPTD5sZOXetDJMu6vKojtnC3rp7jCF0wdw2nho8A@mail.gmail.com>

Hi all,

On Fri, Jan 20, 2017 at 2:56 PM, Nathaniel Smith <njs at pobox.com> wrote:
> On Fri, Jan 20, 2017 at 1:56 PM, Lele Gaifax <lele at metapensiero.it> wrote:
>> Hi all,
>>
>> do installers like pip and conda consult trove classifiers, or more generally
>> is there a way to "mark" a package published on PyPI as installable only in a
>> Python 3 environment?
>>

>  For sdists, historically there was no way to do this, but as of a
> few months ago this was fixed and pip now *does* honor
> python_requires, and PyPI has been updated to expose this information
> so pip can see it before downloading the sdists:
>
>     https://github.com/pypa/pip/pull/3846
>     https://github.com/python/peps/pull/56
>     https://github.com/pypa/pip/pull/3877
>
> Since this is a pretty recent change, you probably want to put a
> runtime check inside your setup.py for old Python versions, so if
> someone uses an old version of pip then they will at least get a nice
> explanation.
>
> IPython's an example of a python-3-only project that uses this
> belt-and-suspenders approach:
>
> https://github.com/ipython/ipython/blob/f060158665189fd456e54a5a864080efe28b4fdf/setup.py#L29
> https://github.com/ipython/ipython/blob/f060158665189fd456e54a5a864080efe28b4fdf/setup.py#L248
>

I'm also in the (slow) process of writing the various step you can
take when python_requires is not honored by older systems and how to
mitigate that:

https://github.com/python3statement/python3statement.github.io/pull/55

Feedback (and help) would be welcome.

In particular only system where pip 9.0+ is installed will understand
the metadata now exposed by PyPI,
so when/if you fail in your setup.py, a useful tip is to ask user to
upgrade pip.

There should be some snipets of code you copy and past in your
setup.py in this PR as well.

Thanks.
-- 
M

From b at sashk.xyz  Fri Jan 20 22:57:38 2017
From: b at sashk.xyz (sashk)
Date: Fri, 20 Jan 2017 22:57:38 -0500
Subject: [Distutils] Compiling and installing C executable in setup.py
Message-ID: <7637991484971058@web18h.yandex.ru>

Hello,

I've got a python package which requires to install C executable, which will need to go into proper place, i.e. same 
place where scripts would go ($VIRTUAL_ENV/bin, /usr/local/bin, etc). 

What would be the correct way to implement this? 

I found couple examples, but they work only when using pip to install  the package, but fail when I try to use 
python setup.py install: they will build the executable, but will not install it during the "install bdist_egg" stage. 

Regards,
-sashk

From robin at reportlab.com  Tue Jan 24 06:09:35 2017
From: robin at reportlab.com (Robin Becker)
Date: Tue, 24 Jan 2017 11:09:35 +0000
Subject: [Distutils] recommended strategy for missing development tools
Message-ID: <d47a0c22-7ede-7e1f-fc25-1b42f6449cb0@chamonix.reportlab.co.uk>

A reportlab user says his pip install fails to create an importable C extension. 
He said

"The platform is Mac OS X 10.11.6, aka "El Capitan". Pip didn't complain"

which is probably true since the extension is not required for reportlab's main 
usage.

I tried to reproduce on OS X 10.10.5, but my machine has xcode installed and the 
extension was correctly produced.

I don't have any code in setup.py to prevent compilation; is there a way to 
alert users to the non-build of the extension(s).
-- 
Robin Becker

From njs at pobox.com  Tue Jan 24 06:13:51 2017
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 24 Jan 2017 03:13:51 -0800
Subject: [Distutils] recommended strategy for missing development tools
In-Reply-To: <d47a0c22-7ede-7e1f-fc25-1b42f6449cb0@chamonix.reportlab.co.uk>
References: <d47a0c22-7ede-7e1f-fc25-1b42f6449cb0@chamonix.reportlab.co.uk>
Message-ID: <CAPJVwBn350SEddTt0wEYXQyj8Vf-9odkk928jxx=vn-bRgd+9Q@mail.gmail.com>

On Tue, Jan 24, 2017 at 3:09 AM, Robin Becker <robin at reportlab.com> wrote:
> A reportlab user says his pip install fails to create an importable C
> extension. He said
>
> "The platform is Mac OS X 10.11.6, aka "El Capitan". Pip didn't complain"
>
> which is probably true since the extension is not required for reportlab's
> main usage.
>
> I tried to reproduce on OS X 10.10.5, but my machine has xcode installed and
> the extension was correctly produced.
>
> I don't have any code in setup.py to prevent compilation; is there a way to
> alert users to the non-build of the extension(s).

I'm having some trouble figuring out exactly what you're asking... is
the question: "if my setup.py encounters a non-fatal error while being
run by pip, how can I print a message for the user to see while still
letting the build continue?"?

-n

-- 
Nathaniel J. Smith -- https://vorpus.org

From chris at withers.org  Wed Jan 25 04:04:00 2017
From: chris at withers.org (Chris Withers)
Date: Wed, 25 Jan 2017 09:04:00 +0000
Subject: [Distutils] latest setuptools appears to require six in a breaking
 way
Message-ID: <f237e64f-b3a8-4443-3b70-6de6d718283f@withers.org>

Hi All,

Anyone else seen this?

|File 
"/home/travis/virtualenv/python2.6.9/lib/python2.6/site-packages/setuptools/__init__.py", 
line 10, in <module> from six.moves import filter, map ImportError: No 
module named six.moves|


Started happening in my nightly builds on travis after the 34.0.2 
release of setuptools. I've filed an issue here since I suspect it isn't 
just me:

https://github.com/pypa/setuptools/issues/945

cheers,

Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170125/fb898f87/attachment.html>

From robin at reportlab.com  Wed Jan 25 05:23:39 2017
From: robin at reportlab.com (Robin Becker)
Date: Wed, 25 Jan 2017 10:23:39 +0000
Subject: [Distutils] recommended strategy for missing development tools
In-Reply-To: <CAPJVwBn350SEddTt0wEYXQyj8Vf-9odkk928jxx=vn-bRgd+9Q@mail.gmail.com>
References: <d47a0c22-7ede-7e1f-fc25-1b42f6449cb0@chamonix.reportlab.co.uk>
 <CAPJVwBn350SEddTt0wEYXQyj8Vf-9odkk928jxx=vn-bRgd+9Q@mail.gmail.com>
Message-ID: <44c52371-726b-9c7e-a255-92428589ea88@chamonix.reportlab.co.uk>

On 24/01/2017 11:13, Nathaniel Smith wrote:
> On Tue, Jan 24, 2017 at 3:09 AM, Robin Becker <robin at reportlab.com> wrote:
>> A reportlab user says his pip install fails to create an importable C
>> extension. He said
>>
>> "The platform is Mac OS X 10.11.6, aka "El Capitan". Pip didn't complain"
>>
>> which is probably true since the extension is not required for reportlab's
>> main usage.
>>
>> I tried to reproduce on OS X 10.10.5, but my machine has xcode installed and
>> the extension was correctly produced.
>>
>> I don't have any code in setup.py to prevent compilation; is there a way to
>> alert users to the non-build of the extension(s).
>
> I'm having some trouble figuring out exactly what you're asking... is
> the question: "if my setup.py encounters a non-fatal error while being
> run by pip, how can I print a message for the user to see while still
> letting the build continue?"?
>
> -n
>
Sorry to be so vague, but I am not the person who ran pip install so I'm not 
exactly sure what happened. The setup.py in question complains if the extension 
files cannot be found, but does not error. If the files can be found and I 
assume they can because they're now part of the same repository then the 
extensions are set up and added to the final setup call. Since pip did not 
complain I assume either the setup.py failed to find the extension source or the 
compile failed silently somehow.

Unfortunately I don't have a test system where I can turn off xcode to see what 
happens. I'm wondering if setup fails silently somehow.
-- 
Robin Becker

From chris at simplistix.co.uk  Wed Jan 25 05:41:13 2017
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 25 Jan 2017 10:41:13 +0000
Subject: [Distutils] latest setuptools appears to require six in a breaking
 way
Message-ID: <77ef0649-1cfa-73fb-5043-98506df9ed41@simplistix.co.uk>

Hi All,

Anyone else seen this?

|File 
"/home/travis/virtualenv/python2.6.9/lib/python2.6/site-packages/setuptools/__init__.py", 
line 10, in <module> from six.moves import filter, map ImportError: No 
module named six.moves|


Started happening in my nightly builds on travis after the 34.0.2 
release of setuptools. I've filed an issue here since I suspect it isn't 
just me:

https://github.com/pypa/setuptools/issues/945

cheers,

Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170125/35c8d9ea/attachment.html>

From thomas at kluyver.me.uk  Wed Jan 25 05:49:01 2017
From: thomas at kluyver.me.uk (Thomas Kluyver)
Date: Wed, 25 Jan 2017 10:49:01 +0000
Subject: [Distutils] recommended strategy for missing development tools
In-Reply-To: <44c52371-726b-9c7e-a255-92428589ea88@chamonix.reportlab.co.uk>
References: <d47a0c22-7ede-7e1f-fc25-1b42f6449cb0@chamonix.reportlab.co.uk>
 <CAPJVwBn350SEddTt0wEYXQyj8Vf-9odkk928jxx=vn-bRgd+9Q@mail.gmail.com>
 <44c52371-726b-9c7e-a255-92428589ea88@chamonix.reportlab.co.uk>
Message-ID: <1485341341.1621161.858942992.7EC4A57D@webmail.messagingengine.com>

On Wed, Jan 25, 2017, at 10:23 AM, Robin Becker wrote:
> The setup.py in question complains if the
> extension files cannot be found, but does not error.

There was a change in pip at some point which means that it only shows
the output from setup.py if running it fails (i.e. it has a non-zero
exit status). I think this was largely because setup.py output is much
more verbose than you usually want, especially if pip installs many
packages at once.

It's nice in some cases where setup.py displays warnings that aren't
really a problem. In one project I maintain, a particular submodule
requires Python 3, while the rest of the code can be used on Python 2 as
well; people with older versions of pip have complained that it 'fails
to install', which it doesn't, because the warning looks like a fatal
error.

However, the flip side of this is that, as far as I know, setup.py has
to choose between succeeding silently, or failing and showing the user
an error message.

Thomas

From alirefaee20 at hotmail.com  Wed Jan 25 06:41:59 2017
From: alirefaee20 at hotmail.com (ali refaee)
Date: Wed, 25 Jan 2017 11:41:59 +0000
Subject: [Distutils] Enquire for python
Message-ID: <VI1PR0601MB2176BBFA9E9419B9C100C65DB9740@VI1PR0601MB2176.eurprd06.prod.outlook.com>

Dear Sir/Madam
I?m beginner in python, how can I get python modules?
when run some function
such as (import numpy as np)
message
ImportError: No module named numpy

another ex
import matplotlib.pyplot as plt
ImportError: No module named matplotlib.pyplot

how we  install modules?
where are the modules?
can I get modules to run python?

Sent from Windows Mail

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170125/10e07ceb/attachment-0001.html>

From robin at reportlab.com  Wed Jan 25 09:46:10 2017
From: robin at reportlab.com (Robin Becker)
Date: Wed, 25 Jan 2017 14:46:10 +0000
Subject: [Distutils] latest setuptools appears to require six in a
 breaking way
In-Reply-To: <77ef0649-1cfa-73fb-5043-98506df9ed41@simplistix.co.uk>
References: <77ef0649-1cfa-73fb-5043-98506df9ed41@simplistix.co.uk>
Message-ID: <241b596e-0239-dd04-49c7-bca9cba7a773@chamonix.reportlab.co.uk>


On 25/01/2017 10:41, Chris Withers wrote:
> Hi All,
>
> Anyone else seen this?
>
> |File
> "/home/travis/virtualenv/python2.6.9/lib/python2.6/site-packages/setuptools/__init__.py",
> line 10, in <module> from six.moves import filter, map ImportError: No module
> named six.moves|
>
My colleague just came across this when creating a new venv.

We then installed six, but our setup then went on to complain about packaging 
and then finally appdirs before it went through.

Each time the fix was to manually install the thing that pip complained about.


>
> Started happening in my nightly builds on travis after the 34.0.2 release of
> setuptools. I've filed an issue here since I suspect it isn't just me:
>
> https://github.com/pypa/setuptools/issues/945
>
> cheers,
>
> Chris
>
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>


-- 
Robin Becker

From bussonniermatthias at gmail.com  Wed Jan 25 11:44:42 2017
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Wed, 25 Jan 2017 08:44:42 -0800
Subject: [Distutils] latest setuptools appears to require six in a
 breaking way
In-Reply-To: <241b596e-0239-dd04-49c7-bca9cba7a773@chamonix.reportlab.co.uk>
References: <77ef0649-1cfa-73fb-5043-98506df9ed41@simplistix.co.uk>
 <241b596e-0239-dd04-49c7-bca9cba7a773@chamonix.reportlab.co.uk>
Message-ID: <CANJQusXp_9wMDf_FyDubKQRsHZwgXy6L0+ipo+byrcSPLeuK0A@mail.gmail.com>

AFAICT this is not really an issue as this is on the release notes:

https://setuptools.readthedocs.io/en/latest/history.html#v34-0-0

You need to upgrade pip  (cf 3rd paragraph):

> #581: Instead of vendoring the growing list of dependencies that Setuptools requires to function, Setuptools now requires these dependencies just like any other project. Unlike other projects, however, Setuptools cannot rely on setup_requires to demand the dependencies it needs to install because its own machinery would be necessary to pull those dependencies if not present (a bootstrapping problem). As a result, Setuptools no longer supports self upgrade or installation in the general case. Instead, users are directed to use pip to install and upgrade using the wheel distributions of setuptools.

> Users are welcome to contrive other means to install or upgrade Setuptools using other means, such as pre-installing the Setuptools dependencies with pip or a bespoke bootstrap tool, but such usage is not recommended and is not supported.

 >As discovered in #940, not all versions of pip will successfully
install Setuptools from its pre-built wheel. If you encounter issues
with ?No module named six? or ?No module named packaging?, especially
following a line ?Running setup.py egg_info for package setuptools?,
then your pip is not new enough.

> There?s an additional issue in pip where setuptools is upgraded concurrently with other source packages, described in pip #4253. The proposed workaround is to always upgrade Setuptools first prior to upgrading other packages that would upgrade Setuptools.

-- 
M

On Wed, Jan 25, 2017 at 6:46 AM, Robin Becker <robin at reportlab.com> wrote:
>
> On 25/01/2017 10:41, Chris Withers wrote:
>>
>> Hi All,
>>
>> Anyone else seen this?
>>
>> |File
>>
>> "/home/travis/virtualenv/python2.6.9/lib/python2.6/site-packages/setuptools/__init__.py",
>> line 10, in <module> from six.moves import filter, map ImportError: No
>> module
>> named six.moves|
>>
> My colleague just came across this when creating a new venv.
>
> We then installed six, but our setup then went on to complain about
> packaging and then finally appdirs before it went through.
>
> Each time the fix was to manually install the thing that pip complained
> about.
>
>
>>
>> Started happening in my nightly builds on travis after the 34.0.2 release
>> of
>> setuptools. I've filed an issue here since I suspect it isn't just me:
>>
>> https://github.com/pypa/setuptools/issues/945
>>
>> cheers,
>>
>> Chris
>>
>>
>>
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>
>
> --
> Robin Becker
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

From chris at simplistix.co.uk  Wed Jan 25 12:20:18 2017
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 25 Jan 2017 17:20:18 +0000
Subject: [Distutils] latest setuptools appears to require six in a
 breaking way
In-Reply-To: <CANJQusXp_9wMDf_FyDubKQRsHZwgXy6L0+ipo+byrcSPLeuK0A@mail.gmail.com>
References: <77ef0649-1cfa-73fb-5043-98506df9ed41@simplistix.co.uk>
 <241b596e-0239-dd04-49c7-bca9cba7a773@chamonix.reportlab.co.uk>
 <CANJQusXp_9wMDf_FyDubKQRsHZwgXy6L0+ipo+byrcSPLeuK0A@mail.gmail.com>
Message-ID: <8692c831-688b-35ad-f5c7-644a1638300a@simplistix.co.uk>

On 25/01/2017 16:44, Matthias Bussonnier wrote:
> AFAICT this is not really an issue as this is on the release notes:
>
> https://setuptools.readthedocs.io/en/latest/history.html#v34-0-0

Just because it's on the release notes doesn't mean its not an issue.

>> #581: Instead of vendoring the growing list of dependencies that Setuptools requires to function, Setuptools now requires these dependencies just like any other project. Unlike other projects, however, Setuptools cannot rely on setup_requires to demand the dependencies it needs to install because its own machinery would be necessary to pull those dependencies if not present (a bootstrapping problem). As a result, Setuptools no longer supports self upgrade or installation in the general case. Instead, users are directed to use pip to install and upgrade using the wheel distributions of setuptools.

For such a fundamental part of the python packaging ecosystem, that's a 
pretty disappointing change. Not supporting self-upgrade seems like a 
pretty massive regression, and the problems here are a result.

None of the problems I've had involve explicit installation of 
setuptools, it seems to be dragged in as a dependency somewhere along 
the way. The lack of ability to self-update in that situation is going 
to cause major pain.

>  >As discovered in #940, not all versions of pip will successfully
> install Setuptools from its pre-built wheel. If you encounter issues
> with ?No module named six? or ?No module named packaging?, especially
> following a line ?Running setup.py egg_info for package setuptools?,
> then your pip is not new enough.

Right, so what's the recommended one-step way to set up a virtualenv now 
in Py 2.6-3.6?

>> There?s an additional issue in pip where setuptools is upgraded concurrently with other source packages, described in pip #4253. The proposed workaround is to always upgrade Setuptools first prior to upgrading other packages that would upgrade Setuptools.

That workaround is quite painful if you maintain any number of libraries 
that have CI scripts that will need to be hacked up to support the 
workaround and then reverted after the issue is finally fixed.

Sorry if this comes across as negative, I know it's not always easy to 
judge the outcome of changes before a big release, but I do hope 
something can be done to mitigate the pain of library maintainers doing 
the right thing and running CI...

cheers,

Chris

From brett at python.org  Wed Jan 25 12:35:54 2017
From: brett at python.org (Brett Cannon)
Date: Wed, 25 Jan 2017 17:35:54 +0000
Subject: [Distutils] Enquire for python
In-Reply-To: <VI1PR0601MB2176BBFA9E9419B9C100C65DB9740@VI1PR0601MB2176.eurprd06.prod.outlook.com>
References: <VI1PR0601MB2176BBFA9E9419B9C100C65DB9740@VI1PR0601MB2176.eurprd06.prod.outlook.com>
Message-ID: <CAP1=2W7hX3dTNcHNGoXYx31cWB36GQ69GPWyD+oqK8Ba5H8p3w@mail.gmail.com>

Instructions can be found at https://packaging.python.org/installing/

On Wed, 25 Jan 2017 at 03:42 ali refaee <alirefaee20 at hotmail.com> wrote:

> Dear Sir/Madam
> I?m beginner in python, how can I get python modules?
> when run some function
> such as (import numpy as np)
> message
> ImportError: No module named numpy
>
> another ex
> import matplotlib.pyplot as plt
> ImportError: No module named matplotlib.pyplot
>
> how we  install modules?
> where are the modules?
> can I get modules to run python?
>
> Sent from Windows Mail
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170125/c16992f2/attachment.html>

From bussonniermatthias at gmail.com  Wed Jan 25 12:51:41 2017
From: bussonniermatthias at gmail.com (Matthias Bussonnier)
Date: Wed, 25 Jan 2017 09:51:41 -0800
Subject: [Distutils] latest setuptools appears to require six in a
 breaking way
In-Reply-To: <8692c831-688b-35ad-f5c7-644a1638300a@simplistix.co.uk>
References: <77ef0649-1cfa-73fb-5043-98506df9ed41@simplistix.co.uk>
 <241b596e-0239-dd04-49c7-bca9cba7a773@chamonix.reportlab.co.uk>
 <CANJQusXp_9wMDf_FyDubKQRsHZwgXy6L0+ipo+byrcSPLeuK0A@mail.gmail.com>
 <8692c831-688b-35ad-f5c7-644a1638300a@simplistix.co.uk>
Message-ID: <CANJQusWs1W5ZqPSammCeDZ5p0dgcg9hdQvpZz8PF51xVg754kA@mail.gmail.com>

> Just because it's on the release notes doesn't mean its not an issue.

Sorry if I miss expressed myself, let me retry:

If it is on the release notes, then it is likely because the
maintainers are aware. And if they released this version with this in
the release notes, then trying to fill a bug will likely result in it
being dismissed as duplicate.
I tried to be too concise.

I don't disagree that this change is likely to be quite painful, and I
have not contributed enough to setuptools to contradict the choice the
maintainers have had. I just linked to (and cited) the release notes,
gave a reason as I was aware (since yesterday, made /r/python top 10)
of why you were seeing what you are seeing.

I do understand the difficulty to upgrade all the CI script and
library. I've been in the process of upgrading our test suite and test
of all the packages I maintain to make sure we don't use
DeprecatedBehavior of 3.6 (Oh invalid backslash escape sequence is fun
in docstring that give example with windows Path). So yes that a
couple of weeks of low productivity. But at the same time that's the
price I chose to pay to support such a wide range of versions and have
sofware from 2 differents decades to work together.

I want to add that if your system is still using old pip/setuptools
version then it might soon break for other reasons. Setuptools < 24
(IIRC) and pip < 9 cannot handle Python 3 only Sdist correctly. So if
one of your libraries you use publish a Python 3 only sdist (which is
likely to happen at some point in time) then your CI will install the
latest python 3 only sdist on python 2 instead of the latest Python 2
compatible sdist. Which is likely to fail as well.

So at some point upgrading pip and setuptools as a first step of CI
integration might be anyway necessary.

Hope this problem will be either accepted or solved soon.

Thanks
-- 
M





On Wed, Jan 25, 2017 at 9:20 AM, Chris Withers <chris at simplistix.co.uk> wrote:
> On 25/01/2017 16:44, Matthias Bussonnier wrote:
>>
>> AFAICT this is not really an issue as this is on the release notes:
>>
>> https://setuptools.readthedocs.io/en/latest/history.html#v34-0-0
>
>
> Just because it's on the release notes doesn't mean its not an issue.
>
>>> #581: Instead of vendoring the growing list of dependencies that
>>> Setuptools requires to function, Setuptools now requires these dependencies
>>> just like any other project. Unlike other projects, however, Setuptools
>>> cannot rely on setup_requires to demand the dependencies it needs to install
>>> because its own machinery would be necessary to pull those dependencies if
>>> not present (a bootstrapping problem). As a result, Setuptools no longer
>>> supports self upgrade or installation in the general case. Instead, users
>>> are directed to use pip to install and upgrade using the wheel distributions
>>> of setuptools.
>
>
> For such a fundamental part of the python packaging ecosystem, that's a
> pretty disappointing change. Not supporting self-upgrade seems like a pretty
> massive regression, and the problems here are a result.
>
> None of the problems I've had involve explicit installation of setuptools,
> it seems to be dragged in as a dependency somewhere along the way. The lack
> of ability to self-update in that situation is going to cause major pain.
>
>>  >As discovered in #940, not all versions of pip will successfully
>> install Setuptools from its pre-built wheel. If you encounter issues
>> with ?No module named six? or ?No module named packaging?, especially
>> following a line ?Running setup.py egg_info for package setuptools?,
>> then your pip is not new enough.
>
>
> Right, so what's the recommended one-step way to set up a virtualenv now in
> Py 2.6-3.6?
>
>>> There?s an additional issue in pip where setuptools is upgraded
>>> concurrently with other source packages, described in pip #4253. The
>>> proposed workaround is to always upgrade Setuptools first prior to upgrading
>>> other packages that would upgrade Setuptools.
>
>
> That workaround is quite painful if you maintain any number of libraries
> that have CI scripts that will need to be hacked up to support the
> workaround and then reverted after the issue is finally fixed.
>
> Sorry if this comes across as negative, I know it's not always easy to judge
> the outcome of changes before a big release, but I do hope something can be
> done to mitigate the pain of library maintainers doing the right thing and
> running CI...
>
> cheers,
>
> Chris

From p.f.moore at gmail.com  Wed Jan 25 14:11:40 2017
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 25 Jan 2017 19:11:40 +0000
Subject: [Distutils] Virtalenv and setuptools [was: latest setuptools
 appears to require six in a breaking way]
Message-ID: <CACac1F_U6vdRLAacK_cW4+bNwkeMc384josepUXRHMX-GChQGQ@mail.gmail.com>

On 25 January 2017 at 17:20, Chris Withers <chris at simplistix.co.uk> wrote:
> Right, so what's the recommended one-step way to set up a virtualenv now in
> Py 2.6-3.6?

This is the point I would consider most significant here. Virtualenv
is deliberately built to allow use offline - pip, wheel and setuptools
are bundled so that it's possible to create a virtualenv without
needing Internet access. This change to setuptools will, if I
understand it, break that expectation.

While it's not a common scenario, I think it's something that should
be considered. Going forward, I see a number of options for
virtualenv:

1. Bundle all of setuptools' dependencies as well.
2. Drop the "no internet required" constraint - if we do this, it may
be reasonable to only bundle pip, and get latest versions of
everything else from PyPI.
3. Drop auto-installing setuptools (it's not needed unless you're
installing from sdist, and it's only a "pip install setuptools" away
for people who need it).
4. Document the changed behaviour by saying that no internet is
required as long as you use --no-setuptools.

Thoughts, anyone? Is the situation common enough to warrant anything
other than (4)? It used to be for me, when pip didn't cache downloads
and I had a secured proxy to deal with, but now I'd be OK with (4).

Paul

From donald at stufft.io  Wed Jan 25 14:15:13 2017
From: donald at stufft.io (Donald Stufft)
Date: Wed, 25 Jan 2017 14:15:13 -0500
Subject: [Distutils] Virtalenv and setuptools [was: latest setuptools
 appears to require six in a breaking way]
In-Reply-To: <CACac1F_U6vdRLAacK_cW4+bNwkeMc384josepUXRHMX-GChQGQ@mail.gmail.com>
References: <CACac1F_U6vdRLAacK_cW4+bNwkeMc384josepUXRHMX-GChQGQ@mail.gmail.com>
Message-ID: <FAD643FC-2B2E-44C2-94CD-66AB7343DB61@stufft.io>


> On Jan 25, 2017, at 2:11 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> 
> On 25 January 2017 at 17:20, Chris Withers <chris at simplistix.co.uk> wrote:
>> Right, so what's the recommended one-step way to set up a virtualenv now in
>> Py 2.6-3.6?
> 
> This is the point I would consider most significant here. Virtualenv
> is deliberately built to allow use offline - pip, wheel and setuptools
> are bundled so that it's possible to create a virtualenv without
> needing Internet access. This change to setuptools will, if I
> understand it, break that expectation.
> 
> While it's not a common scenario, I think it's something that should
> be considered. Going forward, I see a number of options for
> virtualenv:
> 
> 1. Bundle all of setuptools' dependencies as well.
> 2. Drop the "no internet required" constraint - if we do this, it may
> be reasonable to only bundle pip, and get latest versions of
> everything else from PyPI.
> 3. Drop auto-installing setuptools (it's not needed unless you're
> installing from sdist, and it's only a "pip install setuptools" away
> for people who need it).
> 4. Document the changed behaviour by saying that no internet is
> required as long as you use --no-setuptools.
> 
> Thoughts, anyone? Is the situation common enough to warrant anything
> other than (4)? It used to be for me, when pip didn't cache downloads
> and I had a secured proxy to deal with, but now I'd be OK with (4).
> 

If we get PEP 518 landed in pip I think that (3) is the right step forward.

?
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170125/828293b2/attachment.html>

From p.f.moore at gmail.com  Wed Jan 25 15:00:27 2017
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 25 Jan 2017 20:00:27 +0000
Subject: [Distutils] Virtalenv and setuptools [was: latest setuptools
 appears to require six in a breaking way]
In-Reply-To: <FAD643FC-2B2E-44C2-94CD-66AB7343DB61@stufft.io>
References: <CACac1F_U6vdRLAacK_cW4+bNwkeMc384josepUXRHMX-GChQGQ@mail.gmail.com>
 <FAD643FC-2B2E-44C2-94CD-66AB7343DB61@stufft.io>
Message-ID: <CACac1F9NydMp_DZnitnHzWV7+6ZwtrLwz_QBdFo7aJQo6jST8g@mail.gmail.com>

On 25 January 2017 at 19:15, Donald Stufft <donald at stufft.io> wrote:
> If we get PEP 518 landed in pip I think that (3) is the right step forward.

Agreed. With that in mind (do we have any timescale for that work?) I
suggest we do (4) for now and switch to (3) once PEP 518 is in place.
Paul

From donald at stufft.io  Wed Jan 25 15:02:22 2017
From: donald at stufft.io (Donald Stufft)
Date: Wed, 25 Jan 2017 15:02:22 -0500
Subject: [Distutils] Virtalenv and setuptools [was: latest setuptools
 appears to require six in a breaking way]
In-Reply-To: <CACac1F9NydMp_DZnitnHzWV7+6ZwtrLwz_QBdFo7aJQo6jST8g@mail.gmail.com>
References: <CACac1F_U6vdRLAacK_cW4+bNwkeMc384josepUXRHMX-GChQGQ@mail.gmail.com>
 <FAD643FC-2B2E-44C2-94CD-66AB7343DB61@stufft.io>
 <CACac1F9NydMp_DZnitnHzWV7+6ZwtrLwz_QBdFo7aJQo6jST8g@mail.gmail.com>
Message-ID: <F7D9EE0E-56D0-4CFF-A1F7-007A9BFA858E@stufft.io>


> On Jan 25, 2017, at 3:00 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> 
> On 25 January 2017 at 19:15, Donald Stufft <donald at stufft.io> wrote:
>> If we get PEP 518 landed in pip I think that (3) is the right step forward.
> 
> Agreed. With that in mind (do we have any timescale for that work?) I
> suggest we do (4) for now and switch to (3) once PEP 518 is in place.
> Paul


Since we have a PR up for PEP 518 already, I hope to make sure it lands in the next release of pip. I?d likely go as far to say that as soon as it lands I?ll probably release pip again. Beyond that I don?t have a specific timescale.

?
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170125/be0d7c2b/attachment.html>

From thomas at kluyver.me.uk  Wed Jan 25 15:04:08 2017
From: thomas at kluyver.me.uk (Thomas Kluyver)
Date: Wed, 25 Jan 2017 20:04:08 +0000
Subject: [Distutils] Virtalenv and setuptools [was: latest setuptools
 appears to require six in a breaking way]
In-Reply-To: <F7D9EE0E-56D0-4CFF-A1F7-007A9BFA858E@stufft.io>
References: <CACac1F_U6vdRLAacK_cW4+bNwkeMc384josepUXRHMX-GChQGQ@mail.gmail.com>
 <FAD643FC-2B2E-44C2-94CD-66AB7343DB61@stufft.io>
 <CACac1F9NydMp_DZnitnHzWV7+6ZwtrLwz_QBdFo7aJQo6jST8g@mail.gmail.com>
 <F7D9EE0E-56D0-4CFF-A1F7-007A9BFA858E@stufft.io>
Message-ID: <1485374648.3888747.859553968.37073BB7@webmail.messagingengine.com>

On Wed, Jan 25, 2017, at 08:02 PM, Donald Stufft wrote:

> Since we have a PR up for PEP 518 already, 



Here's the link if anyone wants to take a look:

https://github.com/pypa/pip/pull/4144


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170125/9be0fefd/attachment.html>

From p.f.moore at gmail.com  Wed Jan 25 16:02:10 2017
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 25 Jan 2017 21:02:10 +0000
Subject: [Distutils] Virtalenv and setuptools [was: latest setuptools
 appears to require six in a breaking way]
In-Reply-To: <1485374648.3888747.859553968.37073BB7@webmail.messagingengine.com>
References: <CACac1F_U6vdRLAacK_cW4+bNwkeMc384josepUXRHMX-GChQGQ@mail.gmail.com>
 <FAD643FC-2B2E-44C2-94CD-66AB7343DB61@stufft.io>
 <CACac1F9NydMp_DZnitnHzWV7+6ZwtrLwz_QBdFo7aJQo6jST8g@mail.gmail.com>
 <F7D9EE0E-56D0-4CFF-A1F7-007A9BFA858E@stufft.io>
 <1485374648.3888747.859553968.37073BB7@webmail.messagingengine.com>
Message-ID: <CACac1F-=hzkkFj2awFN7hTcEFtnew20UTPQvxhBW3hH=i3Bfyw@mail.gmail.com>

On 25 January 2017 at 20:04, Thomas Kluyver <thomas at kluyver.me.uk> wrote:
> On Wed, Jan 25, 2017, at 08:02 PM, Donald Stufft wrote:
>
> Since we have a PR up for PEP 518 already,
>
>
> Here's the link if anyone wants to take a look:
> https://github.com/pypa/pip/pull/4144

Thanks. I've been seeing the PR comments in passing, but haven't
really looked in detail. Is there anything I can do to help this
along?

If Donald is looking at "the next release of pip" for this (and
virtualenv will presumably be released in sync, as usual) there's not
much point doing anything shorter term than that for virtualenv, so it
looks like option (3) it is :-)

Paul

From thomas at kluyver.me.uk  Wed Jan 25 16:26:18 2017
From: thomas at kluyver.me.uk (Thomas Kluyver)
Date: Wed, 25 Jan 2017 21:26:18 +0000
Subject: [Distutils] Virtalenv and setuptools [was: latest setuptools
 appears to require six in a breaking way]
In-Reply-To: <CACac1F-=hzkkFj2awFN7hTcEFtnew20UTPQvxhBW3hH=i3Bfyw@mail.gmail.com>
References: <CACac1F_U6vdRLAacK_cW4+bNwkeMc384josepUXRHMX-GChQGQ@mail.gmail.com>
 <FAD643FC-2B2E-44C2-94CD-66AB7343DB61@stufft.io>
 <CACac1F9NydMp_DZnitnHzWV7+6ZwtrLwz_QBdFo7aJQo6jST8g@mail.gmail.com>
 <F7D9EE0E-56D0-4CFF-A1F7-007A9BFA858E@stufft.io>
 <1485374648.3888747.859553968.37073BB7@webmail.messagingengine.com>
 <CACac1F-=hzkkFj2awFN7hTcEFtnew20UTPQvxhBW3hH=i3Bfyw@mail.gmail.com>
Message-ID: <1485379578.602700.859643336.02F91305@webmail.messagingengine.com>

On Wed, Jan 25, 2017, at 09:02 PM, Paul Moore wrote:
> Thanks. I've been seeing the PR comments in passing, but haven't
> really looked in detail. Is there anything I can do to help this
> along?

I believe it's working, but it's the first significant change I've made
in pip, so any review is welcome.

Thanks,
Thomas

From p.f.moore at gmail.com  Wed Jan 25 16:34:41 2017
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 25 Jan 2017 21:34:41 +0000
Subject: [Distutils] Virtalenv and setuptools [was: latest setuptools
 appears to require six in a breaking way]
In-Reply-To: <1485379578.602700.859643336.02F91305@webmail.messagingengine.com>
References: <CACac1F_U6vdRLAacK_cW4+bNwkeMc384josepUXRHMX-GChQGQ@mail.gmail.com>
 <FAD643FC-2B2E-44C2-94CD-66AB7343DB61@stufft.io>
 <CACac1F9NydMp_DZnitnHzWV7+6ZwtrLwz_QBdFo7aJQo6jST8g@mail.gmail.com>
 <F7D9EE0E-56D0-4CFF-A1F7-007A9BFA858E@stufft.io>
 <1485374648.3888747.859553968.37073BB7@webmail.messagingengine.com>
 <CACac1F-=hzkkFj2awFN7hTcEFtnew20UTPQvxhBW3hH=i3Bfyw@mail.gmail.com>
 <1485379578.602700.859643336.02F91305@webmail.messagingengine.com>
Message-ID: <CACac1F_0AkhKFvnVb46mQJYhfkshYdPWaWLHD58PrS4QnDH8JA@mail.gmail.com>

On 25 January 2017 at 21:26, Thomas Kluyver <thomas at kluyver.me.uk> wrote:
> On Wed, Jan 25, 2017, at 09:02 PM, Paul Moore wrote:
>> Thanks. I've been seeing the PR comments in passing, but haven't
>> really looked in detail. Is there anything I can do to help this
>> along?
>
> I believe it's working, but it's the first significant change I've made
> in pip, so any review is welcome.

Cool - I'll try to make some time, maybe over the weekend.
Paul

From brunson at brunson.com  Wed Jan 25 12:27:13 2017
From: brunson at brunson.com (Eric Brunson)
Date: Wed, 25 Jan 2017 17:27:13 +0000
Subject: [Distutils] latest setuptools appears to require six in a
 breaking way
In-Reply-To: <CANJQusXp_9wMDf_FyDubKQRsHZwgXy6L0+ipo+byrcSPLeuK0A@mail.gmail.com>
References: <77ef0649-1cfa-73fb-5043-98506df9ed41@simplistix.co.uk>
 <241b596e-0239-dd04-49c7-bca9cba7a773@chamonix.reportlab.co.uk>
 <CANJQusXp_9wMDf_FyDubKQRsHZwgXy6L0+ipo+byrcSPLeuK0A@mail.gmail.com>
Message-ID: <01000159d6aac819-bf81468c-b03e-48eb-8ebc-505913cbbfc8-000000@email.amazonses.com>

It wasn't until recently the I realized how quickly releases to setuptools
and pip are being made, starting back in mid Dec when much of our
dependency resolution started failing.  There were three releases in the
past two days.  Four major and 22 minor releases in the past two months.
While I applaud the speed of development and the improvement in these
tools, don't you feel that breaking changes should be advertised better
before release or perhaps we should slow down the cadence for release?

I think an expectation that every setuptools user in the community start
their day by checking to see if there was a release in the past 24 hours is
a little unreasonable.  I've spent a dozen hours since 32.0.0 resolving
breakage in my own projects and assisting other developers in my org with
their setuptools issues, all the while pushing setuptools as the best
practice to do development and distribution.  Is this period of breaking
changes a short term thing that we just have to tough out for a few more
weeks?

Thanks,
e.

On Wed, Jan 25, 2017 at 9:45 AM Matthias Bussonnier <
bussonniermatthias at gmail.com> wrote:

> AFAICT this is not really an issue as this is on the release notes:
>
> https://setuptools.readthedocs.io/en/latest/history.html#v34-0-0
>
> You need to upgrade pip  (cf 3rd paragraph):
>
> > #581: Instead of vendoring the growing list of dependencies that
> Setuptools requires to function, Setuptools now requires these dependencies
> just like any other project. Unlike other projects, however, Setuptools
> cannot rely on setup_requires to demand the dependencies it needs to
> install because its own machinery would be necessary to pull those
> dependencies if not present (a bootstrapping problem). As a result,
> Setuptools no longer supports self upgrade or installation in the general
> case. Instead, users are directed to use pip to install and upgrade using
> the wheel distributions of setuptools.
>
> > Users are welcome to contrive other means to install or upgrade
> Setuptools using other means, such as pre-installing the Setuptools
> dependencies with pip or a bespoke bootstrap tool, but such usage is not
> recommended and is not supported.
>
>  >As discovered in #940, not all versions of pip will successfully
> install Setuptools from its pre-built wheel. If you encounter issues
> with ?No module named six? or ?No module named packaging?, especially
> following a line ?Running setup.py egg_info for package setuptools?,
> then your pip is not new enough.
>
> > There?s an additional issue in pip where setuptools is upgraded
> concurrently with other source packages, described in pip #4253. The
> proposed workaround is to always upgrade Setuptools first prior to
> upgrading other packages that would upgrade Setuptools.
>
> --
> M
>
> On Wed, Jan 25, 2017 at 6:46 AM, Robin Becker <robin at reportlab.com> wrote:
> >
> > On 25/01/2017 10:41, Chris Withers wrote:
> >>
> >> Hi All,
> >>
> >> Anyone else seen this?
> >>
> >> |File
> >>
> >>
> "/home/travis/virtualenv/python2.6.9/lib/python2.6/site-packages/setuptools/__init__.py",
> >> line 10, in <module> from six.moves import filter, map ImportError: No
> >> module
> >> named six.moves|
> >>
> > My colleague just came across this when creating a new venv.
> >
> > We then installed six, but our setup then went on to complain about
> > packaging and then finally appdirs before it went through.
> >
> > Each time the fix was to manually install the thing that pip complained
> > about.
> >
> >
> >>
> >> Started happening in my nightly builds on travis after the 34.0.2
> release
> >> of
> >> setuptools. I've filed an issue here since I suspect it isn't just me:
> >>
> >> https://github.com/pypa/setuptools/issues/945
> >>
> >> cheers,
> >>
> >> Chris
> >>
> >>
> >>
> >> _______________________________________________
> >> Distutils-SIG maillist  -  Distutils-SIG at python.org
> >> https://mail.python.org/mailman/listinfo/distutils-sig
> >>
> >
> >
> > --
> > Robin Becker
> > _______________________________________________
> > Distutils-SIG maillist  -  Distutils-SIG at python.org
> > https://mail.python.org/mailman/listinfo/distutils-sig
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170125/46b5f2d8/attachment.html>

From donald at stufft.io  Wed Jan 25 19:52:13 2017
From: donald at stufft.io (Donald Stufft)
Date: Wed, 25 Jan 2017 19:52:13 -0500
Subject: [Distutils] latest setuptools appears to require six in a
 breaking way
In-Reply-To: <01000159d6aac819-bf81468c-b03e-48eb-8ebc-505913cbbfc8-000000@email.amazonses.com>
References: <77ef0649-1cfa-73fb-5043-98506df9ed41@simplistix.co.uk>
 <241b596e-0239-dd04-49c7-bca9cba7a773@chamonix.reportlab.co.uk>
 <CANJQusXp_9wMDf_FyDubKQRsHZwgXy6L0+ipo+byrcSPLeuK0A@mail.gmail.com>
 <01000159d6aac819-bf81468c-b03e-48eb-8ebc-505913cbbfc8-000000@email.amazonses.com>
Message-ID: <07C01F10-F638-4337-9397-D315A7432798@stufft.io>


> On Jan 25, 2017, at 12:27 PM, Eric Brunson <brunson at brunson.com> wrote:
> 
> It wasn't until recently the I realized how quickly releases to setuptools and pip are being made, starting back in mid Dec when much of our dependency resolution started failing.  There were three releases in the past two days.  Four major and 22 minor releases in the past two months.  While I applaud the speed of development and the improvement in these tools, don't you feel that breaking changes should be advertised better before release or perhaps we should slow down the cadence for release?
> 
> I think an expectation that every setuptools user in the community start their day by checking to see if there was a release in the past 24 hours is a little unreasonable.  I've spent a dozen hours since 32.0.0 resolving breakage in my own projects and assisting other developers in my org with their setuptools issues, all the while pushing setuptools as the best practice to do development and distribution.  Is this period of breaking changes a short term thing that we just have to tough out for a few more weeks?
> 
> Thanks,
> e.
> 


I don?t believe that pip is really releasing that quickly. We generally make 1-2 ?major? versions a year that include breaking changes, 2-4 ?minor? releases a year that add new features, and 6-10 patch releases that fix bugs. To me that feels like a pretty decent pace of balancing not breaking people and getting new changes into people?s hands and getting rid of broken or less optimal parts of the code.

Now, setuptools is releasing faster than pip is and whether that?s a good thing or not I don?t know. That?s a question for Jason largely :)


?
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170125/9d5fd07b/attachment-0001.html>

From matthew.brett at gmail.com  Wed Jan 25 20:26:38 2017
From: matthew.brett at gmail.com (Matthew Brett)
Date: Wed, 25 Jan 2017 17:26:38 -0800
Subject: [Distutils] latest setuptools appears to require six in a
 breaking way
In-Reply-To: <07C01F10-F638-4337-9397-D315A7432798@stufft.io>
References: <77ef0649-1cfa-73fb-5043-98506df9ed41@simplistix.co.uk>
 <241b596e-0239-dd04-49c7-bca9cba7a773@chamonix.reportlab.co.uk>
 <CANJQusXp_9wMDf_FyDubKQRsHZwgXy6L0+ipo+byrcSPLeuK0A@mail.gmail.com>
 <01000159d6aac819-bf81468c-b03e-48eb-8ebc-505913cbbfc8-000000@email.amazonses.com>
 <07C01F10-F638-4337-9397-D315A7432798@stufft.io>
Message-ID: <CAH6Pt5oK8yztHv5N-BpXm1Ky1-Tu5+JXHqpcoqevHG=PLY8WHg@mail.gmail.com>

Hi,

On Wed, Jan 25, 2017 at 4:52 PM, Donald Stufft <donald at stufft.io> wrote:
>
> On Jan 25, 2017, at 12:27 PM, Eric Brunson <brunson at brunson.com> wrote:
>
> It wasn't until recently the I realized how quickly releases to setuptools
> and pip are being made, starting back in mid Dec when much of our dependency
> resolution started failing.  There were three releases in the past two days.
> Four major and 22 minor releases in the past two months.  While I applaud
> the speed of development and the improvement in these tools, don't you feel
> that breaking changes should be advertised better before release or perhaps
> we should slow down the cadence for release?
>
> I think an expectation that every setuptools user in the community start
> their day by checking to see if there was a release in the past 24 hours is
> a little unreasonable.  I've spent a dozen hours since 32.0.0 resolving
> breakage in my own projects and assisting other developers in my org with
> their setuptools issues, all the while pushing setuptools as the best
> practice to do development and distribution.  Is this period of breaking
> changes a short term thing that we just have to tough out for a few more
> weeks?
>
> Thanks,
> e.
>
>
>
> I don?t believe that pip is really releasing that quickly. We generally make
> 1-2 ?major? versions a year that include breaking changes, 2-4 ?minor?
> releases a year that add new features, and 6-10 patch releases that fix
> bugs. To me that feels like a pretty decent pace of balancing not breaking
> people and getting new changes into people?s hands and getting rid of broken
> or less optimal parts of the code.
>
> Now, setuptools is releasing faster than pip is and whether that?s a good
> thing or not I don?t know. That?s a question for Jason largely :)

It sounds like we need to get some continuous integration to bear on
this problem.

How about the following suggestion: before new releases, make release
candidates for pip, wheel, setuptools and virtualenv, upload to pypi,
announce here.

Then we the dependent projects can have a continuous integration entry
which tests our normal pip install with the --pre versions of these
packages, to screen for trouble.   I think that would catch many of
the problems, and it doesn't seem like it would be much work for the
package maintainers.

Cheers,

Matthew

From donald at stufft.io  Fri Jan 27 14:12:43 2017
From: donald at stufft.io (Donald Stufft)
Date: Fri, 27 Jan 2017 14:12:43 -0500
Subject: [Distutils] latest setuptools appears to require six in a
 breaking way
In-Reply-To: <01000159e151a1c3-76931ef9-eb73-4226-8a8b-3502971aaeb4-000000@email.amazonses.com>
References: <77ef0649-1cfa-73fb-5043-98506df9ed41@simplistix.co.uk>
 <241b596e-0239-dd04-49c7-bca9cba7a773@chamonix.reportlab.co.uk>
 <CANJQusXp_9wMDf_FyDubKQRsHZwgXy6L0+ipo+byrcSPLeuK0A@mail.gmail.com>
 <01000159d6aac819-bf81468c-b03e-48eb-8ebc-505913cbbfc8-000000@email.amazonses.com>
 <07C01F10-F638-4337-9397-D315A7432798@stufft.io>
 <01000159e151a1c3-76931ef9-eb73-4226-8a8b-3502971aaeb4-000000@email.amazonses.com>
Message-ID: <223F58CC-2B5B-4272-9E53-1761FB9BA734@stufft.io>


> On Jan 27, 2017, at 2:05 PM, Eric Brunson <brunson at brunson.com> wrote:
> 
> On Wed, Jan 25, 2017 at 5:52 PM Donald Stufft <donald at stufft.io <mailto:donald at stufft.io>> wrote:
> 
>> On Jan 25, 2017, at 12:27 PM, Eric Brunson <brunson at brunson.com <mailto:brunson at brunson.com>> wrote:
>> 
>> It wasn't until recently the I realized how quickly releases to setuptools and pip are being made, starting back in mid Dec when much of our dependency resolution started failing.  There were three releases in the past two days.  Four major and 22 minor releases in the past two months.  While I applaud the speed of development and the improvement in these tools, don't you feel that breaking changes should be advertised better before release or perhaps we should slow down the cadence for release?
>> 
>> I think an expectation that every setuptools user in the community start their day by checking to see if there was a release in the past 24 hours is a little unreasonable.  I've spent a dozen hours since 32.0.0 resolving breakage in my own projects and assisting other developers in my org with their setuptools issues, all the while pushing setuptools as the best practice to do development and distribution.  Is this period of breaking changes a short term thing that we just have to tough out for a few more weeks?
>> 
>> Thanks,
>> e.
>> 
> 
> 
> I don?t believe that pip is really releasing that quickly. We generally make 1-2 ?major? versions a year that include breaking changes, 2-4 ?minor? releases a year that add new features, and 6-10 patch releases that fix bugs. To me that feels like a pretty decent pace of balancing not breaking people and getting new changes into people?s hands and getting rid of broken or less optimal parts of the code.
> 
> Now, setuptools is releasing faster than pip is and whether that?s a good thing or not I don?t know. That?s a question for Jason largely :)
> 
> Hey Donald,
> 
> Thanks for the reply.
> 
> Doesn't pip rely heavily on setuptools?  I understand they have different origins, but I thought that since pip was moved under the purview of PYPA a lot of work was being done to converge the projects. When I run a pip -e one of the last message I see is "running setuptools.py develop", which isn't really a dependency, but can certainly cause people to infer that the problem is with pip and not know setuptools. Even the release notes Matthias references mentions pip as though it might be affected.
> 
> If pip doesn't rely on setuptools, does that mean we have two separate and possibly different dependency resolution algorithms?
> 
> Sincerely,
> e.


The answer here is? complex.

Pip itself does not depend on setuptools though we _do_ depend on pkg_resources, but like all of our dependencies we bundle a copy internally and use that. Our uses of pkg_resources is primarily to inspect the currently installed packages to see what versions of what have been installed.

If you?re installing from a Wheel file with pip, then you do not need to have setuptools installed at all (although if the wheel depends on setuptools we will obviously install it then).

If you?re installing from a sdist, *then* you need setuptools installed because we?re going to execute the ``setup.py`` that is inside of that sdist and it will use setuptools (and we do some gross hacks to ensure it *always* uses setuptools, even if the project imports distutils).


We are careful to pass in to setuptools the correct arguments that disables setuptools own handling of dependencies and we implement our own resolver. In the cases setuptools is being used (besides our bundled pkg_resources) it is primarily being used as a build tool. That being said, setuptools currently handles setup_requires on it?s own so the setuptools resolver handles that (though PEP 518 provides a way that in the future pip will use so that pip can handle build requirements instead of needing setuptools to do it). There is also a little known ``?egg`` parameter to pip that tells pip to stop preventing setuptools from doing it?s own resolving? but that is deprecated and is either removed or will be removed in the next version.

So yes, they do have two separate and different dependency resolution algorithms (for instance, pip won?t install pre-releases by default unless it?s all that is available while setuptools will) but when you?re using ``pip install`` the setuptools one is carefully disabled *except* for setup_requires.

?
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170127/5b7c529c/attachment.html>

From brunson at brunson.com  Fri Jan 27 14:05:40 2017
From: brunson at brunson.com (Eric Brunson)
Date: Fri, 27 Jan 2017 19:05:40 +0000
Subject: [Distutils] latest setuptools appears to require six in a
 breaking way
In-Reply-To: <07C01F10-F638-4337-9397-D315A7432798@stufft.io>
References: <77ef0649-1cfa-73fb-5043-98506df9ed41@simplistix.co.uk>
 <241b596e-0239-dd04-49c7-bca9cba7a773@chamonix.reportlab.co.uk>
 <CANJQusXp_9wMDf_FyDubKQRsHZwgXy6L0+ipo+byrcSPLeuK0A@mail.gmail.com>
 <01000159d6aac819-bf81468c-b03e-48eb-8ebc-505913cbbfc8-000000@email.amazonses.com>
 <07C01F10-F638-4337-9397-D315A7432798@stufft.io>
Message-ID: <01000159e151a1e2-91b6d4d1-8ccc-4740-a2a8-38d7c363a92b-000000@email.amazonses.com>

On Wed, Jan 25, 2017 at 5:52 PM Donald Stufft <donald at stufft.io> wrote:


On Jan 25, 2017, at 12:27 PM, Eric Brunson <brunson at brunson.com> wrote:

It wasn't until recently the I realized how quickly releases to setuptools
and pip are being made, starting back in mid Dec when much of our
dependency resolution started failing.  There were three releases in the
past two days.  Four major and 22 minor releases in the past two months.
While I applaud the speed of development and the improvement in these
tools, don't you feel that breaking changes should be advertised better
before release or perhaps we should slow down the cadence for release?

I think an expectation that every setuptools user in the community start
their day by checking to see if there was a release in the past 24 hours is
a little unreasonable.  I've spent a dozen hours since 32.0.0 resolving
breakage in my own projects and assisting other developers in my org with
their setuptools issues, all the while pushing setuptools as the best
practice to do development and distribution.  Is this period of breaking
changes a short term thing that we just have to tough out for a few more
weeks?

Thanks,
e.



I don?t believe that pip is really releasing that quickly. We generally
make 1-2 ?major? versions a year that include breaking changes, 2-4 ?minor?
releases a year that add new features, and 6-10 patch releases that fix
bugs. To me that feels like a pretty decent pace of balancing not breaking
people and getting new changes into people?s hands and getting rid of
broken or less optimal parts of the code.

Now, setuptools is releasing faster than pip is and whether that?s a good
thing or not I don?t know. That?s a question for Jason largely :)


Hey Donald,

Thanks for the reply.

Doesn't pip rely heavily on setuptools?  I understand they have different
origins, but I thought that since pip was moved under the purview of PYPA a
lot of work was being done to converge the projects. When I run a pip -e
one of the last message I see is "running setuptools.py develop", which
isn't really a dependency, but can certainly cause people to infer that the
problem is with pip and not know setuptools. Even the release notes
Matthias references mentions pip as though it might be affected.

If pip doesn't rely on setuptools, does that mean we have two separate and
possibly different dependency resolution algorithms?

Sincerely,
e.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170127/ac3ecc95/attachment-0001.html>

From brunson at brunson.com  Fri Jan 27 14:54:58 2017
From: brunson at brunson.com (Eric Brunson)
Date: Fri, 27 Jan 2017 19:54:58 +0000
Subject: [Distutils] latest setuptools appears to require six in a
 breaking way
In-Reply-To: <01000159e151a1e2-91b6d4d1-8ccc-4740-a2a8-38d7c363a92b-000000@email.amazonses.com>
References: <77ef0649-1cfa-73fb-5043-98506df9ed41@simplistix.co.uk>
 <241b596e-0239-dd04-49c7-bca9cba7a773@chamonix.reportlab.co.uk>
 <CANJQusXp_9wMDf_FyDubKQRsHZwgXy6L0+ipo+byrcSPLeuK0A@mail.gmail.com>
 <01000159d6aac819-bf81468c-b03e-48eb-8ebc-505913cbbfc8-000000@email.amazonses.com>
 <07C01F10-F638-4337-9397-D315A7432798@stufft.io>
 <01000159e151a1e2-91b6d4d1-8ccc-4740-a2a8-38d7c363a92b-000000@email.amazonses.com>
Message-ID: <01000159e17ec33c-f5ae4b8f-ddf3-4ce1-90d7-6bfc959d3163-000000@email.amazonses.com>

Donald,

Thank you so much for taking the time to explain the situation so
thoroughly.  I guess it's pretty easy to get confused by where the tools
overlap and where they are disjoint.  I think I understand better.

To be sure, my original email was focused on the setuptools instability
we've seen in the past 6 to 8 weeks, as we rely on that for our CI and
deployment automation.  I hope is a growing pain and things will settle
down in the near term.  I know there are big changes going on and these
sorts of things can't always be painless.

Sincerely,
e.

On Fri, Jan 27, 2017 at 12:14 PM Eric Brunson <brunson at brunson.com> wrote:

> On Wed, Jan 25, 2017 at 5:52 PM Donald Stufft <donald at stufft.io> wrote:
>
>
> On Jan 25, 2017, at 12:27 PM, Eric Brunson <brunson at brunson.com> wrote:
>
> It wasn't until recently the I realized how quickly releases to setuptools
> and pip are being made, starting back in mid Dec when much of our
> dependency resolution started failing.  There were three releases in the
> past two days.  Four major and 22 minor releases in the past two months.
> While I applaud the speed of development and the improvement in these
> tools, don't you feel that breaking changes should be advertised better
> before release or perhaps we should slow down the cadence for release?
>
> I think an expectation that every setuptools user in the community start
> their day by checking to see if there was a release in the past 24 hours is
> a little unreasonable.  I've spent a dozen hours since 32.0.0 resolving
> breakage in my own projects and assisting other developers in my org with
> their setuptools issues, all the while pushing setuptools as the best
> practice to do development and distribution.  Is this period of breaking
> changes a short term thing that we just have to tough out for a few more
> weeks?
>
> Thanks,
> e.
>
>
>
> I don?t believe that pip is really releasing that quickly. We generally
> make 1-2 ?major? versions a year that include breaking changes, 2-4 ?minor?
> releases a year that add new features, and 6-10 patch releases that fix
> bugs. To me that feels like a pretty decent pace of balancing not breaking
> people and getting new changes into people?s hands and getting rid of
> broken or less optimal parts of the code.
>
> Now, setuptools is releasing faster than pip is and whether that?s a good
> thing or not I don?t know. That?s a question for Jason largely :)
>
>
> Hey Donald,
>
> Thanks for the reply.
>
> Doesn't pip rely heavily on setuptools?  I understand they have different
> origins, but I thought that since pip was moved under the purview of PYPA a
> lot of work was being done to converge the projects. When I run a pip -e
> one of the last message I see is "running setuptools.py develop", which
> isn't really a dependency, but can certainly cause people to infer that the
> problem is with pip and not know setuptools. Even the release notes
> Matthias references mentions pip as though it might be affected.
>
> If pip doesn't rely on setuptools, does that mean we have two separate and
> possibly different dependency resolution algorithms?
>
> Sincerely,
> e.
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170127/9aed5ff2/attachment.html>

From ben+python at benfinney.id.au  Sun Jan 29 20:19:23 2017
From: ben+python at benfinney.id.au (Ben Finney)
Date: Mon, 30 Jan 2017 12:19:23 +1100
Subject: [Distutils] =?utf-8?q?Dependency_on_=E2=80=98pkg=5Fresources?=
 =?utf-8?q?=E2=80=99_only_=28not_Setuptools=29_=28was=3A_latest_setuptools?=
 =?utf-8?q?_appears_to_require_six_in_a_breaking_way=29?=
References: <77ef0649-1cfa-73fb-5043-98506df9ed41@simplistix.co.uk>
 <241b596e-0239-dd04-49c7-bca9cba7a773@chamonix.reportlab.co.uk>
 <CANJQusXp_9wMDf_FyDubKQRsHZwgXy6L0+ipo+byrcSPLeuK0A@mail.gmail.com>
 <01000159d6aac819-bf81468c-b03e-48eb-8ebc-505913cbbfc8-000000@email.amazonses.com>
 <07C01F10-F638-4337-9397-D315A7432798@stufft.io>
 <01000159e151a1c3-76931ef9-eb73-4226-8a8b-3502971aaeb4-000000@email.amazonses.com>
 <223F58CC-2B5B-4272-9E53-1761FB9BA734@stufft.io>
Message-ID: <854m0huyxw.fsf_-_@benfinney.id.au>

Donald Stufft <donald at stufft.io> writes:

> Pip itself does not depend on setuptools though we _do_ depend on
> pkg_resources [?]

For those confused about what that means: The ?pkg_resources? package is
separate once installed, but is currently only distributed as part of
Setuptools.

So it's reasonable to declare a dependency on ?pkg_resources?, yet that
dependency cannot be satisfied by Pip without installing the entire
Setuptools distribution.

The issue <URL:https://github.com/pypa/setuptools/issues/863> requests
this to be fixed, but AFAIK no changes have been implemented yet.

-- 
 \         ?I think Western civilization is more enlightened precisely |
  `\     because we have learned how to ignore our religious leaders.? |
_o__)                                                ?Bill Maher, 2003 |
Ben Finney


From mikejyt33 at gmail.com  Tue Jan 31 17:05:09 2017
From: mikejyt33 at gmail.com (Michael Strain)
Date: Tue, 31 Jan 2017 16:05:09 -0600
Subject: [Distutils] install questions and help requested. ---pyautogui
Message-ID: <CAP_HtJJQewDZRUEm0rNuC2aQ_v1yV4BcjObtT0cQXk9O0MuXPw@mail.gmail.com>

I am looking for a simple short cut where I can automate a few clicks with
a button via python automate the boring stuff. I go through data then I am
seeking a short cut for the click inorder to be time efficient.

 I am having some errors. I am new to python.


I am using windows.

I have read that I have to run 'pip install pyautogui'

I have used windows cmd. python shell and python command. I get a syntax
error from python and a nothing from windows. Where should I put this? I
found the pyautogui program and downloaded it and unzipped it.


I try to jump straight to the import script of pyautogui like

SyntaxError: multiple statements found while compiling a single statement

Traceback (most recent call last):
  File "<pyshell#1>", line 1, in <module>
    import pyautogui
ModuleNotFoundError: No module named 'pyautogui'

 >>> pip install pyautogui
SyntaxError: invalid syntax


Sincerely,
Michael G. Strain  Jr.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170131/baf62cc9/attachment.html>