Brett suggested I ask the kind folks here. As you may or may not know, there's an old unmaintained "mypy" package on PyPI that attracts a fair amount of downloads from people trying to download mypy the type checker. We then get bug reports and have to explain in our tracker that they have to use "pip install mypy-lang" instead. Query: https://github.com/python/mypy/issues?utf8=%E2%9C%93&q=is%3Aissue++dbutils+ That mypy package was last updated in 2011, and it's a quite forgettable combination of copied open-source packages and a little bit of glue code presumably written by the package author. Both Donald and myself have approached the owner (zsp007@gmail.com) but not received any response. Is there a "higher authority" to whom we can appeal this, or are we just stuck with this situation? As Brett wrote part of the problem, though, is the mypy project has 2244 downloads in the last month which shows it's being used and we don't want to end up in an npm/left_pad situation. (But how many of those downloads are misguided attempts to install mypy-lang?) One possibility, if people aren't happy with me or Jukka taking over owhership of the old mypy package, would be for someone (not me or Jukka) to take ownership of that package just so they can update the PyPI home page for that package with a prominent note telling people looking for Jukka's type checker to use mypy-lang instead. -- --Guido van Rossum (python.org/~guido)
I've fallen into this trap as well, so +1 for the takeover. It might be a good idea to come up with a standardized process for taking over old, unmaintained packages. 16.04.2016, 02:29, Guido van Rossum kirjoitti:
Brett suggested I ask the kind folks here.
As you may or may not know, there's an old unmaintained "mypy" package on PyPI that attracts a fair amount of downloads from people trying to download mypy the type checker. We then get bug reports and have to explain in our tracker that they have to use "pip install mypy-lang" instead.
Query: https://github.com/python/mypy/issues?utf8=%E2%9C%93&q=is%3Aissue++dbutils+
That mypy package was last updated in 2011, and it's a quite forgettable combination of copied open-source packages and a little bit of glue code presumably written by the package author. Both Donald and myself have approached the owner (zsp007@gmail.com <mailto:zsp007@gmail.com>) but not received any response. Is there a "higher authority" to whom we can appeal this, or are we just stuck with this situation?
As Brett wrote part of the problem, though, is the mypy project has 2244 downloads in the last month which shows it's being used and we don't want to end up in an npm/left_pad situation. (But how many of those downloads are misguided attempts to install mypy-lang?)
One possibility, if people aren't happy with me or Jukka taking over owhership of the old mypy package, would be for someone (not me or Jukka) to take ownership of that package just so they can update the PyPI home page for that package with a prominent note telling people looking for Jukka's type checker to use mypy-lang instead.
-- --Guido van Rossum (python.org/~guido <http://python.org/%7Eguido>)
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Hi Guido, Because this sort of thing has come up a lot in the past, and because I've copped trouble for mishandling it in the past, I took the trouble of writing up a formal description of how I handle these sorts of issues: https://docs.google.com/document/d/1elum7ENjQb0dLB4ATfYNtnXYVLUzsKacc0VWnHHJ... I believe Donald follows the same, or a very similar procedure. In short, all that can be done has been done, from my perspective. Someone has published their module, and regardless of any opinion of it, or desire to also use that name, I have to respect that they published first. In the absence of explicit consent from them to do anything, my hands are tied. I've taken unilateral action in the past to my personal detriment. Of course, once I'm no longer a PyPI admin (I look forward to the day so very much) someone else will have to make these decisions. Richard On 16 April 2016 at 09:29, Guido van Rossum <guido@python.org> wrote:
Brett suggested I ask the kind folks here.
As you may or may not know, there's an old unmaintained "mypy" package on PyPI that attracts a fair amount of downloads from people trying to download mypy the type checker. We then get bug reports and have to explain in our tracker that they have to use "pip install mypy-lang" instead.
Query: https://github.com/python/mypy/issues?utf8=%E2%9C%93&q=is%3Aissue++dbutils+
That mypy package was last updated in 2011, and it's a quite forgettable combination of copied open-source packages and a little bit of glue code presumably written by the package author. Both Donald and myself have approached the owner (zsp007@gmail.com) but not received any response. Is there a "higher authority" to whom we can appeal this, or are we just stuck with this situation?
As Brett wrote part of the problem, though, is the mypy project has 2244 downloads in the last month which shows it's being used and we don't want to end up in an npm/left_pad situation. (But how many of those downloads are misguided attempts to install mypy-lang?)
One possibility, if people aren't happy with me or Jukka taking over owhership of the old mypy package, would be for someone (not me or Jukka) to take ownership of that package just so they can update the PyPI home page for that package with a prominent note telling people looking for Jukka's type checker to use mypy-lang instead.
-- --Guido van Rossum (python.org/~guido)
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Why are you so determined to use the name "mypy" in the first place? Seems to me it's a terrible name. It sounds more like a working title for someone's personal implementation of Python, and certainly gives no clue that it has anything to do with type checking. So instead of trying to steal the name "mypy", how about coming up with a new, more appropriate name? -- Greg
Oh well. I wonder if offering money would change the situation. On Fri, Apr 15, 2016 at 5:10 PM, Richard Jones <richard@python.org> wrote:
Hi Guido,
Because this sort of thing has come up a lot in the past, and because I've copped trouble for mishandling it in the past, I took the trouble of writing up a formal description of how I handle these sorts of issues:
https://docs.google.com/document/d/1elum7ENjQb0dLB4ATfYNtnXYVLUzsKacc0VWnHHJ...
I believe Donald follows the same, or a very similar procedure.
In short, all that can be done has been done, from my perspective. Someone has published their module, and regardless of any opinion of it, or desire to also use that name, I have to respect that they published first. In the absence of explicit consent from them to do anything, my hands are tied. I've taken unilateral action in the past to my personal detriment.
Of course, once I'm no longer a PyPI admin (I look forward to the day so very much) someone else will have to make these decisions.
Richard
On 16 April 2016 at 09:29, Guido van Rossum <guido@python.org> wrote:
Brett suggested I ask the kind folks here.
As you may or may not know, there's an old unmaintained "mypy" package on PyPI that attracts a fair amount of downloads from people trying to download mypy the type checker. We then get bug reports and have to explain in our tracker that they have to use "pip install mypy-lang" instead.
Query: https://github.com/python/mypy/issues?utf8=%E2%9C%93&q=is%3Aissue++dbutils+
That mypy package was last updated in 2011, and it's a quite forgettable combination of copied open-source packages and a little bit of glue code presumably written by the package author. Both Donald and myself have approached the owner (zsp007@gmail.com) but not received any response. Is there a "higher authority" to whom we can appeal this, or are we just stuck with this situation?
As Brett wrote part of the problem, though, is the mypy project has 2244 downloads in the last month which shows it's being used and we don't want to end up in an npm/left_pad situation. (But how many of those downloads are misguided attempts to install mypy-lang?)
One possibility, if people aren't happy with me or Jukka taking over owhership of the old mypy package, would be for someone (not me or Jukka) to take ownership of that package just so they can update the PyPI home page for that package with a prominent note telling people looking for Jukka's type checker to use mypy-lang instead.
-- --Guido van Rossum (python.org/~guido)
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
-- --Guido van Rossum (python.org/~guido)
Another thing. Is the search index on pypi.python.org no longer being updated? Searching for mypy-lang still takes you to mypy-lang 0.2.0, even though 0.3.1 has been released many weeks ago. I just updated the typing package and searching for that still shows the old versions. On Fri, Apr 15, 2016 at 6:10 PM, Guido van Rossum <guido@python.org> wrote:
Oh well. I wonder if offering money would change the situation.
On Fri, Apr 15, 2016 at 5:10 PM, Richard Jones <richard@python.org> wrote:
Hi Guido,
Because this sort of thing has come up a lot in the past, and because I've copped trouble for mishandling it in the past, I took the trouble of writing up a formal description of how I handle these sorts of issues:
https://docs.google.com/document/d/1elum7ENjQb0dLB4ATfYNtnXYVLUzsKacc0VWnHHJ...
I believe Donald follows the same, or a very similar procedure.
In short, all that can be done has been done, from my perspective. Someone has published their module, and regardless of any opinion of it, or desire to also use that name, I have to respect that they published first. In the absence of explicit consent from them to do anything, my hands are tied. I've taken unilateral action in the past to my personal detriment.
Of course, once I'm no longer a PyPI admin (I look forward to the day so very much) someone else will have to make these decisions.
Richard
On 16 April 2016 at 09:29, Guido van Rossum <guido@python.org> wrote:
Brett suggested I ask the kind folks here.
As you may or may not know, there's an old unmaintained "mypy" package on PyPI that attracts a fair amount of downloads from people trying to download mypy the type checker. We then get bug reports and have to explain in our tracker that they have to use "pip install mypy-lang" instead.
Query: https://github.com/python/mypy/issues?utf8=%E2%9C%93&q=is%3Aissue++dbutils+
That mypy package was last updated in 2011, and it's a quite forgettable combination of copied open-source packages and a little bit of glue code presumably written by the package author. Both Donald and myself have approached the owner (zsp007@gmail.com) but not received any response. Is there a "higher authority" to whom we can appeal this, or are we just stuck with this situation?
As Brett wrote part of the problem, though, is the mypy project has 2244 downloads in the last month which shows it's being used and we don't want to end up in an npm/left_pad situation. (But how many of those downloads are misguided attempts to install mypy-lang?)
One possibility, if people aren't happy with me or Jukka taking over owhership of the old mypy package, would be for someone (not me or Jukka) to take ownership of that package just so they can update the PyPI home page for that package with a prominent note telling people looking for Jukka's type checker to use mypy-lang instead.
-- --Guido van Rossum (python.org/~guido)
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
-- --Guido van Rossum (python.org/~guido)
-- --Guido van Rossum (python.org/~guido)
On Fri, Apr 15, 2016 at 8:59 PM, Guido van Rossum <guido@python.org> wrote:
Another thing. Is the search index on pypi.python.org no longer being updated? Searching for mypy-lang still takes you to mypy-lang 0.2.0, even though 0.3.1 has been released many weeks ago. I just updated the typing package and searching for that still shows the old versions.
Yes, search and download statistics have both been broken on pypi for a while. Unfortunately it seems that everyone who's in a position to figure out why or fix it is already scrambling to fight other equally high priority fires. There was some more discussion in this thread: https://mail.python.org/pipermail/distutils-sig/2016-March/028464.html -n -- Nathaniel J. Smith -- https://vorpus.org
On 16 April 2016 at 01:10, Richard Jones <richard@python.org> wrote:
Because this sort of thing has come up a lot in the past, and because I've copped trouble for mishandling it in the past, I took the trouble of writing up a formal description of how I handle these sorts of issues:
https://docs.google.com/document/d/1elum7ENjQb0dLB4ATfYNtnXYVLUzsKacc0VWnHHJ...
I believe Donald follows the same, or a very similar procedure.
In short, all that can be done has been done, from my perspective. Someone has published their module, and regardless of any opinion of it, or desire to also use that name, I have to respect that they published first. In the absence of explicit consent from them to do anything, my hands are tied. I've taken unilateral action in the past to my personal detriment.
Of course, once I'm no longer a PyPI admin (I look forward to the day so very much) someone else will have to make these decisions.
I can understand your reasons for not wanting to take unilateral action. I wonder, however, whether it would be reasonable to add an explicit policy to PyPI (probably at the point of the switch to Warehouse) requiring project owners to provide an active email address (where "active" means, say, responding to an annual automated ping email to confirm the project is still alive). There would obviously have to be some sort of "legacy" exemption from this, and maybe a transition process. It still wouldn't help if the author doesn't respond to requests like Guido's, but at least it avoids the possibility that the actual email address is dead. Other than this, I don't think there's much that can be done here. The "first come, first serve" nature of claiming names on PyPI is pretty much part of the culture. Changing that would be a pretty hard sell (as well as being a ton of work that no-one is likely to want to do...) Paul
To what end? As much as old packages cluttering the namespace of pypi is annoying, the only thing that will accomplish is orphaning projects. We cant unilaterally hand over names on pypi to unrelated.. or even related.. projects because they have a name someone else wants. Another solution like adding namespaces to pypi sounds better to me... but then I think about the nightmare of implementing that in a backwards compatible way. On 4/16/2016 04:36, Paul Moore wrote:
I wonder, however, whether it would be reasonable to add an explicit policy to PyPI (probably at the point of the switch to Warehouse) requiring project owners to provide an active email address (where "active" means, say, responding to an annual automated ping email to confirm the project is still alive).
On Apr 16, 2016, at 12:42 PM, Alexander Walters <tritium-list@sdamon.com> wrote:
Another solution like adding namespaces to pypi sounds better to me... but then I think about the nightmare of implementing that in a backwards compatible way.
We already have namespacing sort of, people can make something like dstufft.mypy and that works fine. The main thing that doesn’t exist is there’s no way to claim an entire namespace to prevent someone else from taking something in your namespace. However, I don’t think most people really want that, people like having shorter names that aren’t tied to a specific namespace generally. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On 4/16/2016 12:53, Donald Stufft wrote:
On Apr 16, 2016, at 12:42 PM, Alexander Walters <tritium-list@sdamon.com> wrote:
Another solution like adding namespaces to pypi sounds better to me... but then I think about the nightmare of implementing that in a backwards compatible way. We already have namespacing sort of, people can make something like dstufft.mypy and that works fine. The main thing that doesn’t exist is there’s no way to claim an entire namespace to prevent someone else from taking something in your namespace.
However, I don’t think most people really want that, people like having shorter names that aren’t tied to a specific namespace generally.
----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
My thought when I made that suggestion (which I already dismissed as a nightmare for you brave souls who provide the index free of charge... thank you) was less in the realm of 'tritium.mypackage', where a user would claim their package namespace for all their code, but more along the lines of 'django/my.super.cool.app' or 'flask/my_extension' or 'qa/mypy' and 'qa/flake8' - curated (or open...) namespaces of related packages.
On 16 April 2016 at 17:42, Alexander Walters <tritium-list@sdamon.com> wrote:
To what end?
To the end of ensuring that people can get in touch with project owners.
As much as old packages cluttering the namespace of pypi is annoying, the only thing that will accomplish is orphaning projects.
Not at all. The projects that would be affected by this are those that are already orphaned, in the sense that there's no means to contact the owner.
We cant unilaterally hand over names on pypi to unrelated.. or even related.. projects because they have a name someone else wants.
I'm not proposing anything like that - I guess when I said "requiring project owners to provide an email address..." it read like I was suggesting removing stuff if the owner went away. I hadn't really thought much about what we *do* with projects where the contact email becomes unmonitored (I *so* think it's acceptable to require a working contact email when a project is registered, or maybe when a user registers
Sorry, please ignore that email. I hit "Send" too soon.
We cant unilaterally hand over names on pypi to unrelated.. or even related.. projects because they have a name someone else wants.
What I *meant* to say here was that when a request for a name transfer gets no reply, it's helpful to know if the email address is no longer responding at all. Nothing more than that. That might help anyone making a decision to decide what to do. But no, I agree absolutely we can't just hand over names. Paul On 16 April 2016 at 19:48, Paul Moore <p.f.moore@gmail.com> wrote:
On 16 April 2016 at 17:42, Alexander Walters <tritium-list@sdamon.com> wrote:
To what end?
To the end of ensuring that people can get in touch with project owners.
As much as old packages cluttering the namespace of pypi is annoying, the only thing that will accomplish is orphaning projects.
Not at all. The projects that would be affected by this are those that are already orphaned, in the sense that there's no means to contact the owner.
We cant unilaterally hand over names on pypi to unrelated.. or even related.. projects because they have a name someone else wants.
I'm not proposing anything like that - I guess when I said "requiring project owners to provide an email address..." it read like I was suggesting removing stuff if the owner went away. I hadn't really thought much about what we *do* with projects where the contact email becomes unmonitored (I *so* think it's acceptable to require a working contact email when a project is registered, or maybe when a user registers
We
cant unilaterally hand over names on pypi to unrelated.. or even related.. projects because they have a name someone else wants. What I*meant* to say here was that when a request for a name transfer gets no reply, it's helpful to know if the email address is no longer responding at all. Nothing more than that. That might help anyone making a decision to decide what to do.
But no, I agree absolutely we can't just hand over names.
Paul If what you intend is to just flag the index entries where the owner has a bouncing email as "Heads up, the owner's email doesn't work anymore",
On 4/16/2016 14:52, Paul Moore wrote: then I don't have any problems with that. Though I do wonder how effective that would be in this case. For all we know, in the case of mypy, the maintainer is simply ignoring someone else who is trying to take the name they registered. (I get emails all the time for people trying to get me to sign over my domain names; even though I am not doing much with them, they are my names and my identity, so that is one possible reason.)
On Sat, Apr 16, 2016 at 2:29 AM, Guido van Rossum <guido@python.org> wrote:
Both Donald and myself have approached the owner (zsp007@gmail.com) but not received any response.
Have you considered getting someone to write an email in Chinese? I suspect he did not understand what was asked in Enligsh. Thanks, -- Ionel Cristian Mărieș, http://blog.ionelmc.ro
Hm, interesting idea. I probably know someone at work who can help me translate. In a similar vein, the package distributor is listed as "zuroc" -- would this be someone else or just an alias for the owner? On Sat, Apr 16, 2016 at 11:38 PM, Ionel Cristian Mărieș <contact@ionelmc.ro> wrote:
On Sat, Apr 16, 2016 at 2:29 AM, Guido van Rossum <guido@python.org> wrote:
Both Donald and myself have approached the owner (zsp007@gmail.com) but not received any response.
Have you considered getting someone to write an email in Chinese? I suspect he did not understand what was asked in Enligsh.
Thanks, -- Ionel Cristian Mărieș, http://blog.ionelmc.ro
-- --Guido van Rossum (python.org/~guido)
The idea of expiring out names has been brought up recently to resolve an issue of two packages, one popular and large; another someone's weekend project. The general idea being that a project maintainer should be forced to renew their contact information, or face the possibility of the PyPI name they registered being de-registered and made available for another package to use. Preamble done, let me enumerate why this is just a disaster: 1. PyYAML is a package that would be de-registered in such a scheme. It is a highly used, extremely popular, package that unserializes text into arbitrary python objects. It is a trusted package... and one that hasn't been active in ages. This is prime malware bait. 2. the package tooling already assumes that names will always point to one, and only one package. ever. until the heat death of the universe or the death of the language whichever is first. If I am the one person in the world who actually depends on the 'mypy' (not mypy-lang) package, you have broken that trust. 3. Who in the PSF really wants that bureaucratic nightmare of arbitrating cases when this inevitably messes up, be this system manual or automatic? To the specifics of the mypy-lang package that brought this up... It's like naming your company "Yahoo", and getting upset that yahoo.com is getting a bump in traffic because of your popularity. It is unfortunate that the mypy-lang developers failed to check pypi for name availability before they named their package, but it is by no means a reason to invite malicious code into the index, break the trust of the tooling, or create a bureaucracy to manage when the first two happen.
You've made a strong case, and this is probably not where PyPi should go -- but it would hardly be a disaster:
The idea of expiring out names has been brought up recently to resolve an issue of two packages, one popular and large; another someone's weekend project.
The issue here is not that it's a weekend project, but that it may be an abandoned project. I don't think anyone suggest that we should have a popularity or quality test to see who gets to trump whom with name allocation -- I sure didn't. Which is quite relevant to below:
1. PyYAML is a package that would be de-registered in such a scheme. It is a highly used, extremely popular, package that unserializes text into arbitrary python objects. It is a trusted package... and one that hasn't been active in ages.
and you don't think ANYONE would be willing to take on the miniscule amount of work to maintain the name? Plus there would be any number of other schemes for determining whether a project name is abandoned.
2. the package tooling already assumes that names will always point to one, and only one package. ever. until the heat death of the universe or the death of the language whichever is first.
IIUC, the current scheme allows for a name to be "taken over" by a new package if the original author so desires -- i.e. if the current owner of the mypy name was happy to relinquish it, then "pip install mypy" would get users something totally different 6 months from now. So no -- we don't currently guarantee anything about future use of names. Other that that the original author can do whatever they want with it. 3. Who in the PSF really wants that bureaucratic nightmare of arbitrating
cases when this inevitably messes up, be this system manual or automatic?
I think bureaucratic nightmare is pretty hyperbolic, but yes, there would be some overhead, for sure. And given that no one has the time and motivation to even maintain PyPi at this point -- this will probably kill the idea altogether. To the specifics of the mypy-lang package that brought this up... It's like
naming your company "Yahoo", and getting upset that yahoo.com is getting a bump in traffic because of your popularity.
Again - this is not about minor weekend projects not be "important". It's about potential abandonware -- with domain registration, "Yahoo" can offer to buy the domain from the current holder, or, if the current owner has abandoned it, it will go into the public pool again when they stop paying to maintain the registration. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Mon, Apr 18, 2016 at 4:16 PM, Chris Barker <chris.barker@noaa.gov> wrote:
You've made a strong case, and this is probably not where PyPi should go -- but it would hardly be a disaster:
The idea of expiring out names has been brought up recently to resolve an issue of two packages, one popular and large; another someone's weekend project.
The issue here is not that it's a weekend project, but that it may be an abandoned project. I don't think anyone suggest that we should have a popularity or quality test to see who gets to trump whom with name allocation -- I sure didn't.
Which is quite relevant to below:
1. PyYAML is a package that would be de-registered in such a scheme. It is a highly used, extremely popular, package that unserializes text into arbitrary python objects. It is a trusted package... and one that hasn't been active in ages.
and you don't think ANYONE would be willing to take on the miniscule amount of work to maintain the name? Plus there would be any number of other schemes for determining whether a project name is abandoned.
I have in fact offered but the author refuses to accept help from anyone. They're also the author of the C library (libyaml) and they do not maintain that either. It's actually quite frustrating as someone who wants to fix some of the numerous bugs in the library + improve it and add support for YAML 1.2 which is years old at this point.
2. the package tooling already assumes that names will always point to one, and only one package. ever. until the heat death of the universe or the death of the language whichever is first.
IIUC, the current scheme allows for a name to be "taken over" by a new package if the original author so desires -- i.e. if the current owner of the mypy name was happy to relinquish it, then "pip install mypy" would get users something totally different 6 months from now. So no -- we don't currently guarantee anything about future use of names. Other that that the original author can do whatever they want with it.
3. Who in the PSF really wants that bureaucratic nightmare of arbitrating cases when this inevitably messes up, be this system manual or automatic?
I think bureaucratic nightmare is pretty hyperbolic, but yes, there would be some overhead, for sure. And given that no one has the time and motivation to even maintain PyPi at this point -- this will probably kill the idea altogether.
To the specifics of the mypy-lang package that brought this up... It's like naming your company "Yahoo", and getting upset that yahoo.com is getting a bump in traffic because of your popularity.
Again - this is not about minor weekend projects not be "important". It's about potential abandonware -- with domain registration, "Yahoo" can offer to buy the domain from the current holder, or, if the current owner has abandoned it, it will go into the public pool again when they stop paying to maintain the registration.
-CHB
--
Christopher Barker, Ph.D. Oceanographer
Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker@noaa.gov
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
On Mon, Apr 18, 2016 at 2:31 PM, Ian Cordasco <graffatcolmingov@gmail.com> wrote:
1. PyYAML is a package that would be de-registered in such a scheme. It
and you don't think ANYONE would be willing to take on the miniscule amount of work to maintain the name? Plus there would be any number of other schemes for determining whether a project name is abandoned.
I have in fact offered but the author refuses to accept help from anyone. They're also the author of the C library (libyaml) and they do not maintain that either. It's actually quite frustrating as someone who wants to fix some of the numerous bugs in the library + improve it and add support for YAML 1.2 which is years old at this point.
Interesting third case I hadn't considered -- the original author is still "active", but not actually maintaining the software or accepting help. I don't think there is anything PyPi policy can do about that -- too bad. Time for a fork? -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
Another high profile example of such a project: PIL. 19.04.2016, 00:56, Chris Barker kirjoitti:
On Mon, Apr 18, 2016 at 2:31 PM, Ian Cordasco <graffatcolmingov@gmail.com <mailto:graffatcolmingov@gmail.com>> wrote:
>> 1. PyYAML is a package that would be de-registered in such a scheme. It
> and you don't think ANYONE would be willing to take on the miniscule amount > of work to maintain the name? Plus there would be any number of other > schemes for determining whether a project name is abandoned.
I have in fact offered but the author refuses to accept help from anyone. They're also the author of the C library (libyaml) and they do not maintain that either. It's actually quite frustrating as someone who wants to fix some of the numerous bugs in the library + improve it and add support for YAML 1.2 which is years old at this point.
Interesting third case I hadn't considered -- the original author is still "active", but not actually maintaining the software or accepting help. I don't think there is anything PyPi policy can do about that -- too bad.
Time for a fork?
-CHB
--
Christopher Barker, Ph.D. Oceanographer
Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker@noaa.gov <mailto:Chris.Barker@noaa.gov>
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
On Apr 18, 2016, at 2:31 PM, Ian Cordasco <graffatcolmingov@gmail.com> wrote:
I have in fact offered but the author refuses to accept help from anyone. They're also the author of the C library (libyaml) and they do not maintain that either. It's actually quite frustrating as someone who wants to fix some of the numerous bugs in the library + improve it and add support for YAML 1.2 which is years old at this point.
Since the spectre of malware has been raised in this thread, I feel I should point out that the reverse is also true. Although libyaml / pyyaml are "trusted" today, what happens after the inevitable 0-day RCE drops which the author refuses to patch it? Does PyPI have a responsibility to re-assign the name in that case? Specifically, YAML does have a heritage <http://www.sitepoint.com/anatomy-of-an-exploit-an-in-depth-look-at-the-rails...> of vulnerabilities, even if this specific instance doesn't. -glyph
On Apr 18, 2016, at 6:14 PM, Glyph <glyph@twistedmatrix.com> wrote:
On Apr 18, 2016, at 2:31 PM, Ian Cordasco <graffatcolmingov@gmail.com <mailto:graffatcolmingov@gmail.com>> wrote:
I have in fact offered but the author refuses to accept help from anyone. They're also the author of the C library (libyaml) and they do not maintain that either. It's actually quite frustrating as someone who wants to fix some of the numerous bugs in the library + improve it and add support for YAML 1.2 which is years old at this point.
Since the spectre of malware has been raised in this thread, I feel I should point out that the reverse is also true. Although libyaml / pyyaml are "trusted" today, what happens after the inevitable 0-day RCE drops which the author refuses to patch it? Does PyPI have a responsibility to re-assign the name in that case? Specifically, YAML does have a heritage <http://www.sitepoint.com/anatomy-of-an-exploit-an-in-depth-look-at-the-rails...> of vulnerabilities, even if this specific instance doesn't.
We don’t currently have much in the way of mechanisms to deal with that. Although I could think of a few that we could do which *wouldn’t* require handing over the name and which could generalize out to other maintenance/abandonment problems as well, like (in order of severity): * Add a warning on the PyPI page indicating that the project is abandoned/unmaintained/etc suggesting they find something else (possibly with specific suggestions, like PIL -> Pillow). * Add some mechanism to pip/PyPI that would allow PyPI to provide a message to people installing a particular project (or perhaps a specific version). This could also be exposed to authors who want to mark specific versions of their project as insecure. * Delete the files from PyPI or otherwise prevent them from being discovered by pip (likely paired with the a warning of some kind on the PyPI page). ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On Apr 18, 2016, at 3:21 PM, Donald Stufft <donald@stufft.io> wrote:
On Apr 18, 2016, at 6:14 PM, Glyph <glyph@twistedmatrix.com <mailto:glyph@twistedmatrix.com>> wrote:
On Apr 18, 2016, at 2:31 PM, Ian Cordasco <graffatcolmingov@gmail.com <mailto:graffatcolmingov@gmail.com>> wrote:
I have in fact offered but the author refuses to accept help from anyone. They're also the author of the C library (libyaml) and they do not maintain that either. It's actually quite frustrating as someone who wants to fix some of the numerous bugs in the library + improve it and add support for YAML 1.2 which is years old at this point.
Since the spectre of malware has been raised in this thread, I feel I should point out that the reverse is also true. Although libyaml / pyyaml are "trusted" today, what happens after the inevitable 0-day RCE drops which the author refuses to patch it? Does PyPI have a responsibility to re-assign the name in that case? Specifically, YAML does have a heritage <http://www.sitepoint.com/anatomy-of-an-exploit-an-in-depth-look-at-the-rails...> of vulnerabilities, even if this specific instance doesn't.
We don’t currently have much in the way of mechanisms to deal with that. Although I could think of a few that we could do which *wouldn’t* require handing over the name and which could generalize out to other maintenance/abandonment problems as well, like (in order of severity):
* Add a warning on the PyPI page indicating that the project is abandoned/unmaintained/etc suggesting they find something else (possibly with specific suggestions, like PIL -> Pillow).
This is the sort of thing I had in mind with https://github.com/pypa/warehouse/issues/933 <https://github.com/pypa/warehouse/issues/933> - it seems like any kind of annotation like this should be a matter of last resort and authors should be given every opportunity to respond first.
* Add some mechanism to pip/PyPI that would allow PyPI to provide a message to people installing a particular project (or perhaps a specific version). This could also be exposed to authors who want to mark specific versions of their project as insecure.
* Delete the files from PyPI or otherwise prevent them from being discovered by pip (likely paired with the a warning of some kind on the PyPI page).
----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
Instead of overtaking, how about clearly marking packages as abandoned/maintained clearly pointing out the mark was imposed by community action and listing potential/primary replacements this would have the possibility of also supporting normal abandonment when people voluntary stop maintenance and cannot find a successor its important that community imposed abandonment is not simply removable by doing a minor "noop"-release, after all if abandonment had to be applied by proof of 3rd party, there is need of proof of continued maintenance pip could warn on installation/update -- Ronny Am 18.04.2016 um 23:31 schrieb Ian Cordasco:
On Mon, Apr 18, 2016 at 4:16 PM, Chris Barker <chris.barker@noaa.gov> wrote:
You've made a strong case, and this is probably not where PyPi should go -- but it would hardly be a disaster:
The idea of expiring out names has been brought up recently to resolve an issue of two packages, one popular and large; another someone's weekend project.
The issue here is not that it's a weekend project, but that it may be an abandoned project. I don't think anyone suggest that we should have a popularity or quality test to see who gets to trump whom with name allocation -- I sure didn't.
Which is quite relevant to below:
1. PyYAML is a package that would be de-registered in such a scheme. It is a highly used, extremely popular, package that unserializes text into arbitrary python objects. It is a trusted package... and one that hasn't been active in ages.
and you don't think ANYONE would be willing to take on the miniscule amount of work to maintain the name? Plus there would be any number of other schemes for determining whether a project name is abandoned. I have in fact offered but the author refuses to accept help from anyone. They're also the author of the C library (libyaml) and they do not maintain that either. It's actually quite frustrating as someone who wants to fix some of the numerous bugs in the library + improve it and add support for YAML 1.2 which is years old at this point.
2. the package tooling already assumes that names will always point to one, and only one package. ever. until the heat death of the universe or the death of the language whichever is first.
IIUC, the current scheme allows for a name to be "taken over" by a new package if the original author so desires -- i.e. if the current owner of the mypy name was happy to relinquish it, then "pip install mypy" would get users something totally different 6 months from now. So no -- we don't currently guarantee anything about future use of names. Other that that the original author can do whatever they want with it.
3. Who in the PSF really wants that bureaucratic nightmare of arbitrating cases when this inevitably messes up, be this system manual or automatic?
I think bureaucratic nightmare is pretty hyperbolic, but yes, there would be some overhead, for sure. And given that no one has the time and motivation to even maintain PyPi at this point -- this will probably kill the idea altogether.
To the specifics of the mypy-lang package that brought this up... It's like naming your company "Yahoo", and getting upset that yahoo.com is getting a bump in traffic because of your popularity.
Again - this is not about minor weekend projects not be "important". It's about potential abandonware -- with domain registration, "Yahoo" can offer to buy the domain from the current holder, or, if the current owner has abandoned it, it will go into the public pool again when they stop paying to maintain the registration.
-CHB
--
Christopher Barker, Ph.D. Oceanographer
Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker@noaa.gov
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
On Tue, Apr 19, 2016 at 9:45 AM, Ronny Pfannschmidt < opensource@ronnypfannschmidt.de> wrote:
Instead of overtaking, how about clearly marking packages as abandoned/maintained clearly pointing out the mark was imposed by community action
I think that would be a good idea -- and maybe start with just that -- then we'd learn how big an issue it really was, etc.
and listing potential/primary replacements
that required real work on someone's part -- so not sure when that would actually happen.
its important that community imposed abandonment is not simply removable by doing a minor "noop"-release,
why not? I brought tis all up to address truly abandoned projects -- maybe we want to go some day to the idea of the names being community owned, but that's not the way ti is now -- and if someone makes the effort to do a noop release, then they have no abandoned the name -- maybe aren't maintaining it worth a damn, but there you go. Personally, I think there is no point in anything between the current free for all, and a "curated" package repo -- a curated repo would support the idea that anything on it had met some minimum standard: no malware, some maintenance, some minimum usefulness, etc. It's kind of a shame that there are so many "toy" packages and experiments on PyPi, but in fact, it's worked pretty darn well.. pip could warn on installation/update
I think that would be good too -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
Am 20.04.2016 um 00:38 schrieb Chris Barker:
On Tue, Apr 19, 2016 at 9:45 AM, Ronny Pfannschmidt <opensource@ronnypfannschmidt.de <mailto:opensource@ronnypfannschmidt.de>> wrote:
Instead of overtaking, how about clearly marking packages as abandoned/maintained clearly pointing out the mark was imposed by community action
I think that would be a good idea -- and maybe start with just that -- then we'd learn how big an issue it really was, etc.
and listing potential/primary replacements
that required real work on someone's part -- so not sure when that would actually happen.
its important that community imposed abandonment is not simply removable by doing a minor "noop"-release,
why not? I brought tis all up to address truly abandoned projects -- maybe we want to go some day to the idea of the names being community owned, but that's not the way ti is now -- and if someone makes the effort to do a noop release, then they have no abandoned the name -- maybe aren't maintaining it worth a damn, but there you go.
a community action is supposed to be imposed after extended non-reaction, an next to no effort way to get out of it seems counter such an invsive move. its not given lightly, and it shouldn't be easy to weasel out of it. Actually a noop release is a good indicator that the mark is well-deserved and should be keept. Making an effort to remove a mark without making the reason for its existence go as well is a lie in plain sight. this reminds me of the whole setuptools/distribute situation. -- Ronny
Personally, I think there is no point in anything between the current free for all, and a "curated" package repo -- a curated repo would support the idea that anything on it had met some minimum standard: no malware, some maintenance, some minimum usefulness, etc.
It's kind of a shame that there are so many "toy" packages and experiments on PyPi, but in fact, it's worked pretty darn well..
pip could warn on installation/update
I think that would be good too
-Chris
--
Christopher Barker, Ph.D. Oceanographer
Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker@noaa.gov <mailto:Chris.Barker@noaa.gov>
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 04/20/2016 04:36 PM, Ronny Pfannschmidt wrote:
its not given lightly, and it shouldn't be easy to weasel out of it. Actually a noop release is a good indicator that the mark is well-deserved and should be keept. Making an effort to remove a mark without making the reason for its existence go as well is a lie in plain sight.
A release, any release, is a sign that the maintainer is around and cares about the package enough to act. That should be sufficient to block any takeover attempt.
this reminds me of the whole setuptools/distribute situation.
I don't think that means what you think it does: the situation was resolved amicably, via negotiation with the owner of the original name. The fact that it took longer than some wanted was *not* an indication that it was the wrong outcome. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJXGAaCAAoJEPKpaDSJE9HY3FIQAL87jOyqytF1MuC5SHqiw0y1 p7BYWrLf8MVl+DFRJ0x0s9NPVgvkeF+rbx17WD7k8Eik5iuXzTpjIun0oI9UBkwC iTpogEwDrFg3T/s3bM0ObAphcLFHAY4YFMXsUThODtCHg4qi/04Z9RiMhnabwitP GfzuI33aDZbgkyi3ozEse5rP1xTn3SmhtHwu7k5gQvAmg8ntpXFzP7l1IXNWtPr+ SGXzsgohZwsDdOBwOibbvQ3VQSmeLBPrfsP0xVzNOIh7GEHT6AXMM8qK52Er4vMd SQmR2r0WAShqDm+LadfK/KjrvwSgQILpp1z4jvY6k3PWhtLABEeotDZj5mgRY8Hf 0+NTBuifYo7SHs+ElI4nryDbp48q2DZH2PLdh3WgyMlLzTocXTjQcwOTEZGcbb7t s5u+/cQLzfvg4Z23i7wPGrUrBZvl4YM3YKZfsC3Pl5v9jfwiJm/ay7G7O2NHstdB inI23H427ELXgPLI8Ffi9u7MQ6zTCd0H/XNxaiWAhG4JmMZyFuNqJQAuZ9+cR9N+ WWPEpfpHhWW9pYdxKWPHJLWKoGTu+6qPqIMnCPn3sY8ACnqH7z5puNu+/uAJ3hBt PNVh+wKE4R1wNXHKHaHqVqMH/Ut057hEh3EIOwX5e8W/zCojZy8O6PjmaFW7EGdd BMtyIkp2jgQJMY3ClfP6 =w60N -----END PGP SIGNATURE-----
On Apr 18, 2016, at 5:16 PM, Chris Barker <chris.barker@noaa.gov> wrote:
1. PyYAML is a package that would be de-registered in such a scheme. It is a highly used, extremely popular, package that unserializes text into arbitrary python objects. It is a trusted package... and one that hasn't been active in ages.
and you don't think ANYONE would be willing to take on the miniscule amount of work to maintain the name? Plus there would be any number of other schemes for determining whether a project name is abandoned.
This sort of gets to the core of one of the major questions that has to be answered before you talk about expiring names and such. Who “owns” a name on PyPI? Right now, as implemented, the people who “own” (in terms of ACL) the name on PyPI are the owners of said name. In this vein PyPI has traditionally been first come first serve on names and generally will not release a name to someone else without the current owner’s permission [1]. This leads to projects, even popular ones, sometimes falling into abandonment, sometimes even when you have a new maintainer willing to step up (as is the case with Ian and pyYAML) or in some cases, has already forked the project under another name (as is the case with PIL and Pillow). However, this also means that in general, if you decide you trust the current owner of something on PyPI, you know that it’s not suddenly going to be owned by someone else *unless* the person who you previously trusted decides to give it away to them (and thus, implicitly transferring the trust you granted them to another person). So, before we try and think up some mechanism for expiring names, or forcibly taking a name from the current owner and giving it to another, we [2] would first need to decide that names on PyPI are not owned by the individuals who hold the registration, but are instead owned by the community and the current individuals are only borrowing them. This would mean that instead of having packages wither and get abandoned we could instead transfer them to new maintainers who can keep the project going. However, that of course raises new questions like: * Should we only transfer projects to maintainers who want to keep the current project going? Such as with PIL or pyYAML? - This will be generally less surprising to end users, but it doesn’t solve the problem of rarely used packages “blocking” other’s use of the name. * How do we determine who to give a name to? Surely we shouldn’t hand off a package that people are using to the first random passerby who happens to ask for it and we can probably trust Guido not to be malicious (or else we have bigger problems ;) ), but how do we determine if someone is to be trusted when they inevitably fall somewhere in between? * The current policy is pretty black and white with only a little bit of grey area (what is a “real” package that’s been uploaded to a project, what is a malicious package, etc), but moving over to community owned names would open things up to a far wider set of grey areas, and who is going to arbitrate when a project is abandoned or when it’s “popular” or not? - If folks are unaware, npmjs.org <http://npmjs.org/> just recently had a bit of a kerfuffle over a package on their repository going missing, “left-pad”, due to the grey-ness of their policy. - The current, fairly rigid policy came around because the PyPI admins decided to give what appeared to be an unmaintained library to someone new who had forked the library, after the original author hadn’t responded to an email. This ended up causing our own kerfuffle when it turned out this person *wasn’t* absent and *wasn’t* OK with this. [3]. * If we start giving projects out to other people, what tools should we give people to deal with that? - The latest version of pip has a mode that allows you to specify a number of hashes and it won’t install anything that doesn’t match a hash. That could protect end users from installing something inadvertently, but it’s very opt in and requires hard pins ``==`` so it’s not likely to get a large uptick. - If we mandated semver (or something like it) we could make it so that transferring a name *forced* a major version bump and the new author would be unable to release anything using a smaller major version number, people could then pin to <CURRENT_MAJOR+1 and be assured that somebody new won’t take over a project without their knowing about it. However we haven’t historically mandated this and there are a lot of projects using date based versions that would b very unhappy (in addition to the fact upper pins often are major contributors to being unable to resolve a version tree of dependencies). - If we implemented a signature scheme, we could possibly flag certain key changes as must-require-confirmation or something like it and if users have previously used the old key, they won’t install the new package without first confirming the key change. Anyways, the very first thing we need to decide is if projects are community or individual owned, and if community owned, under what situations are we willing to transfer and what stipulations there should be. Once we have all that, figuring out a mechanism becomes a lot easier (even if that mechanism is manual). [1] The main exception to this rule is when the name is obviously being used for nothing and the current owner is not reachable by one of the PyPI admins. In this case we’ve vacated a name and assigned it to someone else without the current owner’s OK. Other exceptions are things like malicious packages, spam, etc. [2] Who is we? To be honest, I don’t really know. I suspect this is something that could be handled as a PEP perhaps, or maybe this is something that the PSF Board should decide. [3] The whole thing was a bit more complicated than that, there were some process failures on our side as well. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On Apr 18, 2016, at 3:11 PM, Donald Stufft <donald@stufft.io> wrote:
If we mandated semver (or something like it) we could make it so that transferring a name *forced* a major version bump and the new author would be unable to release anything using a smaller major version number, people could then pin to <CURRENT_MAJOR+1 and be assured that somebody new won’t take over a project without their knowing about it. However we haven’t historically mandated this and there are a lot of projects using date based versions that would b very unhappy (in addition to the fact upper pins often are major contributors to being unable to resolve a version tree of dependencies).
Just want to raise my hand to be counted here among the people that would be extremely unhappy if PyPI started mandating semver. -glyph
Here here. As to the concept of a name being community owned vs. registrant owned - I am very opposed to changing the rules mid-stream. If community ownership is to be a thing (and I do not want it to be a thing at all, but if it is), it should be *from this point forward* the names are community owned. On 4/18/2016 18:16, Glyph wrote:
On Apr 18, 2016, at 3:11 PM, Donald Stufft <donald@stufft.io <mailto:donald@stufft.io>> wrote:
If we mandated semver (or something like it) we could make it so that transferring a name *forced* a major version bump and the new author would be unable to release anything using a smaller major version number, people could then pin to <CURRENT_MAJOR+1 and be assured that somebody new won’t take over a project without their knowing about it. However we haven’t historically mandated this and there are a lot of projects using date based versions that would b very unhappy (in addition to the fact upper pins often are major contributors to being unable to resolve a version tree of dependencies).
Just want to raise my hand to be counted here among the people that would be /extremely/ unhappy if PyPI started mandating semver.
-glyph
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
participants (14)
-
Alex Grönholm -
Alexander Walters -
Chris Barker -
Donald Stufft -
Glyph -
Greg Ewing -
Guido van Rossum -
Ian Cordasco -
Ionel Cristian Mărieș -
Nathaniel Smith -
Paul Moore -
Richard Jones -
Ronny Pfannschmidt -
Tres Seaver