Create formal process for claiming 'abandoned' packages
In order to claim a package as being abandoned it should undergo a formal process that includes: * Placement on a PUBLIC list of packages under review for a grace period to be determined by this discussion * Formal attempts via email and social media (twitter, github, et al) to contact the maintainer. * Investigation of the claimant for the rights to the package. The parties attempting to claim a package may not be the best representatives of the community behind that package, or the Python community in general. Why? * Non-reply does not equal consent. * Access to a commonly (or uncommonly) used package poses security and reliability issues. Why: Scenario 1: I could claim ownership of the redis package, providing a certain-to-fail email for the maintainers of PyPI to investigate? Right now the process leads me to think I would succeed in gaining access. If successful, I would gain complete access to a package used by hundreds of projects for persistence storage. Scenario 2: I could claim ownership of the redis package, while Andy McCurdy (maintainer) was on vacation for two weeks, or sabbatical for six weeks. Again, I would gain access because under the current system non-reply equals consent. Reference: In ticket #407 (https://sourceforge.net/p/pypi/support-requests/407/) someone who does not appear to be vetted managed to gain control of the (arguably) abandoned but still extremely popular django-registration on PyPI. They run one of several HUNDRED forks of django-registration, one that is arguably not the most commonly used. My concern is that as django-registration is the leading package for handling system registration for Python's most popular web framework, handing it over without a full investigation of not just the current maintainer but also the candidate maintainer is risky. Regards, Daniel Greenfeld pydanny@gmail.com
On 20 September 2014 04:47, Daniel Greenfeld <pydanny@gmail.com> wrote:
In order to claim a package as being abandoned it should undergo a formal process that includes:
* Placement on a PUBLIC list of packages under review for a grace period to be determined by this discussion
This is not done at present. Can you suggest a public forum that would reach a useful audience?
* Formal attempts via email and social media (twitter, github, et al) to contact the maintainer.
This is done at present, using the contact details registered with pypi. Or other contact methods if that fails. I always default to asking the current maintainer of a package to transfer it to a new maintainer.
* Investigation of the claimant for the rights to the package. The parties attempting to claim a package may not be the best representatives of the community behind that package, or the Python community in general.
I'm not sure how I could do this reasonably given the breadth of packages in the index, and the size and number of Python communities. How could I possibly determine this? In the open source world, how do you vet someone, especially when the original maintainer is unresponsive?
Why?
* Non-reply does not equal consent.
That's a reasonable statement, but if this were to be held then a large number of stagnating package listings would have remained in that state.
* Access to a commonly (or uncommonly) used package poses security and reliability issues.
Why:
Scenario 1:
I could claim ownership of the redis package, providing a certain-to-fail email for the maintainers of PyPI to investigate?
I attempt contact through other channels. I don't rely just on information provided by the requestor.
Scenario 2:
I could claim ownership of the redis package, while Andy McCurdy (maintainer) was on vacation for two weeks, or sabbatical for six weeks. Again, I would gain access because under the current system non-reply equals consent.
I tend to wait one month, but yes a six month sabbatical would be a problem. On the other hand, I do make every attempt to contact
Reference:
In ticket #407 (https://sourceforge.net/p/pypi/support-requests/407/) someone who does not appear to be vetted managed to gain control of the (arguably) abandoned but still extremely popular django-registration on PyPI. They run one of several HUNDRED forks of django-registration, one that is arguably not the most commonly used.
My concern is that as django-registration is the leading package for handling system registration for Python's most popular web framework, handing it over without a full investigation of not just the current maintainer but also the candidate maintainer is risky.
And my counter is that I get a lot of these requests, I do my best to try to contact the original maintainer, and in the absence of any other information I need to take the requestor at their word. In the case of the request above, I contacted the original maintainer directly, using an address I knew to work, and received no response. To me that correlated well with the indication that he wanted nothing to do with the package any longer. Someone keen enough had come forward to provide updated versions of the package, amongst what you claim are hundreds of such forks (recognising that github forks are a very poor method to judge how engaged someone is with a project). In light of that, I granted that person permission to provided updates for that project. Thanks for your thoughts. The procedure I use should be written down, I guess, but I'm the only person who follows it, so the motivation to do so is very low. Richard
On Sep 19, 2014, at 5:55 PM, Richard Jones <richard@python.org> wrote:
On 20 September 2014 04:47, Daniel Greenfeld <pydanny@gmail.com <mailto:pydanny@gmail.com>> wrote: In order to claim a package as being abandoned it should undergo a formal process that includes:
* Placement on a PUBLIC list of packages under review for a grace period to be determined by this discussion
This is not done at present. Can you suggest a public forum that would reach a useful audience?
* Formal attempts via email and social media (twitter, github, et al) to contact the maintainer.
This is done at present, using the contact details registered with pypi. Or other contact methods if that fails.
I always default to asking the current maintainer of a package to transfer it to a new maintainer.
* Investigation of the claimant for the rights to the package. The parties attempting to claim a package may not be the best representatives of the community behind that package, or the Python community in general.
I'm not sure how I could do this reasonably given the breadth of packages in the index, and the size and number of Python communities. How could I possibly determine this? In the open source world, how do you vet someone, especially when the original maintainer is unresponsive?
Why?
* Non-reply does not equal consent.
That's a reasonable statement, but if this were to be held then a large number of stagnating package listings would have remained in that state.
* Access to a commonly (or uncommonly) used package poses security and reliability issues.
Why:
Scenario 1:
I could claim ownership of the redis package, providing a certain-to-fail email for the maintainers of PyPI to investigate?
I attempt contact through other channels. I don't rely just on information provided by the requestor.
Scenario 2:
I could claim ownership of the redis package, while Andy McCurdy (maintainer) was on vacation for two weeks, or sabbatical for six weeks. Again, I would gain access because under the current system non-reply equals consent.
I tend to wait one month, but yes a six month sabbatical would be a problem. On the other hand, I do make every attempt to contact
Reference:
In ticket #407 (https://sourceforge.net/p/pypi/support-requests/407/ <https://sourceforge.net/p/pypi/support-requests/407/>) someone who does not appear to be vetted managed to gain control of the (arguably) abandoned but still extremely popular django-registration on PyPI. They run one of several HUNDRED forks of django-registration, one that is arguably not the most commonly used.
My concern is that as django-registration is the leading package for handling system registration for Python's most popular web framework, handing it over without a full investigation of not just the current maintainer but also the candidate maintainer is risky.
And my counter is that I get a lot of these requests, I do my best to try to contact the original maintainer, and in the absence of any other information I need to take the requestor at their word. In the case of the request above, I contacted the original maintainer directly, using an address I knew to work, and received no response. To me that correlated well with the indication that he wanted nothing to do with the package any longer. Someone keen enough had come forward to provide updated versions of the package, amongst what you claim are hundreds of such forks (recognising that github forks are a very poor method to judge how engaged someone is with a project). In light of that, I granted that person permission to provided updates for that project.
Thanks for your thoughts. The procedure I use should be written down, I guess, but I'm the only person who follows it, so the motivation to do so is very low.
Richard _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Perhaps in Warehouse the procedure can be automated to some degree and a public record of what actions were taken and when? I don’t mean like a public log of the actual email address or email content or anything of the sort. Just like a "attempted to contact on X date", "notified X thing on Y", "No response in X time, transfering ownership" kind of things. Maybe we could create something like python-updates which would be a read only mailing list which just posts a thread per request and updates it with the actions taken and stuff. People who care could subscribe to it without having to get all of distutils-sig or wahtever. Maybe it could even offer package authors the ability to mark a package as "Request For Adoption" saying that they have a package that they wrote, but that they no longer wish to maintain. I don't know, I'm just tossing out some potentional ideas! --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On 20 September 2014 08:22, Donald Stufft <donald@stufft.io> wrote:
Perhaps in Warehouse the procedure can be automated to some degree and a public record of what actions were taken and when? I don’t mean like a public log of the actual email address or email content or anything of the sort. Just like a "attempted to contact on X date", "notified X thing on Y", "No response in X time, transfering ownership" kind of things.
Maybe we could create something like python-updates which would be a read only mailing list which just posts a thread per request and updates it with the actions taken and stuff. People who care could subscribe to it without having to get all of distutils-sig or wahtever.
Maybe it could even offer package authors the ability to mark a package as "Request For Adoption" saying that they have a package that they wrote, but that they no longer wish to maintain.
I don't know, I'm just tossing out some potentional ideas!
Yep, for this kind of thing, "automate" can be a better answer than "document" - it's much easier to delegate (or otherwise hand over the reins) when the process is built into the tools. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Donald Stufft wrote:
Perhaps in Warehouse the procedure can be automated to some degree and a public record of what actions were taken and when? I don’t mean like a public log of the actual email address or email content or anything of the sort. Just like a "attempted to contact on X date", "notified X thing on Y", "No response in X time, transfering ownership" kind of things.
Maybe we could create something like python-updates which would be a read only mailing list which just posts a thread per request and updates it with the actions taken and stuff. People who care could subscribe to it without having to get all of distutils-sig or wahtever.
Maybe it could even offer package authors the ability to mark a package as "Request For Adoption" saying that they have a package that they wrote, but that they no longer wish to maintain.
I don't know, I'm just tossing out some potentional ideas!
In the same vein, but at the more annoying end of the scale (I don't know how big a problem this really is) - build in an expiry date for every package. Every upload or ping from the maintainer extends it by 6-12 months, then automate a couple of emails in the last month before it expires. Nothing more has to happen, except that then it may be claimed by someone else and the communication has already been done. The manual process becomes checking a flag and the rest is automated. Regular emails also helps make sure that you track email address changes better. I also like the "Request for Adoption" idea. Plenty of projects get wide use without developing a community where it's easy to find someone to take over. Cheers, Steve
Hi Richard, On Fri, Sep 19, 2014 at 2:55 PM, Richard Jones <richard@python.org> wrote:
On 20 September 2014 04:47, Daniel Greenfeld <pydanny@gmail.com> wrote:
In order to claim a package as being abandoned it should undergo a formal process that includes:
* Placement on a PUBLIC list of packages under review for a grace period to be determined by this discussion
This is not done at present. Can you suggest a public forum that would reach a useful audience?
What about a page on PyPI that tracks packages undergoing this review? PyPI has a huge audience. "In theory" all this requires is just a few additional fields added.
* Formal attempts via email and social media (twitter, github, et al) to contact the maintainer.
This is done at present, using the contact details registered with pypi. Or other contact methods if that fails.
I always default to asking the current maintainer of a package to transfer it to a new maintainer.
It would be nice to have this documented on PyPI. I would be more than willing to write this down for you.
* Investigation of the claimant for the rights to the package. The parties attempting to claim a package may not be the best representatives of the community behind that package, or the Python community in general.
I'm not sure how I could do this reasonably given the breadth of packages in the index, and the size and number of Python communities. How could I possibly determine this? In the open source world, how do you vet someone, especially when the original maintainer is unresponsive?
Honestly? I'm not sure either. I know the people that I know, and can research a segment of the community. However, I'm well aware that this is a tiny portion of who is actually using python.
Why?
* Non-reply does not equal consent.
That's a reasonable statement, but if this were to be held then a large number of stagnating package listings would have remained in that state
I concur. Which is why I suggested creating a page that tracks packages undergoing the transfer-of-ownership grace period. That would mean more eyes on the issue, as well as provide a means for eventually automating things in order to relieve you of the workload of maintenance.
* Access to a commonly (or uncommonly) used package poses security and reliability issues.
Why:
Scenario 1:
I could claim ownership of the redis package, providing a certain-to-fail email for the maintainers of PyPI to investigate?
I attempt contact through other channels. I don't rely just on information provided by the requestor.
Knowing you, I would be surprised if it were any other way. ;) I believe documenting this process of communication would cast light on the process. And would mean that you could more easily enlist others to help you. I would be honored to document this or any other part of this system.
Reference:
In ticket #407 (https://sourceforge.net/p/pypi/support-requests/407/) someone who does not appear to be vetted managed to gain control of the (arguably) abandoned but still extremely popular django-registration on PyPI. They run one of several HUNDRED forks of django-registration, one that is arguably not the most commonly used.
My concern is that as django-registration is the leading package for handling system registration for Python's most popular web framework, handing it over without a full investigation of not just the current maintainer but also the candidate maintainer is risky.
And my counter is that I get a lot of these requests, I do my best to try to contact the original maintainer, and in the absence of any other information I need to take the requestor at their word. In the case of the request above, I contacted the original maintainer directly, using an address I knew to work, and received no response. To me that correlated well with the indication that he wanted nothing to do with the package any longer. Someone keen enough had come forward to provide updated versions of the package, amongst what you claim are hundreds of such forks (recognising that github forks are a very poor method to judge how engaged someone is with a project). In light of that, I granted that person permission to provided updates for that project.
Thanks for your thoughts. The procedure I use should be written down, I guess, but I'm the only person who follows it, so the motivation to do so is very low.
Having maintained enough projects of my own, I really do understand your point of view. People ask for things, but it's rare that they will actually provide assistance. It's tiring and frustrating, since they always want you to put in more time, usually without offering to help in any way. So let me say right now that I want to help: * I will help with documenting the process. You can tell it to me in any format you want, written or verbal, and then I'll write it up. * I would like to help with modifying PyPI to create a tracking process for transfer-of-ownership. * I would be honored to pitch in for maintenance of this part of things, and can also issue a call for assistance for more help. I know you do a lot of work on PyPI. I can't begin to tell you how much that is appreciated. Let me help you. Sincerely, Danny
On Fri, Sep 19, 2014 at 4:55 PM, Richard Jones <richard@python.org> wrote:
This is done at present, using the contact details registered with pypi. Or other contact methods if that fails. I always default to asking the current maintainer of a package to transfer it to a new maintainer.
Could you clarify when and how you attempted that contact in this case? At the email address on file for me at PyPI, I have received one email from you regarding PyPI, and it was the automated message regarding the Python wiki password breach. Additionally, the requesting party had contacted me, and we had a brief but inconclusive discussion regarding whether it would be a good idea for the package to be resurrected under a new maintainer. The fact that I literally woke up from a nap to find someone else had been assigned as an owner of one of my packages -- even one I've publicly stepped down as maintainer of -- without any notice to me that I can find from the PyPI side (I found out from seeing my name mentioned on Twitter, then saw this email thread), has placed me in a position where my faith in PyPI's security is now exactly zero, and I'm forced to consider whether I want to continue hosting packages there. For now I have removed user 'macropin' from django-registration on PyPI. Do not make any further changes to the package's records/roles/etc. on PyPI unless I request it of you, via GPG-signed mail (my key is available quite publicly courtesy of Django releases).
On further information, it seems the contact attempt was a message to my gmail address, which is not the contact information I have on file for PyPI, and is the address I use for bulk things like mailing lists. I am now more frightened that missing an email to an alternate address (that address gets literally hundreds of messages/day, and I do not attempt to read all of them, hence I don't use it as a contact address for important things) can result in this type of action.
I **strongly** concur with James here. This has flagrantly violated my trust in PyPI. I would much rather packages not be reclaimed than need to think about whether I trust the PyPI maintainers to do it. Alex
On 09/19/2014 04:13 PM, Alex Gaynor wrote:
I **strongly** concur with James here. This has flagrantly violated my trust in PyPI.
I would much rather packages not be reclaimed than need to think about whether I trust the PyPI maintainers to do it.
Having PyPI become full of cruft is not a tenable situation. People make mistakes. Instead of flaming Richard we should figure out how to make the process better for everyone, Richard included. Thank you, Daniel, for taking the lead on that. -- ~Ethan~
On 09/19/2014 04:13 PM, Alex Gaynor wrote:
I **strongly** concur with James here. This has flagrantly violated my
Ethan Furman <ethan <at> stoneleaf.us> writes: trust in
PyPI.
I would much rather packages not be reclaimed than need to think about whether I trust the PyPI maintainers to do it.
Having PyPI become full of cruft is not a tenable situation.
What is the problem with "cruft" exactly? I'm opposed to the idea that a package may be "transferred" to someone else without any explicit authorization from the original maintainer(s) (or author(s)). It is not a matter of waiting time or tools. It is morally unacceptable. If you want to maintain a package and the maintainer doesn't reply, there is a solution: fork the package (under a new name). If you don't want to see "stale" packages, there is a solution: build an optional filter into PyPI that only shows packages which have received an update in the last 12/24/36 months. Regards Antoine.
Antoine Pitrou <antoine@python.org> wrote:
Having PyPI become full of cruft is not a tenable situation.
What is the problem with "cruft" exactly?
I'm opposed to the idea that a package may be "transferred" to someone else without any explicit authorization from the original maintainer(s) (or author(s)). It is not a matter of waiting time or tools. It is morally unacceptable.
I agree. The situation here is totally different from Debian or Fedora, where the maintainers are just repackaging upstream most of the time. Stefan Krah
On Sep 21, 2014, at 7:06 PM, Antoine Pitrou <antoine@python.org> wrote:
On 09/19/2014 04:13 PM, Alex Gaynor wrote:
I **strongly** concur with James here. This has flagrantly violated my
Ethan Furman <ethan <at> stoneleaf.us> writes: trust in
PyPI.
I would much rather packages not be reclaimed than need to think about whether I trust the PyPI maintainers to do it.
Having PyPI become full of cruft is not a tenable situation.
What is the problem with "cruft" exactly?
I'm opposed to the idea that a package may be "transferred" to someone else without any explicit authorization from the original maintainer(s) (or author(s)). It is not a matter of waiting time or tools. It is morally unacceptable.
If you want to maintain a package and the maintainer doesn't reply, there is a solution: fork the package (under a new name).
If you don't want to see "stale" packages, there is a solution: build an optional filter into PyPI that only shows packages which have received an update in the last 12/24/36 months.
Regards
Antoine.
The problem with cruft is they make it more difficult to find things for end users who often times don't know what they are looking for. This is especially bad when you have a once popular library/tool for which the maintainer is no longer available. It's already a daunting task for someone to select a library that does something they need to do if they aren’t already familiar with the ecosystem. Adding "landmines" in the form of projects which look to solve their problem but where there is no-one to help them if they run into a bug or who can release a bug fix is fairly unfriendly. Circling back to django-registration, we can see the extra confusion this can cause when a maintainer stops maintaining a popular package. You end up with a multitude of forks, each slightly incompatible and with different features, bugs, etc. Now in the case of django-registration the author *is* available and wishes to retain control of django-registration, so that's fine, but you can hopefully see the sort of confusion that a maintainer going missing can cause? This isn't a problem that can be automatically solved because there's no programatic way to differentiate between "stable" and "abandoned", especially when you want to consider "abandoned but the author wants to maintain control", "abandoned but the author is willing to give it up", and "abandoned and the author is no longer available" differently. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
Donald Stufft <donald <at> stufft.io> writes:
The problem with cruft is they make it more difficult to find things for end users who often times don't know what they are looking for. This is especially bad when you have a once popular library/tool for which the maintainer is no longer available. It's already a daunting task for someone to select a library that does something they need to do if they aren’t already familiar with the ecosystem. Adding "landmines" in the form of projects which look to solve
their
problem but where there is no-one to help them if they run into a bug or who can release a bug fix is fairly unfriendly.
It's unfriendly if you consider that it's PyPI's job to select packages for users. But it does not seem to be going in that direction (see e.g. the absence of ratings or comments, after a brief appearance). Usually people get their recommendations through the community. If you want to help people through PyPI, you may want to add a friendly, non-scary warning to the PyPI pages of projects which haven't been updated for 24+ months.
Circling back to django-registration, we can see the extra confusion this can cause when a maintainer stops maintaining a popular package. You end up with a multitude of forks, each slightly incompatible and with different features, bugs, etc.
It's inherent to the problem of unmaintained packages. But why would PyPI have any authority over who steps over? PyPI does not have any legitimity to steer those projects. It's not even a controlled software distribution; it's just an index. Regards Antoine.
On Sep 22, 2014, at 9:25 AM, Antoine Pitrou <antoine@python.org> wrote:
Donald Stufft <donald <at> stufft.io> writes:
The problem with cruft is they make it more difficult to find things for end users who often times don't know what they are looking for. This is especially bad when you have a once popular library/tool for which the maintainer is no longer available. It's already a daunting task for someone to select a library that does something they need to do if they aren’t already familiar with the ecosystem. Adding "landmines" in the form of projects which look to solve
their
problem but where there is no-one to help them if they run into a bug or who can release a bug fix is fairly unfriendly.
It's unfriendly if you consider that it's PyPI's job to select packages for users. But it does not seem to be going in that direction (see e.g. the absence of ratings or comments, after a brief appearance).
It's not PyPI's job to "select packages for users", but it is PyPI's job to make it as easy as possible to discover relevant packages and provide them with as much information as is reasonable about that package so that the user can select for themselves which packages to use. I'm not going to get into the debacle that was the ratings system but the lack or existence of such a feature does not change the role of PyPI in any way.
Usually people get their recommendations through the community. If you want to help people through PyPI, you may want to add a friendly, non-scary warning to the PyPI pages of projects which haven't been updated for 24+ months.
Sure, I never stated that transfering ownership was the only possible or even the best way to handle cruft. Personally I'm on the fence about what the best way to handle it is, there are benefits to transfering ownership and downsides. I was only stating what the problem with cruft is.
Circling back to django-registration, we can see the extra confusion this can cause when a maintainer stops maintaining a popular package. You end up with a multitude of forks, each slightly incompatible and with different features, bugs, etc.
It's inherent to the problem of unmaintained packages. But why would PyPI have any authority over who steps over? PyPI does not have any legitimity to steer those projects. It's not even a controlled software distribution; it's just an index.
PyPI inherinently has complete control over who owns what name on PyPI. What limits are put on that control are a matter of policy. It is no less valid for PyPI to transfer ownership than it is for PyPI not to transfer ownership as a matter of Policy. As I said earlier, PyPI isn't hardly the only system like itself that exercises authority over the project names in the case of abandoned projects. As Toshio said that are situations where it makes *obvious* sense to transfer ownership of a project. Using Django as an pretty good example here, There are four people able to make releases there, until fairly recently there were only two if I recall. I don't think anyone would be against PyPI transfering ownership of Django to another active core developer of Django in the event that all of the people with permissions on PyPI were gone in some fashion. There are also cases where it makes *obvious* sense not to do a transfer, such as if some random person asked to transfer pip to their name because we hadn't had a release in a little under 6 months. Given that there are cases where it makes sense to do a transfer, and cases where it doesn't make sense to do a transfer the important part is to figure out, as a matter of policy, where those lines are. To that I think it'd be great to have a documented procedure for doing transfers and even rough guidelines as to when it makes sense to do it, but that ultimately it would be a decision up to the maintainers of PyPI (currently Richard and myself, though I rarely get involved in that side of things). This sort of "BDFL"-esque thing is primarily because edge cases happen often, although the common cases should be able to be handled by the rough guidelines, and because ultimately you have to trust the PyPI maintainers anyways for any reasonable use of PyPI. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
Donald Stufft <donald <at> stufft.io> writes:
PyPI inherinently has complete control over who owns what name on PyPI.
Political authority does not derive from technical control, though.
As Toshio said that are situations where it makes *obvious* sense to transfer ownership of a project. Using Django as an pretty good example here, There are four people able to make releases there, until fairly recently there were only two if I recall. I don't think anyone would be against PyPI transfering ownership of Django to another active core developer of Django in the event that all of the people with permissions on PyPI were gone in some fashion.
Assuming the remaining Django core developers agree on it, then, yes, that can make sense. That's because they are the primary authors of the project (even though they might not have been listed as such on PyPI). The case people are worried about is whether someone who is not part of the original project author(s) or maintainer(s) can get assigned the PyPI project. In that case people should use one of the forks; there's no reason for PyPI to crown a successor.
To that I think it'd be great to have a documented procedure for doing transfers and even rough guidelines as to when it makes sense to do it, but that ultimately it would be a decision up to the maintainers of PyPI (currently Richard and myself, though I rarely get involved in that side of things).
I think the "rough guidelines" should actually be quite strict about it (see above). Also, publicity is important (see Daniel's original post). Regards Antoine.
On Mon, Sep 22, 2014 at 14:16 +0000, Antoine Pitrou wrote:
Donald Stufft <donald <at> stufft.io> writes:
PyPI inherinently has complete control over who owns what name on PyPI.
Political authority does not derive from technical control, though.
valid point IMO.
As Toshio said that are situations where it makes *obvious* sense to transfer ownership of a project. Using Django as an pretty good example here, There are four people able to make releases there, until fairly recently there were only two if I recall. I don't think anyone would be against PyPI transfering ownership of Django to another active core developer of Django in the event that all of the people with permissions on PyPI were gone in some fashion.
Assuming the remaining Django core developers agree on it, then, yes, that can make sense. That's because they are the primary authors of the project (even though they might not have been listed as such on PyPI).
The case people are worried about is whether someone who is not part of the original project author(s) or maintainer(s) can get assigned the PyPI project. In that case people should use one of the forks; there's no reason for PyPI to crown a successor.
It can be considerable effort to communicate new names throughout the community. FWIW I am pretty happy that e.g. for pytest-cov we got a new maintainer because people use that module heavily, it's advertised in many places, part of tox.ini files etc, yet the original author did not respond and did not release updates for years.
To that I think it'd be great to have a documented procedure for doing transfers and even rough guidelines as to when it makes sense to do it, but that ultimately it would be a decision up to the maintainers of PyPI (currently Richard and myself, though I rarely get involved in that side of things).
I think the "rough guidelines" should actually be quite strict about it (see above). Also, publicity is important (see Daniel's original post).
I think a documented procedure makes sense. FWIW i haven't heart of any complaints from original authors/maintainers regarding the way how ownership was transferred, in the last 10 years or so. Many people are not aware that transfering ownership is possible. best, holger
Regards
Antoine.
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
On 23 Sep 2014 00:19, "Antoine Pitrou" <antoine@python.org> wrote:
Donald Stufft <donald <at> stufft.io> writes:
PyPI inherinently has complete control over who owns what name on PyPI.
Political authority does not derive from technical control, though.
As Toshio said that are situations where it makes *obvious* sense to
ownership of a project. Using Django as an pretty good example here, There are four people able to make releases there, until fairly recently there were only two if I recall. I don't think anyone would be against PyPI transfering ownership of Django to another active core developer of Django in the event that all of the people with permissions on PyPI were gone in some fashion.
Assuming the remaining Django core developers agree on it, then, yes, that can make sense. That's because they are the primary authors of the project (even though they might not have been listed as such on PyPI).
The case people are worried about is whether someone who is not part of
original project author(s) or maintainer(s) can get assigned the PyPI
transfer the project.
In that case people should use one of the forks; there's no reason for PyPI to crown a successor.
That's why I consider it important to get the original project's issue tracker involved in the transfer process. I'd also be OK with a process that required an affirmative "Yes" from the project community, defaulting to "No transfer" in the case of a lack of response. Transfers are most needed for highly active projects where a fork could have a lot of ripple effects. I think it's reasonable to interpret "nobody cared enough to say yes or no" as "nobody cares enough for a transfer to be needed - just fork it rather than claiming the name". Regards, Nick.
Hi folks, sorry for the delay, I was on vacation and then catching up on stuff. I've composed a draft of the policy and I welcome your comments (in the doc, please): https://docs.google.com/document/d/1elum7ENjQb0dLB4ATfYNtnXYVLUzsKacc0VWnHHJ... My apologies if I've missed some nuance in a particular contribution to this discussion: please just leave a comment in the doc :) Richard On 23 September 2014 07:49, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 23 Sep 2014 00:19, "Antoine Pitrou" <antoine@python.org> wrote:
Donald Stufft <donald <at> stufft.io> writes:
PyPI inherinently has complete control over who owns what name on PyPI.
Political authority does not derive from technical control, though.
As Toshio said that are situations where it makes *obvious* sense to
ownership of a project. Using Django as an pretty good example here, There are four people able to make releases there, until fairly recently there were only two if I recall. I don't think anyone would be against PyPI transfering ownership of Django to another active core developer of Django in the event that all of the people with permissions on PyPI were gone in some fashion.
Assuming the remaining Django core developers agree on it, then, yes,
can make sense. That's because they are the primary authors of the
(even though they might not have been listed as such on PyPI).
The case people are worried about is whether someone who is not part of
original project author(s) or maintainer(s) can get assigned the PyPI
transfer that project the project.
In that case people should use one of the forks; there's no reason for PyPI to crown a successor.
That's why I consider it important to get the original project's issue tracker involved in the transfer process. I'd also be OK with a process that required an affirmative "Yes" from the project community, defaulting to "No transfer" in the case of a lack of response.
Transfers are most needed for highly active projects where a fork could have a lot of ripple effects. I think it's reasonable to interpret "nobody cared enough to say yes or no" as "nobody cares enough for a transfer to be needed - just fork it rather than claiming the name".
Regards, Nick.
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Hi all, Having had some time to think this over, I will attempt to explain what the current process is, and how I believe I should change it. It's worth noting that I'm the only person who handles support issues for PyPI (years ago Martin von Lowis also did this, and Donald Stufft has handled one or two cases over the years). There's various reasons for this, not the least of which is that direct ssh/database access is often required to investigate ownership issues. When someone requests to take over a listing on PyPI, the process is: * If the request comes in through some means other than the sf.net support tracker, I require the requestor to make the request through that tracker so there is a record, * I ask whether they have contact the current owner, * I personally contact the owner through whatever means I have (sometimes this means using the address listed for the user in PyPI, sometimes that address is not valid so I use other means where possible), * If contact is made, I ask the current owner to grant the requestor ownership if they feel it is appropriate, * If contact is not made after one month, I add the requestor as an owner. Between the support tracker and PyPI's audit log, all those actions are recorded. However, in this instance, two things did not happen: * I did not record that I had attempted to contact James in the tracker, and * I did not use the listed contact address for James in my attempt to contact him, rather using the address I had in my personal address book. I cannot definitively explain why I didn't do the first step. On the second count though, I can only claim laziness combined with my usually handling these requests in a bunch at 5pm or later after a work day (basically, when I can find a few moments to deal with the backlog). Actually, I think I might have been in an airport transit lounge in that particular instance. It was just easier to use the address I knew than to go through the hoops to find out the correct address to use. Not trying to excuse myself, just explain. There's been some suggestions made: * Publicly announcing the intention to make the change is a good one, though again finding an appropriate forum that enough people would actually read is tricky. * Implement some sort of automated process. Given that we've struggled to produce Warehouse over *years* of development, I don't see this happening any time soon. In light of this specific case, I have an additional change that I think I'll implement to attempt to prevent it again: In the instances where the current owner is unresponsive to my attempts to contact them, *and* the project has releases in the index, I will not transfer ownership. In the cases where no releases have been made I will continue to transfer ownership. Your thoughts, as always, are welcome. Thanks to Danny for bringing the issue up, and to James and Alex for presenting their security concerns so clearly. Richard On 20 September 2014 04:47, Daniel Greenfeld <pydanny@gmail.com> wrote:
In order to claim a package as being abandoned it should undergo a formal process that includes:
* Placement on a PUBLIC list of packages under review for a grace period to be determined by this discussion * Formal attempts via email and social media (twitter, github, et al) to contact the maintainer. * Investigation of the claimant for the rights to the package. The parties attempting to claim a package may not be the best representatives of the community behind that package, or the Python community in general.
Why?
* Non-reply does not equal consent. * Access to a commonly (or uncommonly) used package poses security and reliability issues.
Why:
Scenario 1:
I could claim ownership of the redis package, providing a certain-to-fail email for the maintainers of PyPI to investigate? Right now the process leads me to think I would succeed in gaining access. If successful, I would gain complete access to a package used by hundreds of projects for persistence storage.
Scenario 2:
I could claim ownership of the redis package, while Andy McCurdy (maintainer) was on vacation for two weeks, or sabbatical for six weeks. Again, I would gain access because under the current system non-reply equals consent.
Reference:
In ticket #407 (https://sourceforge.net/p/pypi/support-requests/407/) someone who does not appear to be vetted managed to gain control of the (arguably) abandoned but still extremely popular django-registration on PyPI. They run one of several HUNDRED forks of django-registration, one that is arguably not the most commonly used.
My concern is that as django-registration is the leading package for handling system registration for Python's most popular web framework, handing it over without a full investigation of not just the current maintainer but also the candidate maintainer is risky.
Regards,
Daniel Greenfeld pydanny@gmail.com _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
On Fri, Sep 19, 2014 at 9:26 PM, Richard Jones <richard@python.org> wrote:
When someone requests to take over a listing on PyPI, the process is: i * If the request comes in through some means other than the sf.net support tracker, I require the requestor to make the request through that tracker so there is a record,ard * I ask whether they have contact the current owner, * I personally contact the owner through whatever means I have (sometimes this means using the address listed for the user in PyPI, sometimes that address is not valid so I use other means where possible),
This seems like the step where change would be most fruitful. The idea of a public list mentioned before allows a variety of feedback: 1) The maintainer themselves 2) People who know the maintainer and have an alternate method to contact them 3) Other people who know the project and can raise an objection to the exact person who is being added as a new owner Another thought here is that it's often best to use every means of contacting someone that you reasonably have available. So if there's a valid mail in pypi and a valid email in your contacts, use both. The public list idea essentially lets you crowdsource additional methods of contacting the maintainer.
There's been some suggestions made:
* Publicly announcing the intention to make the change is a good one, though again finding an appropriate forum that enough people would actually read is tricky.
If there's no appropriate forum, starting a new one might be the best. "Uploaders to pypi" could certainly be seen as an audience that doesn't match well with any other existing mailing list.
In light of this specific case, I have an additional change that I think I'll implement to attempt to prevent it again: In the instances where the current owner is unresponsive to my attempts to contact them, *and* the project has releases in the index, I will not transfer ownership. In the cases where no releases have been made I will continue to transfer ownership.
This is tricky. There are certainly security issues with allowing just anyone to take over a popular package at any time. But there are also security concerns with letting a package bitrot on pypi. Say that the 4 pypi maintainers of Django or the 6 pypi maintainers of pip became unresponsive (it doesn't even have to be forever... that 6 month sabbatical could correspond with something happening to your co-maintainers as well). And the still active upstream project makes a new security fix that they need to get into the hands of their users ASAP. We don't want pypi to block that update from going out. Even if the project creates a new pypi package name and uploads there, would we really want the last package on pypi that all sorts of old documentation and blog posts on the internet is pointing to to be the insecure one? So I don't think an absolute "we will never transfer ownership once code is released" is a good idea here. It's a good idea to increase the means used to determine if the current maintainer can be reached and it's a good idea to throw extra eyes at vetting whether a transfer is warranted. It may be a good idea to add more criteria around what makes for an allowable transfer (for instance, in my examples, there's still a large, well known canonical upstream even though the specific members of that upstream responsible for uploading to pypi have gone unresponsive. That might be a valid criteria whereas one-coder projects being replaced by other one-coder forks might be a case where you simply say "rename please"). It could help to have other people involved in the decision making for this. At the least, having other people involved will spread responsibility. At best it gives the group additional man-hours to research the facts in the case. One final thought in regards to ticket 407. My impression from reading the notes is that this was not a complete invalidation of the current process. In the end, the current owner was alerted to the takeover attempt and also was in a position to do something about it since they disagreed with what was happening. Those are both points in favor of some pieces of the process (adding the new owner instead of replacing the owner). This might not be sufficient for a malicious attack on a project but it does show that the process does have some good features in terms of dealing with mistakes in communication. -Toshio
Hi all. TL;DR version: I think * an option to enroll in automatic ownership transfer * an option to promote Request for Adoption * don't transfer unless there are no releases on the index will be reasonable to me. On Fri, Sep 19, 2014 at 9:26 PM, Richard Jones <richard@python.org> wrote:
In light of this specific case, I have an additional change that I think I'll implement to attempt to prevent it again: In the instances where the current owner is unresponsive to my attempts to contact them, *and* the project has releases in the index, I will not transfer ownership. In the cases where no releases have been made I will continue to transfer ownership.
I believe this is the best solution, and frankly, people in the OSS world has been forking all these years should someone disagree with the upstream or just believe they are better off with the fork. I am not a lawyer, but one has to look at any legal issue with ownership transfer. I am not trying to scare anyone, but the way I see ownership transfer (or even modifying the index on behalf of me) is the same as asking Twitter or Github to grant me a username simply because the account has zero activity. Between transferring ownership automatically after N trials and the above, I choose the above. Note not everyone is on Github, twitter. Email, er, email send/receive can go wrong. As a somewhat extreme but not entirely rare example, Satoshi Nakamoto and Bitcoin would be an interesting argument. If Bitcoin was published as a package on PyPI, should someone just go and ask for transfer? We know this person shared his codes and the person disappeared. Is Bitcoin mission-critical? People downloaded the code, fork it and started building a community on their own. What I am illustrating here is that not everyone can be in touch. There are people who choose to remain anonymous, or away from popular social network. Toshio Kuratomi <a.badger@gmail.com> wrote:
But there are also security concerns with letting a package bitrot on pypi.
Again, I think that people should simply fork. The best we can do is simply prevent the packages from being downloaded again. Basically, shield all the packages from public. We preserve what people did and had. We can post a notice so the public knows what is going on. Surely it sucks to have to use a fork when Django or Requests are forked and now everyone has to call it something different and rewrite their code. But that's the beginning of a new chapter. The community has to be reformed. It sucks but I think it is better in the long run. You don't have to argue with the original owner anymore in theory. Last, I think it is reasonable to add Request for Adoption to PyPI. Owners who feel obligated to step down can promote the intent over PyPI John
On Sat, Sep 20, 2014 at 12:30 AM, John Wong <gokoproject@gmail.com> wrote:
Hi all.
TL;DR version: I think
* an option to enroll in automatic ownership transfer * an option to promote Request for Adoption * don't transfer unless there are no releases on the index
will be reasonable to me.
On Fri, Sep 19, 2014 at 9:26 PM, Richard Jones <richard@python.org> wrote:
In light of this specific case, I have an additional change that I think I'll implement to attempt to prevent it again: In the instances where the current owner is unresponsive to my attempts to contact them, *and* the project has releases in the index, I will not transfer ownership. In the cases where no releases have been made I will continue to transfer ownership.
I believe this is the best solution, and frankly, people in the OSS world has been forking all these years should someone disagree with the upstream or just believe they are better off with the fork. I am not a lawyer, but one has to look at any legal issue with ownership transfer. I am not trying to scare anyone, but the way I see ownership transfer (or even modifying the index on behalf of me) is the same as asking Twitter or Github to grant me a username simply because the account has zero activity.
Between transferring ownership automatically after N trials and the above, I choose the above. Note not everyone is on Github, twitter. Email, er, email send/receive can go wrong.
As a somewhat extreme but not entirely rare example, Satoshi Nakamoto and Bitcoin would be an interesting argument. If Bitcoin was published as a package on PyPI, should someone just go and ask for transfer? We know this person shared his codes and the person disappeared. Is Bitcoin mission-critical? People downloaded the code, fork it and started building a community on their own. What I am illustrating here is that not everyone can be in touch. There are people who choose to remain anonymous, or away from popular social network.
Toshio Kuratomi <a.badger@gmail.com> wrote:
But there are also security concerns with letting a package bitrot on pypi.
Again, I think that people should simply fork. The best we can do is simply prevent the packages from being downloaded again. Basically, shield all the packages from public. We preserve what people did and had. We can post a notice so the public knows what is going on.
Surely it sucks to have to use a fork when Django or Requests are forked and now everyone has to call it something different and rewrite their code. But that's the beginning of a new chapter. The community has to be reformed. It sucks but I think it is better in the long run. You don't have to argue with the original owner anymore in theory.
Last, I think it is reasonable to add Request for Adoption to PyPI. Owners who feel obligated to step down can promote the intent over PyPI
John
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
I, for one, am happy that this conversation is happening because I wasn't aware of other communities that did this (but was aware that it happened on PyPI). That said, I would really appreciate people's suggestions be contained to improving the process, not towards modifying PyPI. At this point, as I understand it, PyPI is incredibly hard to modify safely for a number of reasons that others are likely better to speak to. Warehouse has a clear definition, design, and goals and I don't know if adding these on after-the-fact in a semi-haphazard way will improve anything. The more useful discussion right now will be to talk about process and how we can improve it and help Richard with it. Cheers
On Sep 20, 2014, at 11:26 AM, Ian Cordasco <graffatcolmingov@gmail.com> wrote:
I, for one, am happy that this conversation is happening because I wasn't aware of other communities that did this (but was aware that it happened on PyPI). That said, I would really appreciate people's suggestions be contained to improving the process, not towards modifying PyPI. At this point, as I understand it, PyPI is incredibly hard to modify safely for a number of reasons that others are likely better to speak to. Warehouse has a clear definition, design, and goals and I don't know if adding these on after-the-fact in a semi-haphazard way will improve anything. The more useful discussion right now will be to talk about process and how we can improve it and help Richard with it.
Cheers _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
For the record, CPAN and npm both have similar things allowing someone to take over an abandoned project. I don’t believe ruby gems has an official policy and it appears that they are hesitant to do this form the threads I’ve seen (Though they mentioned doing it for _why). Most of the Linux distros have some mechanism for someone to claim that a particular package in the distro is no longer maintained and to attempt to take it over, though that is somewhat different. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On 21 September 2014 01:34, Donald Stufft <donald@stufft.io> wrote:
Most of the Linux distros have some mechanism for someone to claim that a particular package in the distro is no longer maintained and to attempt to take it over, though that is somewhat different.
For reference, Fedora's non-responsive maintainer policy: https://fedoraproject.org/wiki/Policy_for_nonresponsive_package_maintainers "Just fork it" usually isn't an acceptable solution in the distro case, as it creates an incredible amount of rework in other packages for the distro maintainers *and* for distro users as they also need to adjust to the new name. Even when *we're* the ones creating the new tool (e.g. the currently ongoing yum -> dnf transition), it's entirely possible that at the end of process, the new tool (or a compatibility shim based on it) will take over the original name to minimise the ripple effects on the rest of the ecosystem. When forks of major upstream packages happen (ranging from relative small ones like PIL -> Pillow to core dependency changes and community splits like XFree86 -> x.org), the amount of rework generated just on a per-distro basis can be significant, let alone after you account for folks consuming upstream directly. (Triggering external rework is also one of the largest costs associated with major platform upgrades, like Python 2 -> 3, Gnome 2->3, KDE 3->4, the Windows 8-bit -> UTF-16-LE transition, Windows security hardening, RHEL/CentOS major version number bumps, etc. In those cases, the main mitigation effort is to maintain the old platform for an extended period, to allow the cost to be spread out over several years, rather than having to be addressed in a small time window). I do like the preferred contact mechanism listed in the Fedora policy guide though: filing a bug against the package in the Fedora issue tracker. Including the relevant project issue tracker in the required points of contact for PyPI ownership transfer could be a good enhancement over doing it via email, and providing a public record of the interaction between the two communities by cross-linking the PyPI bug requesting the transfer, and the project bug notifying them of the project transfer request. The usage of the project issue tracker also makes it clear that going to the PyPI admins *won't help* if you're having a dispute with currently active project maintainers. Adapting Fedora's process wholesale might look something like: 1. Requestor starts with a ticket on the PyPI issue tracker 2. If there isn't already one on the project issue tracker, the requestor is instructed to file it and link it from the PyPI issue 3. The PyPI admins will notify the registered package maintainer of the transfer request (and both tickets) via all available contact mechanisms 4. The requestor must ping the project ticket and the PyPI ticket after 7 (or more) days, and then again after another 7 (or more) days 5. If there's no response 7 (or more) days after the second ping, the requestor must notify distutils-sig in addition to pinging both tickets a third time 6. If there's no response 7 (or more) days after notifying distutils-sig, then the requestor is *added* to the maintainer list for the project Exception: I'm not sure how best to handle cases where the affected project has no public bug tracker. However, the requestor should definitely be required to *set up* a bug tracker for the project in such cases as part of the handover of responsibility. Some advantages of such an approach: 1. It's mostly public - the only "offline" part is the notification from the PyPI admins directly to the existing PyPI package maintainers 2. It should be scalable, as the folks making the transfer request are responsible for doing most of the work. If they don't do the work, the transfer simply doesn't happen 3. The minimum transfer time is 4 weeks, which will cover many instances of vacation (etc), and provide plenty of time for folks to notice the request (including folks that aren't the package maintainer, but may be aware if they're away from an extended period, and haven't arranged for anyone else to handle bug reports in the meantime) I had a harder time finding the relevant Debian policy info, but this seems to be it: https://www.debian.org/doc/manuals/developers-reference/beyond-pkging.html#m... One common element to both systems: both Fedora and Debian have the capacity for maintainers to officially declare a package an "orphan". While I don't think it's urgent, I'd actually really like to see such a feature for PyPI at some point in the future, independently of the ownership transfer questions. A library like my own shell-command, for example, is usable as is, but there's zero chance I'll ever go back to working on it myself, as I no longer like the basic design concept. If I could flag that directly in PyPI, that would be useful in a few ways: 1. Folks that are *considering* using it would get a clear notice that it's unmaintained and perhaps move on to something with an active maintainer 2. Tools like "pip list" could grow a "--orphaned" option to show unmaintained dependencies 3. A simpler "fast path" could be added to the ownership transfer process for already orphaned packages One possible way approach to handling cases where folks are out of touch for extended periods without arranging for someone else to handle responding to upstream maintenance requests is to offer a calendar application where highly engaged package maintainers could explicitly notify the PyPI admins of an extended lack of availability. We probably don't need to go that far (folks that would be engaged enough to do that are likely to also be engaged enough to have other other folks participating on their issue tracker that would be aware of their extended absence), but if we ever decided such a service would be useful, Fedora also maintain a calendar app specifically to allow folks to indicate lack of availability: https://apps.fedoraproject.org/calendar/list/vacation/ (Project page for the Flask app itself: http://fedocal.readthedocs.org) We probably don't need to go that far, but it would be one possible way for project maintainers to explicitly tell the PyPI admins about lack of availability, without needing to change PyPI itself, and without trying to manage such notifications manually via email. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sat, Sep 20, 2014 at 11:34 AM, Donald Stufft <donald@stufft.io> wrote:
For the record, CPAN and npm both have similar things allowing someone to take over an abandoned project.
I don’t believe ruby gems has an official policy and it appears that they are hesitant to do this form the threads I’ve seen (Though they mentioned doing it for _why).
Good information.
Most of the Linux distros have some mechanism for someone to claim that a particular package in the distro is no longer maintained and to attempt to take it over, though is somewhat different.
yeah, I come from distro land but I'm hesitant to point directly at any of our documented policies on this because there are some differences between being a bunch of people working together to make a set of curated and integrated packages vs a loosely associated group of developers who happen to use a shared namespace within a popular service. All distros I can think of have some sort of self-governance whereas pypi is more akin to a bunch of customers making use of a service. Some of the distro policies don't apply very well in this space. Some do, however, so I hope other people who are familiar with their distros will also filter the relevant policy ideas from their realms and put them forward. -Toshio
On Sep 20, 2014, at 06:10 PM, Toshio Kuratomi wrote:
All distros I can think of have some sort of self-governance whereas pypi is more akin to a bunch of customers making use of a service. Some of the distro policies don't apply very well in this space. Some do, however, so I hope other people who are familiar with their distros will also filter the relevant policy ideas from their realms and put them forward.
Debian and Ubuntu have very interesting differences, especially given that one is derived from the other. Debian has a very strong personal maintainership culture, where often one person maintains a package alone. Debian Developers have upload rights for any package, and they can do non-maintainer uploads. Debian also has various policies related to orphaning and adopting packages, but those are mostly for cooperative package ownership transfers. When a maintainer cannot be contacted, there is a missing-in-action process that can be used to wrest ownership for a non-responsive maintainer. Many packages are team maintained, and I personally find these much more productive and enjoyable to work with. A team maintained package doesn't have to worry about ownership transfers because any team member with general upload rights can upload the package, and even non-uploading team members can do everything short of that. Primarily that means prepare the package's vcs so that it's ready to be sponsored by an uploading developer. Ubuntu is different in that no package is maintained by a single person. Essentially they are all team maintained. Rights to upload packages are conferred on the basis of "pockets" and package-sets. So for example, if someone wants to be involved in Zope, they could join the ~ubuntu-zope-dev team and once approved, they'd have upload rights to any package in the Zope package set. There are also pockets such as universe (packages which are available in Ubuntu but without security and other distro-level guarantees), and there is a MOTU (masters of the universe) team that can upload there. At the top of the ladder, core-devs can upload anything. In the context of PyPI, I tend to think that teams can be an answer to a lot of the problem. I'm looking for example at one of the lazr.* packages I co-maintain on PyPI. The package has a number of individual owner roles, but I know that there are probably only two of those people who still care enough about the package to maintain it upstream, or would ever likely upload new versions to PyPI. Handling roles in this way is pretty inconvenient because there might be dozens of packages that some combination of those group of people would be responsible for. If I could create a LAZR team and manage roles within that team, and then assign ownership of a package to that team, it would be easier to administer. That doesn't solve the problem where individuals have a strong preference for personal ownership of PyPI entries, but given that upstreams often are a team effort, I think it would go a long way toward helping easy transition efforts for PyPI ownership. It might even allow for better handling of transitions. For example, if a package owner is not reachable for some period of time, and someone steps up to take it over, you could create a team and put both people in it, then transfer the ownership to that team. Cheers, -Barry
On 21 September 2014 09:06, Barry Warsaw <barry@python.org> wrote:
In the context of PyPI, I tend to think that teams can be an answer to a lot of the problem. I'm looking for example at one of the lazr.* packages I co-maintain on PyPI. The package has a number of individual owner roles, but I know that there are probably only two of those people who still care enough about the package to maintain it upstream, or would ever likely upload new versions to PyPI. Handling roles in this way is pretty inconvenient because there might be dozens of packages that some combination of those group of people would be responsible for. If I could create a LAZR team and manage roles within that team, and then assign ownership of a package to that team, it would be easier to administer.
That doesn't solve the problem where individuals have a strong preference for personal ownership of PyPI entries, but given that upstreams often are a team effort, I think it would go a long way toward helping easy transition efforts for PyPI ownership.
Right, it also better reflects the way a lot of folks are organising their work on GitHub/BitBucket/GitLab/RhodeCode/etc these days - you have a team of folks with relatively broad permissions as part of an "organisation" (e.g. PyPA) and then multiple projects within those teams. In theory, anyone on the developer list for the organisation can commit directly to any of the repos, but in practice we don't. However, adding teams support would mean a *lot* more engineering work on the PyPI/Warehouse side, so it won't be practical until after the Warehouse development effort comes to fruition and takes over the pypi.python.org domain, allowing the legacy PyPI application to be retired. A major part of that effort is actually cleaning up the PyPI APIs to separate "useful and necessary public interface" from "legacy PyPI implementation detail", so that those legacy features can simply never be implemented in Warehouse at all - that's fairly slow going, since it means a fair bit of negotiation with the client tool developers, as folks figure out what is needed and what isn't.
It might even allow for better handling of transitions. For example, if a package owner is not reachable for some period of time, and someone steps up to take it over, you could create a team and put both people in it, then transfer the ownership to that team.
That's sort of what happens now - the requestor is *added* to the admin list, but the previous maintainer remains as co-owner. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sat, Sep 20, 2014 at 1:30 AM, John Wong <gokoproject@gmail.com> wrote:
Hi all.
TL;DR version: I think
* an option to enroll in automatic ownership transfer * an option to promote Request for Adoption * don't transfer unless there are no releases on the index
will be reasonable to me.
On Fri, Sep 19, 2014 at 9:26 PM, Richard Jones <richard@python.org> wrote:
In light of this specific case, I have an additional change that I think I'll implement to attempt to prevent it again: In the instances where the current owner is unresponsive to my attempts to contact them, *and* the project has releases in the index, I will not transfer ownership. In the cases where no releases have been made I will continue to transfer ownership.
I believe this is the best solution, and frankly, people in the OSS world has been forking all these years should someone disagree with the upstream or just believe they are better off with the fork. I am not a lawyer, but one has to look at any legal issue with ownership transfer. I am not trying to scare anyone, but the way I see ownership transfer (or even modifying the index on behalf of me) is the same as asking Twitter or Github to grant me a username simply because the account has zero activity.
This is a great example, however, I think that you're assuming that the answer to the question of whether services like twitter and github (and facebook and email service providers and many other service-customer relationships) should sometimes grant username takeovers is 100% no and I don't believe that's the case. I mean, in the past year there was a loud outcry about facebook not being willing to grant access to an account where the user had died and their family wanted access to the data so as to preserve it. Facebook eventually granted access in that case. Email has historically been transferred quite frequently. When you quit a job or leave a university your email address is often taken from you and, when someone with a similar name or inclination arrives, that address can be given to someone else.
Toshio Kuratomi <a.badger@gmail.com> wrote:
But there are also security concerns with letting a package bitrot on pypi.
Again, I think that people should simply fork. The best we can do is simply prevent the packages from being downloaded again. Basically, shield all the packages from public. We preserve what people did and had. We can post a notice so the public knows what is going on.
Surely it sucks to have to use a fork when Django or Requests are forked and now everyone has to call it something different and rewrite their code. But that's the beginning of a new chapter. The community has to be reformed. It sucks but I think it is better in the long run. You don't have to argue with the original owner anymore in theory.
I'm on the fence over the model that I think you've got in your head here but I think it's more important to talk about why I think demanding people fork is the wrong path to take in my example which I think is much more cut and dried. Let's say you belong to a large project with 50 committers and a user's mailing list that numbers in the thousands of subscribers. The project owns a domain name with a large website and shoot, maybe they even have a legal body that serves as a place to take donations, register trademarks and so forth. You happen to be the release manager. You've been with the project since it was a small 5 person endeavour. While everyone else was busy coding, you specialized in deployment, installation, and, of course, creating a tarball to upload to pypi on every release. People may oca casionally think that they should help you out but hey, you've always done it, no one has reason to complain, and besides, there's this really important bug that they should be working on fixing instead.... So then you die. It's unexpected. Hit by a bus. Eaten by a velociraptor. You know the various hypothetical scenarios. Well, the project is still vibrant. It still has 49 committers. It's still the owner of a trademark and a domain name. It still has thousands of users. Now that it's a necessity, it can even find that has other people to volunteer to replace you as release manager. What it doesn't have, is permission to upload to pypi anymore. I think if someone asked to transfer ownership to another member of the upstream project had the time to research this they'd have no trouble at all deciding that the right course of action would be to transfer ownership. In this scenario all of the facts point towards the upstream being the people who should have rights to upload to pypi and they simply didn't have the foresight to assure that they wouldn't lose that right through an accident. Now what if we start taking some of the features of the scenario away? What if there wasn't a foundation? A trademark? A domain name? What if the release manager disappeared from the internetz and no one knew if he was alive or not? What if he was on a two week vacation but the project found that they had to make an unexpected release to fix to a horrendous security exploit ASAP? I think that pypi works best if we can be accommodating to these circumstances when there's a clear story of whether the requestor is a valid person to take ownership. We can quibble about the criteria for determining that validity but I think we should keep in forefront of our minds that all of our goals is to give the user the code they are searching for and not some other code. In some cases this will mean that we shouldn't transfer ownership when the new owner is a random fork but in others it will mean that we should transfer ownership because everyone expects the code to be this specific fork. -Toshio
participants (15)
-
Alex Gaynor -
Antoine Pitrou -
Barry Warsaw -
Daniel Greenfeld -
Donald Stufft -
Ethan Furman -
holger krekel -
Ian Cordasco -
James Bennett -
John Wong -
Nick Coghlan -
Richard Jones -
Stefan Krah -
Steve Dower -
Toshio Kuratomi