has_security_fixes flag in PyPI
Hello, I'd like to discuss and possibly implement a feature of PyPI that would facilitate in quickly discovering which of a given project's dependencies need to be updated because of new security-related fixes. A little background is in order. Zato has currently 80+ buildout dependencies https://github.com/zatosource/zato/blob/master/code/versions.cfg and I'm betting at least 80 more will be added with time. I'm in a camp that has absolutely no problems with as many dependencies as it's needed if it saves time and means less wheels being reinvented. Once a dependency is in, it's pinned to a concrete version and that version is updated only in a couple of situations 1. Stability fixes that are to do with the functionality currently in use by Zato 2. A dependency offers a new functionality I'd like to make use of 3. A version disappears from PyPI 4. A security fix is available For the last point, it would be really convenient if authors were offered a 'contains security fixes' kind of checkboxes somewhere in the GUI. This would be displayed in a couple of places - On the project's PyPI page, for instance here - https://pypi.python.org/pypi/redis - there could be a 'This version contains security fixes' box right below the download button - Would be added to the Recent Updates feed https://pypi.python.org/pypi?%3Aaction=rss - There would be a new feed at /pypi?%3Aaction=security_rss that would list only these recent uploads that have the flag set As far as the underlying database goes, this would be a single boolean column in the 'releases' table. Such a feature would allow for quickly reacting to any security changes without chasing dozens of mailing lists, Twitter, RSS, asking authors to be notified when they change something etc. Naturally, nothing would force people to actually use it but authors who treat their own work seriously would hopefully find it an interesting addition as well. I'm familiarizing myself with https://bitbucket.org/pypa/pypi right now but I'd like to ask you if such a feature would be accepted at all if I implemented it. Also, it's not a priority one so if someone beats me to it, it's all good with me. cheers and take care, -- Dariusz Suchojad https://zato.io ESB, SOA and cloud integrations in Python
I work in the same office as some of the folks that are working on the victi.ms vulnerability database for Java projects, and they recently asked me for advice about how to add Python support (they've also been discussing the addition of Ruby support with some of the rubygems.org devs). So, rather than doing anything purely Python specific, I suspect we're more likely to focus on collaborating effectively with victi.ms rather than duplicating their work. Near term, major new features aren't likely to be added to the current PyPI code base - the current PyPI development efforts are mostly focused on migrating to a new architecture where the data integrity constraints are strictly enforced at the database layer. Cheers, Nick.
On Sep 21, 2013, at 10:42 AM, Nick Coghlan
I work in the same office as some of the folks that are working on the victi.ms vulnerability database for Java projects, and they recently asked me for advice about how to add Python support (they've also been discussing the addition of Ruby support with some of the rubygems.org devs).
So, rather than doing anything purely Python specific, I suspect we're more likely to focus on collaborating effectively with victi.ms rather than duplicating their work.
Near term, major new features aren't likely to be added to the current PyPI code base - the current PyPI development efforts are mostly focused on migrating to a new architecture where the data integrity constraints are strictly enforced at the database layer.
Cheers, Nick.
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Just to throw my 2 cents in here as a maintainer of both pip and PyPI. Something like this is definitely something I want to have. I'd need to spend some time thinking more about it because It's been on my "future" stack. It's likely that integrating with a trusted third party service is a viable (and good!) option since historically PyPI itself hasn't been in the business of vetting what people upload to it. Any changes to PyPI would require the projects themselves to flag a security issue which won't always happen. A third party project allows a neutral party to handle this. Also as Nick said PyPI itself is mostly in a holding pattern while a 2.0 is being phased in, new features *are* possible but they are all weighed against the amount of effort it will take (x2). ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On 09/21/2013 04:51 PM, Donald Stufft wrote:
Any changes to PyPI would require the projects themselves to flag a security issue which won't always happen. A third party project allows a neutral party to handle this.
One thing I don't fully get is how victi.ms - or any third party - collect information regarding the vulnerabilities? I understand there would be two sources of information? - public vulnerability databases - data submitted by package maintainers themselves (this would have to be routed to a third party somehow)
Also as Nick said PyPI itself is mostly in a holding pattern while a 2.0 is being phased in, new features *are* possible but they are all weighed against the amount of effort it will take (x2).
Sure, I understand it now. cheers, -- Dariusz Suchojad https://zato.io ESB, SOA and cloud integrations in Python
On 09/21/2013 04:51 PM, Donald Stufft wrote:
Any changes to PyPI would require the projects themselves to flag a security issue which won't always happen. A third party project allows a neutral party to handle this.
One thing I don't fully get is how victi.ms - or any third party - collect information regarding the vulnerabilities? I understand there would be two sources of information? - public vulnerability databases - data submitted by package maintainers themselves (this would have to be routed to a third party somehow)
Also as Nick said PyPI itself is mostly in a holding pattern while a 2.0 is being phased in, new features *are* possible but they are all weighed against the amount of effort it will take (x2).
Sure, I understand it now. cheers, -- Dariusz Suchojad https://zato.io ESB, SOA and cloud integrations in Python
On 22 Sep 2013 01:20, "Dariusz Suchojad"
On 09/21/2013 04:51 PM, Donald Stufft wrote:
Any changes to PyPI would require the projects themselves to flag a security issue which won't always happen. A third party project allows a neutral party to handle this.
One thing I don't fully get is how victi.ms - or any third party - collect information regarding the vulnerabilities?
I understand there would be two sources of information?
- public vulnerability databases - data submitted by package maintainers themselves (this would have to be routed to a third party somehow)
victi.ms is still in the process of launching - they want to have at least Java, Python and Ruby support before making a big push to promote it as a resource. I believe the initial intent is for victi.ms to focus on mapping CVE numbers to upstream packages, and then have optional plugins to check Maven builds, Ruby gem dependencies and Python virtual environments for known vulnerable versions. For PyPI integration, I would expect to initially see us as just a consumer of the data, displaying CVE information on PyPI when available, and likely republishing it through the PyPI APIs. Even that would be a big step forward from where we are now :) Cheers, Nick.
Also as Nick said PyPI itself is mostly in a holding pattern while a 2.0 is being phased in, new features *are* possible but they are all weighed against the amount of effort it will take (x2).
Sure, I understand it now.
cheers,
-- Dariusz Suchojad
https://zato.io ESB, SOA and cloud integrations in Python
participants (3)
-
Dariusz Suchojad
-
Donald Stufft
-
Nick Coghlan