[Distutils] Docker Content Trust and PyPI package signing

[Wes Turner]

The Update Framework
| Homepage: http://theupdateframework.com/
| Src: https://github.com/theupdateframework/tuf
| PyPI: https://pypi.python.org/pypi/tuf


Esky does transactional upgrades: https://github.com/cloudmatrix/esky/

Pants freezes apps into single file dists:
https://pantsbuild.github.io/python-readme.html

... probably relevant to python package signining:

pypa/pip
Implement "hook" support for package signature verification. #1035

https://github.com/pypa/pip/issues/1035#issuecomment-20656810
*westurner <https://github.com/westurner> *commented on Jul 9, 2013
<https://github.com/pypa/pip/issues/1035#issuecomment-20656810>

A syntax like the following would be convenient:

pip install --verify-<sig> -e git+https://github.com/pypa/pip#egg=pip

...

   - http://www.pip-installer.org/en/latest/usage.html#pip-install
   - http://www.pip-installer.org/en/latest/usage.html#pip-verify ?
   -
   https://python-packaging-user-guide.readthedocs.org/en/latest/packaging_tutorial.html#create-your-first-release

These may be helpful for creating documentation on this feature and how it
relates to other components of a secure python packaging process:

*Source Repository GPG*

   - https://en.wikipedia.org/wiki/GNU_Privacy_Guard (PGP
   <https://en.wikipedia.org/wiki/Pretty_Good_Privacy>)
   - http://stackoverflow.com/questions/10077996/sign-git-commits-with-gpg
   -
   http://stackoverflow.com/questions/11556184/whats-the-purpose-of-signing-changesets-in-mercurial

*Python Package GPG (./<package>.asc)*

   - http://pythonhosted.org/distlib/tutorial.html#signing-a-distribution
   -

   http://pythonhosted.org/distlib/tutorial.html#verifying-signatures

   For any archive downloaded from an index, you can retrieve any signature
   by just appending*.asc* to the path portion of the download URL for the
   archive, and downloading that.

   -


   https://pypi.python.org/packages/source/p/pip/pip-1.3.1.tar.gz.asc#md5=cbb27a191cebc58997c4da8513863153

*Python Wheel JWS S/MIME (PEP 427
<http://www.python.org/dev/peps/pep-0427/>)*

   - http://www.python.org/dev/peps/pep-0427/#signed-wheel-files
   - https://tools.ietf.org/html/draft-ietf-jose-json-web-signature-11 (html
   <https://tools.ietf.org/id/draft-ietf-jose-json-web-signature-11.html>)
   - https://tools.ietf.org/html/draft-ietf-jose-json-web-key-11 (html
   <https://tools.ietf.org/html/draft-ietf-jose-json-web-key-11.html>)
   - https://en.wikipedia.org/wiki/X.509
   - https://bitbucket.org/dholth/wheel/src/tip/wheel/signatures/__init__.py
   -

   https://distlib.readthedocs.org/en/latest/internals.html#the-wheel-api
   -

   https://payswarm.com/specs/source/web-keys/ (http://json-ld.org)

*Index Mirror DSA (PEP 381 <http://www.python.org/dev/peps/pep-0381/>)*

   - http://www.python.org/dev/peps/pep-0381/#mirror-authenticity
   - https://en.wikipedia.org/wiki/Digital_Signature_Algorithm

*Package Signatures for .deb, .rpm, ...
<https://github.com/pypa/pip/issues/GPG>*

   -
   https://en.wikipedia.org/wiki/List_of_software_package_management_systems
   - http://man.he.net/man8/apt-key
   - http://wiki.debian.org/SecureApt
   - http://linux.die.net/man/5/yum.conf # gpgcheck, localpkg_gpgcheck,
   repo_gpgcheck
   - http://iuscommunity.org/pages/CreatingAGPGKeyandSigningRPMs.html

*Python Package Configuration Management Systems*

   -
   https://github.com/puppetlabs/puppet/blob/master/lib/puppet/provider/package/pip.rb
   -
   https://github.com/opscode/chef/blob/master/lib/chef/provider/package/easy_install.rb
   - https://github.com/saltstack/salt/blob/develop/salt/modules/pip.py
   - https://github.com/ansible/ansible/blob/devel/library/packaging/pip
   -
   http://docs.bcfg2.org/server/plugins/generators/packages.html#handling-gpg-keys

*[Cryptographic] Hash Functions*

   -
   https://en.wikipedia.org/wiki/Cryptographic_hash_function#Verifying_the_integrity_of_files_or_messages
   -
   https://en.wikipedia.org/wiki/Cryptographic_hash_function#Hash_functions_based_on_block_ciphers
   - https://en.wikipedia.org/wiki/Category:Cryptographic_hash_functions
   - https://en.wikipedia.org/wiki/Hash_function_security_summary
   -

   https://en.wikipedia.org/wiki/Category:Broken_hash_functions
   - https://en.wikipedia.org/wiki/MD5
      - https://en.wikipedia.org/wiki/SHA-1
   -


   http://pythonhosted.org/passlib/lib/passlib.hash.html#unix-modular-crypt-hashes
   - http://pythonhosted.org/passlib/modular_crypt_format.html


On Sun, Aug 23, 2015 at 7:19 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> Hi folks,
>
> The recent Docker 1.8 release was the first one to include their new
> content signing system, which is described well in this post:
> https://blog.docker.com/2015/08/content-trust-docker-1-8/
>
> The resign I bring that up here is because the Docker Content Trust
> system is based on The Update Framework, which is the same system
> we've been exploring for PyPI package signing in PEPs 458 and 480.
>
> The part I particularly like is the way they have handled the trust
> establishment process for content signing: they use a "trust on first
> use" model by default, similar to that used in SSH. This means there
> is still a reliance on HTTPS and the CA system, but only for the task
> of bootstrapping TUF in a way that allows new clients to obtain the
> public signing certificate of the repo publisher transparently. Once
> the intial trust relationship with a public repo like PyPI or a
> private repo within a company or other organisation has been
> established, later compromises of the CA system don't provide the
> ability to forge package signatures.
>
> Also of potential interest is the TUF-based signing infrastructure
> that Docker built, Notary: https://github.com/docker/notary
>
> While I don't have a strong personal preference one way or the other,
> finding a way to reuse that does seem like it could be an interesting
> architectural alternative to building signing capabilities directly
> into Warehouse itself.
>
> Regards,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20150824/30cdd53a/attachment-0001.html>