On 7/3/12 9:42 AM, Bohuslav Kabrda wrote:

----- Original Message -----

...

I would like to amend the spec. The hash column of RECORD should be

'sha256:' + urlsafe_b64encode(hashlib.sha256(data))

instead of the hopelessly obsolete md5. With a secure hash function, you can digitally sign RECORD.

Signing packages does sound interesting, but what authority would sign them? The authors of the packages themselves?

Notice that there's already a --sign feature in Distutils, using gpg. Hash in the RECORD file have nothing to do with making sure the package is originated from developer X. Its only purpose is to know if a file on the system was changed Cheers Tarek

On 7/3/12 9:48 AM, Donald Stufft wrote:

On Tuesday, July 3, 2012 at 3:45 AM, Tarek Ziadé wrote:

...

Hash in the RECORD file have nothing to do with making sure the package is originated from developer X. Its only purpose is to know if a file on the system was changed

Using sha256 would enable preventing someone from maliciously changing the file.

If someone has access to that file, it means that he can also change the RECORD file so you have no way of trusting RECORD either.

Similar to how IDS systems capture hashes of binaries to compare against. Of course someone using the system like this would need to protect the filesystem storing the RECORD files accordingly.

I think that's the main issue - where are you going to put the RECORD file ?

I also think that switching to sha256 is pretty low cost with minimal (no?) downsides with some possible upsides. Is there a reason to stay with md5?

The file is two times smaller and faster to create, and md5 does its job at providing a hash for a file. I still fail to see a use case for stronger hashes Cheers Tarek

At Tue, 3 Jul 2012 07:14:43 -0400, Daniel Holth wrote:

It's embarrassing to see md5 used for any reason. You go to pypi, and every download link has an md5 sum of the package, and you think "what is this archaic system that gives me a useless hash, implicated in such fine situations as the Flame malware and ever-improving attacks against md5?" It is irrelevant that it is "probably good enough for this limited use". You might as well use CRC32; it is much shorter.

Yes, you're right, pypi could as well use CRC32. From a security perspective nothing would change, nor if we would switch to sha512, because there is no way to know whether the hash is correct. Without a trust path the hash is pretty useless except for verifying that the download isn't corrupted. And even if we would have trust paths, the md5 attacks are collision attacks, not preimage attacks. That means the security threat you're worrying about is that a developer uploads something to pypi with the intention of replacing that by something else with the same hash without anyone noticing. And although it is worthwhile to protect against such kind of things, you should also ask the question why you're running code from such a developer. And yes, attacks on md5 will only get better, so we should migrate to better hashes in the future. But if there is something to be embarrassed about, it's not the use of md5, but the lack of proper code signing and trust paths between developers. Kind regards, Jeroen Dekkers

On 7/3/12 3:54 PM, Daniel Holth wrote:

I'm going to implement this except I will replace the sha256: with a sha256= There is simply no realistic drawback.

I am -1000 for any change to the RECORD file hashes in PEP 376 unless there's a clear use case.

Strong hashing is a prerequisite for a trust path, and you avoid the need to even think about why it is OK in this specific circumstance that a weak hash is being used. Sorry but I don't understand your use case.

What "strong", "weak" or "trust" means here ? The use case we have is: we need a check sum for every file, that's all. If you want to build a system where you can verify the origin of the files, you need something like a public/private key system. Which is what --sign is for. Otherwise you're just going to make hashes longer for no apparent reason. Cheers Tarek

On 7/3/12 4:32 PM, PJ Eby wrote:

On Tue, Jul 3, 2012 at 8:48 AM, Jeroen Dekkers <jeroen@dekkers.ch <mailto:jeroen@dekkers.ch>> wrote:

And yes, attacks on md5 will only get better, so we should migrate to better hashes in the future.

No, because that's not what the RECORD hashes are for. It's not an intrusion detection system, it's an installer conflict and "oops I edited the wrong file" checker.

People who are upset because md5 is low security are correctly understanding that this system *provides no security*. We are not promising ANY security, so *not* using a secure hash is actually preferable. The goal is data integrity against accidental overwrite by dumb installer tools (e.g. distutils) and accidental edits, not security against malicious tampering.

Yeah I don't really understand this debate over md5 hashes here. I suggest that we emphasis in PEP 376 the fact that the sole purpose is to have a checksum.

_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig

I am just re-using record in wheel files so I can implement a verify function someday. Pay no attention to this backward-compatible change. You can use the checksum you prefer, and if it does not begin with hashfunc= then you know it's an md5. No discussion about adding provides-extra and the reserved extra names for python setup.py test? How about that the environment markers spec says you can use == but (naked version number) (4.0) is the only example given for "exactly this version"? And why is pkg-info called metadata now anyway? Daniel Holth On Jul 3, 2012, at 11:10 AM, Éric Araujo <merwok@netwok.org> wrote:

Le 03/07/2012 10:53, Tarek Ziadé a écrit :

...

On 7/3/12 4:32 PM, PJ Eby wrote:

...

No, because that's not what the RECORD hashes are for. It's not an intrusion detection system, it's an installer conflict and "oops I edited the wrong file" checker.

People who are upset because md5 is low security are correctly understanding that this system *provides no security*. We are not promising ANY security, so *not* using a secure hash is actually preferable. The goal is data integrity against accidental overwrite by dumb installer tools (e.g. distutils) and accidental edits, not security against malicious tampering.

Exactly. Promises of false security do not help users.

...

Yeah I don't really understand this debate over md5 hashes here. I suggest that we emphasis in PEP 376 the fact that the sole purpose is to have a checksum.

Putting that on my list of editions for the PEPs!

Cheers _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig

Fair enough. I should write up provides-extra as an actual patch to the pep. It is just a way to express together with requires-dist what pip install pyramid[test] installs above just pip install pyramid, it is needed for setup tools to generate dist info without losing that information, and my distribute fork has an implementation. Are there really other proposals for including that package[extras] feature in the static metadata? I think that is the only part of my packaging work that needs to amend an existing spec, unless you want to opine on putting a build serial number in the versioning pep.

On Tuesday, July 3, 2012 at 11:57 AM, Jeroen Dekkers wrote:

Sorry for not being clear, but I totally agree. I was replying to the md5 on PyPI are embarrassing part and meant that we should migrate to use better hashes on PyPI in the future.

Off topic for this list, but I would agree, and have attempted to have this happen but have been told that those hashes are not for security but only for a checksum and will not be changed.