[New-bugs-announce] [issue32304] Upload failed (400): Digests do not match on .tar.gz ending with x0d binary code

[Louis Lecaroz]

New submission from Louis Lecaroz <louis at lecaroz.name>:

Hi,
.tar.gz files can end with x0d bytes or whatever you want

When running setup.py sdist upload, depending on the project, the .tar.gz file, as said can sometimes end with x0d. When doing the upload, the line https://github.com/python/cpython/blob/master/Lib/distutils/command/upload.py#L162 (if value and value[-1:] == b'\r') will remove the ending char of the .tar.gz generating a 400 response error from the server like: 

Upload failed (400): Digests do not match, found: 09f23b52764a6802a87dd753009c2d3d, expected: 972b8e9d3dc8cf6ba6b4b1ad5991f013
error: Upload failed (400): Digests do not match, found: 09f23b52764a6802a87dd753009c2d3d, expected: 972b8e9d3dc8cf6ba6b4b1ad5991f013

As this line is generic & run on all key/values, I clearly understand that this check was initially written to eliminate certainly some issues on values in text format. 

But the mistake here, is that you are also changing the content of the 'content' key which contains the .tar.gz as value, and because you remove the ending 0D, you change the .tar.gz content to be uploaded. As consequence, the server will return a 400 error about a wrong digest/crc.

I was able to make the code working with all .tar.gz files by changing this line to:

                if value and value[-1:] == '\r' and not key=='content':

With a such fix, the .tar.gz content will not see its ending \r to be removed & the computed CRC from the server will be the same as computed by md5(content).hexdigest() in upload.py

----------
components: Distutils
messages: 308205
nosy: dstufft, eric.araujo, llecaroz
priority: normal
severity: normal
status: open
title: Upload failed (400): Digests do not match on .tar.gz ending with x0d binary code
type: security
versions: Python 2.7, Python 3.5, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue32304>
_______________________________________