Can't upload sdist: "File already exists"
I have a little package "huffman" where I build an sdist and wheel (python setup.py sdist bdist_wheel) and both seem to get built and can install fine. I can't seem to upload both to PyPI because the "File already exists": $ twine upload dist/* Uploading distributions to https://upload.pypi.org/legacy/ Uploading huffman-0.1.2-py2.py3-none-any.whl Uploading huffman-0.1.2.tar.gz HTTPError: 400 Client Error: File already exists. for url: https://upload.pypi.org/legacy/ Subsequent call to upload *just* the tarball fails the same way. I can't see an sdist anywhere, and uploading it via the website or twine just tells me it's already there...somehow. Asking pip to try to give it to me fails though (the binary works, however): $ pip download --no-cache-dir --only-binary :all: huffman==0.1.2 Collecting huffman==0.1.2 Downloading huffman-0.1.2-py2.py3-none-any.whl Saved ./huffman-0.1.2-py2.py3-none-any.whl Successfully downloaded huffman $ pip download --no-cache-dir --no-binary :all: huffman==0.1.2 Collecting huffman==0.1.2 Could not find a version that satisfies the requirement huffman==0.1.2 (from versions: ) No matching distribution found for huffman==0.1.2 Am I missing something? I am as sure as I can be that I didn't upload it twice; I bumped my version up one because I figured that may have been it. "twine register" that some guides mention just gets shot down with an HTTP "410 [...] simply upload the file" Cheers, Nick
Because you already uploaded a wheel for version 0.1.2 you can't upload any other files for that version, else people could accidentally upload e.g. an sdist with code different from what was already uploaded in the wheel. If you want an sdist then I would do another release as version 0.1.2post1 with the wheel and sdist (or whatever the proper post release version format is; on my phone so a pain to look up right now). On Wed, Dec 21, 2016, 12:30 Nick Timkovich, <prometheus235@gmail.com> wrote:
I have a little package "huffman" where I build an sdist and wheel (python setup.py sdist bdist_wheel) and both seem to get built and can install fine. I can't seem to upload both to PyPI because the "File already exists":
$ twine upload dist/* Uploading distributions to https://upload.pypi.org/legacy/ Uploading huffman-0.1.2-py2.py3-none-any.whl Uploading huffman-0.1.2.tar.gz HTTPError: 400 Client Error: File already exists. for url: https://upload.pypi.org/legacy/
Subsequent call to upload *just* the tarball fails the same way. I can't see an sdist anywhere, and uploading it via the website or twine just tells me it's already there...somehow. Asking pip to try to give it to me fails though (the binary works, however):
$ pip download --no-cache-dir --only-binary :all: huffman==0.1.2 Collecting huffman==0.1.2 Downloading huffman-0.1.2-py2.py3-none-any.whl Saved ./huffman-0.1.2-py2.py3-none-any.whl Successfully downloaded huffman
$ pip download --no-cache-dir --no-binary :all: huffman==0.1.2 Collecting huffman==0.1.2 Could not find a version that satisfies the requirement huffman==0.1.2 (from versions: ) No matching distribution found for huffman==0.1.2
Am I missing something? I am as sure as I can be that I didn't upload it twice; I bumped my version up one because I figured that may have been it. "twine register" that some guides mention just gets shot down with an HTTP "410 [...] simply upload the file"
Cheers, Nick _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
So upload the sdist first in the future? What's the difference between uploading an sdist second with code that differs from what's in the wheel and vice versa, is PyPI doing some checking? Generally, is it OK (recommended/socially acceptable) to upload universal wheels only or is the sdist still nice to have available? I guess maybe for pip, but it considering universal, pure-Python wheels as "binaries" seems quirky. On Thu, Dec 22, 2016 at 10:49 AM, Brett Cannon <brett@python.org> wrote:
Because you already uploaded a wheel for version 0.1.2 you can't upload any other files for that version, else people could accidentally upload e.g. an sdist with code different from what was already uploaded in the wheel. If you want an sdist then I would do another release as version 0.1.2post1 with the wheel and sdist (or whatever the proper post release version format is; on my phone so a pain to look up right now).
On Wed, Dec 21, 2016, 12:30 Nick Timkovich, <prometheus235@gmail.com> wrote:
I have a little package "huffman" where I build an sdist and wheel (python setup.py sdist bdist_wheel) and both seem to get built and can install fine. I can't seem to upload both to PyPI because the "File already exists":
$ twine upload dist/* Uploading distributions to https://upload.pypi.org/legacy/ Uploading huffman-0.1.2-py2.py3-none-any.whl Uploading huffman-0.1.2.tar.gz HTTPError: 400 Client Error: File already exists. for url: https://upload.pypi.org/legacy/
Subsequent call to upload *just* the tarball fails the same way. I can't see an sdist anywhere, and uploading it via the website or twine just tells me it's already there...somehow. Asking pip to try to give it to me fails though (the binary works, however):
$ pip download --no-cache-dir --only-binary :all: huffman==0.1.2 Collecting huffman==0.1.2 Downloading huffman-0.1.2-py2.py3-none-any.whl Saved ./huffman-0.1.2-py2.py3-none-any.whl Successfully downloaded huffman
$ pip download --no-cache-dir --no-binary :all: huffman==0.1.2 Collecting huffman==0.1.2 Could not find a version that satisfies the requirement huffman==0.1.2 (from versions: ) No matching distribution found for huffman==0.1.2
Am I missing something? I am as sure as I can be that I didn't upload it twice; I bumped my version up one because I figured that may have been it. "twine register" that some guides mention just gets shot down with an HTTP "410 [...] simply upload the file"
Cheers, Nick _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Nick Timkovich <prometheus235@gmail.com> writes:
Generally, is it OK (recommended/socially acceptable) to upload universal wheels only or is the sdist still nice to have available? I guess maybe for pip, but it considering universal, pure-Python wheels as "binaries" seems quirky.
None the less, the Wheel format *is* a format intended for installation (so a “binary” format in that terminology). A wheel is not suitable for making changes and re-building from source. So the source distribution still needs to be uploaded as well, to support that important freedom for the recipient. -- \ “Liberty, n. One of imagination's most precious possessions.” | `\ —Ambrose Bierce, _The Devil's Dictionary_, 1906 | _o__) | Ben Finney
On Thu, Dec 22, 2016 at 9:49 AM, Brett Cannon <brett@python.org> wrote:
Because you already uploaded a wheel for version 0.1.2 you can't upload any other files for that version, else people could accidentally upload e.g. an sdist with code different from what was already uploaded in the wheel. If you want an sdist then I would do another release as version 0.1.2post1 with the wheel and sdist (or whatever the proper post release version format is; on my phone so a pain to look up right now).
I'm pretty sure that's not correct. Twine is written to specifically upload the wheel first because PyPI will extract metadata from that and display it on the page. It won't do that if the sdist is uploaded first. I'm not able to reproduce the behaviour Nick is seeing. My only guess is that something changed in Warehouse or the file existed, was deleted, and is now being re-uploaded with the same version. That's not something Warehouse or PyPI allows anymore (republishing with the same version)
On Dec 22, 2016, at 8:54 AM, Ian Cordasco <graffatcolmingov@gmail.com> wrote:
On Thu, Dec 22, 2016 at 9:49 AM, Brett Cannon <brett@python.org> wrote:
Because you already uploaded a wheel for version 0.1.2 you can't upload any other files for that version, else people could accidentally upload e.g. an sdist with code different from what was already uploaded in the wheel. If you want an sdist then I would do another release as version 0.1.2post1 with the wheel and sdist (or whatever the proper post release version format is; on my phone so a pain to look up right now).
I'm pretty sure that's not correct. Twine is written to specifically upload the wheel first because PyPI will extract metadata from that and display it on the page. It won't do that if the sdist is uploaded first.
I'm not able to reproduce the behaviour Nick is seeing. My only guess is that something changed in Warehouse or the file existed, was deleted, and is now being re-uploaded with the same version. That's not something Warehouse or PyPI allows anymore (republishing with the same version)
Similarly; I've *definitely* done sdist releases after wheel releases of the same version in the past. -glyph
There is a fairly new restriction that you can only have *one* sdist per release now. That should not apply at all to Wheels and if it is, then it is a bug. I can’t reproduce this issue though, so I’m going to guess there was some bit of confusion about exact errors here. If someone actually cannot upload 1 sdist + N wheels, please leave the state as is and tell me.
On Dec 22, 2016, at 3:32 PM, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:
On Dec 22, 2016, at 8:54 AM, Ian Cordasco <graffatcolmingov@gmail.com> wrote:
On Thu, Dec 22, 2016 at 9:49 AM, Brett Cannon <brett@python.org> wrote:
Because you already uploaded a wheel for version 0.1.2 you can't upload any other files for that version, else people could accidentally upload e.g. an sdist with code different from what was already uploaded in the wheel. If you want an sdist then I would do another release as version 0.1.2post1 with the wheel and sdist (or whatever the proper post release version format is; on my phone so a pain to look up right now).
I'm pretty sure that's not correct. Twine is written to specifically upload the wheel first because PyPI will extract metadata from that and display it on the page. It won't do that if the sdist is uploaded first.
I'm not able to reproduce the behaviour Nick is seeing. My only guess is that something changed in Warehouse or the file existed, was deleted, and is now being re-uploaded with the same version. That's not something Warehouse or PyPI allows anymore (republishing with the same version)
Similarly; I've *definitely* done sdist releases after wheel releases of the same version in the past.
-glyph
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org <mailto:Distutils-SIG@python.org> https://mail.python.org/mailman/listinfo/distutils-sig <https://mail.python.org/mailman/listinfo/distutils-sig>
— Donald Stufft
On 12/22/2016 12:41 PM, Donald Stufft wrote:
There is a fairly new restriction that you can only have *one* sdist per release now. That should not apply at all to Wheels and if it is, then it is a bug. I can’t reproduce this issue though, so I’m going to guess there was some bit of confusion about exact errors here. If someone actually cannot upload 1 sdist + N wheels, please leave the state as is and tell me.
I am also seeing this problem. My standard files for upload have been a py2 wheel, a py3 wheel, a zip sdist, and a tar sdist, and now the tar sdist is being denied as a duplicate. Do I need to choose between zip and tar sdists? -- ~Ethan~
On 01/05/2017 08:46 AM, Donald Stufft wrote:
On Jan 5, 2017, at 11:45 AM, Ethan Furman wrote:
Do I need to choose between zip and tar sdists?
Yes.
I haven't followed Windows O/Ses for a while now -- are they able to easily access tar files? Or do *nix machines come standard with unzip now? Or is it more along the lines of -- you seem to be a developer, so you should be able to install the one you need? ;) -- ~Ethan~
On Jan 5, 2017, at 12:55 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 01/05/2017 08:46 AM, Donald Stufft wrote:
On Jan 5, 2017, at 11:45 AM, Ethan Furman wrote:
Do I need to choose between zip and tar sdists?
Yes.
I haven't followed Windows O/Ses for a while now -- are they able to easily access tar files? Or do *nix machines come standard with unzip now?
Or is it more along the lines of -- you seem to be a developer, so you should be able to install the one you need? ;)
-- ~Ethan~ _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
.tar.gz is the recommended format but either one is OK. I think that there is still the default tool disparity but getting a tool to open either one is fairly trivial, particularly since you can do ``python3 -m tarfile -e foobar-1.0.tar.gz. That being said, non-automated downloads account for a tiny fraction of the total downloads on PyPI so I wouldn’t worry too much about that use case in general. — Donald Stufft
I never determined what was causing my problem, I couldn't reproduce it on testpypi so I gave up (it's a small package, and if someone wants the source, they can look at the GH link I have in the metadata). Specifically, why does it say that the file exists, but I *can not see it anywhere*, either via the web interface or download it with pip --no-binary? Is it possible that there exists a "huffman-0.1.2.tar.gz" on PyPI from someone else who may have had the package name, uploaded some releases, then deleted it? I am 99% confident that I did not publish/delete (all I do is twine upload dist/*), let alone delete a specific .tar.gz dist (is that even possible to do from the command-line?). The error happened when I tried to do a 0.1.1, figured I messed up, then rebuilt/republished with a bigger number. The information isn't exposed to me (maybe logs somewhere?), so I'll probably just accept PyPI is strange like that. On Thu, Jan 5, 2017 at 12:13 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 01/05/2017 09:58 AM, Donald Stufft wrote:
.tar.gz is the recommended format ...
.tar.gz it is! Thanks!
-- ~Ethan~ _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
On 01/05/2017 11:37 AM, Nick Timkovich wrote:
I never determined what was causing my problem, I couldn't reproduce it on testpypi so I gave up (it's a small package, and if someone wants the source, they can look at the GH link I have in the metadata).
Do you have both a .zip and a .tar source distribution? -- ~Ethan~
No. My build command was: `python setup.py sdist bdist_wheel` which doesn't generate a zip. On Thu, Jan 5, 2017 at 2:13 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 01/05/2017 11:37 AM, Nick Timkovich wrote:
I never determined what was causing my problem, I couldn't reproduce it
on testpypi so I gave up (it's a small package, and if someone wants the source, they can look at the GH link I have in the metadata).
Do you have both a .zip and a .tar source distribution?
-- ~Ethan~ _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
On 6 January 2017 at 07:37, Nick Timkovich <prometheus235@gmail.com> wrote:
No. My build command was: `python setup.py sdist bdist_wheel` which doesn't generate a zip.
We've seen behaviour like this before with the old upload server implementation, where it was possible to get a particular 5xx error on upload that meant the server had registered the file as uploaded, but the upload hadn't actually worked (this bug in the old server was actually the main reason the default client config suggested on packaging.python.org was changed). However, your command output indicates that you're already using the new upload server, so that's unlikely to have been the problem :( Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
participants (8)
-
Ben Finney -
Brett Cannon -
Donald Stufft -
Ethan Furman -
Glyph Lefkowitz -
Ian Cordasco -
Nick Coghlan -
Nick Timkovich