Hi Steve,

Have you considered and rejected filing a bug on bugs.python.org?

Or are you specifically concerned about how to write a test for this behavior that doesn't fill up the disk when the bug is present? On Unixoid systems there's a resource limit you can set to limit disk space IIRC (e.g. on my Mac, `ulimit -a` shows various limits including file size).

Solving this by patching write() seems somewhat complex because you'd really have to patch open() to return an io.IO subclass that has a patched write() method. But I'm sure it can be done in the context of a specific test. I wouldn't try patching it for *all* tests though -- there are many tests that might fail due to such a change. It also wouldn't help for file operations made directly from C code.

--Guido

On Thu, May 21, 2020 at 6:57 AM Steve Barnes <GadgetSteve@live.co.uk> wrote:

The issue is simple and simple enough for a beginner to fall foul of - test procedure:
Change directory to any directory with files in totally a few 10s of megs (ideally but it doesn't matter much).
python -m zipfile -c my_zipfile.zip .

Wait a few minutes and press control-C then do a directory - you will find that my_zipfile.zip is huge (much bigger than the total of the files in the original directory). The same applies to tarfile.

If you fail to hit control-C soon enough (how long depends on the spare space on the current drive) it will crash when it runs out of disk space, possibly, on some systems, requiring a format and re-install of the OS.

The cause of the problem is that when zipfile &/or tarfile is archiving to a file in the current directory it tries to add the target file to the input list and you have an Ouroboros situation. The fix is also simple - exclude the target file from the permitted inputs.

I have seen this happen in the wild, I do know that python is not the only archiving tool that suffers from this problem but that seems to me to be no reason not to address this.

If write were patched to simply count the bytes but not write the file then the problem would not occur so I guess that it would need to be patched in a transparent manner.

Steve Barnes

-----Original Message-----
From: remi.lapeyre@henki.fr <remi.lapeyre@henki.fr>
Sent: 21 May 2020 13:56
To: python-ideas@python.org
Subject: [Python-ideas] Re: How to propose a change with tests where the failing test case (current behaviour) is bad or dangerous

If the file is supposed to be small, would patching write() in the test to accumulate the number of bytes written and fail the test when it goes over what it's supposed to be possible?

What is the issue?
_______________________________________________
Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ORPDN4CITJZSO246HWCKNZUKA4IT3QWC/
Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-leave@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/OT5M4U4RPJAU7HXXFONGTL442OCHGSXY/
Code of Conduct: http://python.org/psf/codeofconduct/