[New-bugs-announce] [issue13392] Writing a pyc file is not atomic under Windows

[Antoine Pitrou]

New submission from Antoine Pitrou <pitrou at free.fr>:

#13146 solved the issue of writing pyc files under POSIX. Under Windows, the problem still exists, as the following buildbot failure shows:

[317/360] test_multiprocessing
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\multiprocessing\forking.py", line 373, in main
    prepare(preparation_data)
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\multiprocessing\forking.py", line 499, in prepare
    '__parents_main__', file, path_name, etc
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\regrtest.py", line 175, in <module>
    import packaging.command
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\packaging\command\__init__.py", line 4, in <module>
    from packaging.util import resolve_name
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\packaging\util.py", line 5, in <module>
    import csv
EOFError: EOF read where not expected
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\multiprocessing\forking.py", line 373, in main
    prepare(preparation_data)
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\multiprocessing\forking.py", line 499, in prepare
    '__parents_main__', file, path_name, etc
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\regrtest.py", line 175, in <module>
    import packaging.command
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\packaging\command\__init__.py", line 4, in <module>
    from packaging.util import resolve_name
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\packaging\util.py", line 5, in <module>
    import csv
EOFError: EOF read where not expected
[etc.]

(from http://www.python.org/dev/buildbot/all/builders/x86%20XP-4%203.x/builds/5551/steps/test/logs/stdio)

Attached patch uses MoveFileEx to perform a (hopefully atomic) rename from a temporary file when creating the pyc file.
The same strategy cannot be created for importlib since MoveFileEx isn't exposed at the Python level (we could delete the tmp file if renaming fails, though; it simply means another process beat us to it, which shouldn't be a problem here).

----------
components: Interpreter Core
files: winimport.patch
keywords: patch
messages: 147540
nosy: brett.cannon, haypo, ncoghlan, neologix, pitrou, python-dev, r.david.murray
priority: normal
severity: normal
stage: patch review
status: open
title: Writing a pyc file is not atomic under Windows
versions: Python 3.3
Added file: http://bugs.python.org/file23674/winimport.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue13392>
_______________________________________