[IronPython] Last change needed for working tempfiles

Martin (gzlist) gzlist at googlemail.com
Thu Feb 18 19:30:14 CET 2010

As mentioned in the release thread[1] tempfiles nearly work in
IronPython 2.6.1 RC 1 now, they go away when explicitly closed or when
the process ends, thanks to adding support[2] for the os.O_TEMPORARY
flag. The current bug[3] is actually fixed as stated in IronPython
2.6.1 RC 1 as can be seen by the following testcase:

def test_named_temporary_file_is_removed_by_close():
    """Temporary files should be removed from the filesystem when closed"""
    import os, tempfile
    tf = tempfile.NamedTemporaryFile()
    f = tf.file
        if os.path.exists(tf.name): raise AssertionError(
            "Temporary file not removed on close")

However, the larger problem with tempfiles isn't quite done yet, as
can be seen by the following failing case:

def test_named_temporary_file_is_removed_by_gc():
    """Unreferenced temporary files should be removed from the filesystem"""
    import os, tempfile, weakref, gc
    tf = tempfile.NamedTemporaryFile(dir=".")
    wf = weakref.ref(tf.file)
        fname = tf.name
        del tf
        if os.path.exists(fname): raise AssertionError(
            "Temporary file not removed on deletion")
        f = wf()
        if f is not None:

Both those tests are from my mini-suite[4] of issues I've needed to
workaround. The underlying problem is in Src\IronPython.Modules\nt.cs

public static object fdopen(CodeContext/*!*/ context, int fd, string
mode, int bufsize) {
    // check for a valid file mode...

    PythonContext pythonContext = PythonContext.GetContext(context);
    PythonFile pf = pythonContext.FileManager.GetFileFromId(pythonContext, fd);
    return pf;

A change like this should prevent the file from being kept alive:

     PythonFile pf = pythonContext.FileManager.GetFileFromId(pythonContext, fd);
+    pythonContext.FileManager.Remove(pf);
     return pf;

However, will that trip the following assertion? Some code might rely on this.

    f = os.fdopen(fd)
    assert fd == f.fileno()

A better fix would be something like:

     PythonFile pf = pythonContext.FileManager.GetFileFromId(pythonContext, fd);
+    pythonContext.FileManager.Weaken(fd);
     return pf;

That needs some new methods in the underlying mappings to turn the
strong reference into a weak one though.

There may be some code that does variations of the following:

IronPython 2.6.1 (2.6.10920.0) on .NET 2.0.50727.3082
Type "help", "copyright", "credits" or "license" for more information.
>>> import tempfile
>>> for _ in range(4096):
...   f = tempfile.NamedTemporaryFile()
Traceback (most recent call last):
  File "C:\Program Files\IronPython 2.6\Lib\tempfile.py", line 444, in
  File "C:\Program Files\IronPython 2.6\Lib\tempfile.py", line 228, in
OSError: [Errno -2146233079] HybridMapping is full

Which might still run into problems depending on how the GC operates,
but isn't very likely.

(I'd have posted this straight on the tracker rather than to list, but
it doesn't seem to do wiki markup and having it eat spaces is rather
deadly for example Python code).


[1]: http://lists.ironpython.com/pipermail/users-ironpython.com/2010-February/012116.html
[2]: http://lists.ironpython.com/pipermail/users-ironpython.com/2009-October/011457.html
[3]: http://ironpython.codeplex.com/WorkItem/View.aspx?WorkItemId=23936
[4]: http://float.endofinternet.org/bazaar/simple/test_noncpython/test_noncpython.py

More information about the Ironpython-users mailing list