File buffer flush issue in multi-process in Windows
javen72 at gmail.com
javen72 at gmail.com
Tue Jan 13 07:06:20 CET 2009
On 1月13日, 下午2时03分, jave... at gmail.com wrote:
> I encountered a very strange issue in file flush operation in Windows.
> Here's the scenario of my application:
> 1. The main thread of my application will create makefiles
> 2. Once a makefile is generated, launch a new process calling
> nmake.exe to run it. The main thread then create another makefile
> until no more makefiles to create.
> 3. The number of new processes is limited by command line options.
> My application has been running for almost a year without any problem.
> But, after I made some changes recently to the content of makefile
> generated, "nmake.exe" in separate process sometimes complains that
> the makefile is not found. When I went into the directory, the
> makefile did exist there.
> Because I didn't change any thing related to file creation and the new
> makefiles are a little bit less than before, I guessed that the
> makefile just created hasn't been flushed to disk because of size
> change so the new process could not see it in a short time.
> So I decided add code to force flush file buffer after writing the
> file content (I didn't flush the file explicitly before). I did it
> like below:
> Fd = open(File, "w")
> The strangest thing happened. The "makefile" missing happened more
> frequently than no flush operation. I searched the web but no answer
> Finally I tried to use Windows file API to create the file via pywin32
> extension. The problem's gone.
> import win32file
> Fd = win32file.CreateFile(
> win32file.WriteFile(Fd, str(Content), None)
> I tried writing small python extension in C to make use Windows API to
> create file like above. It also works well, even I removed the
> FlushFileBuffers() calling.
> Did anyone encounter such problem before? Did I do something wrong in
> Python file operation? If not, I believe that there's a bug in Python
> file buffer mechanism. I'd appreciate any help or hints.
And the same application has no such problem in Linux.
More information about the Python-list