[IronPython] IronPython and file descriptors

Thu Dec 18 13:00:10 CET 2008

Before playing with this, it's probably worth knowing that the following 
can cause exceptions or no exceptions and lost output from the final write.

1 Writing to a PythonFile
2 Getting the PythonFile's stream's file handle
3 Converting the handle to a file descriptor
4 Writing to this file descriptor directly in C using write ---- Fine up 
to this point
5 Writing to the PythonFile again

Since code that does steps 1-4 is likely to also perform 5 it may not be 
worthwhile having file.fileno() return a valid file descriptor unless 
step 5 can be made to work. Though I appreciate that this is problematic 
and that there are good reasons for having PythonFiles wrap FileStreams.

Tom

William Reade wrote:
> Just a quick note: you can't necessarily just P/invoke out to msvcrt, 
> because CPython was built with msvcr71. Sorry :(.
>
> Tom Wright wrote:
>> I'm not sure about sockets. The only example I have found in the wild 
>> so far is with files in PIL.
>>
>> The python implementation seems to have socket.fileno() returning a 
>> file descriptor which comes from <sys/socket.h>'s socketpair function 
>> in the standard C library (see socketmodule.c if you are interested)
>>
>> So, again, it would be possible for people to pass this identifier as 
>> an integer directly to C. Whether people do this in practice is a 
>> different question...
>>
>> Tom
>>
>>
>>
>> Michael Foord wrote:
>>> Dino Viehland wrote:
>>>>
>>>> (now I’ve replied to the correct thread…)
>>>>
>>>> Can you open a feature request on CodePlex? It's certainly an 
>>>> interesting idea to ponder and I'm leaning towards it but there's 
>>>> lots of details to be gotten right.
>>>>
>>>> Do you know if this needs to work w/ sockets as well? (There's also 
>>>> the question of can we make it work with sockets? :))
>>>>
>>>> There'll be a bunch of places we need to update (nt, socket, file, 
>>>> select, etc...) so I think it'll have to wait until 2.1 instead of 
>>>> coming in a minor update like 2.0.1.
>>>>
>>>
>>> Tom will have to comment on whether it is needed for sockets as 
>>> well. Not yet I guess. :-)
>>>
>>> Tom created an issue on Codeplex - I've added a comment to that:
>>>
>>> http://www.codeplex.com/IronPython/WorkItem/View.aspx?WorkItemId=20242
>>>
>>> Michael
>>>>
>>>> *From:* users-bounces at lists.ironpython.com 
>>>> [mailto:users-bounces at lists.ironpython.com] *On Behalf Of *Michael 
>>>> Foord
>>>> *Sent:* Monday, December 15, 2008 12:53 PM
>>>> *To:* Discussion of IronPython
>>>> *Subject:* Re: [IronPython] IronPython and file descriptors
>>>>
>>>> 2008/12/15 Dino Viehland <dinov at microsoft.com 
>>>> <mailto:dinov at microsoft.com>>
>>>>
>>>> Presumably CPython is linking against msvcrt and is using the C 
>>>> abstraction rather than calling the Win32 APIs which return handles 
>>>> directly.
>>>>
>>>> As Curt said the biggest problem for us is that .NET does not 
>>>> expose C file descriptors. Therefore we could fix this by 
>>>> P/Invoking out to msvcrt. For users on other platforms we'd need to 
>>>> either add support for their platform or hope that their handles == 
>>>> C runtime file descriptors. For something like fileno maybe this is 
>>>> ok because it's an interop point only - or are there any 
>>>> non-interop uses of fileno in the world?
>>>>
>>>> What do people think of this? This would be the 1st place where we 
>>>> would add a P/Invoke so I'd want to tread lightly and make sure 
>>>> this is really the right thing to do.
>>>>
>>>>
>>>>
>>>> Well, some way of doing this is needed if certain Python C 
>>>> extensions are to work with Ironclad. Perhaps a global or per 
>>>> engine setting that allows file handles to be associated with a C 
>>>> descriptor. Users of Ironclad could switch it on if they wished and 
>>>> take the consequences.
>>>>
>>>> Michael Foord
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: users-bounces at lists.ironpython.com
>>>> <mailto:users-bounces at lists.ironpython.com>
>>>> [mailto:users-bounces at lists.ironpython.com
>>>> <mailto:users-bounces at lists.ironpython.com>] On Behalf Of Slide
>>>> Sent: Monday, December 15, 2008 10:21 AM
>>>> To: Discussion of IronPython
>>>> Subject: Re: [IronPython] IronPython and file descriptors
>>>>
>>>> On Mon, Dec 15, 2008 at 11:18 AM, Tom Wright
>>>> <tom.wright at resolversystems.com
>>>> <mailto:tom.wright at resolversystems.com>> wrote:
>>>> > Agreed. It is certainly possible with some work to get a file
>>>> descriptor and
>>>> > pass it to C code.
>>>> >
>>>> > This is not the problem, however. Ironclad's aim (eventually) is
>>>> to allow
>>>> > *arbitrary* C extensions to work with Ironpython without
>>>> changing the
>>>> > extensions. Ctypes aim is to allow arbitrary C code to be run by
>>>> python
>>>> > (possibly in other python libraries which you really don't want
>>>> to modify).
>>>> >
>>>> > In CPython .fileno() is a file descriptor and some modules use
>>>> this fact
>>>> > (specifically PIL) by passing file descriptors as integers into
>>>> C code. I do
>>>> > not know how one can make this code work in IronPython unless
>>>> .fileno()
>>>> > returns a file descriptor.
>>>> >
>>>> > The game here is not making a particular C library work with
>>>> IronPython by
>>>> > modifying this library but making as many libraries as possible
>>>> work without
>>>> > modification. Of course whether this is worthwhile depends on
>>>> how many
>>>> > libraries rely on this fact.
>>>> >
>>>> > Sorry - I hope this is a little clearer.
>>>> >
>>>> > Tom
>>>>
>>>>
>>>> How does CPython get the file descriptor for fileno()? Does it just
>>>> return the HANDLE on Windows?
>>>>
>>>> slide
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at lists.ironpython.com <mailto:Users at lists.ironpython.com>
>>>> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at lists.ironpython.com <mailto:Users at lists.ironpython.com>
>>>> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>>>>
>>>>
>>>>
>>>>
>>>> -- 
>>>> http://www.ironpythoninaction.com/
>>>>
>>>> ------------------------------------------------------------------------ 
>>>>
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at lists.ironpython.com
>>>> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>>>
>>>
>>
>> _______________________________________________
>> Users mailing list
>> Users at lists.ironpython.com
>> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>>
>
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com