[C++-sig] Conversion of python files to C++ ostreams

[Jim Bosch]

On Mon, 2010-03-29 at 13:28 -0700, Christopher Bruns wrote:
> I am trying to coax boost.python to automatically convert python files
> to C++ std::ostreams, for methods that take arguments of type
> "std::ostream&".  I have made some progress, and I could use some
> advice on how to go further.
> 
> I created a derived class of std::ostream, called FilestarOstream,
> which takes a std::FILE* as it constructor argument.  Then I created a
> conversion from "python file" to the FilestarOstream class.  Now my
> python module can automatically convert a python file to a
> FilestarOstream for any method that takes a "const FilestarOstream&"
> as an argument.  Ideally I would prefer to automatically convert a
> python file for any method that takes a (non-const) "std::ostream&" as
> an argument.
> 
> I have included my simplified source code below
> 
> My questions:
> 
>   1 - Is there an easier way to do this?  Does boost.python have a
> built in incantation for wrapping methods that take a std::ostream
> reference, to take python files?
> 

Nope.

>   2 - Is there a way to make the type conversion work for non-const
> reference arguments?  (see print_hello_wrapper2() below)
> 

Yes, but it's a bit painful (at least the way I know how to do it).

You have to make your "FilestarOstream" class a Boost.Python "object
manager".  That means it should probably derive publicly from
boost::python::object, and you'll have to specialize a few traits
classes.

For an example, you can look at the Boost.Python source, or some
numpy-based object managers I've thrown together here:

https://svn.boost.org/trac/boost/browser/sandbox/numpy

Making a class that derives publicly from both std::ostream and
boost::python::object may cause some problems, however, as they're not
exactly small classes.  I'm not sure inheritance from object is
necessary.  There's apparently some distinction between the
ObjectWrapper concept in the documentation:

http://www.boost.org/doc/libs/1_42_0/libs/python/doc/v2/ObjectWrapper.html#ObjectWrapper-concept

and the "object_manager" traits classes I discovered in the code.

I'd be curious to hear if you figure any of that out.

>   3 - Why does my print_hello_wrapper3() not work?  It takes a const
> std::ostream reference argument.  I can pass it a FileStarOstream, so
> python knows how to convert a FileStarOstream to a std::ostream.  Plus
> python knows how to convert a python file to a const FileStarOstream
> reference.  So why can't it convert a python file to a const ostream
> reference?
> 

Because you haven't registered a converter.  Wrapping a C++ class for
Python does that for you, but you can also do it manually.  I could tell
you all about it for rvalues, and I know there's such thing as an lvalue
converter (lvalue_from_pytype is one, though it doesn't do what you
want), but I've never written one.  But poke around in the source and
google for custom lvalue converters and you may find something.  I'd
solve problem #2 first, though.

>   4 - In my FilestarOstream_from_pyfile::convertible method, is there
> a way to check that the object is not only a file, but that it is also
> writable?

I suspect you really want to just throw an exception when you encounter
a non-writeable file, and you could probably do that in the
FilestarOstream constructor by setting a Python exception and calling
boost::python::throw_error_already_set().

Having overload support for writeable vs. non-writeable file objects
would probably be harder, but probably still possible (somewhere in how
you implement the lvalue conversion, if it's anything like rvalue
conversion).

> 
>   5 - The PyFile_AsFile docs say "If the caller will ever use the
> returned FILE* object while the GIL is released it must also call the
> PyFile_IncUseCount()  and PyFile_DecUseCount()  functions described
> below as appropriate."  What is "the GIL".  If I call
> PyFile_IncUseCount() when I construct the FileStarOstream, when would
> I call the corresponding PyFile_DecUseCount()?

GIL is the global interpreter lock.  You only need to worry about it if
your program is multithreaded on the C++ side.

In this case, if you make the FileStarOstream an object manager, it will
have to own a reference to the actual underlying Python file object, and
I *think* that will deal with the use count issues for you.

Good luck!

Jim Bosch