[C++-sig] Re: Presenting a Python embedding tutorial for Boost.Python

[Dirk Gerrits]

David Abrahams wrote:
> Dirk Gerrits <dirk at gerrits.homeip.net> writes:
> 
> 
>>David Abrahams wrote:
>>
>>>Dirk Gerrits <dirk at gerrits.homeip.net> writes:
>>>
>>>
>>>>I've finally finished the first draft of the embedding tutorial I was
>>>>writing for Boost.Python. I guess I underestimated the task, because
>>>>it took several days longer than I anticipated, but here it is:
>>>>
>>>>http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/*checkout*/boost-sandbox/boost-sandbox/libs/python/doc/tutorial/doc/embedding.html
>>>>
>>>>Note that I am neither an experienced writer, nor an experienced
>>>>Pythoneer. Any and all proof reading will be greatly appreciated.
>>>
>>>Thanks very much for your efforts!
>>
>>Your welcome. I'm just glad I can help a great library become even
>>better. :)
> 
> 
> Many <g>s!
> 
> 
>>>I have several high-level remarks:
>>>1. Why spend so much time discussing manual reference-counting, if we
>>>   have a better solution?
>>
>>Good point. At the time, I felt it was necessary to understand this
>>to understand the difference between 'borrowed' and 'new'. On
>>retrospect, I suppose this is not really necessary.
> 
> 
> I don't know, maybe it is neccessary. It depends on whether you think
> your audience understands how to use the Python 'C' API or not.  If
> you think many don't, it might be best to write a separate tutorial
> section on how to use handle<>. I know, the project is growing ;-)
> OTOH you've already written much of it.
> 
>>>2. I notice no use of handle_exception(). Boost.Python code can throw
>>>   C++ exceptions which correspond to Python exceptions (e.g. when
>>>   initializing a handle<> from a null pointer), and
>>>   handle_exception() will translate these back into Python
>>>   exceptions... which can finally be printed on the console or
>>>   wherever else you think is appropriate
>>
>>Well I guess that is because I haven't studied Boost.Python's
>>exception facilities in much detail myself. But I agree that this
>>would make a good addition. And a description of null_ok would
>>probably be appropriate too.
> 
> 
> That part should go in the handle<> tutorial ;-)
> 
>>>3. It was always my intention to help people avoid touching PyObject*s
>>>   and the Python 'C' API at all.  Are there any obvious C++
>>>   interfaces that should be added to the library to wrap the
>>>   boilerplate use of the Python 'C' API I see in your examples?
>>
>>A noble goal. :) And you've already got dict, list, long, numeric,
>>object, str, tuple which covers a great deal of ground. My PyTuple_
>>examples are perhaps ill-chosen because python::tuple can do this much
>>easier
> 
> 
> Yes, it really would be best to avoid touching PyTuple_xxx directly.

Agreed.

>>, but I didn't know what else to demonstrate there.
> 
> 
> I also confess to not having any idea what piece of code you're
> talking about ;-)

The two code examples in the manual reference counting section. ;)

> See also http://www.mcmillan-inc.com/embed.html, FWIW.

Thanks for the link. Taught me two things:
1) Any sane person will prefer Boost.Python over the Python C API ;)
2) It is possible to create multiple interpreters, even in different 
threads. This could lead to a very cool python::interpreter class indeed. :)

>>If I scrap the manual reference counting section though, then I've
>>still got the PyRun_String example. The PyRun_ functions are pretty
>>vital for embedding and they are not directly covered by Boost.Python
>>AFAIK.
>>
>>However, adding Boost.Python facilities for the PyRun_ functions
>>probably means adding a load of other Python C API facilites as well.
> 
> 
> Yup. It was always the intention to add these. Dave Hawkes made a
> valiant effort:
> http://mail.python.org/pipermail/c++-sig/2002-June/001503.html, but it
> was in some ways just too much to take into the library at once.

I see. Pity. Let's see how the incremental approach will fare.

>>I would like to have something like this in Boost.Python:
>>
>>object run_python_code(InputIterator begin, InputIterator end);
> 
> 
> These must have char as their value_type?

I suppose so. At least I don't see any unicode/wchar_t in the Python C 
API. ;) I was under the impression that Python supported unicode source 
files though. How does that work then?

Also, while this interface is nicely STL-ish, it is probably not very 
efficient. I don't see how this can be implemented without some 
std::copy to std::vector<char> or std::string. Thoughts?

>>object run_python_code(String str);
> 
> 
> Umm, what type is String?

char const* I suppose. I was more or less writing pseudo-C++. Sorry for 
not mentioning that.

>>Or, if a python::module was added, perhaps this would be more generic:
>>
>>object run_python_code(InputIterator begin, InputIterator end, module
>>mod = main_module());
>>object run_python_code(String str, module mod = main_module());
> 
> 
> We don't really need a new type for that; we /could/ just use
> object. Until module has some useful constructor or member functions,
> there's no reason to add it.

Ah yes of course. Good point.

>>Also, it might be a good idea to wrap Py_Initialize and Py_Finalize in
>>a RAII class?
> 
> 
> Absolutely. Just as soon as Py_Finalize becomes safe for us ;-)
> That might be a good project for you...

I've been meaning to look into BPL's internals in more detail anyway. :) 
I'm not sure if university will leave me with enough spare time though. 
I think I should concentrate on the tutorial first.

>>I'm just not sure where this line of thought will end. It might add a
>>lot of functions and classes to Boost.Python.
>>
>>What do you think?
> 
> 
> I think that's OK, as long as we take small steps.
> 
> We could start with just the minimum needed to do the kinds of
> embedding jobs you're tackling here.

Sounds like a plan.

Dirk Gerrits