[Python-3000] how about switching to a preprocessor? (Re: A better way to initialize PyTypeObject)

[Mike Klaas]

On 12/2/06, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Mike Klaas wrote:
>
> > Reducing the pyrex magic and adding more detailed errors to the code
> > generator should eliminate much of the "code-peering" that is
> > currently necessary (at least, by me) when writing extensions.
>
> Can you expand on that? What magic would you like to
> have removed, and what new error messages would you
> like to see?

Sure.  The "magic" (and I don't mean that derogatorily) I'm talking
about is the Python <-> c type conversion and assumption that all
undeclared variables are python objects.  I don't think that they
should be removed (as having python-compatibility is great), but I
find them an impediment when I'm trying to write mostly c code rather
than mostly python, and a pyrex mode for extension writing might
consider dropping them.  It is very easy to forget a cdef, or make a
typo in a variable name and have code that still works (due to pyrex's
autoconversion) but operates inefficiently (a loop counter being
converted back and forth every iteration, for instance).

The problem is that variable declaration in pyrex means "this is
special", but when you're programming in c, ctypes are not special but
the norm.  I would be happier to see "error: 'count' undeclared"
rather it being a PyObject.  Autoconversion is actually less a problem
when declarations are required (not to mention ridiculously handy).

As for errors, I remember being confused by some the complaints in
parameter list declaration, but I can't think of specific instances at
the moment.  Most of my points of greatest frustration have been when
I'm trying to produce a specific line or two of c code, and it becomes
a question of trying to trick pyrex into generating it (for instance,
using certain types of macros, or trying to pass a PyObject**).  You
could (safely) argue that I'm using the wrong tool if I'm trying to
write too much c in pyrex, but extension writers will need to be able
to do the same things.

I'd like to postfix this all by mentioning that I think that pyrex is
absolutely wonderful and has _huge_ potential.  I build large
distributed systems with python, and I am always impressed how little
of the systems need to be written in c or pyrex to achieve massive
increases in performance.

Ever consider opening up the project to other committers and becoming
the BDFL of pyrex?

-Mike