[Cython] [cython-users] Cython .pxd introspection: listing defined constants

[W. Trevor King]

On Sat, Feb 19, 2011 at 02:04:16PM -0800, Robert Bradshaw wrote:
> On Sat, Feb 19, 2011 at 1:45 PM, W. Trevor King <wking at drexel.edu> wrote:
> > On Sat, Feb 19, 2011 at 12:47:41PM -0800, Robert Bradshaw wrote:
> >> On Sat, Feb 19, 2011 at 11:22 AM, W. Trevor King <wking at drexel.edu> wrote:
> >> > However, the filename <-> module mapping is troublesome for backing
> >> > externally-implemented Python modules (e.g. numpy).  If you wanted to
> >> > write a .pxd file backing numpy.random, how would you go about getting
> >> > your module installed in Cython/Includes/numpy/random.pxd or another
> >> > path that cython would successfully match with `cimport numpy.random`?
> >>
> >> Note that extern blocks (by definition) declare where things come from.
> >
> > They declare where the .pxd file looks for .h files, but not where
> > .pyx files look for the .pxd file.
> 
> Sorry, I should have said extern blocks that make cdef class
> declarations (such as our numpy.pxd).

It doesn't look like there are cdef class declarations in numpy.pxd:

    cython $ grep class Cython/Includes/numpy.pxd 
        ctypedef class numpy.dtype [object PyArray_Descr]:
        ctypedef extern class numpy.flatiter [object PyArrayIterObject]:
        ctypedef extern class numpy.broadcast [object PyArrayMultiIterObject]:
        ctypedef class numpy.ndarray [object PyArrayObject]:
        ctypedef extern class numpy.ufunc [object PyUFuncObject]:

This still doesn't explain how .pxd files specify which external
implemented Python modules they correspond to.

> >> >> >      cdef public struct X:
> >> >> >          int x
> >> >> >          readonly int z
> >> >> >          private int z
> >> >> >
> >> >> > I would perhaps say that non-Pythonable non-private members in public
> >> >> > structs would be a compile error.
> >> >>
> >> >> +1, keep it safe at the beginning.
> >> >
> >> > -1, keep the code clean and the interface consistent ;).  I think the
> >> > struct syntax should be identical to the class syntax, with the
> >> > exception that you can't bind methods to structs.  That's the only
> >> > real difference between structs and classes, isn't it?
> >>
> >> In C++, the only difference between structs and classes is that struct
> >> members are public by default. (Not saying that C++ is always the
> >> model to follow, but it gives precedent). And structs can have
> >> function members, that's how to do OOP in C.
> >
> > Oh.  Even more reason to have identical struct and class handling in
> > Cython ;).
> >
> > It is unclear to me what `cdef public struct` means.  I think it
> > should mean "Python bindings can alter this struct's definition",
> > which doesn't make sense.
> 
> I think it should mean "this struct is accessible from Python (as X)"

Wouldn't that be "cdef readonly struct X"?

> > Shouldn't the syntax for public members be
> >
> >    cdef struct X:
> >        cdef public:
> >            int x
> >            readonly int y
> >            private int z
> 
> -1 on nesting things like this. Rather than make a struct visible from
> Python iff any of its members are,

A struct is visible from python iff it is declared public or readonly:

    cdef public struct X:
        ...

or

    cdef readonly struct X:
        ...

I don't think the visibility of the struct as a whole should have any
effect over the visibility of its members, so you should be able to
specify member visibility explicitly with per-member granularity (as
you currently can for classes).

I was assuming that structs would be public by default (like classes),
but that is obviously configurable.

> I think it makes more sense to put
> the declaration on the struct itself. We could support
> 
> cdef public struct X:
>     int x # public
> 
> cdef readonly struct Y:
>     int y # readonly
> 
> cdef [private] struct Z:
>     int z # private, as we don't even have Z in the Python namespace,
> and no wrapper is created.

The problems with this are:

* It's differnent from how we handle the almost identical class case.
* It makes it impossible to define, for example a public struct with
  C-only attributes:

    cdef public struct X:
        cdef public int a
        cdef private void* ptr

Obviously, public attributes of private structs should raise
compile-time Cython errors.

> >> > If safety with a new feature is a concern, a warning like
> >> > "EXPERIMENTAL FEATURE" in the associated docs and compiler output
> >> > should be sufficient.
> >>
> >> I think the point of "safe" is to start out with a compiler error, and
> >> we can change our minds later, which is better than trying to make
> >> legal statements illegal in the future.
> >
> > Ok, but I still don't understand why the cdefs were removed from the
> > proposed structure members, when they are required for class
> > definitions.
> 
> Because structs can only have c members, so the cdef was entirely
> redundant. They weren't really removed per say, it's just that with
> the exception of cdef classes, "cdef ..." meant "a c declaration
> follows."

But cdef classes can also only have cdef members.

I think it's better to keep cdef meaning "backed by C data", not
necessarily "written using C syntax", since you're trying to do more
with Cython, so it doesn't make sense to force C syntax.

> >> > That would be nice, since the C compiler would (I think) raise an error
> >> > when you try to use an invalid <type> for macro value.
> >>
> >> Const is different than readonly, as readonly specifies the
> >> python-level accessibility.
> >
> > Ah.  Sorry for all the c(p)def/qualifier confusion, but I'm trying to
> > consolidate the way these are handled in Parsing/Nodes/Symtab and I
> > want to make sure I don't implement the wrong interpretation.  Can you
> > clarify how one knows if "public" means "expose a read/write Python
> > interface to this object" or "expose this symbol to external C code"?
> 
> Public has had several different meanings. I wish there were a spec
> and full grammer for Cython, but there's not (yet?). The meaning is
> implicit in the code, and there's enough users out there that we
> should stay backwards compatible. It may be worth doing some
> backwards-incompatible normalization before we hit 1.0, but compiling
> the entire Python grammar is higher priority than that.

Since I'm going to have lots of similar stuff (classes, enums,
structs, unions) all with the same (hopefully) cdef/cpdef/visibility
stuff for members, I'd like to consolidate now.  I will of course, add
special-case code as necessary to support the current syntax, which
can then be removed whenever you think it is appropriate, but writing
separate, near-identical handlers for each type seems like a recipe
for disaster ;).

I'll look to the code for guidance on public, and try to work out the
appropriate meaning during the parse phase.

-- 
This email may be signed or encrypted with GPG (http://www.gnupg.org).
The GPG signature (if present) will be attached as 'signature.asc'.
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20110219/469ed5bf/attachment.pgp>