[Cython] [cython-users] Cython .pxd introspection: listing defined constants
W. Trevor King
wking at drexel.edu
Thu Feb 17 14:29:41 CET 2011
This thread is coming over to cython-dev (and the new cython-devel)
from cython-users because it turns out it will probably require
chaning the Cython code. To get everyone who hasn't been following on
cython-users up to speed, here's a summary of what I'm trying to do:
That's what I was trying to give with this:
On Wed, Feb 09, 2011 at 12:23:25PM -0500, W. Trevor King wrote:
> I'm wrapping an external C library with Cython, so I have `mylib.pxd`:
>
> cdef extern from 'mylib.h'
> enum: CONST_A
> enum: CONST_B
> ...
>
> where I declare each constant macro from the library's header `mylib.h`:
>
> #define CONST_A 1
> #define CONST_B 2
> ...
>
> Now I want to expose those constants in Python, so I have `expose.pyx`:
>
> cimport mylib
>
> CONST_A = mylib.CONST_A
> CONST_B = mylib.CONST_B
> ...
>
> But the last part seems pretty silly. I'd like to do something like
>
> cimport mylib
> import sys
>
> for name in dir(mylib):
> setattr(sys.modules[__name__], name, getattr(mylib, name))
>
> which compiles fine, but fails to import with...
Looking into the Cython internals, everything defined in mylib.pxd is
stored as `Entry`s in a `ModuleScope`, and...
On Wed, Feb 16, 2011 at 03:55:19PM -0800, Robert Bradshaw wrote:
> On Wed, Feb 16, 2011 at 8:17 AM, W. Trevor King <wking at drexel.edu> wrote:
> > What I'm missing is a way to bind the ModuleScope namespace to a name
> > in expose.pyx so that commands like `dir(mylib)` and `getattr(mylib,
> > name)` will work in expose.pyx.
>
> You have also hit into the thorny issue that .pxd files are used for
> many things. They may be pure C library declarations with no Python
> module backing, they may be declarations of (externally implemented)
> Python modules (such as numpy.pxd), or they may be declarations for
> Cython-implemented modules.
> > It seems like it would be easier to generate some kind of wrapper
> > class (PxdModule?) for mylib when it is cimported (at compile time),
> > and then further interactions would take care of themselves (at run
> > time).
>
> Would such an object be created anew for every module that cimports
> the declaration file?
Hmm, That doesn't sound very nice, does it. However, .pxd files
declaring C libraries have no Python-space presence, so that was my
initial idea.
> I have toyed with the idea of subclassing the module object itself for
> better support of C-level attributes from the Python (and Cython)
> namespaces.
Sorry, I don't understand "better support of C-level attributes". Can
you give an example?
> Here's another idea, what if extern blocks could contain cpdef
> declarations, which would automatically generate a Python-level
> wrappers for the declared members (if possible, otherwise an error)?
Ah, this sounds good! Of the three .pxd roles you list above,
external Python modules (e.g. numpy) and Cython-implemented modules
(e.g. matched .pxd/.pyx) both already have a presence in Python-space.
What's missing is a way to give (where possible) declarations of
external C libraries a Python presence. cpdef fills this hole nicely,
since its whole purpose is to expose Python interfaces to
C-based elements.
A side effect of this cpdef change would be that now even bare .pxd
files (no matching .pyx) would have a Python presence, so You could do
something like
cimport mylib as mylib_c
import mylib as mylib_py
import sys
# Access through Python
for name in dir(mylib_py):
setattr(sys.modules[__name__], name, getattr(mylib_py, name))
# Direct C access
cdef get_a():
return mylib_c.CONST_A
Where the Python access would be the new feature, and list all
cpdef-ed stuff. However, from Parsing.py:2369:
error(pos, "C struct/union/enum cannot be declared cpdef")
From pyrex_differences.rst:
If a function is declared :keyword:`cpdef` it can be called from
and overridden by both extension and normal python subclasses.
I believe the reason that cpdef-ed enums and similar are currently
illegal is confusion between "can be called from Python" and "can be
overridden from Python". I think these should be just like methods
already are, in that you can "override" a method by subclassing it,
but not by rebinding the name in the base class:
>>> import pyximport; pyximport.install()
>>> import rectangle as R
>>> r = R.Rectangle(1, 2, 3, 4)
>>> r.area = lambda(self): r.x1'
Traceback (most recent call last):
File "<string>", line 1, in <module>
AttributeError: 'rectangle.Rectangle' object attribute 'area' is read-only
Where rectangle.pyx is a minorly patched version of the last example
from early_binding_for_speed.rst [1] and `area` is a cpdef-ed method.
Why can't enums share this handling, with the enum taking the place of
the method and the enum's module taking the place of the class? After
all, enums have a Python-side tyoe (int or long). Unions don't really
have a Python parallel, but structs do, so long as you can select
which attributes should have (writable) Python interfaces. If we
change the struct declaration syntax to be closer to the `cdef class`
declaration syntax:
cpdef struct Foo:
cpdef public int intA
cpdef readonly int intB
cdef void *ptr
We would both declare the important members of the C struct (as we can
already do in Cython) and also have Cython automatically generate a
Python class wrapping the struct (because of `cpdef struct`). The
Python class would have:
* Cython-generated getter/setter for intA (because of `cpdef public`)
using the standard Python<->int coercion.
* Similar Cython-generated getter for int B (because of `cpdef
readonly`).
* No Python access to ptr (standard C-access still possible through
Cython).
Doing something crazy like `cdef public void *ptr` would raise a
compile-time error.
I'm definately willing to help out with this (if someone will point me
in the right direction), as the enum stuff would fix my original
problem, and the struct stuff would allow me to rip out a bunch of
boilerplate like
cdef class Foo (object):
cdef mylib.Foo _Foo
def _intA_get(self):
return self._Foo.intA
def _intA_set(self, value):
self._Foo.intA = value
intA = property(fget=_intA_get, fset=_intA_set)
def _intB_get(self):
return self._Foo.intB
intB = property(fget=_intB_get)
from my wrapper code.
Thanks,
Trevor
[1] While testing my overriding method example, I found a small typo
in cython-docs' early_binding_for_speed.rst. Patch attached.
--
This email may be signed or encrypted with GPG (http://www.gnupg.org).
The GPG signature (if present) will be attached as 'signature.asc'.
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt
-------------- next part --------------
From 80a52b3c0224e73a969b88fb414b6d026029a85e Mon Sep 17 00:00:00 2001
From: W. Trevor King <wking at drexel.edu>
Date: Thu, 17 Feb 2011 07:57:11 -0500
Subject: [PATCH] `int` -> `cdef int` when declaring local variables in early binding example.
---
src/userguide/early_binding_for_speed.rst | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/userguide/early_binding_for_speed.rst b/src/userguide/early_binding_for_speed.rst
index 07e0047..d44498d 100644
--- a/src/userguide/early_binding_for_speed.rst
+++ b/src/userguide/early_binding_for_speed.rst
@@ -53,7 +53,7 @@ where calls occur within Cython code. For example:
def __init__(self, int x0, int y0, int x1, int y1):
self.x0 = x0; self.y0 = y0; self.x1 = x1; self.y1 = y1
cdef int _area(self):
- int area
+ cdef int area
area = (self.x1 - self.x0) * (self.y1 - self.y0)
if area < 0:
area = -area
@@ -88,7 +88,7 @@ overheads. Consider this code:
def __init__(self, int x0, int y0, int x1, int y1):
self.x0 = x0; self.y0 = y0; self.x1 = x1; self.y1 = y1
cpdef int area(self):
- int area
+ cdef int area
area = (self.x1 - self.x0) * (self.y1 - self.y0)
if area < 0:
area = -area
--
1.7.3.4
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20110217/83dfed52/attachment.pgp>
More information about the cython-devel
mailing list