Mailman 3 April 2013 - cython-devel

Re: [Cython] [cython-users] ANN: XDress v0.1 -- Automatic Code Generator and C/C++ Wrapper
by Dag Sverre Seljebotn April 7, 2013

April 7, 2013

[Crossing over to cython-devel to discuss further development] Wow, my first impression is this looks great! It's a shame that clang wasn't able to give a good enough parse tree for templates. Is gcc-xml still a fork of a really old gcc? Did you look into the code using the gcc plugin architecture that Phillip Heron worked on in the GSoC last year for this? Also, Cython devs, where's the wiki page listing all the wrapper generators in existence? I tried to find it but couldn't.. Dag Sverre On 04/02/2013 08:26 PM, Anthony Scopatz wrote: > Hello All, > > I am spamming the lists which may be interested in a C/C++ automatic > API wrapper / code generator / type system / thing I wrote. I'll keep > future updates more discrete. I'd love to help folks get started with this > and more participation is always welcome! Release notes are below. > > Please visit the docs: http://bit.ly/xdress-code > > Or just grab the repo: http://github.com/scopatz/xdress > > Be Well > Anthony > > ======================== > XDress 0.1 Release Notes > ======================== > XDress is an automatic wrapper generator for C/C++ written in pure > Python. Currently, > xdress may generate Python bindings (via Cython) for C++ classes & functions > and in-memory wrappers for C++ standard library containers (sets, > vectors, maps). > In the future, other tools and bindings will be supported. > > The main enabling feature of xdress is a dynamic type system that was > designed with > the purpose of API generation in mind. > > Release highlights: > > - Dynamic system for specifying types > - Automatically describes C/C++ APIs from source code with no > modifications. > - Python extension module generation (via Cython) from C++ API > descriptions > - Python views into C++ STL containers. Vectors are NumPy arrays > while maps > and sets have custom collections.MutableMapping and > collections.MutableSet > subclasses. > - Command line interface to the above tools. > > Please visit the website for more information: http://bit.ly/xdress-code > > Or grab the code from GitHub: http://github.com/scopatz/xdress > > XDress is free & open source (BSD 2-clause license) and requires Python 2.7, > NumPy 1.5+, PyTables 2.1+, Cython 0.18+, GCC-XML, and lxml. > > New Features > ============ > > Type System > ----------- > This module provides a suite of tools for denoting, describing, and > converting > between various data types and the types coming from various systems. > This is > achieved by providing canonical abstractions of various kinds of types: > > * Base types (int, str, float, non-templated classes) > * Refined types (even or odd ints, strings containing the letter 'a') > * Dependent types (templates such arrays, maps, sets, vectors) > > All types are known by their name (a string identifier) and may be > aliased with > other names. However, the string id of a type is not sufficient to > fully describe > most types. The system here implements a canonical form for all kinds > of types. > This canonical form is itself hashable, being comprised only of strings, > ints, > and tuples. > > Descriptions > ------------ > A key component of API wrapper generation is having a a top-level, abstract > representation of the software that is being wrapped. In C++ there are > three > basic constructs which may be wrapped: variables, functions, and classes. > Here we restrict ourselves to wrapping classes and functions, though > variables > may be added in the future. > > The abstract representation of a C++ class is known as a description > (abbr. desc). > This description is simply a Python dictionary with a specific structure. > This structure makes heavy use of the type system to declare the types > of all needed > parameters. > > Mini-FAQ > ======== > * Why not use an existing solution (eg, SWIG)? > > Their type systems don't support run-time, user provided refinement > types, > and thus are unsuited for verification & validation use cases that > often > arise in computational science. > > Furthermore, they tend to not handle C++ dependent types well (i.e. > vector<T> > does not come back as a np.view(..., dtype=T)). > > * Why GCC-XML and not Clang's AST? > > I tried using Clang's AST (and the remnants of a broken visitor > class remain > in the code base). However, the official Clang AST Python bindings > lack > support for template argument types. This is a really big deal. > Other C++ ASTs > may be supported in the future -- including Clang's. > > * I run xdress and it creates these files, now what?! > > It is your job to integrate the files created by xdress into your > build system. > > Join in the Fun! > ================ > If you are interested in using xdress on your project (and need help), > contributing > back to xdress, starting up a development team, or writing your own code > generation > front end tool on top of the type system and autodescriber, please let > me know. > Participation is very welcome! > > Authors > ======= > XDress was written by `Anthony Scopatz <http://scopatz.com/>`_, who had many > type system discussions with John Bachan over coffee at the Div school, > and was > polished up and released under the encouragement of Christopher > Jordan-Squire at > `PyCon 2013 <https://us.pycon.org/2013/>`_. > > -- > > --- > You received this message because you are subscribed to the Google > Groups "cython-users" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to cython-users+unsubscribe(a)googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > >

4 4

Re: [Cython] [cython-users] Re: Cython classes and attribute cache
by Stefan Behnel April 6, 2013

April 6, 2013

Nils Bruin, 06.04.2013 21:37: > I can confirm that changing Compiler/TypeSlots.py, line 380 from: > > value = "Py_TPFLAGS_DEFAULT|Py_TPFLAGS_CHECKTYPES| > Py_TPFLAGS_HAVE_NEWBUFFER" > > to: > > value = "Py_TPFLAGS_DEFAULT|Py_TPFLAGS_CHECKTYPES| > Py_TPFLAGS_HAVE_NEWBUFFER|Py_TPFLAGS_HAVE_VERSION_TAG" > > indeed makes the difference in timing disappear. For extension classes > that don't mess directly with their tp_dict having this flag should be > entirely safe. > > See http://bugs.python.org/issue1700288 for background on the > introduction of the method cache in python and http://bugs.python.org/issue1878 > for the reason why the flag is not included in Py_TPFLAGS_DEFAULT on > 2.7, but is in 3.*. I wouldn't mind making it an "on by default" compiler directive. That would mean that you could switch it off with a class decorator at need. Any objections? Stefan

1 1

Fwd: Re: JIT compilers for Python, what is the latest news?
by Stefan Behnel April 6, 2013

April 6, 2013

Maybe we should update the docs here? -------- Original-Message -------- Subject: Re: JIT compilers for Python, what is the latest news? Date: Sat, 6 Apr 2013 11:23:43 +0100 From: Joshua Landau <joshua.landau.ws...(a)gmail.com> To: gmane.comp.python.general On 5 April 2013 19:37, Devin Jeanpierre wrote: > On Fri, Apr 5, 2013 at 4:34 AM, John Ladasky wrote: >> On Thursday, April 4, 2013 7:39:16 PM UTC-7, MRAB wrote: >>> Have you looked at Cython? Not quite the same, but still... >> >> I'm already using Numpy, compiled with what is supposed to be a fast >> LAPACK. I don't think I want to attempt to improve on all the work that >> has gone into Numpy. > > There's no reason you can't use both cython and numpy. See: > http://docs.cython.org/src/tutorial/numpy.html Don't use this. Use memoryviews: http://docs.cython.org/src/userguide/memoryviews.html. I have no idea why that doc page isn't headed "DEPRICATED" by now.

1 0

Add support for the offsetof() C macro
by Emmanuel Gil Peyrot April 5, 2013

April 5, 2013

Hi, offsetof() is not supported by current Cython, and I have not found any workaround (except hardcoding offsets for a specific architecture and compiler, but this is obviously wrong). offsetof(struct_type, member) is a C macro that expands to the offset of the member in the struct, in bytes. Could you add support for it, perhaps like sizeof as a special case in the parser? And many thanks for the whole project! ☺ -- Emmanuel Gil Peyrot XMPP: <linkmauve(a)linkmauve.fr> OpenPGP: 24B1D609

3 3

Bug: Returning real value crashes the code, complex value does not
by Martin Fiers April 2, 2013

April 2, 2013

Dear Cython developers, I stumbled upon a strange error when using Cython. I made a minimal working example, see attachment for the two necessary files. (btw I didn't find the e-mail address of Robert Bradshaw so I could not request him for an account on the issue tracker. Is it possible to put the bug on there?) To reproduce the bug: 1) Reboot to Windows :) (the bug only appears on Windows) 2) Run compile_bug.py to generate the Cython extension 3) Try to run the my_func_exposed function: python >>> import complex_double (does not crash) >>> complex_double.my_func_exposed(1,1j) (crashes) >>> complex_double.my_func_exposed(1,1) If I put a breakpoint in the code with gdb, jump in the code, and leave the function again, it does not crash! Also, it is no problem on Linux. It has to do with the fact that in the first case, a real value was used. In the complex-value case, it does not crash. I went through the generated cpp file and I don't see any issues there (the reason I use cpp is because it's in a big project that needs cpp enabled; it is further linked and so on). gcc version used: 4.6.2 (mingw) cython version used: 0.18 (I did pip install Cython) python version used: python 2.7.3 (MSC v.1500 32 bit). Looking forward to hearing from you! With kind regards, Martin -- ----------------------------------------------------------- ir. Martin Fiers Photonics Research Group Universiteit Gent - Ghent University Sint-Pietersnieuwstraat 41 9000 Gent - Belgium T + 32 9 264 34 48 E martin.fiers(a)intec.ugent.be W www.caphesim.com -----------------------------------------------------------

3 3

Re: [Cython] Bug: Returning real value crashes the code, complex value does not
by Robert Bradshaw April 1, 2013

April 1, 2013

On Wed, Mar 27, 2013 at 2:44 AM, Martin Fiers <Martin.Fiers(a)intec.ugent.be> wrote: > On 3/27/2013 3:54 AM, Robert Bradshaw wrote: >> >> On Tue, Mar 26, 2013 at 5:12 PM, Martin Fiers >> <Martin.Fiers(a)intec.ugent.be> wrote: >>> >>> On 3/26/2013 6:48 PM, Robert Bradshaw wrote: >>>> >>>> On Tue, Mar 26, 2013 at 2:52 AM, Martin Fiers >>>> <Martin.Fiers(a)intec.ugent.be> wrote: >>>>> >>>>> Dear Cython developers, >>>>> >>>>> I stumbled upon a strange error when using Cython. I made a minimal >>>>> working >>>>> example, see attachment for the two necessary files. (btw I didn't find >>>>> the >>>>> e-mail address of Robert Bradshaw so I could not request him for an >>>>> account >>>>> on the issue tracker. Is it possible to put the bug on there?) >>>> >>>> Sure. You should have my email now. >>> >>> Thank you! I just sent a mail. >>> >>> Also, thanks for replying so quickly. Replies follow inline. >>> >>>>> To reproduce the bug: >>>>> 1) Reboot to Windows :) (the bug only appears on Windows) >>>>> 2) Run compile_bug.py to generate the Cython extension >>>>> 3) Try to run the my_func_exposed function: >>>>> >>>>> python >>>>>>>> >>>>>>>> import complex_double >>>>> >>>>> (does not crash) >>>>>>>> >>>>>>>> complex_double.my_func_exposed(1,1j) >>>>> >>>>> (crashes) >>>>>>>> >>>>>>>> complex_double.my_func_exposed(1,1) >>>>> >>>>> If I put a breakpoint in the code with gdb, jump in the code, and leave >>>>> the >>>>> function again, it does not crash! Also, it is no problem on Linux. >>>>> >>>>> It has to do with the fact that in the first case, a real value was >>>>> used. >>>>> In >>>>> the complex-value case, it does not crash. I went through the generated >>>>> cpp >>>>> file and I don't see any issues there (the reason I use cpp is because >>>>> it's >>>>> in a big project that needs cpp enabled; it is further linked and so >>>>> on). >>>>> >>>>> gcc version used: 4.6.2 (mingw) >>>>> cython version used: 0.18 (I did pip install Cython) >>>>> python version used: python 2.7.3 (MSC v.1500 32 bit). >>>> >>>> Very strange. Does calling PyComplex_AsCComplex directly produce the >>>> same crash? What about >>> >>> I'm not sure how to call this directly. Do you mean by modifying the >>> generated cpp file and then manually building an extension module? >>> >>>> cdef complex double x = 1.0 >>> >>> This one works. >>> >>>> or >>>> >>>> cdef object py_x = 1.0 >>>> cdef complex double x = py_x >>> >>> This one crashes! >> >> Ah. Try >> >> from cpython.complex cimport Py_complex, PyComplex_AsCComplex >> cdef Py_complex x = PyComplex_AsCComplex(py_x) >> print x.real, x.imag > > Ok. I tried this, and it also crashes. Here's the modification: > > from cpython.complex cimport Py_complex > from cpython.complex cimport PyComplex_AsCComplex > > @cython.cdivision(True) > cdef public double complex my_func(int a, b): > > cdef object py_x = 1.0 > > #cdef double complex x = 1.0 # Does not crash > #cdef double complex x2 = py_x # Crashes for py_x = 1, > not for py_x=1j. > #cdef Py_complex x = PyComplex_AsCComplex(py_x) # Crashes, even for > py_x=1j > #print x.real, x.imag > > And as you can see, the PyComplex_AsCComplex also crashes (SIGSEGV). > I tried to compile with debug information, as in the instructions in > http://docs.cython.org/src/userguide/debugging.html > But I cannot get the line numbers. Probably I need a debug-python version, > but that seems to be very nontrivial on Windows. > > Not sure if I can think of other options to test it and/or track down the > bug... > > Now it even crashes when py_x = 1j. So maybe there's something else going > wrong here too. I wonder if it's a compiler miss-match or something like that. > Regards, > Martin > > P.S. I only replied to you because you didn't put the > cython-devel(a)python.org in the previous mail. Oops. Un-intentional oversight.

3 2