Mailman 3 September 2012 - cython-devel

[PATCH] make memoryviews work when strides is NULL (unfinished)
by Christoph Groth Sept. 19, 2012

Sept. 19, 2012

Hi, Thanks a lot for the recently added generic memoryview support, it's just what we needed to optimize an important part of our program. However, I believe that there's a problem with Cython's support for objects which do not provide "strides" information (because they are C-contiguous). The buffer API documentation [1] is a bit ambiguous on this, but it says "If strides is NULL, the array is interpreted as a standard n-dimensional C-array." In any case, numpy happily accepts such buffers and computes the "strides" itself. A thread [2] on Python's devel mailing list also supports the view that setting strides to NULL is OK even if strides were requested. You might ask: why not simply provide strides when they have been requested? The problem is to find space for them. malloc'ing or PyMem_Alloc'ing memory for strides each time when a buffer is requested is slow, and reserving space for strides within the object is wasteful when "shape" is all that's needed and you have _many_ small objects which expose the buffer interface. The provided patch works for our use case, but someone with a good understanding of Cython's memoryview support should finish it. I believe the patch will break more complicated buffer use cases. [1] http://docs.python.org/dev/c-api/buffer.html [2] http://thread.gmane.org/gmane.comp.python.devel/134387 Thank you for considering fixing this issue in Cython, Christoph --- Cython/Utility/MemoryView_C.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/Cython/Utility/MemoryView_C.c b/Cython/Utility/MemoryView_C.c index 84e55da..c2524b0 100644 --- a/Cython/Utility/MemoryView_C.c +++ b/Cython/Utility/MemoryView_C.c @@ -207,12 +207,6 @@ static int __Pyx_ValidateAndInit_memviewslice( goto fail; } - if (!buf->strides) { - PyErr_SetString(PyExc_ValueError, - "buffer does not supply strides necessary for memoryview."); - goto fail; - } - for(i=0; i<ndim; i++) { spec = axes_specs[i]; @@ -320,8 +314,19 @@ __Pyx_init_memviewslice(struct __pyx_memoryview_obj *memview, goto fail; } + if (buf->strides) { + for (i = 0; i < ndim; i++) { + memviewslice->strides[i] = buf->strides[i]; + } + } else { + Py_ssize_t stride = buf->itemsize; + for (i = ndim - 1; i >= 0; i--) { + memviewslice->strides[i] = stride; + stride *= buf->shape[i]; + } + } + for (i = 0; i < ndim; i++) { - memviewslice->strides[i] = buf->strides[i]; memviewslice->shape[i] = buf->shape[i]; if (buf->suboffsets) { memviewslice->suboffsets[i] = buf->suboffsets[i]; -- 1.7.10.4

2 3

new FFI library for Python
by Saravanan Shanmugham Sept. 17, 2012

Sept. 17, 2012

I have been using CFFI for wrapping a whole bunch of C libraries and I can tell you its way easier than Cython. And a couple of clarifications of misconceptions mentioned on this list. 1. It does not "require" GCC at run time. The extension can be compiled at package/install time using the GCC and can run without need GCC at run time. 2. Its way easier to wrap C libraries than Cython. I would know, since I started with Cython. And after many email back and forth with the cython-users mailing list, I was referred to CFFI which turned out great. The trouble was wrapping C structures and datastructures with Python wrappers. 3. I had to just copy over the C header files as is and had a replace a few #define values with "..." to ask it to automatically get these values during compile time from the compiler. This makes it even more simpler. Now, I am still a Cython lover when it comes optimizing portions of my Python code and having it compiled to C. The question I had in mind is, If I had say a shared math library libmath.so and wrapped it with a CFFI based python wrapper, which includes a portion of python wrapper code that calls into the generated Python/C wrapper shared library. And then wrote a piece of cython code that invoked that CFFI based python wrapper, how will my performance be compared to writing the python wrapper for libmath.so with cython cdef or cpdef wrapper function definitions. Thanks, Sarvi

2 1

usage does not match the options
by B. Clausius Sept. 12, 2012

Sept. 12, 2012

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I found that the options shown by "cython -h" are different to the options parsed by cython. Usage: --warning-error Parser: --warning-errors The parser does not recognize "-q" and "-quiet" (Patch attached) There are other options that are not in the usage: -+ -z, --pre-import --convert-range --no-c-in-traceback --disable-function-redefinition --old-style-globals --debug -h, --help B.C. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iQIcBAEBAgAGBQJQUIG1AAoJEEDg0A3D0GN4lN0QAKicvLAAcVqvJ6+12ErOSLxx hzvJcIvjR9EBys3QMzq6+wfGY8JdzWA4ofn6VI+xtLkFDt9NhlpvB64UN1TUy1Nm zj5IopRVV9VuKAvWxHoxk7QsuR2PmOd3ZR7d+v4xOxncfKmp1NNYYwuSvF9o232M gdABJ3U34hGXhWTtVt21WkFqBgIQFnuWsCCzOokNL/rYuUEu1/AalFgy6Eko8xs8 V55bGXAFEgNCGJb/DPWglsTqoS3XJO4Wr7nt/N7ca6ejHo6Q0acA8GEcBikXM7kA KI7swe/FwcNzz3FR0JflhbnzVltrR/bVGuhtjdElH2CNfSeX0lLIeihl9qmANYiZ oGnl5n2Wmx/jUaFiEVmXa1ry+0xaNU10n7lv9zfAJYHt4xzF0HOmyudmviU34SVi 2w+SVPdBNH/08R0C3JPkcrLxN3L9RkXWzv6+bEsdtTGrWycQyXSLpVJn66OtAeKl LoBJw2wzfibazKIWO7PuNW4VTz5Mcn4m8NG/ujEpw3WUE97mqRZRm9cp3opaeIjh uZeWLUsEgn2GEMAwVrg10ILdverD08RX7ovyUqKCP84W03MV5fBHQISNAW2rS1ig KXf1zm0lYr+W1MN9BnGQtqUTFSzxpnNJZ4KhDio+kAMc4bqbSk6R2ste2XXbS6/j JGCc02zjM7TtSuw33oeJ =8zfk -----END PGP SIGNATURE-----

2 1

cython --create-listing fails
by B. Clausius Sept. 12, 2012

Sept. 12, 2012

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, when running: $ cython --create-listing -o build/temp.linux-x86_64-2.7/pybiklib/gldraw_c.c build/temp.linux-x86_64-2.7/pybiklib/gldraw_c.pyx I get the error: Traceback (most recent call last): File "/usr/bin/cython", line 8, in <module> main(command_line = 1) File "/usr/lib/pymodules/python2.7/Cython/Compiler/Main.py", line 662, in main result = compile(sources, options) File "/usr/lib/pymodules/python2.7/Cython/Compiler/Main.py", line 637, in compile return compile_multiple(source, options) File "/usr/lib/pymodules/python2.7/Cython/Compiler/Main.py", line 609, in compile_multiple result = run_pipeline(source, options) File "/usr/lib/pymodules/python2.7/Cython/Compiler/Main.py", line 472, in run_pipeline context.setup_errors(options, result) File "/usr/lib/pymodules/python2.7/Cython/Compiler/Main.py", line 399, in setup_errors result.listing_file = Utils.replace_suffix(source, ".lis") NameError: global name 'source' is not defined I'm using Cython 0.15.1 . The patch is against git as it seems the code there is same but tested only for 0.15.1. B.C. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iQIcBAEBAgAGBQJQUGjJAAoJEEDg0A3D0GN4Yj0QAKYHfo1C7bwl8a2pZi+DPrrE N0T5KAzu3RjYgacw0MpC5hb+L1X6J5Yi3BAdIp4ertCY6xeKD6+lQt2q+zHK0De3 0CI4ivnh+yH/x5hvjRjDprrpotKc/tre0BWMpKpchw2ohZ21PRDmxpOsjFzTmyC0 V8TO0AG0hzpabzI9qNZRwyBz/GOzfFwyumKBDOv/z/2n2LDDBO+k10orfXrfhjdI uFE6m1YAA4vK0apTi0F6GokJNq+vqT2pRNEzO/Fsek7kRxoNCAUrbzL0aKMj8wsz M4wIUq8TE3R2d5cvEt+ibQK0+7OydoKoaiCd3BfumVsj5SBRio/GcMetJA1lThUn W8e1yPtzPO/GlSatdvRiIJFzyzlX1ZVAfFXD419Gq0/X23/GK+j9k1rHQzg8e3+j 7yVph3DLfZfnQqmBpD3Xl4NCp7zYq7QgfdsB1GNzub/PEatfcv2DVLlnCDCTKgGu j5/r/0dwrOkb1q3SGtONo/dAovSi0oIbW3Wfe4ebB8S1HVo9pFpPDIqc1g3xXRA2 laqc7vBAcE5ZfS5/uDmW74PMV1khDeDkhmhoEGSQbFTSSC40DwufvyBAPMyzjjvf 9JAvOp5FS9RS0020KPKIlOtxQXA+CrJSIbaQZu6m/nmH17a60QnW8kwZ799/Pqgd IsacvSJvpYKHtQbg9JCJ =Hav4 -----END PGP SIGNATURE-----

1 0

Hitting gcc 4.6 bug with Cython 0.17 and pandas
by Wes McKinney Sept. 10, 2012

Sept. 10, 2012

This issue came up earlier today. Everything works fine for me with gcc 4.6.1 and Cython 0.17, but maybe there is a bug being hit in gcc 4.6. Any debugging help would be appreciated (since this affects EC2 AMIs...) http://github.com/pydata/pandas/issues/1880 thanks, Wes

1 0

Cython-0.17 builtin type import bug
by Robert Byrnes Sept. 10, 2012

Sept. 10, 2012

If a builtin type that differs between Python 2 and 3 (e.g., unicode) is imported, then malformed C code is emitted by Cython-0.17: there is an extraneous, unbalanced #if. Here's a small reproducer: shell$ cat foo.pyx cdef extern from "Python.h": ctypedef class __builtin__.unicode [object PyUnicodeObject]: pass shell$ cython foo.pyx warning: foo.pyx:2:4: unicode already a builtin Cython type shell$ gcc -c -fPIC -I/usr/include/python2.6 foo.c foo.c:5:1: error: unterminated #else foo.c: In function 'initfoo': foo.c:574: error: expected declaration or statement at end of input foo.c:535: error: label '__pyx_L1_error' used but not defined This works for Cython-0.16 and earlier. I think this was broken by ... https://github.com/cython/cython/commit/558f4ef1c48ce091fa15b9319eb0f92cd47… This patch for ModuleNode.generate_type_import_call seems to fix it: --- a/Cython/Compiler/ModuleNode.py +++ b/Cython/Compiler/ModuleNode.py @@ -2281,7 +2281,6 @@ class ModuleNode(Nodes.Node, Nodes.BlockNode): module_name = '__Pyx_BUILTIN_MODULE_NAME' if type.name in Code.non_portable_builtins_map: condition, replacement = Code.non_portable_builtins_map[type.name] - code.putln("#if %s" % condition) if objstruct in Code.basicsize_builtins_map: # Some builtin types have a tp_basicsize which differs from sizeof(...): sizeof_objstruct = Code.basicsize_builtins_map[objstruct] Thanks, -- Bob

1 0

Fwd: [numpy] MAINT: silence Cython warnings about changes dtype/ufunc size. (#432)
by Nathaniel Smith Sept. 8, 2012

Sept. 8, 2012

Hi Cythoneers, Ralf just proposed this pull request for numpy, to unconditionally install a warning filter to silence all "numpy.dtype size changed", "numpy.ufunc size changed" warnings that Cython likes to spit out. See the links below for details. Figured you should have a chance to comment, anyway. -n ---------- Forwarded message ---------- From: Ralf Gommers <notifications(a)github.com> Date: Sat, Sep 8, 2012 at 9:00 PM Subject: [numpy] MAINT: silence Cython warnings about changes dtype/ufunc size. (#432) To: numpy/numpy <numpy(a)noreply.github.com> These warnings are visible whenever you import scipy (or another package) that was compiled against an older numpy than is installed. For example compiled against 1.5.1, like current scipy binaries are, and used with 1.7.0. These warnings aren't useful; if numpy would really break its ABI it would be noticed in no time without these warnings. ------------------------------ You can merge this Pull Request by running: git pull https://github.com/rgommers/numpy silence-cython-warnings Or view, comment on, or merge it at: https://github.com/numpy/numpy/pull/432 Commit Summary - MAINT: silence Cython warnings about changes dtype/ufunc size. File Changes - *M* numpy/__init__.py (6) Patch Links - https://github.com/numpy/numpy/pull/432.patch - https://github.com/numpy/numpy/pull/432.diff — Reply to this email directly or view it on GitHub<https://github.com/numpy/numpy/pull/432>.

3 2

Non-profit 501(c)(1) Cython foundation
by Robert Bradshaw Sept. 5, 2012

Sept. 5, 2012

NumFOCUS has graciously offered to help Cython get non-profit US tax status as part of their application as a part of a "Group Exception." Basically, this gives us all the benefits of a non-profit (including being able to receive and spend tax-favored donations) without having to go through the enormous amount of paperwork involved, both upfront (to apply) and ongoing. For more context, see https://groups.google.com/forum/?fromgroups=#!topic/numfocus/moVBdZDLqmI Though we don't have any immediate need of such a status, doing it now will be much easier than trying to do it later (especially if there's a tight timeframe when we recognize the need). I went ahead and applied for an EIN for a "Cython Foundation" and they have some paperwork to sign (see below). Any thoughts/concerns/suggestions? - Robert ---------- Forwarded message ---------- From: Leah Holdridge <lsholdridge(a)gmail.com> Date: Tue, Sep 4, 2012 at 3:32 PM Subject: Group Exemption To: Ondřej Čertík <ondrej.certik(a)gmail.com>, Brian Granger <ellisonbg(a)gmail.com>, Robert Bradshaw <robertwb(a)gmail.com> Cc: Numfocus admin <admin(a)numfocus.org> Hi Ondrej, Brian, and Robert, The good news is; the only items I need from you for the Group Exemption are the 2 documents I've attached needing your signature. The first is the Articles of Association. This is the uniform governing instrument that states your nonprofit tax-exempt purpose in 501(c)(3) language. It also states that you are a subordinate organization under general supervision of NumFOCUS. This is regarding the responsibility NumFOCUS has in ensuring that all it's subordinates continue to qualify for tax-exempt status. I need 2 signatures, it should be officers, but since you are not a formal corporation at this time, it can really be anyone involved. It would be best if they are US citizens. The other document is the letter from you giving written authorization to NumFOCUS to be a central organization. I have listed each of you as president of your group, if your have another title, or don't have one at all, that's no problem. Also, it states in the letter that as a board you have voted on and approved this. Since you have no formal governing body or bylaws, this can be interpreted very loosely, an email or discussion is fine. Ondrej, I didn't have your address, so that needs to be put in SymPy's letter. Brian, I need your EIN, the actual name you registered with the IRS when you applied for your EIN ( I used IPython on the attached documents, but wasn't sure if that needs to be corrected), your mailing address, and brief description with expenditures. Now for the not so good news; I need these back tomorrow or first thing Thursday morning so I can send off the application Thursday. You can print the documents off, sign, and fax back to me at: 800-753-8308. Thanks. Have a good one, Leah

2 2