Mailman 3 July 2011 - cython-devel

PEP 380 approved: syntax for generator delegation
by Stefan Behnel March 6, 2012

March 6, 2012

Hi, PEP 380 ("yield from") recently went through the approval gate. http://www.python.org/dev/peps/pep-0380/ The implementation in CPython isn't rounded up yet and lots of tests are missing from the test suite. But it's agreed that it'll make it into 3.3, and we should try to support it as well. It's a really nice feature. Stefan

2 3

Any news from the IronPython port?
by Stefan Behnel Oct. 8, 2011

Oct. 8, 2011

Hi, subject says it all. The port to IronPython appears to run completely in stealth mode, so I wonder if it has become any usable yet, or if the project was already swept under the carpet and buried for good. Stefan

4 4

Re: [Cython] adding support for __dict__ in extension types
by Lisandro Dalcin Sept. 26, 2011

Sept. 26, 2011

Bringing up this old post... On 21 June 2010 15:41, Robert Bradshaw <robertwb(a)math.washington.edu> wrote: > On Jun 17, 2010, at 9:31 AM, Lisandro Dalcin wrote: > >> If we special case a __dict__ attribute in extension types, i.e: >> >> cdef class Foo: >> cdef dict __dict__ >> >> and fill type->tp_dictoffset, then we can support __dict__ in >> extension types. >> >> What do you think? > > Sounds like a good idea to me. Note that we check tp_dictoffset for > fast dispatching for cpdef methods (which would be correct as a dict > lookup *would* be needed if __dict__ is available). > I still have this patch lying around in my disk. I remember Stefan had some objections. For example, when the user ask for __dict__, a new dict is unconditionally created (in CPython, type dict are allocated on-demand). I propose to get this patch pushed now, and optimize later (however, I really don't know how to safely implement this optimization). -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169

3 4

Vitja's CyFunction branch
by Stefan Behnel Aug. 14, 2011

Aug. 14, 2011

Hi, I wonder what we should do with Vitja's CyFunction branch. He mentioned issues with it in the past (I remember that there was one specific changeset that he considered questionable), and it seems that we found several ways to extend the function support beyond that, which may have an impact on the current way it's implemented. Is this something that makes sense to be included in 0.15, or should it wait to get stable and further extended before being merged? It appears to me that there are reasons for one and the other. However, given that we plan to drop support for (at least) Python 2.3 after this release, I feel that we should try to get out a version that doesn't require much fixing and maintenance. Changing the implementation of functions at this point seems like a rather deep cut. If we agree that it's in a state that we feel comfortable with, I wouldn't object to merging it. Otherwise, I'd suggest to wait until after the release. Comments? Stefan

5 9

Re: [Cython] GCC 4.6 unused-but-set-variable warnings
by Stefan Behnel Aug. 4, 2011

Aug. 4, 2011

[moving this here from cython-users] Nikolaus Rath, 13.06.2011 16:59: > Stefan Behnel writes: >> Nikolaus Rath, 13.06.2011 01:18: >>> Stefan Behnel writes: >>>> Nikolaus Rath, 03.06.2011 23:24: >>>>> Cython 0.14 generated code triggers lots of unused-but-set-variable >>>>> warnings (which are new in GCC 4.6). >>>> >>>> Hmm, that's annoying. We actually generate a lot of useless "dangling >>>> reset to 0" statements for safety (and also as a bit of a "now unused" >>>> hint to the C compiler). >>>> >>>> We could get rid of most of them based on control flow analysis, but I >>>> actually like having them there, just in case we accidentally generate >>>> broken code at some point. A NULL pointer is a lot safer than an >>>> arbitrarily dangling address, especially when it comes to CPython heap >>>> allocated memory (which doesn't necessarily get freed, so you may not >>>> get a quick segfault but a read from dead memory). >>> >>> I don't quite understand. The way to get rid of the warnings is to >>> remove the unnecessary variables. How can this result in a dangling >>> address? You'll get a syntax error from gcc about using an undeclared >>> variable. >> >> Ah, sorry. I misunderstood the meaning of the warning message as >> pointing at unnecessary assignments in general. Could you post a C >> code snippet (and the corresponding Cython code) that triggers this? I >> don't have gcc 4.6 available for testing. > > Sure. It's apparently triggered by unused function arguments: > > $ cat test.pyx > class Operations(object): > def handle_exc(self, fn, exc): > pass > > $ cython test.pyx > > $ gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c test.c -o test.o -D_FILE_OFFSET_BITS=64 -I/usr/include/fuse -Werror -Wall -Wextra -Wconversion -Wno-unused-parameter -Wno-sign-conversion -fno-strict-aliasing test.c: > test.c:454:13: warning: variable ‘__pyx_v_exc’ set but not used [-Wunused-but-set-variable] > test.c:453:13: warning: variable ‘__pyx_v_fn’ set but not used [-Wunused-but-set-variable] > test.c:452:13: warning: variable ‘__pyx_v_self’ set but not used [-Wunused-but-set-variable] http://trac.cython.org/cython_trac/ticket/704 I think this is something that the control flow analysis could deal with. Basically, every parameter that turns out to be unused in the function (and this most likely only happens with parameters) could be ignored, unless it requires a type conversion with potential side effects. In that (unlikely) case, we could still drop the final assignment to the variable itself. Stefan

2 15

Cython 0.15 release candidate
by Robert Bradshaw Aug. 2, 2011

Aug. 2, 2011

Cython has seen an enormous amount of development since 0.14.1. If you are not already using the latest version from the development repository, we encourage you to try out the release candidate: http://cython.org/release/Cython-0.15rc0.tar.gz - Robert

8 12

OpenMP problem
by Vitja Makarov Aug. 2, 2011

Aug. 2, 2011

I've tried openmp support with simple example: from cython.parallel cimport prange def mul(values): ret = 1 for i in prange(values): ret *= i return ret And cython crashes: ((b04e040...)) vitja@vitja-laptop:~/work/cython-vitek/zzz$ make mul.so /usr//bin/python ../cython.py --gdb -v mul.pyx -o mul.c Compiling /home/vitja/work/cython-vitek/zzz/mul.pyx Error compiling Cython file: ------------------------------------------------------------ ... from cython.parallel cimport prange def mul(values): ret = 1 for i in prange(values): ^ ------------------------------------------------------------ mul.pyx:5:26: stop argument must be numeric or a pointer (perhaps if a numeric literal is too big, use 1000LL) Error compiling Cython file: ------------------------------------------------------------ ... from cython.parallel cimport prange def mul(values): ret = 1 for i in prange(values): ^ ------------------------------------------------------------ mul.pyx:5:19: Compiler crash in AnalyseExpressionsTransform ModuleNode.body = StatListNode(mul.pyx:1:0) StatListNode.stats[0] = DefNode(mul.pyx:3:0, modifiers = [...]/0, name = u'mul', num_required_args = 1, reqd_kw_flags_cname = '0', used = True) File 'Nodes.py', line 337, in analyse_expressions: StatListNode(mul.pyx:4:4, is_terminator = True) File 'Nodes.py', line 6553, in analyse_expressions: ParallelRangeNode(mul.pyx:5:19, assigned_nodes = [...]/1, is_parallel = True, is_prange = True, valid_keyword_arguments = [...]/3) Compiler crash traceback from this point on: File "/home/vitja/work/cython-vitek/Cython/Compiler/Nodes.py", line 6553, in analyse_expressions self.index_type, node.type) File "/home/vitja/work/cython-vitek/Cython/Compiler/PyrexTypes.py", line 2624, in widest_numeric_type elif type1.rank < type2.rank: AttributeError: 'PyObjectType' object has no attribute 'rank' make: *** [mul.c] Ошибка 1 rm mul.c ((b04e040...)) vitja@vitja-laptop:~/work/cython-vitek/zzz$ -- vitja.

5 7

Entry.is_arg question
by Vitja Makarov July 31, 2011

July 31, 2011

What is the meaning of Entry.is_arg flag? For generic python arguments it's set to 0, it makes it very tricky to determinate whether entry is argument: https://github.com/cython/cython/blob/master/Cython/Compiler/FlowControl.py… Can we have is_args flag set for all kind of args? And have all the args inside scope.arg_entries? -- vitja.

1 0

problematic code in C++ pyregr tests
by Stefan Behnel July 31, 2011

July 31, 2011

Hi, I noticed a couple of problems in the latest pyregr test run for py3k. https://sage.math.washington.edu:8091/hudson/job/cython-devel-tests-pyregr-… 1) test_contextlib.cpp:16926: warning: deprecated conversion from string constant to ‘char*’ This repeated warning refers (I assume) to each literal in this code: """ char* __pyx_import_star_type_names[] = { "__pyx_scope_struct_2_test_contextmanager_finally", "__pyx_scope_struct_19_test_contextmanager_as_decorator", [...] "__pyx_scope_struct_6_test_contextmanager_except", "__pyx_scope_struct_12_testWithCondition", 0 }; """ This looks like a standard C++ WTF to me. I noticed that Lisandro keeps casting string literals in his code, so I wonder if that's something we need to do here as well? 2) test_cgi.cpp:2494: error: ‘None’ was not declared in this scope There seems to be a problem with the dict-loop optimisation. This is how the following code gets translated: """ * for k, v in dict(form).items(): # <<<<<<<<<<<<<< __Pyx_INCREF(None); __Pyx_XDECREF(__pyx_t_1); __pyx_t_1 = None; __pyx_t_5 = 0; __pyx_t_6 = PyDict_Size(__pyx_t_1); while (1) { /* __pyx_t_7 allocated */ /* __pyx_t_8 allocated */ if (unlikely(__pyx_t_6 != PyDict_Size(__pyx_t_1))) { PyErr_SetString(PyExc_RuntimeError, "dictionary changed size during iteration"); {__pyx_filename = __pyx_f[0]; __pyx_lineno = 114; """ Looks like the literal doesn't get evaluated. I'll look into this. 3) warning: TestClassUtilityCode:18:10: 'cpdef_method' redeclared warning: TestClassUtilityCode:29:10: 'cpdef_cname_method' redeclared 4) test_pep263.cpp:520: error: redefinition of ‘PyObject* __pyx_kp_b_1’ test_pep263.cpp:519: error: ‘PyObject* __pyx_kp_b_1’ previously declared here test_pep263.cpp:526: error: redefinition of ‘PyObject* __pyx_kp_b_2’ test_pep263.cpp:525: error: ‘PyObject* __pyx_kp_b_2’ previously declared here Triggered by this code, which refers to the same byte string literals twice in both cases: """ * self.assertEqual( # <<<<<<<<<<<<<< * "".encode("utf-8"), * b'\xd0\x9f\xd0\xb8\xd1\x82\xd0\xbe\xd0\xbd' */ /* __pyx_t_1 allocated */ __pyx_t_1 = PyObject_GetAttr(__pyx_v_self, __pyx_n_s__assertEqual); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 9; __pyx_clineno = __LINE__; goto __pyx_L1_error;} __Pyx_GOTREF(__pyx_t_1); /* __pyx_t_2 allocated */ __pyx_t_2 = PyTuple_New(2); if (unlikely(!__pyx_t_2)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 9; __pyx_clineno = __LINE__; goto __pyx_L1_error;} __Pyx_GOTREF(((PyObject *)__pyx_t_2)); __Pyx_INCREF(((PyObject *)__pyx_kp_b_1)); PyTuple_SET_ITEM(__pyx_t_2, 0, ((PyObject *)__pyx_kp_b_1)); __Pyx_GIVEREF(((PyObject *)__pyx_kp_b_1)); __Pyx_INCREF(((PyObject *)__pyx_kp_b_1)); PyTuple_SET_ITEM(__pyx_t_2, 1, ((PyObject *)__pyx_kp_b_1)); __Pyx_GIVEREF(((PyObject *)__pyx_kp_b_1)); * self.assertEqual( # <<<<<<<<<<<<<< * "\".encode("utf-8"), * b'\\\xd0\x9f' */ /* __pyx_t_3 allocated */ __pyx_t_3 = PyObject_GetAttr(__pyx_v_self, __pyx_n_s__assertEqual); if (unlikely(!__pyx_t_3)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 13; __pyx_clineno = __LINE__; goto __pyx_L1_error;} __Pyx_GOTREF(__pyx_t_3); /* __pyx_t_2 allocated */ __pyx_t_2 = PyTuple_New(2); if (unlikely(!__pyx_t_2)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 13; __pyx_clineno = __LINE__; goto __pyx_L1_error;} __Pyx_GOTREF(((PyObject *)__pyx_t_2)); __Pyx_INCREF(((PyObject *)__pyx_kp_b_2)); PyTuple_SET_ITEM(__pyx_t_2, 0, ((PyObject *)__pyx_kp_b_2)); __Pyx_GIVEREF(((PyObject *)__pyx_kp_b_2)); __Pyx_INCREF(((PyObject *)__pyx_kp_b_2)); PyTuple_SET_ITEM(__pyx_t_2, 1, ((PyObject *)__pyx_kp_b_2)); __Pyx_GIVEREF(((PyObject *)__pyx_kp_b_2)); """ 5) there are also several compiler crashes in that log 6) several tests bail out with """ File "runtests.py", line 900, in run_test except (unittest.SkipTest, support.ResourceDenied): AttributeError: 'module' object has no attribute 'ResourceDenied' """ I guess that's an incompatibility of our test runner with the latest Py3k. 7) g++: /.../workspace/BUILD/pyregr/cpp/test_inspect.o: No such file or directory g++: no input files There are loads of these - not sure what might trigger them. As I said, I'll look into the dict looping bug. If any of the other bugs is obvious to someone and not too hard to fix, it would be nice to get it done for the release. However, I think none of them is a real blocker. Stefan

2 1

Cython 0.15 release
by Robert Bradshaw July 31, 2011

July 31, 2011

We're long overdue for a release, and this week would be a good one for me to push one out. Hudson https://sage.math.washington.edu:8091/hudson is looking in pretty good shape, and though I know we've got a big pile of stuff currently in progress, we've also got a big backlog of stuff to get out. I'd like to finish looking at https://github.com/cython/cython/pull/38 , are there any other changes that people want to urgently get in? Also, I've started http://wiki.cython.org/ReleaseNotes-0.15 , feel free to edit if you think anything should be highlighted. - Robert

7 25