Mailman 3 April 2011 - cython-devel

Re: [Cython] adding support for __dict__ in extension types
by Lisandro Dalcin Sept. 26, 2011

Sept. 26, 2011

Bringing up this old post... On 21 June 2010 15:41, Robert Bradshaw <robertwb(a)math.washington.edu> wrote: > On Jun 17, 2010, at 9:31 AM, Lisandro Dalcin wrote: > >> If we special case a __dict__ attribute in extension types, i.e: >> >> cdef class Foo: >> cdef dict __dict__ >> >> and fill type->tp_dictoffset, then we can support __dict__ in >> extension types. >> >> What do you think? > > Sounds like a good idea to me. Note that we check tp_dictoffset for > fast dispatching for cpdef methods (which would be correct as a dict > lookup *would* be needed if __dict__ is available). > I still have this patch lying around in my disk. I remember Stefan had some objections. For example, when the user ask for __dict__, a new dict is unconditionally created (in CPython, type dict are allocated on-demand). I propose to get this patch pushed now, and optimize later (however, I really don't know how to safely implement this optimization). -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169

3 4

Hudson pyregr testing takes too long
by Stefan Behnel May 5, 2011

May 5, 2011

Hi, I just noticed that the CPython pyregr tests have jumped up from ~14 minutes for a run to ~4 hours when we added generator support. https://sage.math.washington.edu:8091/hudson/view/cython-devel/job/cython-d… I currently have no idea why that is (well, it's likely because we compile more tests now, but Vitja's branch ran the tests in ~30 minutes). It would be great if someone could find the time to analyse this problem. The current run time makes it basically impossible to keep these tests enabled. Stefan

2 12

Fused Types
by mark florisson May 4, 2011

May 4, 2011

Hey, I've been working a bit on fused types (http://wiki.cython.org/enhancements/fusedtypes), and I've got it to generate code for every permutation of specific types. Currently it only works for cdef functions. Now in SimpleCallNode I'm trying to use PyrexTypes.best_match() to find the function with the best signature, but it doesn't seem to take care of coercions, e.g. it calls 'assignable_from' on the dst_type (e.g. char *), but for BuiltinObjectType (a str) this returns false. Why is this the case, don't calls to overloaded C++ methods need to dispatch properly here also? Other issues are public and api declarations. So should we support that at all? We could define a macro that calls a function that does the dispatch. So e.g. for this ctypedef cython.fused_type(typeA, typeB) dtype cdef func(dtype x): ... we would get two generated functions, say, __pyx_typeA_func and __pyx_typeB_func. So we could have a macro get_func(dtype) or something that then substitutes __pyx_get_func(#dtype), where __pyx_get_func returns the pointer to the right function based on the type names. I'm not sure we should support it, right now I just put the mangled names in the header. At least the cdef functions will be sharable between Cython implementation files. I also noticed that for cdef functions with optional argument it gets a struct as argument, but this struct doesn't seem to end up in the header file when the function is declared public. I believe that also the typedefs for ctypedef-ed things in the .pyx file don't make it there when used to type a cdef function's arguments. Should that be fixed? Cheers, Mark

7 82

prange CEP updated
by Dag Sverre Seljebotn May 4, 2011

May 4, 2011

I've done a pretty major revision to the prange CEP, bringing in a lot of the feedback. Thread-private variables are now split in two cases: i) The safe cases, which really require very little technical knowledge -> automatically inferred ii) As an advanced feature, unsafe cases that requires some knowledge of threading -> must be explicitly declared I think this split simplifies things a great deal. I'm rather excited over this now; this could turn out to be a really user-friendly and safe feature that would not only allow us to support OpenMP-like threading, but be more convenient to use in a range of common cases. http://wiki.cython.org/enhancements/prange <http://wiki.cython.org/enhancements/prange#preview> Dag Sverre

3 63

Pull request emails
by Dag Sverre Seljebotn April 30, 2011

April 30, 2011

Finally think I figured out how to get pull request emails (thanks to Gael V). From https://github.com/organizations/cython/teams/24445: """ Owners do not receive notifications for the organization's repos by default. To receive notifications, create a team and add the owners and repos for which notifications are desired. """ I created Reviewers and added me, Stefan & Robert for now. https://github.com/organizations/cython/teams/54516 Dag Sverre

2 1

speed.pypy.org
by Stefan Behnel April 30, 2011

April 30, 2011

Hi, I'm currently discussing with Maciej Fijalkowski (PyPy) how to get Cython running on speed.pypy.org (that's what I wrote "cythonrun" for). If it works out well, we may have it up in a couple of days. I would expect that Cython won't be a big winner in this game, given that it will only compile plain untyped Python code. It's also going to fail entirely in some of the benchmarks. But I think it's worth having it up there, simply as a way for us to see where we are performance-wise and to get quick (nightly) feed-back about optimisations we try. The benchmark suite is also a nice set of real-world Python code that will allow us to find compliance issues. Stefan

3 12

Fwd: Proposal for a common benchmark suite
by Stefan Behnel April 30, 2011

April 30, 2011

[Seems like this was meant to go to this list but didn't make it there. Forwarding from python-dev] -------- Original-Message -------- Subject: Proposal for a common benchmark suite Date: Thu, 28 Apr 2011 20:55:19 +0200 From: DasIch Hello, As announced in my GSoC proposal I'd like to announce which benchmarks I'll use for the benchmark suite I will work on this summer. As of now there are two benchmark suites (that I know of) which receive some sort of attention, those are the ones developed as part of the PyPy project[1] which is used for http://speed.pypy.org and the one initially developed for Unladen Swallow which has been continued by CPython[2]. The PyPy benchmarks contain a lot of interesting benchmarks some explicitly developed for that suite, the CPython benchmarks have an extensive set of microbenchmarks in the pybench package as well as the previously mentioned modifications made to the Unladen Swallow benchmarks. I'd like to "simply" merge both suites so that no changes are lost. However I'd like to leave out the waf benchmark which is part of the PyPy suite, the removal was proposed on pypy-dev for obvious deficits[3]. It will be easier to add a better benchmark later than replacing it at a later point. Unless there is a major issue with this plan I'd like to go forward with this. .. [1]: https://bitbucket.org/pypy/benchmarks .. [2]: http://hg.python.org/benchmarks .. [3]: http://mailrepository.com/pypy-dev.codespeak.net/msg/3627509/

1 0

Re: [Cython] [cython-users] using generators
by Robert Bradshaw April 28, 2011

April 28, 2011

On Wed, Apr 27, 2011 at 11:42 PM, Stefan Behnel <stefan_ml(a)behnel.de> wrote: > Robert Bradshaw, 27.04.2011 22:41: >> >> On Wed, Apr 27, 2011 at 1:19 PM, Mad wrote: >>> >>> Hi list, >>> i just read, that generators are supported in cython 0.15 - great news >>> and brw, manny thanks. but is it somehow possible to use them with >>> cython 0.14 too? >>> It would be nice, if there's a patch (or the patch of 0.14 -> 0.15 >>> could be applied to the 0.14 source) to get them working, If this is >>> not possible, >> >> No, that's not possible--there's a lot that went on between here and >> there (it's not something you can just patch in). Any reason you're >> trying to stick with 0.14? >> >>> is 0.15 in a usable state? and how could i get it? >> >> It hasn't been released yet, and the development branch (on github) is >> relatively stable but not quite ready yet. As for the state of >> generators, Vitja and Stefan can comment more than I can, but there >> are still some corner cases with exception throwing that don't quite >> work like Python, and there's also a regression with respect to inline >> generators. I'm not sure if either of these is enough to be a blocker >> for a release. > > Don't think they are. As you said, the exception handling is for corner > cases (which exceptions should always be anyway) and I consider the > behaviour "close enough" now to not be a major issue. Vitja can comment on > this, too. The only case where I know that it currently diverges is > exception chaining in Python 3. And that should be fixable. > > Inline generators will require some work to become usable again. > Specifically, the expression scope that I had previously implemented for > comprehensions (and reused for generator expressions) doesn't work with the > then already created function scope of the generator. This needs to be fixed > somehow, either by moving the declarations over to the expression scope or > by directly using the function scope instead. In the latter case, it would > also be nice to merge this with the scoped comprehensions as well. > > So, agreed that these are regressions, but certainly less important than > getting generators and coroutines out in the wild. OK, I've added the inline generator tests to the bug list so we can see where we stand. I'd like to get Sage back up and compiling as well. Any thoughts on a release timeline? Any other known instabilities? - Robert

1 0

HTML Coverage reports
by Stefan Behnel April 28, 2011

April 28, 2011

Hi, I enabled HTML coverage report generation for the test suite. It's now available from the "HTML Coverage Reports" link of the coverage job: https://sage.math.washington.edu:8091/hudson/job/cython-devel-tests-py27-co… From a quick look, the test coverage seems not too bad and we are mostly lacking error tests. However, especially the branch coverage analysis hints at a couple of cases that are worth looking into. Compile time value calculation is an obvious area that is worth improving, and there are clearly some important untested code sections in ExprNodes.py: https://sage.math.washington.edu:8091/hudson/job/cython-devel-tests-py27-co… Stefan

1 0

Compiler error when @staticmethod is used in cdef class
by Bogdan Opanchuk April 27, 2011

April 27, 2011

Hello, I am using Cython 0.14.1 with Python 2.7.1 on OSX 10.6.7 The following code: --- cdef class Test: @staticmethod def func(): print "Static method" --- gives error when being cythoned: "Method func has wrong number of arguments (0 declared, 1 or more expected)" And if one adds some parameters to this method (like "def func(a, b):"), cython crashes with exception: --- ... long stack trace ... File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Cython-0.14-py2.7-macosx-10.6-intel.egg/Cython/Compiler/TypeSlots.py", line 100, in fixed_arg_type return self.format_map[self.fixed_arg_format[i]] KeyError: 'T' --- So, is there a way to create static methods in cdef classes? This workaround seems to work, but it is a bit ugly: --- def func(a, b): print "Static method" cdef class Test: func = func --- Best regards, Bogdan

2 1