Numpy/Cython Google Summer of Code project idea
Hi all, after the Scipy/Sage Days 8 meeting, we were all very impressed by the progress made by Cython. For those not familiar with it, Cython: http://www.cython.org/ is an evolved version of Pyrex (which is used by numpy and scipy) with lots of improvements. We'd like to position Cython as the preferred way of writing most, if not all, new extension code written for numpy and scipy, as it is easier to write, get right, debug (when you still get it wrong) and maintain than writing to the raw Python-C API. A specific project along these lines, that would be very beneficial for numpy could be: - Creating new matrix types in cython that match the cvxopt matrices. The creation of new numpy array types with efficient code would be very useful. - Rewriting the existing ndarray subclasses that ship with numpy, such as record arrays, in cython. In doing this, benchmarks of the relative performance of the new code should be obtained. Another possible project would be the addition to Cython of syntactic support for array expressions, multidimensional indexing, and other features of numpy. This is probably more difficult than the above, as it would require fairly detailed knowledge of both the numpy C API and the Cython internals, but would ultimately be extremely useful. Any student interested in this should quickly respond on the list; such a project would likely be co-mentored by people on the Numpy and Cython teams, since it is likely to require expertise from both ends. Cheers, f
A specific project along these lines, that would be very beneficial for numpy could be:
- Creating new matrix types in cython that match the cvxopt matrices. The creation of new numpy array types with efficient code would be very useful.
- Rewriting the existing ndarray subclasses that ship with numpy, such as record arrays, in cython. In doing this, benchmarks of the relative performance of the new code should be obtained.
What level of experience do you think would be necessary for the student for this? I've got a fair amount of Python & C experience, and I've used numpy and Pyrex (but not Cython) in the past. I wouldn't mind putting in some time to become familiar with the particulars before submitting a project proposal.
Hi Alan, On Thu, Mar 6, 2008 at 3:18 PM, Alan McIntyre <alan.mcintyre@gmail.com> wrote:
A specific project along these lines, that would be very beneficial for numpy could be:
- Creating new matrix types in cython that match the cvxopt matrices. The creation of new numpy array types with efficient code would be very useful.
- Rewriting the existing ndarray subclasses that ship with numpy, such as record arrays, in cython. In doing this, benchmarks of the relative performance of the new code should be obtained.
What level of experience do you think would be necessary for the student for this? I've got a fair amount of Python & C experience, and I've used numpy and Pyrex (but not Cython) in the past. I wouldn't mind putting in some time to become familiar with the particulars before submitting a project proposal.
I don't want to put words in others' mouths, since I wouldn't be the one mentoring such a project. I suspect that you'd need to be reasonably familiar with some C programming, because at this point in the game this project might require debugging generated C code and perhaps diving into Cython itself. In the long term we'd like Cython to be so python-like and friendly that those who are *not* C experts can use it effectively for Numpy programming, but that isn't currently the case. As far as Pyrex/cython, if you know pyrex, you'll be OK with cython. It's only better than pyrex, but is as far as I know mostly, if not fully compatible with pyrex. Cheers f
Folks, I may be asked to help port an application written in C that runs on Macs to run on a Windows-based PC. I have two questions: 1) Does anyone have recommendations for a free, open source C compiler for Windows? 2) Is there a step-by-step guide or tutorial for writing a Python module to act as a wrapper for the C code? Thanks. Brandon Nuttall
Hi, You can check on mingw, but also the Visual C++ Express edition (although it is not open source, but I don't see why it should be). To write a module, you can check ctypes (and there is a tutorial in the scipy's cookbook ;)) Matthieu 2008/3/7, Nuttall, Brandon C <bnuttall@uky.edu>:
Folks,
I may be asked to help port an application written in C that runs on Macs to run on a Windows-based PC. I have two questions:
1) Does anyone have recommendations for a free, open source C compiler for Windows?
2) Is there a step-by-step guide or tutorial for writing a Python module to act as a wrapper for the C code?
Thanks.
Brandon Nuttall
_______________________________________________ SciPy-user mailing list SciPy-user@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-user
-- French PhD student Website : http://matthieu-brucher.developpez.com/ Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn : http://www.linkedin.com/in/matthieubrucher
Also check out http://pymag.phparch.com The December issue (which you can purchase separately) has a thorough introduction for ctypes, with a science library bent! I have been wrapping libraries like mad since! Gabriel On Fri, Mar 07, 2008 at 08:08:13PM +0100, Matthieu Brucher wrote:
Hi,
You can check on mingw, but also the Visual C++ Express edition (although it is not open source, but I don't see why it should be). To write a module, you can check ctypes (and there is a tutorial in the scipy's cookbook ;))
Matthieu
2008/3/7, Nuttall, Brandon C <[1]bnuttall@uky.edu>:
Folks,
I may be asked to help port an application written in C that runs on Macs to run on a Windows-based PC. I have two questions:
1) Does anyone have recommendations for a free, open source C compiler for Windows?
2) Is there a step-by-step guide or tutorial for writing a Python module to act as a wrapper for the C code?
Thanks.
Brandon Nuttall
_______________________________________________ SciPy-user mailing list [2]SciPy-user@scipy.org [3]http://projects.scipy.org/mailman/listinfo/scipy-user
-- French PhD student Website : [4]http://matthieu-brucher.developpez.com/ Blogs : [5]http://matt.eifelle.com and [6]http://blog.developpez.com/?blog=92 LinkedIn : [7]http://www.linkedin.com/in/matthieubrucher
References
Visible links 1. mailto:bnuttall@uky.edu 2. mailto:SciPy-user@scipy.org 3. http://projects.scipy.org/mailman/listinfo/scipy-user 4. http://matthieu-brucher.developpez.com/ 5. http://matt.eifelle.com/ 6. http://blog.developpez.com/?blog=92 7. http://www.linkedin.com/in/matthieubrucher
_______________________________________________ SciPy-user mailing list SciPy-user@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-user
I mean the January issue . . . Gabriel On Fri, Mar 07, 2008 at 12:39:32PM -0700, Gabriel Gellner wrote:
Also check out http://pymag.phparch.com The December issue (which you can purchase separately) has a thorough introduction for ctypes, with a science library bent! I have been wrapping libraries like mad since!
Gabriel
On Fri, Mar 07, 2008 at 08:08:13PM +0100, Matthieu Brucher wrote:
Hi,
You can check on mingw, but also the Visual C++ Express edition (although it is not open source, but I don't see why it should be). To write a module, you can check ctypes (and there is a tutorial in the scipy's cookbook ;))
Matthieu
2008/3/7, Nuttall, Brandon C <[1]bnuttall@uky.edu>:
Folks,
I may be asked to help port an application written in C that runs on Macs to run on a Windows-based PC. I have two questions:
1) Does anyone have recommendations for a free, open source C compiler for Windows?
2) Is there a step-by-step guide or tutorial for writing a Python module to act as a wrapper for the C code?
Thanks.
Brandon Nuttall
_______________________________________________ SciPy-user mailing list [2]SciPy-user@scipy.org [3]http://projects.scipy.org/mailman/listinfo/scipy-user
-- French PhD student Website : [4]http://matthieu-brucher.developpez.com/ Blogs : [5]http://matt.eifelle.com and [6]http://blog.developpez.com/?blog=92 LinkedIn : [7]http://www.linkedin.com/in/matthieubrucher
References
Visible links 1. mailto:bnuttall@uky.edu 2. mailto:SciPy-user@scipy.org 3. http://projects.scipy.org/mailman/listinfo/scipy-user 4. http://matthieu-brucher.developpez.com/ 5. http://matt.eifelle.com/ 6. http://blog.developpez.com/?blog=92 7. http://www.linkedin.com/in/matthieubrucher
_______________________________________________ SciPy-user mailing list SciPy-user@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-user mailing list SciPy-user@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-user
Hi,
You can check on mingw, but also the Visual C++ Express edition (although it is not open source, but I don't see why it should be). To write a module, you can check ctypes (and there is a tutorial in the scipy's cookbook ;)) Note that you won't be able to easily build extensions with Visual C++ express edition (or any VS which is not VS 2003 for that matter), at least for the official binaries for python 2.5 (that will change for
Matthieu Brucher wrote: python 2.6, which is in alpha stage right now). mingw32 is directly supported by distutils. And since it is gcc, it should be more familiar if you are coming from mac os X. cheers, David
On Sat, 08 Mar 2008, David Cournapeau apparently wrote:
Note that you won't be able to easily build extensions with Visual C++ express edition (or any VS which is not VS 2003 for that matter), at least for the official binaries for python 2.5 (that will change for python 2.6, which is in alpha stage right now).
How will this change? (Aside from changing to VS 2008, I mean.) Can you point to a discussion? Thank you, Alan Isaac
Alan G Isaac wrote:
How will this change? (Aside from changing to VS 2008, I mean.) Can you point to a discussion?
That's the change: you will be able to use the compiler used for official binaries for free. For python 3k (or later), there may be a bigger change: since posix functions are hopelessly broken in VS runtime (which is at least one of the reasons why you cannot safely change compilers on windows), it is being rewritten with the win32 api, which is guaranteed to be ABI compatible across compilers by Microsoft, I guess. But this will take time, and is certainly no fun. http://mail.python.org/pipermail/python-3000/2008-February/012201.html cheers, David
On Fri, Mar 7, 2008 at 4:57 AM, Fernando Perez <fperez.net@gmail.com> wrote:
On Thu, Mar 6, 2008 at 3:18 PM, Alan McIntyre <alan.mcintyre@gmail.com> wrote:
A specific project along these lines, that would be very beneficial for numpy could be:
- Creating new matrix types in cython that match the cvxopt matrices. The creation of new numpy array types with efficient code would be very useful.
- Rewriting the existing ndarray subclasses that ship with numpy, such as record arrays, in cython. In doing this, benchmarks of the relative performance of the new code should be obtained.
What level of experience do you think would be necessary for the student for this? I've got a fair amount of Python & C experience, and I've used numpy and Pyrex (but not Cython) in the past. I wouldn't mind putting in some time to become familiar with the particulars before submitting a project proposal.
I don't want to put words in others' mouths, since I wouldn't be the one mentoring such a project. I suspect that you'd need to be reasonably familiar with some C programming, because at this point in the game this project might require debugging generated C code and perhaps diving into Cython itself. In the long term we'd like Cython to be so python-like and friendly that those who are *not* C experts can use it effectively for Numpy programming, but that isn't currently the case.
As far as Pyrex/cython, if you know pyrex, you'll be OK with cython. It's only better than pyrex, but is as far as I know mostly, if not fully compatible with pyrex.
I don't have a problem doing any of that, so I'll try to re-familiarize myself my numpy and Cython and see how hard it looks. However, if somebody is already planning on submitting a proposal for one or both of these projects, please let me know so we're not competing with each other needlessly. If anybody is interested in mentoring these, and can offer any advice (or suggestions for other things that need doing), I'd be glad to hear from you. Thanks, Alan
Hi Alan On Mon, Mar 24, 2008 at 8:50 PM, Alan McIntyre <alan.mcintyre@gmail.com> wrote:
On Fri, Mar 7, 2008 at 4:57 AM, Fernando Perez <fperez.net@gmail.com> wrote:
As far as Pyrex/cython, if you know pyrex, you'll be OK with cython. It's only better than pyrex, but is as far as I know mostly, if not fully compatible with pyrex.
I don't have a problem doing any of that, so I'll try to re-familiarize myself my numpy and Cython and see how hard it looks. However, if somebody is already planning on submitting a proposal for one or both of these projects, please let me know so we're not competing with each other needlessly. If anybody is interested in mentoring these, and can offer any advice (or suggestions for other things that need doing), I'd be glad to hear from you.
Cython integration with NumPy would be tremendously useful. I mentioned earlier the Sage GSOC project: http://wiki.cython.org/DagSverreSeljebotn/soc/details It would be good to talk to them (they hang out on #sage-devel on freenode.net) to hear what the current status is. As a start, we need - Multi-dimensional indexing print x[0,1] print x[:3,1] x[0,1] = 1 x[:,1] = [1,2,3] - Fancy indexing print x[[0,3,5]] x[[0,3,5]] = [12,15,18] - Broadcasting (maybe nothing needs to be done to get this working, I haven't investigated) x = array([1,2,3]) x = x + 3 The SAGE project addresses a fairly high-level abstraction, and, while that sounds like a good plan in the long run, a more numpy-specific solution would benefit us too, and *may* be easier to implement. Regards, Stéfan
On Mon, Mar 24, 2008 at 8:35 PM, Stéfan van der Walt <stefan@sun.ac.za> wrote:
Cython integration with NumPy would be tremendously useful. I mentioned earlier the Sage GSOC project:
http://wiki.cython.org/DagSverreSeljebotn/soc/details
It would be good to talk to them (they hang out on #sage-devel on freenode.net) to hear what the current status is. As a start, we need
Thanks, Stéfan; I missed your mention of that earlier; I'll have a look and pop into #sage-devel.
participants (8)
-
Alan G Isaac -
Alan McIntyre -
David Cournapeau -
Fernando Perez -
Gabriel Gellner -
Matthieu Brucher -
Nuttall, Brandon C -
Stéfan van der Walt