[Python-Dev] 2.5 and beyond

[Fernando Perez]

Thomas Heller wrote:

> I would like to ask about the possibility to add some improvements to
> ctypes
> in Python 2.5, although the feature freeze is now in effect.  Hopefully
> former third-party libraries can have the freeze relaxed somewhat;-).
> 
> I intend to do these changes, the first is a small and trivial one, but
> allows a lot of flexibility:

[...]

I'd just like to provide a bit of context for Thomas' request (disclaimer:
he did NOT ask me to write this, nor did anyone else).  I understand the
release managers' need to be strict with the freeze, but perhaps knowing
what's behind this particular change will help them make a more informed
decision.

Numpy (http://numpy.scipy.org/) is the new python array package for
numerical computing, which has been developed at enormous effort by Travis
Oliphant (with community help) over the last year, as a way to unify the
old Numeric package (written by Jim Hugunin, of Jython and IronPython fame)
and Numarray (written by the Hubble telescope team).  The effect of numpy
in the community, even in its current pre-1.0 form, has been tremendous. 
There is a real, pressing need in the scientific world for open source and
technically superior replacements to Matlab and IDL, the propietary 800-lb
gorillas of the field.  Many major research institutions across the world
are seriously looking at python as fulfilling this role, but the previous
situation of a divided library (Numeric/numarray) was keeping a lot of
people on the fence.

With Travis' effort and numpy maturing towards a 1.0 release right around
the time of python 2.5, a LOT of people have come out of the woodwork to
contribute code, ideas, documentation, etc.  There is a real sense that the
combination of python2.5 (with better 64-bit and __index__ support) and
numpy will provide a significant advancement for scientific computing with
modern, high-level tools.

In this particular community, the need to interface with low-level existing
libraries is probably much more common than in other fields.  There are
literally millions of lines of C/C++ code for scientific work which we have
to use efficiently, and this is an everyday need for us.  While there are a
number of tools for this (SWIG, Boost::Python, pyrex, scipy.weave,...),
very recently people have discovered how useful ctypes can be for this
task.  One of the Google SoC projects (http://2006.planet-soc.com/blog/140)
started trying to wrap libsvm with SWIG and a week of frustrated efforts
led nowhere.  Albert then discovered ctypes and in a few hours was up and
running.  This has generated a lot of interest in the numpy crowd for
ctypes, and people would really, really like to see python2.5 come 'out of
the box' with as solid a support as possible from ctypes for numpy array
interfacing.

Ultimately the decision is up to the release team, I know that.  But at
least with this info, I hope you can understand:

1. why this is important to this community

2. why the timing isn't ideal: it is only /very/ recently that the numpy
team 'discovered' how much ctypes could truly help with a necessary (and
often very unpleasant) task in the numerical/python world.


Thanks for reading,


f