Mailman 3 December 2005 - SciPy-Dev

newcore atlas info on Gentoo
by Darren Dale 20 Jan '06

20 Jan '06

In order to get scipy-0.3.2 to build on Gentoo Linux, I use site.cfg in the distutils module. This is necessary because Gentoo has renamed the libf77blas.* libraries to libblas.* in order to allow users to easily switch between different libraries, ATLAS and Intel's MKL for example. When I compile newcore using the old site.cfg, my atlas info is properly reported. However, when I compile newscipy, the atlas info is reportedly unavailable. I looked into system_info.atlas_info and friends, and discovered that _lib_names is not informed by site.cfg, it is hardcoded with ['f77blas', 'cblas'], or ['ptf77blas','ptcblas'], etc. If I remove the 'f77', my atlas info is correctly reported. Darren

3 10

Example of power of new data-type descriptors.
by Travis Oliphant 07 Jan '06

07 Jan '06

I'd like more people to know about the new power that is in scipy core due to the general data-type descriptors that can now be used to define numeric arrays. Towards that effort here is a simple example (be sure to use latest SVN -- there were a coupld of minor changes that improve usability made recently). Notice this example does not use a special "record" array subclass. This is just a regular array. I'm kind of intrigued (though not motivated to pursue) the possibility of accessing (or defining) databases directly into scipy_core arrays using the record functionality. # Define a new data-type descriptor >>> import scipy >>> dtype = scipy.dtypedescr({'names': ['name', 'age', 'weight'], 'formats': ['S30', 'i2', 'f4']}) >>> a = scipy.array([('Bill',31,260),('Fred',15,135)], dtype=dtype) # the argument to dtypedescr could have also been placed here as the argument to dtype >>> print a['name'] [Bill Fred] >>> print a['age'] [31 15] >>> print a['weight'] [ 260. 135.] >>> print a[0] ('Bill', 31, 260.0) >>> print a[1] ('Fred', 15, 135.0) It seems to me there are some very interesting possibilities with this new ability. The record array subclass adds an improved scalar type (the record) and attribute access to get at the fields: (e.g. a.name, a.age, and a.weight). But, if you don't need attribute access you can use regular arrays to do a lot of what you might need a record array to accomplish for you. I'd love to see what people come up with using this new facility. The new array PEP for Python basically proposes adding a very simple array object (just the basic PyArrayObject * of Numeric with a bare-bones type-object table) plus this new data-type descriptor object to Python and a very few builtin data-type descriptors (perhaps just object initially). This would basically add the array interface to Python directly and allow people to start using it generally. The PEP is slow going because it is not on my priority list right now because it is not essential to making scipy_core work well. But, I would love to have more people ruminating on the basic ideas which I think are crystallizing. Best wishes for a new year, -Travis Oliphant

4 4

chararry array method
by Christopher Hanley 04 Jan '06

04 Jan '06

Hi Travis, Should the array method in chararry have a shape parameter? Chris

3 6

Moving random.py
by Travis Oliphant 01 Jan '06

01 Jan '06

Hey Robert. I temporariliy moved random back under basic for now. I did this because it's really important to me that rand, randn, fft, and ifft are in the scipy namespace. I don't want you to think I'm unsupportive of the general changes you are trying to see, though. While in principle it would be nice to not do any import magic. There are three problems that I'd like to see solved that I think the import magic is trying to accomplish: 1) making some names visible in the scipy namespace (like fft, rand, etc.) 2) having some mechanism for scipy_core to use the tools in full scipy should it be installed (like svd, for example). 3) having some means to speed up importing. I think that there are two views of how scipy should behave that need to be resolved. The first view is where scipy is at right now in that when the user imports scipy the whole kit-and-kaboodle gets imported at the same time. This view was developed back when it was envisoned that scipy alone would be the MATLAB-like replacement. I think it is more clear now that IPython is better for interactive work and Matplotlib (and Enthought's coming tools) are better for plotting, that something else should provide the "environment for computing" besides the simple command import scipy. The second view is that scipy should just be a simple name-space package and not try to do any magic. Users will primarily use scipy for writing code and will import the needed functions from the right places. Most of what scipy is has been done to support interactive work, so that one does not have to do: import scipy import scipy.integrate import scipy.special in order to get access to the sub-packages. Perhaps this should be re-thought and something like this magic moved to IPython. -Travis

6 13

Sorting speed
by Francesc Altet 31 Dec '05

31 Dec '05

Hi, It seems that scipy_core (0.9.0.1713) is far more slower than numarray (1.5.0) when sorting arrays: In [43]: t5=timeit.Timer('a=sc.empty(shape=10000);a.sort()', 'import scipy.base as sc') In [44]: t5.repeat(3,100) Out[44]: [0.40603208541870117, 0.41605615615844727, 0.39800286293029785] In [45]: t4=timeit.Timer('a=na.array(None, shape=10000);a.sort()', 'import numarray as na') In [46]: t4.repeat(3,100) Out[46]: [0.090479135513305664, 0.086208105087280273, 0.086167097091674805] i.e. numarray is roughly 5x faster than scipy_core. As PyTables uses extensively this kind of sorting for indexing, it would be nice to see scipy_core with at least the speed of numarray in this sense. Nevertheless, as this only affects to the array package in the core, and there are no plans to migrate to scipy_core anytime soon, this is not a high priority for me. Just wanted to point it out. Thanks, -- >0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-"

5 14

Indexing array performance
by Francesc Altet 31 Dec '05

31 Dec '05

Hi, I'm doing some timings on array indexing feature. It's quite shocking to me that, provided empty() and arange() are faster in scipy_core than its counterparts in numarray: In [110]: t1 = timeit.Timer('a=empty(shape=10000);a=arange(10000)','from scipy.base import empty, arange') In [111]: t1.repeat(3,10000) Out[111]: [0.74018502235412598, 0.76141095161437988, 0.71947312355041504] In [112]: t2 = timeit.Timer('a=array(None,shape=10000);a=arange(10000)','from numarray import array, arange') In [113]: t2.repeat(3,10000) Out[113]: [2.3724348545074463, 2.4109888076782227, 2.3820669651031494] however, the next code seems to be slower in scipy_core: In [114]: t3 = timeit.Timer('a=empty(shape=10000);a[arange(10000)]','from scipy.base import empty, arange') In [115]: t3.repeat(3,1000) Out[115]: [3.5126161575317383, 3.5309510231018066, 3.5558919906616211] In [116]: t4 = timeit.Timer('a=array(None,shape=10000);a[arange(10000)]','from numarray import array, arange') In [117]: t4.repeat(3,1000) Out[117]: [2.0824751853942871, 2.1258058547973633, 2.0946059226989746] It seems like if the index array feature can be further optimized in scipy_core. -- >0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-"

2 3

Changes allow string -> number casting
by Travis Oliphant 30 Dec '05

30 Dec '05

I just checked in to SVN, the changes needed to allow strings (and unicode) to be converted to numbers: >>> print array(['3.0','4','5','6']).astype(float) array([ 3., 4., 5., 6.]) It is not the fastest-possible approach (it goes through the Python int, long, float, and complex __new__ methods), but it works. -Travis

1 0

New release of scipy_core
by Travis Oliphant 30 Dec '05

30 Dec '05

I'd like to place a target date of Jan 4 for the new release of scipy_core. With a release of scipy occurring by Jan 6. Please have all changes to scipy_core and scipy in-place by then. We need to get a re-factored scipy_core and scipy out quickly. I think the packaging is finally stabilized and free of too much fanciness. -Travis

1 0

Re: [SciPy-dev] [SciPy-user] docstrings; corefft, corelinalg
by Travis Oliphant 30 Dec '05

30 Dec '05

Steve Schmerler wrote: > Available subpackages > --------------------- > > > SciPy: A scientific computing package for Python > ================================================ > > Available subpackages > --------------------- > > >In [20]: >##################################################################################################################################### > >... no infos about scipy subpackages (of course `help(scipy)` has some >infos) > > These subpackage docs were at one time auto-generated. I'm not sure what the status of that is right now. It could be that the info.py file is wrong in the sub-packages. > >2) I think the doc strings of corefft and corelinalg should contain some > words explaining their existence. New scipy users would wonder > > > Good idea. >Is it possible to move all additional functionality of corefft and >corelinalg to linalg and fftpack (e.g corelinalg.eigh which seems to >have no equivalent function in linalg). > > Should be possible. The linalg approach uses f2py blas wrappers. I'm sure there is already a wrapper around the underlying function that handles eigh. Lots of little things to do so dive on in :-) Thanks for the feedback and suggestions. -Travis

2 1

Some tests for function_base.py
by Hugo Gamboa 30 Dec '05

30 Dec '05

Hello, Since I was trying to fix the unwrap function I decided to add some more tests to the function_base.py In the process I've also suggest some corrections to the code: unwrap: was failing by using the old typecode histogram: one variable was cleaning the input data histogram: the bin generation was not adequate creating unbalanced bin counts. I've cross checked with matplotlib code (both are very similar) and corrected the histogram functions. Hope this will be usefull. Hugo Gamboa

2 1