I was experimenting with np.min_scalar_type to make sure it worked as
expected, and found some unexpected results for integers between 2**63
and 2**64-1. I would have expected np.min_scalar_type(2**64-1) to
return uint64. Instead, I get object. Further experimenting showed
that the largest integer for which np.min_scalar_type will return uint64
is 2**63-1. Is this expected behavior?
On python 2.7.2 on a 64-bit linux machine:
>>> import numpy as np
>>> np.version.full_version
'2.0.0.dev-55472ca'
>>> np.min_scalar_type(2**8-1)
dtype('uint8')
>>> np.min_scalar_type(2**16-1)
dtype('uint16')
>>> np.min_scalar_type(2**32-1)
dtype('uint32')
>>> np.min_scalar_type(2**64-1)
dtype('O')
>>> np.min_scalar_type(2**63-1)
dtype('uint64')
>>> np.min_scalar_type(2**63)
dtype('O')
I get the same results on a Windows XP machine running python 2.7.2 and
numpy 1.6.1.
Kathy

Dear all,
I was just looking into numpy.einsum and encountered an issue which might
be worth pointing out in the documentation.
Let us say you wish to evaluate something like this (repeated indices a
summed)
D[alpha, alphaprime] = A[alpha, beta, sigma] * B[alphaprime, betaprime,
sigma] * C[beta, betaprime]
with einsum as
einsum('abs,cds,bd->ac', A, B, C)
then it is not exactly clear which order einsum evaluates the contractions
(or if it does it in one go). This can be very important since you can do
it in several ways, one of which has the least computational complexity.
The most efficient way of doing it is to contract e.g. A and C and then
contract that with B (or exchange A and B). A quick test on my labtop says
2.6 s with einsum and 0.13 s for two tensordots with A and B being D x D x
2 and C is D x D for D = 96. This scaling seems to explode for higher
dimensions, whereas it is much better with the two independent contractions
(i believe it should be O(D^3)).For D = 512 I could do it in 5 s with two
contractions, whereas I stopped waiting after 60 s for einsum (i guess
einsum probably is O(D^4) in this case).
I had in fact thought of making a function similar to einsum for a while,
but after I noticed it dropped it. I think, however, that there might still
be room for a tool for evaluating more complicated expressions efficiently.
I think the best way would be for the user to enter an expression like the
one above which is then evaluated in the optimal order. I know how to do
this (theoretically) if all the repeated indices only occur twice (like the
expression above), but for the more general expression supported by einsum
I om not sure how to do it (haven't thought about it). Here I am thinking
about stuff like x[i] = a[i] * b[i] and their more general counterparts (at
first glance this seems to be a simpler problem than full contractions). Do
you think there is a need/interest for this kind of thing? In that case I
would like the write it / help write it. Much of it, I think, can be
reduced to decomposing the expression into existing numpy operations (e.g.
tensordot). How to incorporate issues of storage layout etc, however, I
have no idea.
In any case I think it might be nice to write explicitly how the expression
in einsum is evaluated in the docs.
Søren Gammelmark
PhD-student
Department of Physics and Astronomy
Aarhus University

I know I know, that's pretty outrageous to even suggest, but please
bear with me, I am stumped as you may be:
2-D data file here:
http://dl.dropbox.com/u/139035/data.npy
Then:
In [3]: data.mean()
Out[3]: 3067.0243839999998
In [4]: data.max()
Out[4]: 3052.4343
In [5]: data.shape
Out[5]: (1000, 1000)
In [6]: data.min()
Out[6]: 3040.498
In [7]: data.dtype
Out[7]: dtype('float32')
A mean value calculated per loop over the data gives me 3045.747251076416
I first thought I still misunderstand how data.mean() works, per axis
and so on, but did the same with a flattenend version with the same
results.
Am I really soo tired that I can't see what I am doing wrong here?
For completion, the data was read by a osgeo.gdal dataset method called
ReadAsArray()
My numpy.__version__ gives me 1.6.1 and my whole setup is based on
Enthought's EPD.
Best regards,
Michael

hi everyone, I run python 2.7.2. in Eclipse (recently upgraded from 2.6). I
have a problem with installing matplotlib (I found the version for python
2.7. MacOs 10.3, no later versions). If I run python in terminal using arch
-i386 python, and then
from matplotlib.pylab import *
and similar stuff, everything works fine. If I run python in eclipse or
just without arch -i386, I can import matplotlib as
from matplotlib import *
but actually nothing gets imported. If I do it in the same way as above, I
get the message
no matching architecture in universal wrapper
which means there's conflict of versions or something like that. I tried
reinstalling the interpreter and adding matplotlib to forced built-ins, but
nothing helped. For some reason I didn't have this problem with numpy and
tkinter.
Any suggestions are appreciated.

After having saved data, I need to know/remember the data dtype to
restore it correctly.
Is there a way to save the dtype with the data?
(I guess the header parameter of savedata could help, but they are
only available in v2.0+ )
I would like to save several related structured array and a dictionary
of parameters into a TEXT file.
Is there an easy way to do that?
(maybe xml file, or maybe archive zip file of other files, or ..... )
Any recommendation is helpful.
Regards,
--
Emmanuel

Course "Python for Scientists and Engineers" in Chicago
=======================================================
There will be a comprehensive Python course for scientists and engineers
in Chicago end of February / beginning of March 2012. It consists of a 3-day
intro and a 2-day advanced section. Both sections can be taken separately
or combined.
More details below and here: http://www.dabeaz.com/chicago/science.html
Please let friends or colleagues who might be interested in such a
course know about it.
3-Day Intro Section
-------------------
- Overview of Scientific and Technical Libraries for Python.
- Numerical Calculations with NumPy
- Storage and Processing of Large Amounts of Data
- Graphical Presentation of Scientific Data with matplotlib
- Object Oriented Programming for Scientific and Technical Projects
- Open Time for Problem Solving
2-Day Advanced Section
----------------------
- Extending Python with Other Languages
- Unit Testing
- Version Control with Mercurial
The Details
-----------
The course is hosted by David Beazley (http://www.dabeaz.com).
Date: Feb 27 - Mar 2, 2012
Location: Chicago, IL, USA
Trainer: Mike Müller
Course Language: English
Link: http://www.dabeaz.com/chicago/science.html

With N.seterr(all='raise'):
>>> from scipy import special
>>> import scipy
>>> special.erf(26.6)
1.0
>>> scipy.__version__
'0.11.0.dev-81dc505'
>>> import numpy as N
>>> N.seterr(all='raise')
{'over': 'warn', 'divide': 'warn', 'invalid': 'warn', 'under': 'ignore'}
>>> special.erf(26.5)
1.0
>>> special.erf(26.6)
Traceback (most recent call last):
File "<pyshell#7>", line 1, in <module>
special.erf(26.6)
FloatingPointError: underflow encountered in erf
>>> special.erf(26.7)
1.0
What is so special in 26.6?
I have this error also with previous versions of scipy
Nadav.

Dear List,
I frequently work with MATLAB and it is necessary for me many a times
to adapt MATLAB codes for NumPy arrays.
While for most practical purposes it works fine, I think there might
be a lot of 'under the hood' things that I might be missing when I
make the translations from MATLAB to Python.
Are there any 'best practices' for working on this transition?
Thanks

Is there a way to save a structured array in a text file?
My problem is not so much in the saving procedure, but rather in the
'reloading' procedure.
See below
In [3]: import numpy as np
In [4]: r = np.ones(3,dtype=[('name', '|S5'), ('foo', '<i8'), ('bar', '<f8')])
In [5]: r.tofile('toto.txt',sep='\n')
bash-4.2$ cat toto.txt
('1', 1, 1.0)
('1', 1, 1.0)
('1', 1, 1.0)
In [7]: r2 = np.fromfile('toto.txt',sep='\n',dtype=r.dtype)
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
/home/cls1fs/clseng/10/<ipython-input-7-b07ba265ede7> in <module>()
----> 1 r2 = np.fromfile('toto.txt',sep='\n',dtype=r.dtype)
ValueError: Unable to read character files of that array type
--
Emmanuel

Sorry,
that i use this way to send an answer to Tony Yu , Nadav Horesh , Chris Barker.
When iam direct answering on Your e-mail i get an error 5.
I think i did a mistake.
Your ideas are very helpfull and the code is very fast.
Thank You
elodw