Hy everybody,
I'm wondering what is the (best) way to apply the same function to multiple
arrays.
For example, in the following code:
from numpy import *
def f(arr):
return arr*2
a = array( [1,1,1] )
b = array( [2,2,2] )
c = array( [3,3,3] )
d = array( [4,4,4] )
a = f(a)
b = f(b)
c = f(c)
d = f(d)
I would like to replace :
a = f(a)
b = f(b)
c = f(c)
d = f(d)
with something like that, but which really modify a,b,c and d:
for x in [a,b,c,d]:
x = f(x)
So having something like a pointer on the arrays.
Thanks for your help !
David
--
Hi,
This is with numpy 1.6.1 under Linux x86_64, testing the upcast mechanism of
"scalar + array":
>>> import numpy; print (numpy.array(3, dtype=numpy.complex128) +
numpy.ones(3, dtype=numpy.float32)).dtype
complex64
Since it has to upcast my array (float32 is not "compatible enough" with
complex128), why does it upcast it to complex64 instead of complex128?
As far as I can tell 1.4.x and 1.5.x versions of numpy are indeed upcasting
to complex128.
Thanks,
-=- Olivier
I was working on adding some test cases in numpy for the argmin/max
functions with some datetime64s. I found that on my 32-bit machine, it
fails to parse a date past the Y2.038k date. I find this odd because the
datetime is supposed to be 64-bits, but I guess there is some arch-dependent
code somewhere?
Ben Root
I keep getting this error: #warning "Using deprecated NumPy
API, disable it by #defining NPY_NO_DEPRECATED_API" and a bunch of
undefined variables when comping cython code with numpy
I am new to cython and numpy and I am having trouble getting a *.pyx extension
to compile with this in the header:
from __future__ import division
import numpy as np
cimport numpy as np
DTYPE = np.int
ctypedef np.int_t DTYPE_t
I complie to C code using:
cython -a montcarl.pyx
And it seems to work fine.
I then attempt to compile using:
gcc -I /Developer/SDKs/MacOSX10.6.sdk/System/Library/Frameworks/
Python.framework/Versions/2.6/include/python2.6 -I /usr/local/lib/
python2.7/site-packages/numpy/core/include -shared -pthread -fPIC -
fwrapv -O2 -fno-strict-aliasing
-o montcar.so montcarl.c
And I get a bunch of undefined variables:
spoust-m:old spoust$ gcc $CFLAGS -o montcar.so montcarl.c
In file included from /usr/local/lib/python2.7/site-packages/numpy/
core/include/numpy/ndarraytypes.h:1965,
from /usr/local/lib/python2.7/site-packages/numpy/
core/include/numpy/ndarrayobject.h:17,
from /usr/local/lib/python2.7/site-packages/numpy/
core/include/numpy/arrayobject.h:14,
from montcarl.c:225:
/usr/local/lib/python2.7/site-packages/numpy/core/include/numpy/
npy_deprecated_api.h:11:2: warning: #warning "Using deprecated NumPy
API, disable it by #defining NPY_NO_DEPRECATED_API"
Undefined symbols:
"_PyLong_AsLong", referenced from:
___Pyx_PyInt_AsLong in ccvkIDLQ.o
___Pyx_PyInt_AsLong in ccvkIDLQ.o
...
Any ideas on how to get this to compile and how to get rid of these
numpy undefined variables? The cython guys directed me here.
Towards a reasonable behavior with regard to local times, I've made the
default repr for datetimes use the C standard library to print them in a
local ISO format. Combined with the ISO8601-prescribed behavior of
interpreting datetime strings with no timezone specifier to be in local
times, this allows the following cases to behave reasonably:
>>> np.datetime64('now')
numpy.datetime64('2011-06-15T15:16:51-0500','s')
>>> np.datetime64('2011-06-15T18:00')
numpy.datetime64('2011-06-15T18:00-0500','m')
As noted in another thread, there can be some extremely surprising behavior
as a consequence:
>>> np.array(['now', '2011-06-15'], dtype='M')
array(['2011-06-15T15:18:26-0500', '2011-06-14T19:00:00-0500'],
dtype='datetime64[s]')
Having the 15th of June print out as 7pm on the 14th of June is probably not
what one would generally expect, so I've come up with an approach which
hopefully deals with this in a good way.
One firm principal of the datetime in NumPy is that it is always stored as a
POSIX time (referencing UTC), or a TAI time. There are two categories of
units that can be used, which I will call *date units* and *time units*. The
date units are 'Y', 'M', 'W', and 'D', while the time units are 'h', 'm',
's', ..., 'as'. Time zones are only applied to datetimes stored in time
units, so there's a qualitative difference between date and time units with
respect to string conversions and calendar operations.
I would like to place an 'unsafe' casting barrier between the date units and
the time units, so that the above conversion from a date into a datetime
will raise an error instead of producing a confusing result. This only
applies to datetimes and not timedeltas, because for timedeltas the day <->
hour case is fine, it is just the year/month <-> other units which has
issues, and that is already treated with an 'unsafe' casting barrier.
Two new functions will facilitate the conversions between datetimes with
date units and time units:
date_as_datetime(datearray, hour, minute, second, microsecond,
timezone='local', unit=None, out=None), which converts the provided dates
into datetimes at the specified time, according to the specified timezone.
If 'unit' is specified, it controls the output unit, otherwise it is the
units in 'out' or the amount of precision specified in the function.
datetime_as_date(datetimearray, timezone='local', out=None), which converts
the provided datetimes into dates according to the specified timezone.
In both functions, timezone can be any of 'UTC', 'TAI', 'local', '+/-####',
or a datetime.tzinfo object. The latter will allow NumPy datetimes to work
with the pytz library for flexible time zone support.
I would also like to extend the 'today' input string parsing to accept
strings like 'today 12:30' to allow a convenient way to express different
local times occurring today, mostly useful for interactive usage.
I welcome any comments on this design, particularly if you can find a case
where this doesn't produce a reasonable behavior.
Cheers,
Mark
Hello all,
I just opened a pull request to add scipy.signal._peak_finding.find_peaks,
which finds the relative maxima in a 1d ndarray. The algorithm behind this
function has been discussed on the scipy list, but here's a quick recap: The
idea is basically convolve the function with a wavelet of varying widths
(default is ricker, or "mexican hat" wavelet) and identify as peaks those
locations where there are relative maxima (meaning they are larger than
their nearest neighbor on each side) at each of several widths, and have a
sufficiently high SNR. More details are provided in the docstrings of the
functions. Pull request at https://github.com/scipy/scipy/pull/85.
I believe the code quality/docstrings/test coverage are pretty good, however
it would be of great benefit for someone with experience with similar
algorithms to review the code.
-Jacob
PS I've sent this to both the scipy and numpy lists, apologies to those who
get it twice.
Hi all,
Myself and several colleagues have recently started work on a Python library
for solar physics <http://www.sunpy.org/>, in order to provide an
alternative to the current mainstay for solar
physics<http://www.lmsal.com/solarsoft/>,
which is written in IDL.
One of the first steps we have taken is to create a Python
port<https://github.com/sunpy/sunpy/blob/master/benchmarks/time_test3.py>of
a popular benchmark for IDL (time_test3) which measures performance
for a
variety of (primarily matrix) operations. In our initial attempt, however,
Python performs significantly poorer than IDL for several of the tests. I
have attached a graph which shows the results for one machine: the x-axis is
the test # being compared, and the y-axis is the time it took to complete
the test, in milliseconds. While it is possible that this is simply due to
limitations in Python/Numpy, I suspect that this is due at least in part to
our lack in familiarity with NumPy and SciPy.
So my question is, does anyone see any places where we are doing things very
inefficiently in Python?
In order to try and ensure a fair comparison between IDL and Python there
are some things (e.g. the style of timing and output) which we have
deliberately chosen to do a certain way. In other cases, however, it is
likely that we just didn't know a better method.
Any feedback or suggestions people have would be greatly appreciated.
Unfortunately, due to the proprietary nature of IDL, we cannot share the
original version of time_test3, but hopefully the comments in time_test3.py
will be clear enough.
Thanks!
Keith
Hi,
I am trying out the latest version of numpy 2.0 dev:
np.__version__
Out[44]: '2.0.0.dev-aded70c'
I am trying to import CSV data that looks like this:
date,system,pumping,rgt,agt,sps,eskom_import,temperature,wind,pressure,weather
2007-01-01 00:30,481.9,,,,,481.9,15,SW,1040,Fine
2007-01-01 01:00,471.9,,,,,471.9,15,SW,1040,Fine
2007-01-01 01:30,455.9,,,,,455.9,,,,
etc.
by using the following code:
convertdict = {0: lambda s: np.datetime64(s, 'm'), 1: lambda s: float(s or 0),
2: lambda s: float(s or 0), 3: lambda s: float(s or 0), 4: lambda s: float(s or
0), 5: lambda s: float(s or 0), 6: lambda s: float(s or 0), 7: lambda s: float(s
or 0), 8: str, 9: str, 10: str}
dt = [('date', np.datetime64),('system', float), ('pumping', float),('rgt',
float), ('agt', float), ('sps', float) ,('eskom_import', float), ('temperature',
float), ('wind', str), ('pressure', float), ('weather', str)]
a = np.recfromcsv(fp, dtype=dt, converters=convertdict, usecols=range(0-11),
names=True)
The dtype it generates for a.date is 'object':
array([2007-01-01T00:30+0200, 2007-01-01T01:00+0200, 2007-01-01T01:30+0200,
..., 2007-12-31T23:00+0200, 2007-12-31T23:30+0200,
2008-01-01T00:00+0200], dtype=object)
But I need it to be datetime64, like in this example (but including hrs and
minutes):
array(['2011-07-11', '2011-07-12', '2011-07-13', '2011-07-14',
'2011-07-15', '2011-07-16', '2011-07-17'], dtype='datetime64[D]')
It seems that the CSV import creates an embedded object datetype for 'date'
rather than a datetime64 data type. Any ideas on how to fix this?
Grové
Hi all,
Matthew Brett showed me an interesting code snippet this evening:
# Construct input data
In [15]: x
Out[15]:
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11]])
# Fancy indexing with 1D boolean array
In [16]: x[np.array([True, False, True])]
Out[16]:
array([[0, 1, 2],
[6, 7, 8]])
# Fancy indexing with 2D boolean array
In [17]: x[np.array([[True, False, True]])]
Out[18]: array([0, 2])
I guess it's been a long day, but why does this work at all?
I expected the first example to break, because the 1D mask does not
match the number of rows in x. In the second example, I expected an
error because the 2D mask was not of the same shape as x. But, oddly,
both work. There's also no attempt at broadcasting indexes.
Any hints would be appreciated!
Thanks
Stéfan
Hello,
as said in the subject, the following code produces an error. Is it normal ?
**********************************************************
A = r_[1]
file_buffer = StringIO()
A.tofile(file_buffer)
IOError: first argument must be a string or open file
**********************************************************
