Mailman 3 May 2010 - NumPy-Discussion

Poll: Semantics for % in Cython
by Dag Sverre Seljebotn May 4, 2010

May 4, 2010

(First off, is it OK to continue polling the NumPy list now and then on Cython language decisions? Or should I expect that any interested Cython users follow the Cython list?) In Python, if I write "-1 % 5", I get 4. However, in C if I write "-1 % 5" I get -1. The question is, what should I get in Cython if I write (a % b) where a and b are cdef ints? Should I [ ] Get 4, because it should behave just like in Python, avoiding surprises when adding types to existing algorithms (this will require extra logic and be a bit slower) [ ] Get -1, because they're C ints, and besides one isn't using Cython if one doesn't care about performance Whatever we do, this also affects the division operator, so that one in any case will have a==(a//b)*b+a%b. (Orthogonal to this, we can introduce compiler directives to change the meaning of the operator from the default in a code blocks, and/or make special functions for the semantics that are not chosen as default.) -- Dag Sverre

10 16

incremental histogram
by Neal Becker May 4, 2010

May 4, 2010

I have coded in c++ a histogram object that can be used as: h += my_sample or h += my_vector This is very useful in simulations which are looping and developing results incrementally. It would me great to have such a feature in numpy.

2 3

datetime failure on py3k, string type issue.
by Charles R Harris May 4, 2010

May 4, 2010

The following fails after fixing datetime_data assert_equal(datetime_data(a.dtype), ('us', 1, 1, 1)) The problem is that 'us' is unicode and the function call yields bytes. The question is: should datetime units use unicode when compiled on python >= 3k? Chuck

1 0

ndimage.label - howto force SWIG to use int32 - even on 64bit Linux ?
by Sebastian Haase May 4, 2010

May 4, 2010

Hi, I wanted to write some C code to accept labels as they come from ndimage.label. For some reason ndimage.label produces its output as an int32 array - even on my 64bit system . BTW, could this be considered a bug ? Now, if I use the typemaps of numpy.i I can choose between NPY_LONG and NPY_INT. But those are sometimes 32 sometimes 64 bit, depending on the system. Any ideas ... ? Thanks, Sebastian Haase

6 10

&*(*&$%# Macros
by Charles R Harris May 3, 2010

May 3, 2010

Hi Travis, Could you remove this macro? It isn't py3k compliant because of the PyCobject and we have too many public macros already. Please make it part of the interface if it needs to be exposed, otherwise make it an inline function somewhere. #define PyDataType_GetDatetimeMetaData(descr) \ ((descr->metadata == NULL) ? NULL : \ ((PyArray_DatetimeMetaData *)(PyCObject_AsVoidPtr( \ PyDict_GetItemString(descr->metadata, NPY_METADATA_DTSTR))))) Chuck

2 2

Broken numpy tests in python3k
by Charles R Harris May 3, 2010

May 3, 2010

They seem to be string/unicode related. 1) This error remains after replacing the disappeared built in function "file" with "open" ====================================================================== ERROR: test_universal_newline (test_io.TestLoadTxt) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.1/site-packages/numpy/lib/tests/test_io.py", line 412, in test_universal_newline data = np.loadtxt(name) File "/usr/local/lib/python3.1/site-packages/numpy/lib/npyio.py", line 632, in loadtxt first_vals = split_line(first_line) File "/usr/local/lib/python3.1/site-packages/numpy/lib/npyio.py", line 610, in split_line line = line.split(comments)[0].strip() TypeError: Can't convert 'bytes' object to str implicitly 2) Not sure about this one, it doesn't show up for Python 2k ====================================================================== ERROR: test_type_check.TestDateTimeData.test_basic ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python3.1/site-packages/nose/case.py", line 177, in runTest self.test(*self.arg) File "/usr/local/lib/python3.1/site-packages/numpy/lib/tests/test_type_check.py", line 382, in test_basic assert_equal(datetime_data(a.dtype), ('us', 1, 1, 1)) File "/usr/local/lib/python3.1/site-packages/numpy/lib/type_check.py", line 641, in datetime_data result = func(obj) TypeError: this function takes at least 2 arguments (1 given) Chuck

1 0

Question about numpy.arange()
by Gökhan Sever May 3, 2010

May 3, 2010

Hello, Is "b" an expected value? I am suspecting another floating point arithmetic issue. I[1]: a = np.arange(1.6, 1.8, 0.1, dtype='float32') I[2]: a O[2]: array([ 1.60000002, 1.70000005], dtype=float32) I[3]: b = np.arange(1.7, 1.8, 0.1, dtype='float32') I[4]: b O[4]: array([ 1.70000005, 1.79999995], dtype=float32) A bit conflicting with the np.arange docstring: "* Values are generated within the half-open interval ``[start, stop)`` (in other words, the interval including `start` but excluding `stop`). * " Thanks. -- Gökhan

6 6

numpy + MKL and AMD64 CPU, recognized as 32 bit
by Andrzej Giniewicz May 2, 2010

May 2, 2010

Hi, from what I've been reading, and testing myself, 64bit CPU's from AMD and actually it seems like almost all 64bit other than IA64, use em64t in MKL - but in Numpy (at least 1.4.1), this architecture is picked only for Xeon - though older Xeons were not 64 bit iirc - I'm talking about numpy/distutils/system_info.py file, mkl_info class, __init__ function. The fragment: from cpuinfo import cpu l = 'mkl' # use shared library if cpu.is_Itanium(): plt = '64' #l = 'mkl_ipf' elif cpu.is_Xeon(): plt = 'em64t' #l = 'mkl_em64t' else: plt = '32' #l = 'mkl_ia32' with this, my AMD64 that works with em64t (that's version I have installed and that passes all tests, that is MKL 10.2.5.035) is recognized as 32 bit. Wouldn't it be better to check for is_64bit instead of is_Xeon? Thanks in advance for hints, Andrzej.

1 0

bug, issue with genfromtxt
by Vincent Davis May 2, 2010

May 2, 2010

I ran into this issue and it was discussed on the pystatsmodels mailing list. Here is the setup Running on a Mac 10.6 Using Office 2008 Saving an spreadsheet using excel "save as" a csv file. Try to import using genfromtxt fails, report a EOL error I thought this was because the EOL was wrong, It seems the file has '\r' as the line ending (this may be wrong) anyway I changed it to '\n' and it works fine. I am told (on the pystatsmodels mailing list) that this is actually because the file is in unicode and that genfromtxt does not read the EOL correctly. To me it is a bug because one might expect a user to what to save a file from excel and read it using genfromtxt. And for useres with little experience the problem is not obvious. I guess this is not a problem with py3? ORIGINAL ATTEMPT datatype = [('date','|S9'),('gpd','i8'),('temp','i8' ('precip','f16')] data = np.genfromtxt('waterdata.csv', delimiter=',', skip_header=1, dtype=datatype) Traceback (most recent call last): File "/Applications/WingIDE.app/Contents/MacOS/src/debug/tserver/_sandbox.py", line 1, in <module> # Used internally for debug sandbox under external interpreter File "/Library/Frameworks/EPD64.framework/Versions/6.1/lib/ python2.6/site-packages/numpy/lib/io.py", line 1048, in genfromtxt raise IOError('End-of-file reached before encountering data.') IOError: End-of-file reached before encountering data. THIS DOES NOT WORK >>> s = file('data_with_CR.csv','r') >>> data = np.genfromtxt(s, delimiter=",", skip_header=1, dtype=None) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Library/Frameworks/EPD64.framework/Versions/6.1/lib/python2.6/site-packages/numpy/lib/io.py", line 1048, in genfromtxt raise IOError('End-of-file reached before encountering data.') IOError: End-of-file reached before encountering data. >>> data = np.genfromtxt(s, delimiter=",", , dtype=None) File "<stdin>", line 1 data = np.genfromtxt(s, delimiter=",", , dtype=None) THIS DOES WORK >>> s = file('data_with_CR.csv','U') >>> data = np.genfromtxt(s, delimiter=",", skip_header=1, dtype=None) >>> data array([('1/1/00', 8021472, 52, 0.02), ('1/2/00', 9496016, 46, 0.059999999999999998), ('1/3/00', 8478792, 29, 0.0), ..., ('12/29/02', 10790000, 61, 0.0), ('12/30/02', 9501000, 44, 0.0), ('12/31/02', 9288000, 53, 0.0)], dtype=[('f0', '|S8'), ('f1', '<i8'), ('f2', '<i8'), ('f3', '<f8')]) *Vincent Davis 720-301-3003 * vincent(a)vincentdavis.net my blog <http://vincentdavis.net> | LinkedIn<http://www.linkedin.com/in/vincentdavis>

1 0