NumPy-Discussion October 2012

numpy-discussion@python.org

74 participants
62 discussions

fromfile with sys.stdin?
by Michael Aye 19 Oct '12

19 Oct '12

Is it possible to use sys.stdin as input for numpy.fromfile? I can't make it work. This simple example: ---- import sys import numpy as np # I know the stream format, so I skip parsing the header keys = ['lot','lon','tb','qual'] dt = [(key,'f8') for key in keys] # skip the header, it has 'end' in the last line while 'end' not in sys.stdin.readline(): pass # pass the stdin filepointer to fromfile arr = np.fromfile(sys.stdin, dtype = dt) print arr ---- produces this error: ---- (develop+)[maye@luna1 ~/src/diviner]$ pread des=~/data/data.des < ~/data/mydata.txt | python test_fromfile.py Traceback (most recent call last): File "test_fromfile.py", line 10, in <module> arr = np.fromfile(sys.stdin, dtype = dt) IOError: could not seek in file ---- I also tried to catch the binary output stream into a binary file and read that with fromfile without any problem with the exact same method like here, skipping over the header and so on, so it should not be any problem with the rest of the code. Any hints welcome!

1 0

numpy.arrange not returning expected results (bug?)
by Simon Lieschke 18 Oct '12

18 Oct '12

Hi, I've discovered calling numpy.arange(1.1, 17.1) and numpy(1.1, 16.1) both return the same results. Could this be a numpy bug, or is there some behaviour I'm possibly not aware of here? I've pasted in the results of an interactive Python session comparing and contrasting these with some other similar calls. I'm using NumPy 1.6.2. Thanks in advance, Simon 14:21:46.77 @ C:\Users\simon >python Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy as np >>> np.arange(1.1, 17.1) array([ 1.1, 2.1, 3.1, 4.1, 5.1, 6.1, 7.1, 8.1, 9.1, 10.1, 11.1, 12.1, 13.1, 14.1, 15.1, 16.1]) >>> np.arange(1.1, 16.1) array([ 1.1, 2.1, 3.1, 4.1, 5.1, 6.1, 7.1, 8.1, 9.1, 10.1, 11.1, 12.1, 13.1, 14.1, 15.1, 16.1]) >>> np.arange(1.1, 15.1) array([ 1.1, 2.1, 3.1, 4.1, 5.1, 6.1, 7.1, 8.1, 9.1, 10.1, 11.1, 12.1, 13.1, 14.1]) >>> np.arange(1, 17) array([ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]) >>> np.arange(1, 16) array([ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]) >>> np.arange(1, 15) array([ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14])

2 1

how is y += x computed when y.strides = (0, 8) and x.strides=(16, 8) ?
by Sebastian Walter 17 Oct '12

17 Oct '12

Hi, I'm using numpy 1.6.1 on Ubuntu 12.04.1 LTS. A code that used to work with an older version of numpy now fails with an error. Were there any changes in the way inplace operations like +=, *=, etc. work on arrays with non-standard strides? For the script: ------- start of code ------- import numpy x = numpy.arange(6).reshape((3,2)) y = numpy.arange(2) print 'x=\n', x print 'y=\n', y u,v = numpy.broadcast_arrays(x, y) print 'u=\n', u print 'v=\n', v print 'v.strides=\n', v.strides v += u print 'v=\n', v # expectation: v = [[6,12], [6,12], [6,12]] print 'u=\n', u print 'y=\n', y # expectation: y = [6,12] ------- end of code ------- I get the output -------- start of output --------- x= [[0 1] [2 3] [4 5]] y= [0 1] u= [[0 1] [2 3] [4 5]] v= [[0 1] [0 1] [0 1]] v.strides= (0, 8) v= [[4 6] [4 6] [4 6]] u= [[0 1] [2 3] [4 5]] y= [4 6] -------- end of output -------- I would have expected that v += u performs an element-by-element += v[0,0] += u[0,0] # increments y[0] v[0,1] += u[0,1] # increments y[1] v[1,0] += u[1,0] # increments y[0] v[1,1] += u[1,1] # increments y[1] v[2,0] += u[2,0] # increments y[0] v[2,1] += u[2,1] # increments y[1] yielding the result y = [6,12] but instead one obtains y = [4, 6] which could be the result of v[2,0] += u[2,0] # increments y[0] v[2,1] += u[2,1] # increments y[1] Is this the intended behavior? regards, Sebastian

5 5

A change with minor compatibility questions
by Travis Oliphant 17 Oct '12

17 Oct '12

Hey all, https://github.com/numpy/numpy/pull/482 is a pull request that changes the hash function for numpy void scalars. These are the objects returned from fully indexing a structured array: array[i] if array is a 1-d structured array. Currently their hash function just hashes the pointer to the underlying data. This means that void scalars can be used as keys in a dictionary but the behavior is non-intuitive because another void scalar with the same data but pointing to a different region of memory will hash differently. The pull request makes it so that two void scalars with the same data will hash to the same value (using the same algorithm as a tuple hash). This pull request also only allows read-only scalars to be hashed. There is a small chance this will break someone's code if they relied on this behavior. I don't believe anyone is currently relying on this behavior -- but I've been proven wrong before. What do people on this list think? Should we raise a warning in the next release when a hash function on a void scalar is called or just make the change, put it in the release notes and make a few people change their code if needed. The problem was identified by a couple of users of NumPy currently which is why I think that people who have tried using numpy void scalars as keys aren't doing it right now but are instead converting them to tuples first. -Travis

4 4

distutils & C++ & Fortran
by Pauli Virtanen 16 Oct '12

16 Oct '12

Hi, I'd like to link both C++ and Fortran code into a single Python extension, using numpy.distutils (for scipy.special). But it seems the distutils-based tools actually cannot do this? Does someone know if there's a way to work around this? -- Pauli Virtanen

3 3

array flags not copied/deepcopied
by Byron Blay 16 Oct '12

16 Oct '12

Copying / deepcopying an array does not copy the writeable flag: >>> import numpy >>> from copy import deepcopy >>> a = numpy.array([1,2,3,4,5]) >>> a.flags.writeable = False >>> b = deepcopy(a) >>> b.flags.writeable True Is this a bug? Should I raise an issue on github? Many thanks, Byron

2 1

Fwd: [numpy] ENH: Initial implementation of a 'neighbor' calculation (#303)
by Nathaniel Smith 15 Oct '12

15 Oct '12

[Manual PR notification] ---------- Forwarded message ---------- From: timcera Date: Sat, Jun 9, 2012 at 10:13 PM Subject: [numpy] ENH: Initial implementation of a 'neighbor' calculation (#303) To: njsmith <njs(a)pobox.com> Each element is assigned the result of a function based on it's neighbors. Neighbors are selected based on a weight array. It uses the new pad routines to pad arrays if neighboring values are required that would be off the edge of the input array. Will be great to have the masked array settled because right now you can only sort of exclude from the neighborhood using a zero in the weight array. Zero or np.IGNORE don't affect np.sum, but functions like np.mean and np.std would give different answers. Because of this my early implementations of neighbor included an optional mask array along with the weight array, but I decided would be best to wait for the new masked arrays. This in some ways could be considered a generalization of a convolution, and comparison with existing numpy/scipy convolution results are included in the tests. The advantage to neighbor is that any function that accepts a 1-d array, and returns a single result, can be used instead of convolution only using summation. The convolution functions require the weight array to be flipped to get the same answer as neighbor. You can merge this Pull Request by running: git pull https://github.com/timcera/numpy neighbor Or you can view, comment on it, or merge it online at: https://github.com/numpy/numpy/pull/303 -- Commit Summary -- * ENH: Initial implementation of a 'neighbor' calculation where the each -- File Changes -- M numpy/lib/__init__.py (2) A numpy/lib/neighbor.py (305) A numpy/lib/tests/test_neighbor.py (278) -- Patch Links -- https://github.com/numpy/numpy/pull/303.patch https://github.com/numpy/numpy/pull/303.diff --- Reply to this email directly or view it on GitHub: https://github.com/numpy/numpy/pull/303

7 14

[ANN] CFP: SciPy India 2012 -- Dec 27-29 -- IIT Bombay
by Jarrod Millman 15 Oct '12

15 Oct '12

Hello, The CFP for SciPy India 2012, to be held in IIT Bombay from December 27-29 is open. Please spread the word! Scipy.in is a conference providing opportunities to spread the use of the Python programming language in the Scientific Computing community in India. It provides a unique opportunity to interact with the "Who's who" of the Python for Scientific Computing fraternity and learn, understand, participate, and contribute to Scientific Computing using Python. Attendees of the conference and participants of the sprints planned will be able to access and review the tools available. They will also be able to learn domain-specific applications and how the tools apply to a plethora of application problems. One of the goals of the conference is to combine education, engineering, and science with computing through the medium of Python. This conference also aims to spread the use of Python for Scientific Computing in various fields and among different communities. Call for Papers ================ We look forward to your submissions on the use of Python for Scientific Computing and Education. This includes pedagogy, exploration, modeling and analysis from both applied and developmental perspectives. We welcome contributions from academia as well as industry. Submission of Papers ===================== If you wish to present your paper using this platform, please submit an abstract of 300 to 700 words describing the topic, including its relevance to scientific computing. Based on the number and quality of the submissions, the conference organizers will allot 10 - 30 minutes for each accepted talk. In addition to these talks, there will be an open session of lightning talks, during which any attendee who wishes to talk on a pertinent topic is invited to do a presentation not exceeding five minutes in duration. If you wish to present a talk at the conference, please follow the guidelines below. Submission Guidelines ====================== - Submit your proposals at scipy(a)fossee.in - Submissions whose main purpose is to promote a commercial product or service will be refused. - All accepted proposals must be presented at the SciPy conference by at least one author. Important Dates ================ - Call for proposals start: 27th September 2012, Thursday - Call for proposals end: 1st November 2012, Thursday - List of accepted proposals will be published: 19th November 2012, Monday - Submission of first presentation: 10th December 2012, Monday - Submission of final presentation(with final changes): 20th December 2012, Thursday

1 0

Compiling against serial ATLAS
by Jared O'Connell 14 Oct '12

14 Oct '12

Hello! I am trying to build numpy 1.6.2 with python 2.7 in Ubuntu 12.04 and wish to link against serial ATLAS rather than the threaded version. I have modified site.cfg accordingly by uncommenting out the following lines: [blas_opt] libraries = f77blas, cblas, atlas [lapack_opt] libraries = lapack, f77blas, cblas, atlas However when I run python setup.py build it still links to the threaded versions eg. python setup.py build Running from numpy source directory.F2PY Version 2 blas_opt_info: blas_mkl_info: libraries mkl,vml,guide not found in /usr/local/lib libraries mkl,vml,guide not found in /usr/lib NOT AVAILABLE atlas_blas_threads_info: Setting PTATLAS=ATLAS libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib Setting PTATLAS=ATLAS customize GnuFCompiler Could not locate executable g77 Could not locate executable f77 customize IntelFCompiler Could not locate executable ifort Could not locate executable ifc customize LaheyFCompiler Could not locate executable lf95 customize PGroupFCompiler Could not locate executable pgfortran customize AbsoftFCompiler Could not locate executable f90 customize NAGFCompiler Found executable /usr/bin/f95 customize VastFCompiler customize CompaqFCompiler Could not locate executable fort customize IntelItaniumFCompiler Could not locate executable efort Could not locate executable efc customize IntelEM64TFCompiler customize Gnu95FCompiler Found executable /usr/bin/gfortran customize Gnu95FCompiler customize Gnu95FCompiler using config compiling '_configtest.c': /* This file is generated from numpy/distutils/system_info.py */ void ATL_buildinfo(void); int main(void) { ATL_buildinfo(); return 0; } C compiler: gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC compile options: '-c' gcc: _configtest.c gcc -pthread _configtest.o -L/usr/lib/atlas-base -lptf77blas -lptcblas -latlas -o _configtest ATLAS version 3.8.4 built by buildd on Sat Sep 10 23:12:12 UTC 2011: UNAME : Linux crested 2.6.24-29-server #1 SMP Wed Aug 10 15:58:57 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux INSTFLG : -1 0 -a 1 ARCHDEFS : -DATL_OS_Linux -DATL_ARCH_HAMMER -DATL_CPUMHZ=1993 -DATL_USE64BITS -DATL_GAS_x8664 F2CDEFS : -DAdd_ -DF77_INTEGER=int -DStringSunStyle CACHEEDGE: 393216 F77 : gfortran, version GNU Fortran (Ubuntu/Linaro 4.6.1-9ubuntu2) 4.6.1 F77FLAGS : -fomit-frame-pointer -mfpmath=387 -O2 -falign-loops=4 -Wa,--noexecstack -fPIC -m64 SMC : gcc, version gcc (Ubuntu/Linaro 4.6.1-9ubuntu2) 4.6.1 SMCFLAGS : -fomit-frame-pointer -mfpmath=387 -O2 -falign-loops=4 -Wa,--noexecstack -fPIC -m64 SKC : gcc, version gcc (Ubuntu/Linaro 4.6.1-9ubuntu2) 4.6.1 SKCFLAGS : -fomit-frame-pointer -mfpmath=387 -O2 -falign-loops=4 -Wa,--noexecstack -fPIC -m64 success! removing: _configtest.c _configtest.o _configtest Setting PTATLAS=ATLAS FOUND: libraries = ['ptf77blas', 'ptcblas', 'atlas'] library_dirs = ['/usr/lib/atlas-base'] language = c define_macros = [('ATLAS_INFO', '"\\"3.8.4\\""')] include_dirs = ['/usr/include/atlas'] FOUND: libraries = ['ptf77blas', 'ptcblas', 'atlas'] library_dirs = ['/usr/lib/atlas-base'] language = c define_macros = [('ATLAS_INFO', '"\\"3.8.4\\""')] include_dirs = ['/usr/include/atlas'] lapack_opt_info: lapack_mkl_info: mkl_info: libraries mkl,vml,guide not found in /usr/local/lib libraries mkl,vml,guide not found in /usr/lib NOT AVAILABLE NOT AVAILABLE atlas_threads_info: Setting PTATLAS=ATLAS libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib libraries lapack_atlas not found in /usr/local/lib libraries lapack not found in /usr/lib/atlas-base numpy.distutils.system_info.atlas_threads_info Setting PTATLAS=ATLAS Setting PTATLAS=ATLAS FOUND: libraries = ['lapack', 'ptf77blas', 'ptcblas', 'atlas'] library_dirs = ['/usr/lib/atlas-base/atlas', '/usr/lib/atlas-base'] language = f77 define_macros = [('ATLAS_INFO', '"\\"3.8.4\\""')] include_dirs = ['/usr/include/atlas'] FOUND: libraries = ['lapack', 'ptf77blas', 'ptcblas', 'atlas'] library_dirs = ['/usr/lib/atlas-base/atlas', '/usr/lib/atlas-base'] language = f77 define_macros = [('ATLAS_INFO', '"\\"3.8.4\\""')] include_dirs = ['/usr/include/atlas'] ... Any ideas? best, Jared

1 0

Re: [Numpy-discussion] Welcome to the "NumPy-Discussion" mailing list (Digest mode)
by Jared O'Connell 14 Oct '12

14 Oct '12

1 0

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

NumPy-Discussion October 2012