NumPy-Discussion November 2010

numpy-discussion@python.org

102 participants
101 discussions

Re: [Numpy-discussion] How to export a CObject from NumPy to my own module?
by NumPyStudent 25 Nov '10

25 Nov '10

Well, I believe I should problably use the PyArray_API instead (http://www.velocityreviews.com/forums/t358140-re-dynamically-loaded-librari…), since it seems it's implemented in NumPy for this exact reason(?). Can anyone point me in the direction of a good step-step guide og using this api to export function pointers from NumPy to your own modules? NumPyStudent wrote: > > Hi > > I haven't worked very much with NumPy and Python and hope some assistance > :-) > > I need access to a functionpointer in my own Python module. > This functionpointer is available in a modified NumPy module. > I would like to pass the functionpointer to my module when I import it in > the Python runtime. > Before importing my module I've already imported the NumPy module, and > this is fine by me - > I don't necessarily need my own module to handle the import of NumPy... > > My question concerns how I should modify NumPy source code to be able to > use the method of passing CObjects as described here: > http://docs.python.org/release/2.5.2/ext/using-cobjects.html > > Can anyone tell me where I should add the C-Object in NumPy? > I tried to add it to the "multiarray" module (in the file > multiarraymodule.c), but it dosen't look like this module is available for > import (as in PyImport_ImportModule("multiarray"); in my own module - the > return value of this is null). > However, if I try PyImport_ImportModule("numpy"); it seems like I get a > module imported, but this module does not have the CObject added. This is > not surprising, since I added it to the "multiarray" module. > > I believe I can solve my problem by adding the CObject to the "numpy" > module instead, but in what file or files should I make modifications to > do this? > > Alternatively, how can I make the "multiarray" module available for import > by my own module? > > BR. > NumPyStudent > -- View this message in context: http://old.nabble.com/How-to-export-a-CObject-from-NumPy-to-my-own-module--… Sent from the Numpy-discussion mailing list archive at Nabble.com.

3 4

Give Mark Wiebe numpy commit rights.
by Charles R Harris 24 Nov '10

24 Nov '10

Hi All, I'd like to give Mark Wiebe <mwwiebe(a)gmail.com> commit rights. That does bring up the question of commit rights proliferation, so along with that I'd like to suggest time limits on commit rights, something along the lines of "no commits for a year, lose rights". It's nothing punitive, it's just a matter of keeping some control over what happens to the repository. Chuck

2 2

question for numpy import error
by 명광민 24 Nov '10

24 Nov '10

I receive the following error when I try to import numpy =========================================================== [node0]:/home/koojy/KMM> python2.6 Python 2.6.4 (r264:75706, Jul 26 2010, 16:55:18) [GCC 3.4.6 20060404 (Red Hat 3.4.6-10)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python2.6/site-packages/numpy/__init__.py", line 132, in <module> import add_newdocs File "/usr/local/lib/python2.6/site-packages/numpy/add_newdocs.py", line 9, in <module> from lib import add_newdoc File "/usr/local/lib/python2.6/site-packages/numpy/lib/__init__.py", line 13, in <module> from polynomial import * File "/usr/local/lib/python2.6/site-packages/numpy/lib/polynomial.py", line 17, in <module> from numpy.linalg import eigvals, lstsq File "/usr/local/lib/python2.6/site-packages/numpy/linalg/__init__.py", line 47, in <module> from linalg import * File "/usr/local/lib/python2.6/site-packages/numpy/linalg/linalg.py", line 22, in <module> from numpy.linalg import lapack_lite ImportError: libgfortran.so.1: cannot open shared object file: No such file or directory >>> ============================================================ OS info : Linux node0 2.6.9-78.ELsmp #1 SMP Thu Jul 24 23:54:48 EDT 2008 x86_64 x86_64 x86_64 GNU/Linux numpy install : following the "Building by hand" of Contents of below site. http://www.scipy.org/Installing_SciPy/Linux#head-40c26a5b93b9afc7e3241e1d7f… and To build with gfortran: python setup.py build –fcompiler=gnu95 Any help on this error would be appreciated. K.M. Myung ================================== Korea Environmental Science & Technology Institute, inc. phone : 82-2-2113-0705 direct : 82-70-7098-2644 fax : 82-2-2113-0706 e-mail : kmmyung(a)kesti.co.kr ==================================

1 0

numpy.test() errors
by Nils Wagner 23 Nov '10

23 Nov '10

Hi all, There are some new test errors ====================================================================== ERROR: Test with missing and filling values ---------------------------------------------------------------------- Traceback (most recent call last): File "/data/home/nwagner/local/lib/python2.5/site-packages/numpy/lib/tests/test_io.py", line 947, in test_user_filling_values test = np.genfromtxt(StringIO(data), **kwargs) File "/data/home/nwagner/local/lib/python2.5/site-packages/numpy/lib/npyio.py", line 1285, in genfromtxt key = names.index(key) AttributeError: 'tuple' object has no attribute 'index' ====================================================================== ERROR: test_user_missing_values (test_io.TestFromTxt) ---------------------------------------------------------------------- Traceback (most recent call last): File "/data/home/nwagner/local/lib/python2.5/site-packages/numpy/lib/tests/test_io.py", line 931, in test_user_missing_values **basekwargs) File "/data/home/nwagner/local/lib/python2.5/site-packages/numpy/lib/npyio.py", line 1657, in mafromtxt return genfromtxt(fname, **kwargs) File "/data/home/nwagner/local/lib/python2.5/site-packages/numpy/lib/npyio.py", line 1285, in genfromtxt key = names.index(key) AttributeError: 'tuple' object has no attribute 'index' ====================================================================== ERROR: test_io.test_gzip_load ---------------------------------------------------------------------- Traceback (most recent call last): File "/data/home/nwagner/local/lib/python2.5/site-packages/nose-0.11.1-py2.5.egg/nose/case.py", line 183, in runTest self.test(*self.arg) File "/data/home/nwagner/local/lib/python2.5/site-packages/numpy/lib/tests/test_io.py", line 1255, in test_gzip_load assert_array_equal(np.load(f), a) File "/data/home/nwagner/local/lib/python2.5/site-packages/numpy/lib/npyio.py", line 327, in load fid = seek_gzip_factory(file) File "/data/home/nwagner/local/lib/python2.5/site-packages/numpy/lib/npyio.py", line 66, in seek_gzip_factory g.name = f.name AttributeError: GzipFile instance has no attribute 'name' ---------------------------------------------------------------------- Ran 3066 tests in 12.458s FAILED (KNOWNFAIL=4, errors=3) <nose.result.TextTestResult run=3066 errors=3 failures=0> >>> numpy.__version__ '2.0.0.dev-12d0200' Nils

5 6

Theano 0.3 Released
by Frédéric Bastien 23 Nov '10

23 Nov '10

====================== Announcing Theano 0.3 ====================== This is an important release. The upgrade is recommended for everybody using Theano 0.1. For those using the bleeding edge version in the mercurial repository, we encourage you to update to the `0.3` tag. This is the first major release of Theano since 0.1. Version 0.2 development started internally but it was never advertised as a release. What's New ---------- There have been too many changes since 0.1 to keep track of them all. Below is a *partial* list of changes since 0.1. * GPU code using NVIDIA's CUDA framework is now generated for many Ops. * Some interface changes since 0.1: * A new "shared variable" system which allows for reusing memory space between Theano functions. * A new memory contract has been formally written for Theano, for people who want to minimize memory copies. * The old module system has been deprecated. * By default, inputs to a Theano function will not be silently downcasted (e.g. from float64 to float32). * An error is now raised when using the result of a logical operation of a Theano variable in an 'if' (i.e. an implicit call to __nonzeros__). * An error is now raised when we receive a non-aligned ndarray as input to a function (this is not supported). * An error is raised when the list of dimensions passed to dimshuffle() contains duplicates or is otherwise not sensible. * Call NumPy BLAS bindings for gemv operations in addition to the already supported gemm. * If gcc is unavailable at import time, Theano now falls back to a Python-based emulation mode after raising a warning. * An error is now raised when tensor.grad is called on a non-scalar Theano variable (in the past we would implicitly do a sum on the tensor to make it a scalar). * Added support for "erf" and "erfc" functions. * The current default value of the parameter axis of theano.{max,min, argmax,argmin,max_and_argmax} is deprecated. We now use the default NumPy behavior of operating on the entire tensor. * Theano is now available from PyPI and installable through "easy_install" or "pip". You can download Theano from http://pypi.python.org/pypi/Theano. Description ----------- Theano is a Python library that allows you to define, optimize, and efficiently evaluate mathematical expressions involving multi-dimensional arrays. It is built on top of NumPy. Theano features: * tight integration with NumPy: a similar interface to NumPy's. numpy.ndarrays are also used internally in Theano-compiled functions. * transparent use of a GPU: perform data-intensive computations up to 140x faster than on a CPU (support for float32 only). * efficient symbolic differentiation: Theano can compute derivatives for functions of one or many inputs. * speed and stability optimizations: avoid nasty bugs when computing expressions such as log(1+ exp(x) ) for large values of x. * dynamic C code generation: evaluate expressions faster. * extensive unit-testing and self-verification: includes tools for detecting and diagnosing bugs and/or potential problems. Theano has been powering large-scale computationally intensive scientific research since 2007, but it is also approachable enough to be used in the classroom (IFT6266 at the University of Montreal). Resources --------- About Theano: http://deeplearning.net/software/theano/ About NumPy: http://numpy.scipy.org/ About Scipy: http://www.scipy.org/ Acknowledgments --------------- I would like to thank all contributors of Theano. For this particular release, the people who have helped resolve many outstanding issues: (in alphabetical order) Frederic Bastien, James Bergstra, Guillaume Desjardins, David-Warde Farley, Ian Goodfellow, Pascal Lamblin, Razvan Pascanu and Josh Bleecher Snyder. Also, thank you to all NumPy and Scipy developers as Theano builds on its strength. All questions/comments are always welcome on the Theano mailing-lists ( http://deeplearning.net/software/theano/ )

1 0

categorical distributions
by Hagen Fürstenau 23 Nov '10

23 Nov '10

Hi, numpy doesn't seem to have a function for sampling from simple categorical distributions. The easiest solution I could come up with was something like >>> from numpy.random import multinomial >>> multinomial(1, [.5, .3, .2]).nonzero()[0][0] 1 but this is bound to be inefficient as soon as the vector of probabilities gets large, especially if you want to draw multiple samples. Have I overlooked something or should this be added? - Hagen

3 5

Matlab IO Warning in mio5.py
by Ben Longstaff 22 Nov '10

22 Nov '10

Is the warning message produced by the test script below an issue I need to address if I am to use the numpy and scipy libraries? The script below gives me the warning below on both windows 7 and vista Thanks for your time Ben Python Script #!/path/to/python import re, string, sys from numpy import * import scipy.weave print "Hello, World!" Warning Message C:\Python26\lib\site-packages\scipy\io\matlab\mio5.py:90: RuntimeWarning: __builtin__.file size changed, may indicate binary incompatibility from mio5_utils import VarReader5 Hello, World!

6 10

indexing question
by Ernest Adrogué 22 Nov '10

22 Nov '10

Hi, Suppose an array of shape (N,2,2), that is N arrays of shape (2,2). I want to select an element (x,y) from each one of the subarrays, so I get a 1-dimensional array of length N. For instance: In [228]: t=np.arange(8).reshape(2,2,2) In [229]: t Out[229]: array([[[0, 1], [2, 3]], [[4, 5], [6, 7]]]) In [230]: x=[0,1] In [231]: y=[1,1] In [232]: t[[0,1],x,y] Out[232]: array([1, 7]) This way, I get the elements (0,1) and (1,1) which is what I wanted. The question is: is it possible to omit the [0,1] in the index? Thanks in advance. -- Ernest

4 10

Does np.std() make two passes through the data?
by Keith Goodman 22 Nov '10

22 Nov '10

Does np.std() make two passes through the data? Numpy: >> arr = np.random.rand(10) >> arr.std() 0.3008736260967052 Looks like an algorithm that makes one pass through the data (one for loop) wouldn't match arr.std(): >> np.sqrt((arr*arr).mean() - arr.mean()**2) 0.30087362609670526 But a slower two-pass algorithm would match arr.std(): >> np.sqrt(((arr - arr.mean())**2).mean()) 0.3008736260967052 Is there a way to get the same result as arr.std() in one pass (cython for loop) of the data?

4 19

Slow element-by-element access?
by Bruce Sherwood 22 Nov '10

22 Nov '10

A colleague showed me a program using Numeric with Python 2.5 which ran much faster than the same program using numpy with Python 2.7. I distilled this down to a simple test case, characterized by a "for" loop in which he does an element-by-element calculation involving arrays: from numpy import arange # or from Numeric import arange from time import clock # Numeric 0.24 seconds; 15 times as fast as numpy # numpy 3.6 seconds N = 100000 a = arange(N) b = arange(N) t = clock() for i in range(1,N-1): pass tpass = clock()-t t = clock() for i in range(1,N-1): b[i] = a[i]-t*(a[i+1]-2*a[i]+a[i-1])+a[i]*a[i]*t t = clock()-t print t-tpass His calculation b[i] = a[i]-t*(a[i+1]-2*a[i]+a[i-1])+a[i]*a[i]*t is 15 times faster with Numeric than with numpy. It is of course the case that he should have done a single array calculation rather than use a "for" loop, and I've explained that to him. The array calculation runs at about the same speed in numpy as in Numeric. Nevertheless, I'm surprised that the element-by-element calculation is so very slow, and surely there are situations where individual element access is important in a numpy calculation. Is this a known issue? Does it matter? I was unable to find any discussion of this. Bruce Sherwood

4 4

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

NumPy-Discussion November 2010