Re: [Numpy-discussion] Numpy-discussion Digest, Vol 17, Issue 15

Thanks I been trying to compile a code that uses random,pylab and numpy with py2exe the code of setup.py(compiles mycode.py into mycode.exe) follows #Start here from distutils.core import setup import py2exe import pylab import numpy import glob import scipy import random import os setup( console=['mycode.py'],options={'py2exe': {"skip_archive":1,'packages':['matplotlib','pytz']',}},data_files=[ matplotlib.get_py2exe_datafiles()]) #End here It works well for codes that uses pylab only. But It i add more modules i get trouble Is there any good books on how to use py2exe? On 2/7/08, numpy-discussion-request@scipy.org < numpy-discussion-request@scipy.org> wrote:
Send Numpy-discussion mailing list submissions to numpy-discussion@scipy.org
To subscribe or unsubscribe via the World Wide Web, visit http://projects.scipy.org/mailman/listinfo/numpy-discussion or, via email, send a message with subject or body 'help' to numpy-discussion-request@scipy.org
You can reach the person managing the list at numpy-discussion-owner@scipy.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of Numpy-discussion digest..."
Today's Topics:
1. f2py compiled module not found by python (Chris) 2. Re: f2py compiled module not found by python (Pearu Peterson) 3. Re: Bug in numpy all() function (Robert Kern) 4. Re: f2py compiled module not found by python (Chris) 5. Re: random enhancement (Robert Kern) 6. Re: Bug in numpy all() function (Anne Archibald) 7. Re: Bug in numpy all() function (Robert Kern) 8. Re: Numpy-discussion Digest, Vol 17, Issue 13 (Steven H. Rogers) 9. isn't it a bug? (matrix multiplication) (dmitrey) 10. Re: isn't it a bug? (matrix multiplication) (Alan G Isaac)
----------------------------------------------------------------------
Message: 1 Date: Wed, 6 Feb 2008 18:35:35 +0000 (UTC) From: Chris <listservs@mac.com> Subject: [Numpy-discussion] f2py compiled module not found by python To: numpy-discussion@scipy.org Message-ID: <loom.20080206T183501-435@post.gmane.org> Content-Type: text/plain; charset=utf-8
Hello,
I'm trying to build a package on Linux (Ubuntu) that contains a fortran module, built using f2py. However, despite the module building and installing without error, python cannot seem to see it (see log below). This works fine on Windows and Mac; the problem only seems to happen on Linux:
In [1]: import PyMC ----------------------------------------------- exceptions.ImportError Traceback (most recent call last)
/home/tianhuil/?ipython console>
/usr/lib/python2.4/site-packages/PyMC/__init__.py
/home/tianhuil/?string>
/usr/lib/python2.4/site-packages/PyMC/MCMC.py
ImportError: No module named flib /usr/lib/python2.4/site-packages/PyMC/MCMC.py
Notice that the module exists in the site-packages directory:
tianhuil <at> tianhuil:/usr/lib/python2.4/site-packages/PyMC$ ll total 432 drwxr-xr-x 2 root root 4096 2008-02-03 17:24 Backends -rwxrwx--- 1 root root 195890 2008-02-03 17:24 flib.so -rwxrwx--- 1 root root 259 2008-02-03 17:14 __init__.py -rw-r--r-- 1 root root 473 2008-02-03 17:24 __init__.pyc -rwxrwx--- 1 root root 10250 2008-02-03 17:14 Matplot.py -rw-r--r-- 1 root root 7516 2008-02-03 17:24 Matplot.pyc -rwxrwx--- 1 root root 98274 2008-02-03 17:14 MCMC.py -rw-r--r-- 1 root root 79039 2008-02-03 17:24 MCMC.pyc drwxr-xr-x 2 root root 4096 2008-02-03 17:24 Tests -rwxrwx--- 1 root root 6631 2008-02-03 17:14 TimeSeries.py -rw-r--r-- 1 root root 5043 2008-02-03 17:24 TimeSeries.pyc
------------------------------
Message: 2 Date: Wed, 6 Feb 2008 20:58:07 +0200 (EET) From: "Pearu Peterson" <pearu@cens.ioc.ee> Subject: Re: [Numpy-discussion] f2py compiled module not found by python To: "Discussion of Numerical Python" <numpy-discussion@scipy.org> Message-ID: <61111.85.166.27.136.1202324287.squirrel@cens.ioc.ee> Content-Type: text/plain;charset=iso-8859-1
On Wed, February 6, 2008 8:35 pm, Chris wrote:
Hello,
I'm trying to build a package on Linux (Ubuntu) that contains a fortran module, built using f2py. However, despite the module building and installing without error, python cannot seem to see it (see log below). This works fine on Windows and Mac; the problem only seems to happen on Linux:
Can you import flib module directly? That is, what happens if you execute cd .../PyMC PYTHONPATH=. python -c 'import flib' ? Pearu
------------------------------
Message: 3 Date: Wed, 06 Feb 2008 14:03:20 -0600 From: Robert Kern <robert.kern@gmail.com> Subject: Re: [Numpy-discussion] Bug in numpy all() function To: Discussion of Numerical Python <numpy-discussion@scipy.org> Message-ID: <47AA1288.6070700@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed
Dan Goodman wrote:
Hi all,
I think this is a bug (I'm running Numpy 1.0.3.1):
from numpy import * def f(x): return False
all(f(x) for x in range(10)) True
I guess the all function doesn't know about generators?
Yup. It works on arrays and things it can turn into arrays by calling the C API equivalent of numpy.asarray(). There's a ton of magic and special cases in asarray() in order to interpret nested Python sequences as arrays. That magic works fairly well when we have sequences with known lengths; it fails utterly when given an arbitrary iterator of unknown length. So we punt. Unfortunately, what happens then is that asarray() sees an object that it can't interpret as a sequence to turn into a real array, so it makes a rank-0 array with the iterator object as the value. This evaluates to True.
It's possible that asarray() should raise an exception for generators, but it would be a special case. We wouldn't be able to test for arbitrary iterables.
-- Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
------------------------------
Message: 4 Date: Wed, 6 Feb 2008 20:13:04 +0000 (UTC) From: Chris <listservs@mac.com> Subject: Re: [Numpy-discussion] f2py compiled module not found by python To: numpy-discussion@scipy.org Message-ID: <loom.20080206T201205-452@post.gmane.org> Content-Type: text/plain; charset=us-ascii
Pearu Peterson <pearu <at> cens.ioc.ee> writes:
This works fine on Windows and Mac; the problem only seems to happen on Linux:
Can you import flib module directly? That is, what happens if you execute cd .../PyMC PYTHONPATH=. python -c 'import flib'
It gives a "no module named flib" error.
------------------------------
Message: 5 Date: Wed, 06 Feb 2008 14:15:18 -0600 From: Robert Kern <robert.kern@gmail.com> Subject: Re: [Numpy-discussion] random enhancement To: Discussion of Numerical Python <numpy-discussion@scipy.org> Message-ID: <47AA1556.8070708@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed
Neal Becker wrote:
One thing missing from random is a mechanism to share a single underlying rng with other code that is not part of numpy.random.
For example, I have code that generates distributions that expect a mersenne twister (the shared, underlying rng) to be passed in as a constructor argument.
numpy.random shares a single rng amongst it's own distributions, but I don't see any way to share with others.
Are you talking about C or Python?
In Python, just instantiate numpy.random.RandomState() and pass it around. All of the functions in numpy.random are just aliases to the methods on a global RandomState() instance.
C is a problem because the module is implemented in Pyrex, and RandomState is an extension type. I've tried working on exposing the C API as a PyCObject like numpy does, but it is incredibly tedious and, furthermore, is unlikely to capture the higher-level methods like multivariate_normal(). I believe that Cython has a way to automatically expose the C API of a Pyrex/Cython module, but I haven't had the time to investigate it.
For everything but the higher level methods like multivariate_normal(), we might be able to expose the pointer to the rk_state struct on the RandomState object as a PyCObject and punt on exposing the API. The C user can copy the randomkit.[ch] and distributions.[ch] files into their own code and operate on the rk_state pointer with those functions. We may be thwarted by symbol conflicts on some platforms, but I'm not sure.
Contributions are, of course, welcome.
-- Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
------------------------------
Message: 6 Date: Wed, 6 Feb 2008 15:18:38 -0500 From: "Anne Archibald" <peridot.faceted@gmail.com> Subject: Re: [Numpy-discussion] Bug in numpy all() function To: "Discussion of Numerical Python" <numpy-discussion@scipy.org> Message-ID: <ce557a360802061218o2b7ffabblca5919d1a3b47bb7@mail.gmail.com> Content-Type: text/plain; charset=UTF-8
On 06/02/2008, Robert Kern <robert.kern@gmail.com> wrote:
I guess the all function doesn't know about generators?
Yup. It works on arrays and things it can turn into arrays by calling the C API equivalent of numpy.asarray(). There's a ton of magic and special cases in asarray() in order to interpret nested Python sequences as arrays. That magic works fairly well when we have sequences with known lengths; it fails utterly when given an arbitrary iterator of unknown length. So we punt. Unfortunately, what happens then is that asarray() sees an object that it can't interpret as a sequence to turn into a real array, so it makes a rank-0 array with the iterator object as the value. This evaluates to True.
It's possible that asarray() should raise an exception for generators, but it would be a special case. We wouldn't be able to test for arbitrary iterables.
Would it be possible for asarray() to pull out the first element from the iterable, make an array out of it, then assume that all other values out of the iterable will have the same shape (raising an error, of course, when they aren't)? I guess this has high foot-shooting potential, but is it that much worse than numpy's shpe-guessing generally?
It would be handy to be able to use an iterable to fill an array, so that you'd never need to store the values in anything else first:
a = N.array((sin(N.pi*x/n) for x in xrange(n)))
Anne
------------------------------
Message: 7 Date: Wed, 06 Feb 2008 14:51:29 -0600 From: Robert Kern <robert.kern@gmail.com> Subject: Re: [Numpy-discussion] Bug in numpy all() function To: Discussion of Numerical Python <numpy-discussion@scipy.org> Message-ID: <47AA1DD1.7090504@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed
On 06/02/2008, Robert Kern <robert.kern@gmail.com> wrote:
I guess the all function doesn't know about generators? Yup. It works on arrays and things it can turn into arrays by calling
Anne Archibald wrote: the C API
equivalent of numpy.asarray(). There's a ton of magic and special cases in asarray() in order to interpret nested Python sequences as arrays. That magic works fairly well when we have sequences with known lengths; it fails utterly when given an arbitrary iterator of unknown length. So we punt. Unfortunately, what happens then is that asarray() sees an object that it can't interpret as a sequence to turn into a real array, so it makes a rank-0 array with the iterator object as the value. This evaluates to True.
It's possible that asarray() should raise an exception for generators, but it would be a special case. We wouldn't be able to test for arbitrary iterables.
Would it be possible for asarray() to pull out the first element from the iterable, make an array out of it, then assume that all other values out of the iterable will have the same shape (raising an error, of course, when they aren't)? I guess this has high foot-shooting potential, but is it that much worse than numpy's shpe-guessing generally?
I'm skeptical. Personally, it comes down to this: if you provide code that implements this safely and efficiently without making a confusing API, I'm more than happy to consider it for inclusion. But I'm not going to spend time trying to write the code.
It would be handy to be able to use an iterable to fill an array, so that you'd never need to store the values in anything else first:
a = N.array((sin(N.pi*x/n) for x in xrange(n)))
If n is large enough that storage matters,
a = N.sin(N.linspace(0, np.pi, n))
is always faster, more memory efficient, and more readable. Remember that the array will have to be dynamically resized as we go through the iterator. The memory movement is going to wipe out much of the benefit of having an iterator in the first place.
For 1D arrays, remember that we have numpy.fromiter() already, so we can test this.
In [39]: import numpy as np
In [40]: from math import sin
In [41]: n = 10
In [42]: %timeit np.fromiter((sin(np.pi*x/n) for x in xrange(n)), float) 100000 loops, best of 3: 11.5 ?s per loop
In [43]: %timeit np.sin(np.linspace(0, np.pi, n)) 10000 loops, best of 3: 26.1 ?s per loop
In [44]: n = 100
In [45]: %timeit np.fromiter((sin(np.pi*x/n) for x in xrange(n)), float) 10000 loops, best of 3: 84 ?s per loop
In [46]: %timeit np.sin(np.linspace(0, np.pi, n)) 10000 loops, best of 3: 32.3 ?s per loop
In [47]: n = 1000
In [48]: %timeit np.fromiter((sin(np.pi*x/n) for x in xrange(n)), float) 1000 loops, best of 3: 794 ?s per loop
In [49]: %timeit np.sin(np.linspace(0, np.pi, n)) 10000 loops, best of 3: 91.8 ?s per loop
So, for n=10, the generator wins, but is n=10 really the case that you want to use a generator for?
-- Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
------------------------------
Message: 8 Date: Wed, 06 Feb 2008 18:51:10 -0700 From: "Steven H. Rogers" <steve@shrogers.com> Subject: Re: [Numpy-discussion] Numpy-discussion Digest, Vol 17, Issue 13 To: Discussion of Numerical Python <numpy-discussion@scipy.org> Message-ID: <47AA640E.2030006@shrogers.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
matthew yeomans wrote:
Is it possible to compile numpy with py2exe?
Matthew Yeomans
If you mean to generate a Windows executable containing py2exe, the answer is yes. The process isn't what is usually thought of as compilation as it just packages the Python interpreter, your application code, and required libraries for easy distribution.
# Steve
------------------------------
Message: 9 Date: Thu, 07 Feb 2008 15:41:41 +0200 From: dmitrey <dmitrey.kroshko@scipy.org> Subject: [Numpy-discussion] isn't it a bug? (matrix multiplication) To: Discussion of Numerical Python <numpy-discussion@scipy.org> Message-ID: <47AB0A95.8050809@scipy.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
from numpy import array a = array((1.0, 2.0))
b = c = 15 b = b*a#ok c *= a#ok
d = array(15) e = array(15) d = d*a#this works ok e *= a#this intended to be same as prev line, but yields error: Traceback (innermost last): File "<stdin>", line 1, in <module> ValueError: invalid return array shape
------------------------------
Message: 10 Date: Thu, 7 Feb 2008 09:10:59 -0500 From: Alan G Isaac <aisaac@american.edu> Subject: Re: [Numpy-discussion] isn't it a bug? (matrix multiplication) To: Discussion of Numerical Python <numpy-discussion@scipy.org> Message-ID: <Mahogany-0.67.0-664-20080207-091059.00@american.edu> Content-Type: TEXT/PLAIN; CHARSET=UTF-8
On Thu, 07 Feb 2008, dmitrey apparently wrote:
a = array((1.0, 2.0)) e = array(15) e *= a # ... yields error:
You are trying to stuff in two values where you have only allocated space for 1.
Cheers, Alan Isaac
------------------------------
_______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
End of Numpy-discussion Digest, Vol 17, Issue 15 ************************************************
-- Kollox Ghal Xejn

Matthew, please do not reply to the digests. Think of them as read-only. If you want to start a new thread, send your mail, with a descriptive Subject line, to numpy-discussion@scipy.org . If you want to reply to individual messages, please turn digest delivery *off* and receive and respond to messages normally. Thank you. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

matthew yeomans wrote:
Thanks I been trying to compile a code that uses random,pylab and numpy with py2exe the code of setup.py(compiles mycode.py into mycode.exe) follows
#Start here from distutils.core import setup import py2exe import pylab import numpy import glob import scipy import random import os
setup( console=['mycode.py'],options={'py2exe': {"skip_archive":1,'packages':['matplotlib','pytz']',}},data_files=[matplotlib.get_py2exe_datafiles()])
#End here
It works well for codes that uses pylab only. But It i add more modules i get trouble
Is there any good books on how to use py2exe?
Matthew: I've only used py2exe with a couple of imports (numpy and os). You're importing an awful lot. Slimming this down to only import the things you really need from each module may help, e.g. from numpy import foo from random import bar etc. # Steve
participants (3)
-
matthew yeomans
-
Robert Kern
-
Steven H. Rogers