From lists at onerussian.com  Thu Aug  1 11:32:39 2013
From: lists at onerussian.com (Yaroslav Halchenko)
Date: Thu, 1 Aug 2013 11:32:39 -0400
Subject: [Numpy-discussion] Now benchmarking multiple branches (master +
	maintenance/1.[67].x)
In-Reply-To: <20130506143241.GV5140@onerussian.com>
References: <20130506143241.GV5140@onerussian.com>
Message-ID: <20130801153239.GY27621@onerussian.com>

I am glad to announce that now you can see benchmark timing plots for
multiple branches, thus being able to spot regressions in maintenance
branches and compare enhancements in relation to previous releases.

e.g.
* improving upon 1.7.x but still lacking behind 1.6.x
  http://www.onerussian.com/tmp/numpy-vbench/vb_vb_core.html#numpy-identity-100
  http://www.onerussian.com/tmp/numpy-vbench/vb_vb_function_base.html#percentile
  ...

* what seems to be a regression caught/fixed in 1.7.x:
  http://www.onerussian.com/tmp/numpy-vbench/vb_vb_indexing.html#a-indexes-float32

* or not (yet) fixed in 1.7.x
  http://www.onerussian.com/tmp/numpy-vbench/vb_vb_io.html#strided-assign-complex64

summary table generation is not yet adjusted for this multi-branch
changes, so there might be misleading results there:
http://www.onerussian.com/tmp/numpy-vbench/index.html#benchmarks-performance-analysis

Cheers,
-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate,     Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        


From joferkington at gmail.com  Thu Aug  1 13:24:11 2013
From: joferkington at gmail.com (Joe Kington)
Date: Thu, 1 Aug 2013 12:24:11 -0500
Subject: [Numpy-discussion] Fwd: Python Session at AGU 2013
Message-ID: <CACe1pJfoUKs=Zpx8X-HY8+QqC5Z1JKbT4L9oVErhBkrUeN1hMQ@mail.gmail.com>

For anyone attending the AGU (American Geophysical Union) fall meeting this
year, there will be a session on python and "big data" in the earth
sciences. Abstract submission is still open until Aug. 6th. See below for
more info.

Cheers,
-Joe

---------- Forwarded message ----------
From: IRIS Webmaster <webmaster at iris.washington.edu>
Date: Thu, Aug 1, 2013 at 11:18 AM
Subject: [iris-bulk] Python Session at AGU 2013
To: bulkmail at iris.washington.edu


Forwarded on behalf of:
Lion Krischer
LMU Munich
krischer at geophysik.uni-muenchen.de


Dear members of the IRIS community,

with the deadline for abstract submission to the AGU Fall Meeting 2013
approaching fast, I wanted to point out a session revolving around the
Python programming language.

If you will be attending the meeting and are using Python for your research
or workflows, please consider submitting an abstract to the IN-034 session
until *next week Tuesday, August 6th*.

https://fallmeeting.agu.org/2013/scientific-program/session-search/sessions/in034-ultra-scale-earth-systems-analyses-using-python/

It aims to promote the use of Python in the earth-science community.

All the best,

Lion Krischer and Thomas Lecocq
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130801/eb33a2c0/attachment.html>

From mdroe at stsci.edu  Thu Aug  1 14:06:35 2013
From: mdroe at stsci.edu (Michael Droettboom)
Date: Thu, 1 Aug 2013 14:06:35 -0400
Subject: [Numpy-discussion] ANN: matplotlib 1.3.0 released
Message-ID: <51FAA3AB.6020506@stsci.edu>

On behalf of a veritable army of super coders, I'm pleased to announce 
the release of matplotlib 1.3.0.


  Downloads

Downloads are available here:

<http://matplotlib.org/downloads.html>http://matplotlib.org/downloads.html

as well as through |pip|. Check with your distro for when matplotlib 
1.3.0 will become packaged for your environment.

(Note: Mac .dmg installers are still forthcoming due to some issues with 
the new installation approach.)


  Important known issues

matplotlib no longer ships with its Python dependencies, including 
dateutil, pytz, pyparsing and six. When installing from source or |pip|, 
|pip| will install these for you automatically. When installing from 
packages (on Linux distributions, MacPorts, homebrew etc.) these 
dependencies should also be handled automatically. The Windows binary 
installers do not include or install these dependencies.

You may need to remove any old matplotlib installations before 
installing 1.3.0 to ensure matplotlib has access to the latest versions 
of these dependencies.

The following backends have been removed: QtAgg (Qt version 3.x only), 
FlktAgg and Emf.

For a complete list of removed features, see 
<http://matplotlib.org/api/api_changes.html#changes-in-1-3>http://matplotlib.org/api/api_changes.html#changes-in-1-3


  What's new

  * xkcd-style sketch plotting
  * webagg backend for displaying and interacting with plots in a web
    browser
  * event plots
  * triangular grid interpolation
  * control of baselines in stackplot
  * many improvements to text and color handling

For a complete list of what's new, see

<http://matplotlib.org/users/whats_new.html#new-in-matplotlib-1-3>http://matplotlib.org/users/whats_new.html#new-in-matplotlib-1-3

Have fun, and enjoy matplotlib!

Michael Droettboom

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130801/4bb8d2b8/attachment.html>

From rowen at uw.edu  Thu Aug  1 14:53:55 2013
From: rowen at uw.edu (Russell E. Owen)
Date: Thu, 01 Aug 2013 11:53:55 -0700
Subject: [Numpy-discussion] ANN: matplotlib 1.3.0 released
References: <51FAA3AB.6020506@stsci.edu>
Message-ID: <rowen-5BF9EC.11535401082013@news.gmane.org>

In article <51FAA3AB.6020506 at stsci.edu>,
 Michael Droettboom <mdroe at stsci.edu> wrote:

> On behalf of a veritable army of super coders, I'm pleased to announce 
> the release of matplotlib 1.3.0.
> 
> 
>   Downloads
> 
> Downloads are available here:
> 
> <http://matplotlib.org/downloads.html>http://matplotlib.org/downloads.html
> 
> as well as through |pip|. Check with your distro for when matplotlib 
> 1.3.0 will become packaged for your environment.
> 
> (Note: Mac .dmg installers are still forthcoming due to some issues with 
> the new installation approach.)
> 
> 
>   Important known issues
> 
> matplotlib no longer ships with its Python dependencies, including 
> dateutil, pytz, pyparsing and six. When installing from source or |pip|, 
> |pip| will install these for you automatically. When installing from 
> packages (on Linux distributions, MacPorts, homebrew etc.) these 
> dependencies should also be handled automatically. The Windows binary 
> installers do not include or install these dependencies.

An unofficial Mac binary is available from here:
<http://www.astro.washington.edu/users/rowen/python/>

Known issues:
- This may break existing installations of pytz and python-dateutil 
(especially if those were installed by the matplotlib 1.2.1 Mac binary 
installer). For safety, reinstall those after installing matplotlib.
- Like the Windows binaries, it does not include pytz, python-dateutil, 
six or pyparsing. You will have to install those manually (e.g. with pip 
or easy_install).
- Much of the test code is missing, for unknown reasons. Thus I was not 
able to run most of its unit tests.

So...use at your own risk. At this point I have no idea if or when there 
will be an official Mac binary installer. I'm afraid I don't have time 
to track down the issues right now.

-- Russell



From cwg at falma.de  Thu Aug  1 18:34:11 2013
From: cwg at falma.de (Christoph Groth)
Date: Fri, 02 Aug 2013 00:34:11 +0200
Subject: [Numpy-discussion] Announcing tinyarray
Message-ID: <8738qtvy58.fsf@falma.de>

Hello,

Almost two years ago a discussion on this list took place under the
subject "speeding up operations on small vectors".  The issue was the
rather high constant time and space overhead of small NumPy arrays.  I'm
aware that in general one should combine small arrays into large ones,
but there are cases where this is not feasible.

So, today, I'm pleased to announce a Python package that provides an
alternative for those who would like or have to use many small arrays of
numbers in Python: Tinyarray

http://kwant-project.org/tinyarray/
https://pypi.python.org/pypi/tinyarray/

"""
Tinyarray is a numerical array module for Python.  The multi-dimensional
arrays it provides are best thought of as (possibly nested) tuples of
numbers that, unlike Python's built-in tuples, support mathematical
operations.  Like tuples, tinyarrays are hashable and immutable and thus
can be used as dictionary keys.  The module's interface is a subset of
that of NumPy and hence should be familiar to many Python programmers.
Tinyarray has been heavily optimized for small arrays: For example,
common operations on 1-d arrays of length 3 run up to 35 times faster
than with NumPy.  When storing many small arrays, memory consumption is
reduced by a factor of 3.  In summary, Tinyarray is a more efficient
alternative to NumPy when many separate small numerical arrays are to be
used.
"""

I will be happy to hear comments and suggestions, and I will gladly
accept useful patches.  As I will be traveling the next days, I may not
be very responsive.

Best,
Christoph



From pav at iki.fi  Fri Aug  2 13:58:44 2013
From: pav at iki.fi (Pauli Virtanen)
Date: Fri, 02 Aug 2013 20:58:44 +0300
Subject: [Numpy-discussion] unit tests / developing numpy
In-Reply-To: <FDC9CF5C-4C36-4DE9-911B-77254073DC18@skogoglandskap.no>
References: <mailman.11.1374858002.25379.numpy-discussion@scipy.org>
	<FDC9CF5C-4C36-4DE9-911B-77254073DC18@skogoglandskap.no>
Message-ID: <ktgs0d$ke0$1@ger.gmane.org>

Hi,

29.07.2013 10:43, Graeme B. Bell kirjoitti:
[clip]
> $ ls build/testenv/
> bin  lib64
>
[clip]
>>>> import os
>>>> from distutils.sysconfig import get_python_lib
>>>> get_python_lib(prefix=os.path.abspath('build/testenv'))
>
> '/ssd-space/home/X/github/numpy/build/testenv/lib/python2.7/site-packages'
>
> I'm running this in a fresh python client in the directory 'github/numpy'.

Thanks, the issue seems to be due to some patch Fedora applied to its 
Python version. The fix seems to be to change on line 234 in runtests.py 
the line

      site_dir = get_python_lib(prefix=dst_dir)

to

      site_dir = get_python_lib(prefix=dst_dir, plat_specific=True)


-- 
Pauli Virtanen



From jaime.frio at gmail.com  Mon Aug  5 01:49:20 2013
From: jaime.frio at gmail.com (=?ISO-8859-1?Q?Jaime_Fern=E1ndez_del_R=EDo?=)
Date: Sun, 4 Aug 2013 22:49:20 -0700
Subject: [Numpy-discussion] Questions on GUFUNCS
Message-ID: <CAPOWHWkBywL4CvRGGZor1Xg4WwK0zfkbQc+Pcyut1CgUjB5CYg@mail.gmail.com>

Hi!

I have spent the last couple of weeks playing around with GUFUNCS, and am
literally blown away by the power that a C compiler and NumPy put at the
tip of my fingers! I still have many questions, but the following ones are
the most pressing in my current state of amazed ignorance:

1. **The data argument to the GUFUNC loop function** Where in the source
code can I find example of it's use? I remember reading that many UFUNCS
are just the same looping, with a pointer to a different function passed in
this argument. I think I understand the idea, but would like to see it
implemented.

2. **Is there a place for initializations?** I have a GUFNC in mind that
will need, in each iteration, a small dynamically allocated array, which
can be reused by other iterations. Rather than malloc-ing it for every
loop, I am thinking of wrapping the GUFUNC in a Python interface that
creates a numpy array for this, and passes it down as another parameter. Is
there an easy way to keep this all bundled in the C code, e.g. by defining
some initialization code to be run before doing any looping?

3. **What happens with missing types?** Say I only register a function that
takes unsigned ints. What happens if I try to call it with ints? I figure
it will complain, but not if it is the other way around and the type can be
safely casted. Is that really so? Can this behavior be in any way
configured? Is it documented anywhere?

4. **.src files** The prime example of GUFUNC implementation I have found
is "umath_linalg.c.src" Correct me if I am wrong, but this is simply C code
plus the special syntax to generate multiple versions of the same functions
changing only a few types and names, using the @tag@ syntax. While the way
it works seems clear, I had never seen this done like this before. Is this
standard or just a numpy thing? How do you parse and expand this code to
its full glory before compiling?

Thanks!

Jaime

-- 
(\__/)
( O.o)
( > <) Este es Conejo. Copia a Conejo en tu firma y ay?dale en sus planes
de dominaci?n mundial.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130804/ed3ec110/attachment.html>

From grb at skogoglandskap.no  Mon Aug  5 04:15:10 2013
From: grb at skogoglandskap.no (Graeme B. Bell)
Date: Mon, 5 Aug 2013 08:15:10 +0000
Subject: [Numpy-discussion] NumPy-Discussion Digest, Vol 83, Issue 3
In-Reply-To: <mailman.11.1375549202.20028.numpy-discussion@scipy.org>
References: <mailman.11.1375549202.20028.numpy-discussion@scipy.org>
Message-ID: <418A2B25-86A7-41B8-AA4F-CD9E30697F04@skogoglandskap.no>


Pauli,

Thanks very much for investigating this and fixing it. Your patch works perfectly for me. :) 

Runtests.py now automatically uses "NumPy version 1.8.0.dev-af12c09". 
The only comment I have is that the line number patched was different on my copy (line 192).

Perhaps we can add  "run  'python runtest.py'" to the development workflow page, to guide new contributors? 
http://docs.scipy.org/doc/numpy/dev/gitwash/development_workflow.html

Graeme. 

> Thanks, the issue seems to be due to some patch Fedora applied to its 
> Python version. The fix seems to be to change on line 234 in runtests.py 
> the line
> 
>      site_dir = get_python_lib(prefix=dst_dir)
> 
> to
> 
>      site_dir = get_python_lib(prefix=dst_dir, plat_specific=True)



From jaakko.luttinen at aalto.fi  Mon Aug  5 05:53:27 2013
From: jaakko.luttinen at aalto.fi (Jaakko Luttinen)
Date: Mon, 5 Aug 2013 12:53:27 +0300
Subject: [Numpy-discussion] Installation bug on Python 3.3?
Message-ID: <51FF7617.2000309@aalto.fi>

Hi,

I'm trying to install NumPy 1.7.1 for Python 3.3 using

    pip install numpy

However, I get the following error after a while:

    error: numpy.egg-info/dependency_links.txt: Operation not supported

Is this a bug or am I doing something wrong? If it matters, I'm using
virtualenv as I do not have root permission on this computer.

Thanks for any help!
Jaakko

PS. The end of the error log:
----------------------------------------

Command /home/jluttine/.virtualenvs/bayespy-3.3/bin/python3.3 -c "import
setuptools;__file__='/home/jluttine/.virtualenvs/bayespy-3.3/build/numpy/setup.py';exec(compile(open(__file__).read().replace('\r\n',
'\n'), __file__, 'exec'))" install --record
/tmp/pip-309wd8-record/install-record.txt
--single-version-externally-managed --install-headers
/home/jluttine/.virtualenvs/bayespy-3.3/include/site/python3.3 failed
with error code 1 in /home/jluttine/.virtualenvs/bayespy-3.3/build/numpy

Exception information:
Traceback (most recent call last):
  File
"/home/jluttine/.virtualenvs/bayespy-3.3/lib/python3.3/site-packages/pip-1.2.1-py3.3.egg/pip/basecommand.py",
line 107, in main
    status = self.run(options, args)
  File
"/home/jluttine/.virtualenvs/bayespy-3.3/lib/python3.3/site-packages/pip-1.2.1-py3.3.egg/pip/commands/install.py",
line 261, in run
    requirement_set.install(install_options, global_options)
  File
"/home/jluttine/.virtualenvs/bayespy-3.3/lib/python3.3/site-packages/pip-1.2.1-py3.3.egg/pip/req.py",
line 1166, in install
    requirement.install(install_options, global_options)
  File
"/home/jluttine/.virtualenvs/bayespy-3.3/lib/python3.3/site-packages/pip-1.2.1-py3.3.egg/pip/req.py",
line 589, in install
    cwd=self.source_dir, filter_stdout=self._filter_install,
show_stdout=False)
  File
"/home/jluttine/.virtualenvs/bayespy-3.3/lib/python3.3/site-packages/pip-1.2.1-py3.3.egg/pip/util.py",
line 612, in call_subprocess
    % (command_desc, proc.returncode, cwd))
pip.exceptions.InstallationError: Command
/home/jluttine/.virtualenvs/bayespy-3.3/bin/python3.3 -c "import
setuptools;__file__='/home/jluttine/.virtualenvs/bayespy-3.3/build/numpy/setup.py';exec(compile(open(__file__).read().replace('\r\n',
'\n'), __file__, 'exec'))" install --record
/tmp/pip-309wd8-record/install-record.txt
--single-version-externally-managed --install-headers
/home/jluttine/.virtualenvs/bayespy-3.3/include/site/python3.3 failed
with error code 1 in /home/jluttine/.virtualenvs/bayespy-3.3/build/numpy


From jaakko.luttinen at aalto.fi  Mon Aug  5 09:09:18 2013
From: jaakko.luttinen at aalto.fi (Jaakko Luttinen)
Date: Mon, 5 Aug 2013 16:09:18 +0300
Subject: [Numpy-discussion] Installation bug on Python 3.3?
In-Reply-To: <51FF7617.2000309@aalto.fi>
References: <51FF7617.2000309@aalto.fi>
Message-ID: <51FFA3FE.3060506@aalto.fi>

I was able to install by downloading the package for version 1.7.1 from
github and then running

    python3.3 setup.py install

No errors given. So, the problem might be related to pip and the fact
that python3.3 is installed locally in my personal home folder which is
in a different filesystem than the other python versions. So it might be
because of some hardlinks over different partitions or something similar.

Anyway, it seems that I have the same problem installing other python
packages too, so it is not related to numpy but either python 3.3, pip,
virtualenv or my system set up.

But still, any help is appreciated.

-Jaakko

On 08/05/2013 12:53 PM, Jaakko Luttinen wrote:
> Hi,
> 
> I'm trying to install NumPy 1.7.1 for Python 3.3 using
> 
>     pip install numpy
> 
> However, I get the following error after a while:
> 
>     error: numpy.egg-info/dependency_links.txt: Operation not supported
> 
> Is this a bug or am I doing something wrong? If it matters, I'm using
> virtualenv as I do not have root permission on this computer.
> 
> Thanks for any help!
> Jaakko
> 
> PS. The end of the error log:
> ----------------------------------------
> 
> Command /home/jluttine/.virtualenvs/bayespy-3.3/bin/python3.3 -c "import
> setuptools;__file__='/home/jluttine/.virtualenvs/bayespy-3.3/build/numpy/setup.py';exec(compile(open(__file__).read().replace('\r\n',
> '\n'), __file__, 'exec'))" install --record
> /tmp/pip-309wd8-record/install-record.txt
> --single-version-externally-managed --install-headers
> /home/jluttine/.virtualenvs/bayespy-3.3/include/site/python3.3 failed
> with error code 1 in /home/jluttine/.virtualenvs/bayespy-3.3/build/numpy
> 
> Exception information:
> Traceback (most recent call last):
>   File
> "/home/jluttine/.virtualenvs/bayespy-3.3/lib/python3.3/site-packages/pip-1.2.1-py3.3.egg/pip/basecommand.py",
> line 107, in main
>     status = self.run(options, args)
>   File
> "/home/jluttine/.virtualenvs/bayespy-3.3/lib/python3.3/site-packages/pip-1.2.1-py3.3.egg/pip/commands/install.py",
> line 261, in run
>     requirement_set.install(install_options, global_options)
>   File
> "/home/jluttine/.virtualenvs/bayespy-3.3/lib/python3.3/site-packages/pip-1.2.1-py3.3.egg/pip/req.py",
> line 1166, in install
>     requirement.install(install_options, global_options)
>   File
> "/home/jluttine/.virtualenvs/bayespy-3.3/lib/python3.3/site-packages/pip-1.2.1-py3.3.egg/pip/req.py",
> line 589, in install
>     cwd=self.source_dir, filter_stdout=self._filter_install,
> show_stdout=False)
>   File
> "/home/jluttine/.virtualenvs/bayespy-3.3/lib/python3.3/site-packages/pip-1.2.1-py3.3.egg/pip/util.py",
> line 612, in call_subprocess
>     % (command_desc, proc.returncode, cwd))
> pip.exceptions.InstallationError: Command
> /home/jluttine/.virtualenvs/bayespy-3.3/bin/python3.3 -c "import
> setuptools;__file__='/home/jluttine/.virtualenvs/bayespy-3.3/build/numpy/setup.py';exec(compile(open(__file__).read().replace('\r\n',
> '\n'), __file__, 'exec'))" install --record
> /tmp/pip-309wd8-record/install-record.txt
> --single-version-externally-managed --install-headers
> /home/jluttine/.virtualenvs/bayespy-3.3/include/site/python3.3 failed
> with error code 1 in /home/jluttine/.virtualenvs/bayespy-3.3/build/numpy
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 



From opensource.numpy at user.fastmail.fm  Mon Aug  5 13:09:04 2013
From: opensource.numpy at user.fastmail.fm (Hugo Gagnon)
Date: Mon, 05 Aug 2013 13:09:04 -0400
Subject: [Numpy-discussion] distutils seems to pick the wrong fortran
	compiler flags
Message-ID: <1375722544.30925.6147807.2DD2CDBE@webmail.messagingengine.com>

Hi,

I'm using f2py shipping with EPD 32 bit on 64 bit Mac OS 10.8.  The
command "f2py -c -m plot3d --fcompiler=gnu95 plot3d.f90" compiles the
objects files in 32 bit, which is good, but fails at the linkage step
with a "file was built for i386 which is not the architecture being
linked (x86_64)" warning.  See this post for a full output:
http://stackoverflow.com/questions/18003399/wrapping-32-bit-libraries-with-f2py-gfortran-on-mac-os-10-8

My solution was to hack numpy/distutils/fcompiler/gnu.py by explicitly
adding the "-m32" flag to the "compiler_f77", "compiler_f90" and
"linker_so" keys of the "executables" dictionary of the "Gnu95FCompiler"
class (line 251).  I'm sure there's a nicer way to do that though...

-- 
  Hugo Gagnon


From charlesr.harris at gmail.com  Mon Aug  5 13:16:37 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 5 Aug 2013 11:16:37 -0600
Subject: [Numpy-discussion] Questions on GUFUNCS
In-Reply-To: <CAPOWHWkBywL4CvRGGZor1Xg4WwK0zfkbQc+Pcyut1CgUjB5CYg@mail.gmail.com>
References: <CAPOWHWkBywL4CvRGGZor1Xg4WwK0zfkbQc+Pcyut1CgUjB5CYg@mail.gmail.com>
Message-ID: <CAB6mnxJk0XiiJ=W8ooxGxfhvD071DXosw06=WPMb+aMcyn48_A@mail.gmail.com>

On Sun, Aug 4, 2013 at 11:49 PM, Jaime Fern?ndez del R?o <
jaime.frio at gmail.com> wrote:

> Hi!
>
> I have spent the last couple of weeks playing around with GUFUNCS, and am
> literally blown away by the power that a C compiler and NumPy put at the
> tip of my fingers! I still have many questions, but the following ones are
> the most pressing in my current state of amazed ignorance:
>
> 1. **The data argument to the GUFUNC loop function** Where in the source
> code can I find example of it's use? I remember reading that many UFUNCS
> are just the same looping, with a pointer to a different function passed in
> this argument. I think I understand the idea, but would like to see it
> implemented.
>
> 2. **Is there a place for initializations?** I have a GUFNC in mind that
> will need, in each iteration, a small dynamically allocated array, which
> can be reused by other iterations. Rather than malloc-ing it for every
> loop, I am thinking of wrapping the GUFUNC in a Python interface that
> creates a numpy array for this, and passes it down as another parameter. Is
> there an easy way to keep this all bundled in the C code, e.g. by defining
> some initialization code to be run before doing any looping?
>
> 3. **What happens with missing types?** Say I only register a function
> that takes unsigned ints. What happens if I try to call it with ints? I
> figure it will complain, but not if it is the other way around and the type
> can be safely casted. Is that really so? Can this behavior be in any way
> configured? Is it documented anywhere?
>
> 4. **.src files** The prime example of GUFUNC implementation I have found
> is "umath_linalg.c.src" Correct me if I am wrong, but this is simply C code
> plus the special syntax to generate multiple versions of the same functions
> changing only a few types and names, using the @tag@ syntax. While the
> way it works seems clear, I had never seen this done like this before. Is
> this standard or just a numpy thing? How do you parse and expand this code
> to its full glory before compiling?
>

It's a numpy thing. The code can be expanded like so if you are working in
the repository:
```
$ python numpy/distutils/conv_template.py <filename>
```

Chuck

>
> Thanks!
>
> Jaime
>
> --
> (\__/)
> ( O.o)
> ( > <) Este es Conejo. Copia a Conejo en tu firma y ay?dale en sus planes
> de dominaci?n mundial.
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130805/e9fd80be/attachment.html>

From raul at virtualmaterials.com  Mon Aug  5 16:00:16 2013
From: raul at virtualmaterials.com (Raul Cota)
Date: Mon, 05 Aug 2013 14:00:16 -0600
Subject: [Numpy-discussion] Problems building from source
Message-ID: <52000450.2@virtualmaterials.com>

Hello,

I had not updated my code for a few months. I updated today to the 
latest source and I cannot build anymore,
(Windows, Python 2.6)


When I do,
python setup.py build


I get,

"""
Running from numpy source directory.
Traceback (most recent call last):
   File "setup.py", line 192, in <module>
     setup_package()
   File "setup.py", line 169, in setup_package
     from numpy.distutils.core import setup
   File 
"C:\Users\Raul\Documents\GitHub\numpy\numpy\distutils\__init__.py", line
37, in <module>
     from numpy.testing import Tester
   File 
"C:\Users\Raul\Documents\GitHub\numpy\numpy\testing\__init__.py", line 13
, in <module>
     from .utils import *
   File "C:\Users\Raul\Documents\GitHub\numpy\numpy\testing\utils.py", 
line 13, i
n <module>
     from numpy.core import float32, empty, arange
   File "C:\Users\Raul\Documents\GitHub\numpy\numpy\core\__init__.py", 
line 6, in
  <module>
     from . import multiarray
ImportError: cannot import name multiarray

"""

I can see
numpy\numpy\core\__init__.py

changed the imports to be 'relative' based from when it used to work.

Any hints or reference to a related discussion ? All I found was pr 3178 
"2to3 apply import fixer"


Raul





From charlesr.harris at gmail.com  Mon Aug  5 16:17:41 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 5 Aug 2013 14:17:41 -0600
Subject: [Numpy-discussion] Problems building from source
In-Reply-To: <52000450.2@virtualmaterials.com>
References: <52000450.2@virtualmaterials.com>
Message-ID: <CAB6mnxLp7-sLN=CxkOsyWsHmLv61DSv_=fu1jeg2xXYHp5WrBg@mail.gmail.com>

On Mon, Aug 5, 2013 at 2:00 PM, Raul Cota <raul at virtualmaterials.com> wrote:

> Hello,
>
> I had not updated my code for a few months. I updated today to the
> latest source and I cannot build anymore,
> (Windows, Python 2.6)
>
>
> When I do,
> python setup.py build
>
>
> I get,
>
> """
> Running from numpy source directory.
> Traceback (most recent call last):
>    File "setup.py", line 192, in <module>
>      setup_package()
>    File "setup.py", line 169, in setup_package
>      from numpy.distutils.core import setup
>    File
> "C:\Users\Raul\Documents\GitHub\numpy\numpy\distutils\__init__.py", line
> 37, in <module>
>      from numpy.testing import Tester
>    File
> "C:\Users\Raul\Documents\GitHub\numpy\numpy\testing\__init__.py", line 13
> , in <module>
>      from .utils import *
>    File "C:\Users\Raul\Documents\GitHub\numpy\numpy\testing\utils.py",
> line 13, i
> n <module>
>      from numpy.core import float32, empty, arange
>    File "C:\Users\Raul\Documents\GitHub\numpy\numpy\core\__init__.py",
> line 6, in
>   <module>
>      from . import multiarray
> ImportError: cannot import name multiarray
>
> """
>
> I can see
> numpy\numpy\core\__init__.py
>
> changed the imports to be 'relative' based from when it used to work.
>
> Any hints or reference to a related discussion ? All I found was pr 3178
> "2to3 apply import fixer"
>

I don't think anyone else has seen this problem, although I don't know who
else uses window. Have you checked with a clean build and clean install?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130805/2fa7b2a6/attachment.html>

From raul at virtualmaterials.com  Mon Aug  5 16:35:26 2013
From: raul at virtualmaterials.com (Raul Cota)
Date: Mon, 05 Aug 2013 14:35:26 -0600
Subject: [Numpy-discussion] Problems building from source
In-Reply-To: <CAB6mnxLp7-sLN=CxkOsyWsHmLv61DSv_=fu1jeg2xXYHp5WrBg@mail.gmail.com>
References: <52000450.2@virtualmaterials.com>
	<CAB6mnxLp7-sLN=CxkOsyWsHmLv61DSv_=fu1jeg2xXYHp5WrBg@mail.gmail.com>
Message-ID: <52000C8E.5090308@virtualmaterials.com>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130805/1c74650c/attachment.html>

From aron at ahmadia.net  Mon Aug  5 17:06:54 2013
From: aron at ahmadia.net (Aron Ahmadia)
Date: Mon, 5 Aug 2013 16:06:54 -0500
Subject: [Numpy-discussion] Correct way to query NumPy for linktime BLAS and
	LAPACK
Message-ID: <CAPhiW4gOR2i4TvdxXg0-SJPkSiT+eAuh8zc_2CA7HM2WbBic=A@mail.gmail.com>

Dear NumPy Developers,

In the Clawpack/* repositories [1], we use a mixture of Fortran and Python
source, currently glued together using f2py.

Occasionally, we'll need to link the Fortran code directly against LAPACK.
 In particular, we're using dgeev and dgesv to solve several different
Riemann problems [2,3].

In the past, we've relied on either the operating system or the user to
provide these link commands for us, but it would be ideal in the future if
we could query NumPy for how it is linked.  Currently, the only information
I can find is in the hidden __config__ module of NumPy's distutils module:

    numpy.distutils.__config__.blas_opt_info['extra_link_args']
    numpy.distutils.__config__.lapack_opt_info['extra_link_args']

This seems to suggest that we shouldn't be relying on this information
being available in future versions of NumPy (or at least, not in this
location).  That said, we'd still probably like to use this to avoid the
possibility of multiple BLAS/LAPACK libraries being linked in to our builds.

Any comments?

Thanks,
Aron

[1] https://github.com/clawpack/clawpack
[2]
https://github.com/clawpack/riemann/blob/master/src/rp1_layered_shallow_water.f90#L687
[3]
https://github.com/clawpack/riemann/blob/master/src/rpn2_layered_shallow_water.f90#L478
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130805/6e7cc883/attachment.html>

From nouiz at nouiz.org  Tue Aug  6 08:57:36 2013
From: nouiz at nouiz.org (=?ISO-8859-1?Q?Fr=E9d=E9ric_Bastien?=)
Date: Tue, 6 Aug 2013 08:57:36 -0400
Subject: [Numpy-discussion] Correct way to query NumPy for linktime BLAS
 and LAPACK
In-Reply-To: <CAPhiW4gOR2i4TvdxXg0-SJPkSiT+eAuh8zc_2CA7HM2WbBic=A@mail.gmail.com>
References: <CAPhiW4gOR2i4TvdxXg0-SJPkSiT+eAuh8zc_2CA7HM2WbBic=A@mail.gmail.com>
Message-ID: <CADKKbtjJ8LuTL27Ty0MW6sQaBELov-RFbyjCvTfjgAjv5ZNUpA@mail.gmail.com>

Hi,

In Theano, we use the information in this
dictionnary: numpy.distutils.__config__.blas_opt_info. We do this for a few
years already, so I don't know how much future proof it is, but I would
expect that they aren't going to change this shortly.

We use this dict for the default configuration, but still we allow the user
to provide its own library and it work well. In case you don't know Theano,
it is a compiler that generate dynamically c code, compile them as python
module and load them in the python interpreter. So it happen that numpy and
Theano module use different version of BLAS. Up to now, I never heard a
problem about this.

Don't forget that many different BLAS version use different internal symbol
for the BLAS function and just provide an official function for the
interface. So if we mix different BLAS, it work. But I'm not sure if what
will happen if we link with different version of the same BLAS project,
like different MKL version. Maybe just on the them will get imported if the
library name is the same.

HTH

Fred


On Mon, Aug 5, 2013 at 5:06 PM, Aron Ahmadia <aron at ahmadia.net> wrote:

> Dear NumPy Developers,
>
> In the Clawpack/* repositories [1], we use a mixture of Fortran and Python
> source, currently glued together using f2py.
>
> Occasionally, we'll need to link the Fortran code directly against LAPACK.
>  In particular, we're using dgeev and dgesv to solve several different
> Riemann problems [2,3].
>
> In the past, we've relied on either the operating system or the user to
> provide these link commands for us, but it would be ideal in the future if
> we could query NumPy for how it is linked.  Currently, the only information
> I can find is in the hidden __config__ module of NumPy's distutils module:
>
>     numpy.distutils.__config__.blas_opt_info['extra_link_args']
>     numpy.distutils.__config__.lapack_opt_info['extra_link_args']
>
> This seems to suggest that we shouldn't be relying on this information
> being available in future versions of NumPy (or at least, not in this
> location).  That said, we'd still probably like to use this to avoid the
> possibility of multiple BLAS/LAPACK libraries being linked in to our builds.
>
> Any comments?
>
> Thanks,
> Aron
>
> [1] https://github.com/clawpack/clawpack
> [2]
> https://github.com/clawpack/riemann/blob/master/src/rp1_layered_shallow_water.f90#L687
> [3]
> https://github.com/clawpack/riemann/blob/master/src/rpn2_layered_shallow_water.f90#L478
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130806/597075a8/attachment.html>

From jaime.frio at gmail.com  Tue Aug  6 18:19:02 2013
From: jaime.frio at gmail.com (=?ISO-8859-1?Q?Jaime_Fern=E1ndez_del_R=EDo?=)
Date: Tue, 6 Aug 2013 15:19:02 -0700
Subject: [Numpy-discussion] Bug in gufuncs affecting umath_linalg
Message-ID: <CAPOWHWmtUdWqk1us-e06Zt3Y9YHmXsvNRxGRbLa5mh+CaBp0eQ@mail.gmail.com>

Hi,

I think I have found an "undocumented feature" of the gufuncs machinery. I
have filed a bug report:

https://github.com/numpy/numpy/issues/3582

Some more background on what i am seeing...

I have coded a gufunc with signature '(r,c,p),(g,g,g,q)->(r,c,q)'. It is a
color map, i.e. a transformation of a 3-dimensional array of p channels
(with p=3, an RGB image of r rows and c columns), into a 3-dimensional
array of q channels (with q=4, a CMYK image of the same size), via a
p-dimensional look-up-table (LUT).

For all practical purposes, the LUT always has the first three dimensions
identical, hence the repeated g's in the signature.

The function registered with this signature, receives the expected values
in the dimensions argument: 'n', 'r', 'c', 'p', 'g', 'q', with 'n' being
the length of the gufunc loop.

But there is a problem with the steps argument. As expected I get a 13 item
long array: 3 main loop strides, 3 strides (r, c, p) for the first
argument, 4 strides (g, g, g, q) for the second, and 3 strides (r, c, q)
for the return. Everything is OK except for the strides for the repeating
'g's: instead of getting three different stride values, the first two are
the same as the last. This does not happen if I modify the signature to be
'(r,c,p),(i,j,k,q)->(r,c,q)', which is the workaround I am implementing in
my code for the time being. I have also managed to repeat the behavior in
repeated dimensions on other arguments, e.g. '(r,r,p),(i,j,k,q)->(r,c,p)'
shows the same issue for the strides of the first argument.

I have seen that the gufunc version of umath_linalg makes use of a similar,
repeated index scheme, e.g. in the 'solve' and 'det' gufuncs. At least for
these two, the tests in place do not catch the error. For solve, the tests
run these two cases (the results below come from the traditional linalg, as
I am running numpy 1.7.1):

>>> np.linalg.solve([[1, 2], [3, 4]], [[4, 3], [2, 1]])
array([[-6., -5.],
       [ 5.,  4.]])
>>> np.linalg.solve([[1+2j,2+3j], [3+4j,4+5j]], [[4+3j,3+2j], [2+1j,1+0j]])
array([[-6. +0.00000000e+00j, -5. +0.00000000e+00j],
       [ 5. -3.46944695e-16j,  4. -3.46944695e-16j]])

But because of their highly strucutred nature, these particular test cases
give the same result if you get the strides wrong (!!!):

>>> np.linalg.solve([[1, 2], [2, 3]], [[4, 3], [3, 2]])
array([[-6., -5.],
       [ 5.,  4.]])
>>> np.linalg.solve([[1+2j,2+3j], [2+3j,3+4j]], [[4+3j,3+2j], [3+2j,2+1j]])
array([[-6. -1.09314267e-15j, -5. -1.09314267e-15j],
       [ 5. +1.08246745e-15j,  4. +1.08246745e-15j]])

As for the determinant, no abolute check of the return value is performed:
the return of 'det' is compared to the product of the return of 'eigvals',
which also has the '(m, m)' signature, and interprets the data equally
wrong.

For my particular issue, I am simply going to register the gufunc with
non-repeating dimensions, check for equality in a Python wrapper, and
discard the repeated values in my C code. Not sure what is the best way of
going about umath_linalg. Probably better to fix the issue in the gufunc
machinery than to patch umath_lnalg.

If there's any way, other than reporting it, in which I can help getting
this fixed, I'll be more than happy to do it. But for this job I am clearly
unqualified labor, and would need to work under someone else's command.

Regards,

Jaime

-- 
(\__/)
( O.o)
( > <) Este es Conejo. Copia a Conejo en tu firma y ay?dale en sus planes
de dominaci?n mundial.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130806/3c7bc6f6/attachment.html>

From dalke at dalkescientific.com  Tue Aug  6 22:32:29 2013
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Wed, 7 Aug 2013 04:32:29 +0200
Subject: [Numpy-discussion] chebyshev, laguerre,
	... exec() template code at each import
Message-ID: <D37A1790-EB87-4EA4-8D1F-D890199F4CA7@dalkescientific.com>

Hi all,

  I mostly develop software related cheminformatics. There
isn't much direct overlap between the tools of that field
and NumPy and SciPy provide, but it's increasing with the
use of scikit-learn and pandas.

  I tend to write command-line tools which indirectly
import numpy. I've noticed that 25% of the "import numpy"
cost of about 0.081 seconds is due to the chebyshev, laguerre,
legendre, hermite_e, and hermite_e modules. Each module
takes about 0.004 seconds to import.

  This is because each of them does a:

exec(polytemplate.substitute(name='Chebyshev', nick='cheb', domain='[-1,1]'))

during import. It appears that *everyone* takes a 0.02 second
overhead during "import numpy" in order to simplify maintenance.
This balance doesn't seem correct, given the number of people
who use numpy vs. how rarely the polytemplate changes.


Last year I submitted a patch which pre-computed all of those
templates, so they would only be byte-compiled once.


I knew (and still know) almost nothing about git/github, so
Scott Sinclair kindly took it up and made it a pull request at:

   https://github.com/numpy/numpy/pull/334


I see that there's been no activity for at least 10 months.

Is there anything more I can do to encourage that this patch
be accepted?

Cheers,

				Andrew
				dalke at dalkescientific.com




From charlesr.harris at gmail.com  Tue Aug  6 22:37:10 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Tue, 6 Aug 2013 20:37:10 -0600
Subject: [Numpy-discussion] chebyshev, laguerre,
 ... exec() template code at each import
In-Reply-To: <D37A1790-EB87-4EA4-8D1F-D890199F4CA7@dalkescientific.com>
References: <D37A1790-EB87-4EA4-8D1F-D890199F4CA7@dalkescientific.com>
Message-ID: <CAB6mnxKqWO7H-gBV2FSwMQDJje3TXYpdZDU_-wVaJcLTgtmeqA@mail.gmail.com>

On Tue, Aug 6, 2013 at 8:32 PM, Andrew Dalke <dalke at dalkescientific.com>wrote:

> Hi all,
>
>   I mostly develop software related cheminformatics. There
> isn't much direct overlap between the tools of that field
> and NumPy and SciPy provide, but it's increasing with the
> use of scikit-learn and pandas.
>
>   I tend to write command-line tools which indirectly
> import numpy. I've noticed that 25% of the "import numpy"
> cost of about 0.081 seconds is due to the chebyshev, laguerre,
> legendre, hermite_e, and hermite_e modules. Each module
> takes about 0.004 seconds to import.
>
>   This is because each of them does a:
>
> exec(polytemplate.substitute(name='Chebyshev', nick='cheb',
> domain='[-1,1]'))
>
> during import. It appears that *everyone* takes a 0.02 second
> overhead during "import numpy" in order to simplify maintenance.
> This balance doesn't seem correct, given the number of people
> who use numpy vs. how rarely the polytemplate changes.
>
>
> Last year I submitted a patch which pre-computed all of those
> templates, so they would only be byte-compiled once.
>
>
> I knew (and still know) almost nothing about git/github, so
> Scott Sinclair kindly took it up and made it a pull request at:
>
>    https://github.com/numpy/numpy/pull/334
>
>
> I see that there's been no activity for at least 10 months.
>
> Is there anything more I can do to encourage that this patch
> be accepted?
>
>
Hi Andrew,

I haven't forgotten and intend to look at it before the next release.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130806/140f4e99/attachment.html>

From dalke at dalkescientific.com  Wed Aug  7 09:26:51 2013
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Wed, 7 Aug 2013 15:26:51 +0200
Subject: [Numpy-discussion] import overhead of numpy.testing
In-Reply-To: <CAB6mnxKqWO7H-gBV2FSwMQDJje3TXYpdZDU_-wVaJcLTgtmeqA@mail.gmail.com>
References: <D37A1790-EB87-4EA4-8D1F-D890199F4CA7@dalkescientific.com>
	<CAB6mnxKqWO7H-gBV2FSwMQDJje3TXYpdZDU_-wVaJcLTgtmeqA@mail.gmail.com>
Message-ID: <142AB120-3E54-44AC-8DCF-5470A2FC8AB3@dalkescientific.com>

On Aug 7, 2013, at 4:37 AM, Charles R Harris wrote:
> I haven't forgotten and intend to look at it before the next release.

Thanks!

On a related topic, last night I looked into deferring the
import for numpy.testing. This is the only other big place
where numpy's import overhead might be reduced without
breaking backwards compatibility.

I made a _DeferredTester [1] and replaced the 10 __init__.py
uses of:

from .testing import Tester
test = Tester().test
bench = Tester().bench

to use the _DeferredTester instead.

With that in place the "import numpy" time (best of 5)
goes from 0.0796 seconds to 0.0741 seconds, or 7%.

That 0.0796 includes the 0.02 seconds for the exec()
of the polynomial templates. Without that 0.02 seconds
in the baseline would give a 10% speedup. [2]

Would this sort of approach be acceptable to NumPy?
If so, I could improve the patch to make it be acceptable.

The outstanding code issues to be resolve before making
a pull request are:

1) The current wrapper uses *args and **kwargs to forward
   any test() and bench() calls to the actual function.
   As a result, parameter introspection doesn't work.

2) The current wrapper doesn't have a __doc__

3) The only way to fix 1) and 2) is to copy the signatures
   and docstring from the actual Tester() implementation,
   which causes code/docstring duplication.

4) I don't know if it's appropriate to add my _DeferredTester
   to numpy.core vs. some other place in the code base.


If you want to see the patch, I followed the NumPy instructions at
 http://docs.scipy.org/doc/numpy/dev/gitwash/git_development.html
and made an experimental fork at
 https://github.com/adalke/numpy/tree/no-default-tester-import

I have no git/github experience beyond what I did for this
patch, so let me know if there are problems in what I did.

Cheers,

				Andrew
				dalke at dalkescientific.com


[1]

Inside of numpy/core/__init__.py I added

class _DeferredTester(object):
   def __init__(self, package_filename):
       import os
       self._package_path = os.path.dirname(package_filename)
   def test(self, *args, **kwargs):
       from ..testing import Tester
       return Tester(self._package_path).test(*args, **kwargs)
   def bench(self, *args, **kwargs):
       from ..testing import Tester
       return Tester(self._package_path).bench(*args, **kwargs)
   def get_test_and_bench(self):
       return self.test, self.bench

It's used like this:

from ..core import _DeferredTester
test, bench = _DeferredTester(__file__).get_test_and_bench()


That's admittedly ugly. It could also be:

test = _DeferredTester(__file__).test
bench = _DeferredTester(__file__).bench


[2]

Is an import speedup (on my laptop) of 0.0055 seconds
important? I obviously think so. 

This time affects everyone who uses NumPy, even if
incidentally, as in my case. I don't actually use
NumPy, but I use a chemistry toolkit with a C++
core that imports NumPy in order to have access to
its array data structure, even though I don't make
use of that ability.

If there are 1,000,000 "import numpy"s per day, then
that's 90 minutes of savings per day.

Yes, I could also switch to an SSD and the overhead
will decrease. But on the other hand, I've also worked
on a networked file system for a cluster where
"python -c pass" took over a second to run, because
Lustre is lousy with lots of metadata requests. (See
http://www.nas.nasa.gov/hecc/support/kb/Lustre-Best-Practices_226.html )

In that case we switched to a zip importer, but you
get my point that the 0.0055 seconds is also a function
of the filesystem time, and that performance varies.



From mail.till at gmx.de  Wed Aug  7 09:52:26 2013
From: mail.till at gmx.de (Till Stensitzki)
Date: Wed, 7 Aug 2013 13:52:26 +0000 (UTC)
Subject: [Numpy-discussion] Releaseplan for 1.8?
Message-ID: <loom.20130807T155120-219@post.gmane.org>

Hi,
there already a plan to release 1.8? I would like to play around with gufuncs.

greetings 
Till



From aron at ahmadia.net  Thu Aug  8 10:42:52 2013
From: aron at ahmadia.net (Aron Ahmadia)
Date: Thu, 8 Aug 2013 09:42:52 -0500
Subject: [Numpy-discussion] Correct way to query NumPy for linktime BLAS
 and LAPACK
In-Reply-To: <CADKKbtjJ8LuTL27Ty0MW6sQaBELov-RFbyjCvTfjgAjv5ZNUpA@mail.gmail.com>
References: <CAPhiW4gOR2i4TvdxXg0-SJPkSiT+eAuh8zc_2CA7HM2WbBic=A@mail.gmail.com>
	<CADKKbtjJ8LuTL27Ty0MW6sQaBELov-RFbyjCvTfjgAjv5ZNUpA@mail.gmail.com>
Message-ID: <CAPhiW4jBHmVb6a6TSKtveCk+VbBacLggmYBT2xbLRoFQiEhFFg@mail.gmail.com>

Hi Fred,

Thanks for the feedback.  We'll try this out in Clawpack moving forward.

Regards,
Aron


On Tue, Aug 6, 2013 at 7:57 AM, Fr?d?ric Bastien <nouiz at nouiz.org> wrote:

> Hi,
>
> In Theano, we use the information in this
> dictionnary: numpy.distutils.__config__.blas_opt_info. We do this for a few
> years already, so I don't know how much future proof it is, but I would
> expect that they aren't going to change this shortly.
>
> We use this dict for the default configuration, but still we allow the
> user to provide its own library and it work well. In case you don't know
> Theano, it is a compiler that generate dynamically c code, compile them as
> python module and load them in the python interpreter. So it happen that
> numpy and Theano module use different version of BLAS. Up to now, I never
> heard a problem about this.
>
> Don't forget that many different BLAS version use different internal
> symbol for the BLAS function and just provide an official function for the
> interface. So if we mix different BLAS, it work. But I'm not sure if what
> will happen if we link with different version of the same BLAS project,
> like different MKL version. Maybe just on the them will get imported if the
> library name is the same.
>
> HTH
>
> Fred
>
>
> On Mon, Aug 5, 2013 at 5:06 PM, Aron Ahmadia <aron at ahmadia.net> wrote:
>
>> Dear NumPy Developers,
>>
>> In the Clawpack/* repositories [1], we use a mixture of Fortran and
>> Python source, currently glued together using f2py.
>>
>> Occasionally, we'll need to link the Fortran code directly against
>> LAPACK.  In particular, we're using dgeev and dgesv to solve several
>> different Riemann problems [2,3].
>>
>> In the past, we've relied on either the operating system or the user to
>> provide these link commands for us, but it would be ideal in the future if
>> we could query NumPy for how it is linked.  Currently, the only information
>> I can find is in the hidden __config__ module of NumPy's distutils module:
>>
>>     numpy.distutils.__config__.blas_opt_info['extra_link_args']
>>     numpy.distutils.__config__.lapack_opt_info['extra_link_args']
>>
>> This seems to suggest that we shouldn't be relying on this information
>> being available in future versions of NumPy (or at least, not in this
>> location).  That said, we'd still probably like to use this to avoid the
>> possibility of multiple BLAS/LAPACK libraries being linked in to our builds.
>>
>> Any comments?
>>
>> Thanks,
>> Aron
>>
>> [1] https://github.com/clawpack/clawpack
>> [2]
>> https://github.com/clawpack/riemann/blob/master/src/rp1_layered_shallow_water.f90#L687
>> [3]
>> https://github.com/clawpack/riemann/blob/master/src/rpn2_layered_shallow_water.f90#L478
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130808/37a5cb4c/attachment.html>

From fperez.net at gmail.com  Thu Aug  8 21:36:53 2013
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 8 Aug 2013 18:36:53 -0700
Subject: [Numpy-discussion] [ANN] IPython 1.0 is finally released,
	nearly 12 years in the making!
Message-ID: <CAHAreOpCpBVC4oBf_G_K4ZZ_Ni42zsZ-L2GMF0Af7tm_ChXNLw@mail.gmail.com>

Hi all,

I am incredibly thrilled, on behalf of the amazing IPython Dev Team,
to announce the official release of IPython 1.0 today, an effort
nearly 12 years in the making.  The previous version (0.13) was
released on June 30, 2012, and in this development cycle we had:

~12 months of work.
~700 pull requests merged.
~600 issues closed (non-pull requests).
contributions from ~150 authors.
~4000 commits.


# A little context

What does "1.0" mean for IPython? Obviously IPython has been a staple
of the scientific Python community for years, and we've made every
effort to make it a robust and production ready tool for a long time,
so what exactly do we mean by tagging this particular release as 1.0?
Basically, we feel that the core design of IPython, and the scope of
the project, is where we want it to be.

What we have today is what we consider a reasonably complete, design-
and scope-wise, IPython 1.0: an architecture for interactive
computing, that can drive kernels in a number of ways using a
well-defined protocol, and rich and powerful clients that let users
control those kernels effectively. Our different clients serve
different needs, with the old workhorse of the terminal still being
very useful, but much of our current development energy going into the
Notebook, obviously.  The Notebook enables interactive exploration to
become Literate Computing, bridging the gaps from individual work to
collaboration and publication, all with an open file format that is a
direct record of the underlying communication protocol.

There are obviously plenty of open issues (many of them very
important) that need fixing, and large and ambitious new lines of
development for the years to come.  But the work of the last four
years, since the summer of 2009 when Brian Granger was able to devote
a summer (thanks to funding from the NiPy project - nipy.org) to
refactoring the old IPython core code, finally opened up or
infrastructure for real innovation. By disentangling what was a useful
but impenetrable codebase, it became possible for us to start building
a flexible, modern system for interactive computing that abstracted
the old REPL model into a generic protocol that kernels could use to
talk to clients. This led at first to the creation of the Qt console,
and then to the Notebook and out-of-process terminal client.  It also
allowed us to (finally!) unify our parallel computing machinery with
the rest of the interactive system, which Min Ragan-Kelley pulled off
in a development tour de force that involved rewriting in a few weeks
a huge and complex Twisted-based system.

We are very happy with how the Notebook work has turned out, and it
seems the entire community agrees with us, as the uptake has been
phenomenal.  Back from the very first "IPython 0.0.1" that I started
in 2001:

https://gist.github.com/fperez/1579699

there were already hints of tools like Mathematica: it was my everyday
workhorse as a theoretical physicist and I found its Notebook
environment invaluable. But as a grad student trying out "just an
afternoon hack" (IPython was my very first Python program as I was
learning the language), I didn't have the resources, skills or vision
to attempt building an entire notebook system, and to be honest the
tools of the day would have made that enterprise a miserable one.  But
those ideas were always driving our efforts, and as IPython started
becoming a project with a team, we made multiple attempts to get a
good Notebook built around IPython.  Those interested can read an old
blog post of mine with the history
(http://blog.fperez.org/2012/01/ipython-notebook-historical.html).
The short story is that in 2011, on our sixth attempt, Brian was again
able to devote a focused summer into using our client-server
architecture and, with the stack of the modern web (Javascript, CSS,
websockets, Tornado, ...), finally build a robust system for Literate
Computing across programming languages.

Today, thanks to the generous support and vision of Josh Greenberg at
the Alfred P. Sloan Foundation, we are working very hard on building
the notebook infrastructure, and this release contains major advances
on that front.  We have high hopes for what we'll do next; as a
glimpse of the future that this enables, now there is a native Julia
kernel that speaks to our clients, notebook included:
https://github.com/JuliaLang/IJulia.jl.


# Team

I can't stress enough how impressed I am with the work people are
doing in IPython, and what a privilege it is to work with colleagues
like these.  Brian Granger and Min Ragan-Kelley joined IPython around
2005, initially working on the parallel machinery, but since ~ 2009
they have become the heart of the project. Today Min is our top
committer and knows our codebase better than anyone else, and I can't
imagine better partners for an effort like this.

And from regulars in our core team like Thomas Kluyver, Matthias
Bussonnier, Brad Froehle and Paul Ivanov to newcomers like Jonathan
Frederic and Zach Sailer, in addition to the many more whose names are
in our logs, we have a crazy amount of energy being poured into
IPython.  I hope we'll continue to harness it productively!

The full list of contributors to this release can be seen here:

http://ipython.org/ipython-doc/rel-1.0.0/whatsnew/github-stats-1.0.html


# Release highlights

* nbconvert: this is the major piece of new functionality in this
cycle, and was an explicit part of our roadmap
(https://github.com/ipython/ipython/wiki/Roadmap:-IPython). nbconvert
is now an IPython subcommand to convert notebooks into other formats
such as HTML or LaTeX, but more importantly, it's a very flexible
system that lets you write custom templates to generate new output
with arbitrary control over the formatting and transformations that
are applied to the input.

We want to stress that despite the fact that a huge amount of work
went into nbconvert, this should be considered a *tech preview*
release. We've come to realize how complex this problem is, and while
we'll make every effort to keep the high-level command-line syntax and
APIs as stable as possible, it is quite likely that the internals will
continue to evolve, possibly in backwards-incompatible ways.  So if
you start building services and libraries that make heavy use of the
nbconvert internals, please be prepared for some turmoil in the months
to come, and ping us on the dev list with questions or concerns.

* Notebook improvements: there has been a ton of polish work in the
notebook at many levels, though the file format remains unchanged from
0.13, so you shouldn't have any problems sharing notebooks with
colleagues still using 0.13.

  - Autosave: probably the most oft-requested feature, the notebook
server now autosaves your files!  You can still hit Ctrl-S to force a
manual save (which also creates a special 'checkpoint' you can come
back to).

  - The notebook supports raw_input(), and thus also %debug. This was
probably the main deficiency of the notebook as a client compared to
the terminal/qtconsole, and it has been finally fixed.

  - Add %%html, %%svg, %%javascript, and %%latex cell magics for
writing raw output in notebook cells.
  - Fix an issue parsing LaTeX in markdown cells, which required users
to type \\\, instead of \\.
  -Images support width and height metadata, and thereby 2x scaling
(retina support).
  - %%file has been renamed %%writefile (%%file) is deprecated.

* The input transofrmation code has been updated and rationalized.
This is a somewhat specialized part of IPython, but of importance to
projects that build upon it for custom environments, like Sympy and
Sage.

Our full release notes are here:

http://ipython.org/ipython-doc/rel-1.0.0/whatsnew/version1.0.html

and the gory details are here:

http://ipython.org/ipython-doc/rel-1.0.0/whatsnew/github-stats-1.0.html


# Installation

Installation links and instructions are at: http://ipython.org/install.html
And IPython is also on PyPI: http://pypi.python.org/pypi/ipython


# Requirements

IPython 1.0 requires Python ? 2.6.5 or ? 3.2.1. It does not support
Python 3.0, 3.1, or 2.5.


# Acknowledgments

Last but not least, we'd like to acknowledge the generous support of
those who make it possible for us to spend our time working on
IPython.  In particular, the Alfred P. Sloan Foundation today lets us
have a solid team working full-time on the project, and without the
support of Enthought Inc at multiple points in our history, we
wouldn't be where we are today.

The full list of our support is here:

http://ipython.org/index.html#support


Thanks to everyone! Please enjoy IPython 1.0, and report all bugs as usual!

Fernando, on behalf of the IPython Dev Team.

-- 
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail


From scopatz at gmail.com  Fri Aug  9 03:36:06 2013
From: scopatz at gmail.com (Anthony Scopatz)
Date: Fri, 9 Aug 2013 00:36:06 -0700
Subject: [Numpy-discussion] ANN: XDress v0.3 Release
Message-ID: <CAPk-6T5NC2ncik=xrTu6A4bOh37cSU78NktXp5G79_spq15jrg@mail.gmail.com>

Hello All,

I am pleased to announce the latest the latest and greatest release of
xdress, which incorporates a lot of feedback received at SciPy 2013.
 Please see the release notes below for more information.

And a HUGE congratulations to the IPython team on their v1.0 release today!
 Please continue taking over the world ;).

Be Well
Anthony

========================
XDress 0.3 Release Notes
========================
XDress is an automatic wrapper generator for C/C++ written in pure Python.
Currently,
xdress may generate Python bindings (via Cython) for C++ classes,
functions, and
certain variable types. It also contains idiomatic wrappers for C++
standard library
containers (sets, vectors, maps). In the future, other tools and bindings
will be
supported.

The main enabling feature of xdress is a dynamic type system that was
designed with
the purpose of API generation in mind.

Release highlights:

  - C++ template class & function support
  - Automatic docstring generation via dOxygen
  - A complete type system implementation refactor
  - Wrapped names may be different than source names,
    allowing automatic PEP-8-ification
  - A plethora of useful bug fixes

Please visit the website for more information: http://xdress.org/

Ask questions on the mailing list: xdress at googlegroups.com

Download the code from GitHub: http://github.com/xdress/xdress

XDress is free & open source (BSD 2-clause license) and requires Python 2.7,
NumPy 1.5+, Cython 0.19+, and optionally GCC-XML, pycparser, dOxygen, and
lxml.

New Features
============

C++ Template Support
---------------------
The most requested feature following SciPy 2013 was full and complete
support
for C++ templates.  It is here!  Template instantiations are now
automatically
discovered, described, and wrapped.  The only exception to this is that
template member functions are *not* automatically discovered due to a
defect in
GCC-XML.  Template methods must, therefore, be added by hand in a sidecar
file.
See the FAQ for more details.

dOxygen Docstrings
------------------
A new plugin called ``xdress.doxygen`` has been added which automatically
scrapes the source code for documentation using dOxygen.  These are then
automatically inserted as docstrings into the wrappers that are generated.
These
docstrings follow the numpydoc convention.

Type System Refactor
--------------------
The type system has undergone a complete reactor to make it more
user-friendly
and expose a lot of its internals.  Previously, the type system was a
collection
of functions and module-level private containers.  All of this has been
moved into
a new ``TypeSystem`` class whose attributes and methods are almost all
public.
This enables a number of potentially useful features.

* Type systems may be customized in the xdressrc.py file.
* Type systems may be customized in side car files.
* Type systems can now persist and load themselves.  This enables saving
  a type system when wrapping one project and loading it in as a dependency
  in future projects.
* Easier testing

As a consequence, most other parts of xdress must now hold a reference to a
type
system object.  Before, it was sufficient to import the type system module.

API Named Tuple
---------------
A new named tuple class ``apiname`` has been added.  This allows for a
broader
selection of how users can choose to write names in the classes, functions,
and
variables list in the run control file.  Possible options now include::

    from xdress.utils import apiname
    classes = [
        (srcname, srcfile,),
        (srcname, srcfile, tarfile,),
        (srcname, srcfile, tarfile, tarname),
        [srcname, srcfile, tarfile, tarname],
        {'srcname': a, 'srcfile': b, 'tarfile': c, 'tarname': d},
        apiname(srcname, srcfile, tarfile, tarname),
        ]

This allows the target name (ie the name in the wrapper) to be distinct from
the name that is in the C/C++ source code (ie the source name).  This is
very
useful for customizing APIs because xdress takes care of the mapping for
you!

PEP-8 Compliance Plugin
-----------------------
A new plugin called ``xdress.pep8names`` has been added which ensures that
all target names for API elements are PEP-8 compliant for that type.  If the
target name is not PEP-8 complaint then it converts it to a compliant
version
for you.  Subversive?  That's the point!  Functions and variables are
converted
to lowercase_with_underscores while classes are converted to CapCase.

Description Filtering Plugin
----------------------------
Sometimes autoall will expose too much.  To avoid certain API elements while
still automatically finding other names, the ``xdress.descfilter`` plugin
allows
you to skip methods, attributes, and types that the you doesn't feel are
appropriate
to expose.  This is also a useful stop-gap for data types that have not yet
been
implemented into the xdress type system.
Tab Completion in BASH
----------------------
Through the power and magic of the argcomplete package, all arguments and
options
for all plugins now have tab-completion enabled in BASH.  Install
argcomplete
and then add the following lines to your ``~/.bashrc`` file::

    # Enable completion for xdress
    eval "$(register-python-argcomplete xdress)"

Major Bug Fixes
===============

* Enums and functions pointers were noticeably absent from the GCC-XML
describer.
  This has been rectified.

* Classes and structs are now registered into the type system earlier.
 This allows
  them to be used in other plugins.  This includes stlwrap.  It is now
possible
  and easy to create custom numpy dtypes for any struct or C++ class.

* Fixed segfaults with custom dtypes

* Fixed build issue when only vectors were present in stlcontainers.

* The global namespace is now correctly represented by None, rather than
'::',
  when using GCC-XML.

Join in the Fun!
================
If you are interested in using xdress on your project (and need help),
contributing
back to xdress, starting up a development team, or writing your own code
generation
plugin tool, please let us know. Participation is very welcome!

Authors
=======
- [Anthony Scopatz](http://scopatz.com/)
- Spencer Lyon
- John Wiggins *
- Matt McCormick *
- Brad Buran

An * indicates a first time contributor.

Links
=====

1. Homepage - http://xdress.org/
2. Mailing List - xdress at googlegroups.com
3. GitHub Organization - https://github.com/xdress
4. numpydoc - https://pypi.python.org/pypi/numpydoc
5. argcomplete - https://argcomplete.readthedocs.org/en/latest/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130809/10057bb9/attachment.html>

From gael.varoquaux at normalesup.org  Fri Aug  9 03:42:42 2013
From: gael.varoquaux at normalesup.org (=?ISO-8859-1?Q?Ga=EBl_Varoquaux?=)
Date: Fri, 9 Aug 2013 09:42:42 +0200
Subject: [Numpy-discussion] np.load crashing in Anaconda Python 3.3.2
Message-ID: <CANOOnVLcvHLEAYkKv_0ZQ_ChtainkFg0eBe+StLKZa6HamCGmw@mail.gmail.com>

Hi list,

Just a quick heads up: on Python Anaconda 3.3.2 np.load leads to a
segfault. I am not on a computer for a week, but I saw that just before
leaving. If I remember right, it is a simple matter of doing:

a = np.zeros(10)
np.save('a.npy', a)
np.load('a.npy')

I haven't submitted a bug report, because I didn't have time to check
things properly. I suspect that it might be a problem of the compilation,
rather then numpy itself, so the Anaconda team might want to check.

Sorry for the incomplete report. I thought that it was better to submit an
incomplete report now than wait a week.

Ga?l
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130809/a3e14635/attachment.html>

From valentin at haenel.co  Fri Aug  9 18:06:14 2013
From: valentin at haenel.co (Valentin Haenel)
Date: Sat, 10 Aug 2013 00:06:14 +0200
Subject: [Numpy-discussion] a list of all available integer dtypes
Message-ID: <20130809220613.GA31290@kudu.in-berlin.de>

Hi,

I have a quick question: Is there a way to get a list of all available
Numpy integer dtypes programatically?

thanks,

V-


From robert.kern at gmail.com  Fri Aug  9 18:17:27 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 9 Aug 2013 23:17:27 +0100
Subject: [Numpy-discussion] a list of all available integer dtypes
In-Reply-To: <20130809220613.GA31290@kudu.in-berlin.de>
References: <20130809220613.GA31290@kudu.in-berlin.de>
Message-ID: <CAF6FJitU-TQhTpdc7xu-G-seytnKB1U1p_gzZaS=XCEv3QH2Bw@mail.gmail.com>

On Fri, Aug 9, 2013 at 11:06 PM, Valentin Haenel <valentin at haenel.co> wrote:
>
> Hi,
>
> I have a quick question: Is there a way to get a list of all available
> Numpy integer dtypes programatically?

[~]
|8> def all_dtypes(cls):
..>     for sub in cls.__subclasses__():
..>         try:
..>             sub(0)
..>         except TypeError:
..>             pass
..>         else:
..>             yield sub
..>         for subsub in all_dtypes(sub):
..>             yield subsub
..>

[~]
|10> list(all_dtypes(np.integer))
[numpy.int8,
 numpy.int16,
 numpy.int32,
 numpy.int64,
 numpy.int64,
 numpy.timedelta64,
 numpy.uint8,
 numpy.uint16,
 numpy.uint32,
 numpy.uint64,
 numpy.uint64]

--
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130809/26dc7ab0/attachment.html>

From matthew.brett at gmail.com  Fri Aug  9 18:20:15 2013
From: matthew.brett at gmail.com (Matthew Brett)
Date: Fri, 9 Aug 2013 15:20:15 -0700
Subject: [Numpy-discussion] a list of all available integer dtypes
In-Reply-To: <CAF6FJitU-TQhTpdc7xu-G-seytnKB1U1p_gzZaS=XCEv3QH2Bw@mail.gmail.com>
References: <20130809220613.GA31290@kudu.in-berlin.de>
	<CAF6FJitU-TQhTpdc7xu-G-seytnKB1U1p_gzZaS=XCEv3QH2Bw@mail.gmail.com>
Message-ID: <CAH6Pt5pUDWBk5PFwAEF01XW+KkPAKHwo=yXw3XxfbZT7RO0X5Q@mail.gmail.com>

Hi,

On Fri, Aug 9, 2013 at 3:17 PM, Robert Kern <robert.kern at gmail.com> wrote:
> On Fri, Aug 9, 2013 at 11:06 PM, Valentin Haenel <valentin at haenel.co> wrote:
>>
>> Hi,
>>
>> I have a quick question: Is there a way to get a list of all available
>> Numpy integer dtypes programatically?
>
> [~]
> |8> def all_dtypes(cls):
> ..>     for sub in cls.__subclasses__():
> ..>         try:
> ..>             sub(0)
> ..>         except TypeError:
> ..>             pass
> ..>         else:
> ..>             yield sub
> ..>         for subsub in all_dtypes(sub):
> ..>             yield subsub
> ..>
>
> [~]
> |10> list(all_dtypes(np.integer))
> [numpy.int8,
>  numpy.int16,
>  numpy.int32,
>  numpy.int64,
>  numpy.int64,
>  numpy.timedelta64,
>  numpy.uint8,
>  numpy.uint16,
>  numpy.uint32,
>  numpy.uint64,
>  numpy.uint64]

or (at least for the standard int dtypes):

np.sctypes['int'] + np.sctypes['uint']

Cheers,

Matthew


From david.reed.c at gmail.com  Fri Aug  9 18:48:53 2013
From: david.reed.c at gmail.com (David Reed)
Date: Fri, 9 Aug 2013 18:48:53 -0400
Subject: [Numpy-discussion] Nanmean review
Message-ID: <91C3FC67-3D37-4674-BEB1-0E086604E232@gmail.com>


Hello,

This is my first time contributing and I was hoping to get a review of a change I made.  Here is the comparison link.

https://github.com/dvreed77/numpy/compare/nanmean


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130809/a8b486f6/attachment.html>

From valentin at haenel.co  Fri Aug  9 18:57:35 2013
From: valentin at haenel.co (Valentin Haenel)
Date: Sat, 10 Aug 2013 00:57:35 +0200
Subject: [Numpy-discussion] a list of all available integer dtypes
In-Reply-To: <CAH6Pt5pUDWBk5PFwAEF01XW+KkPAKHwo=yXw3XxfbZT7RO0X5Q@mail.gmail.com>
References: <20130809220613.GA31290@kudu.in-berlin.de>
	<CAF6FJitU-TQhTpdc7xu-G-seytnKB1U1p_gzZaS=XCEv3QH2Bw@mail.gmail.com>
	<CAH6Pt5pUDWBk5PFwAEF01XW+KkPAKHwo=yXw3XxfbZT7RO0X5Q@mail.gmail.com>
Message-ID: <20130809225734.GA5285@kudu.in-berlin.de>

Hi,

* Matthew Brett <matthew.brett at gmail.com> [2013-08-10]:
> On Fri, Aug 9, 2013 at 3:17 PM, Robert Kern <robert.kern at gmail.com> wrote:
> > On Fri, Aug 9, 2013 at 11:06 PM, Valentin Haenel <valentin at haenel.co> wrote:
> >> I have a quick question: Is there a way to get a list of all available
> >> Numpy integer dtypes programatically?
> >
> > [~]
> > |8> def all_dtypes(cls):

[snip

> or (at least for the standard int dtypes):
> 
> np.sctypes['int'] + np.sctypes['uint']

Thanks Matthew and Robert. That was very helpful.

V-


From ralf.gommers at gmail.com  Sat Aug 10 03:21:11 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sat, 10 Aug 2013 09:21:11 +0200
Subject: [Numpy-discussion] Nanmean review
In-Reply-To: <91C3FC67-3D37-4674-BEB1-0E086604E232@gmail.com>
References: <91C3FC67-3D37-4674-BEB1-0E086604E232@gmail.com>
Message-ID: <CABL7CQjTuzvueb-73RrBmXVahvUBRkiJeXShQcGS2HjRg9EFNQ@mail.gmail.com>

On Sat, Aug 10, 2013 at 12:48 AM, David Reed <david.reed.c at gmail.com> wrote:

>
> Hello,
>
> This is my first time contributing and I was hoping to get a review of a
> change I made.  Here is the comparison link.
>
> https://github.com/dvreed77/numpy/compare/nanmean
>

Hi David, your contribution looks good and adding nanmean is a good idea.
However, your code happens to overlap for almost 100% with an already
submitted pull request: https://github.com/numpy/numpy/pull/3534. I suggest
that you compare your implementation with the one in PR-3534, and maybe you
can add an example there or fix some inconsistency.

This doesn't seem all that consistent by the way. The PR doesn't have an
example, so I don't know if it behaves the same:

    >>> np.nanmean([1, np.nan, np.inf])       inf
>>> np.nanmean([1, np.nan, np.inf, np.NINF])       nan
Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130810/384dfa58/attachment.html>

From ralf.gommers at gmail.com  Sat Aug 10 04:28:47 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sat, 10 Aug 2013 10:28:47 +0200
Subject: [Numpy-discussion] import overhead of numpy.testing
In-Reply-To: <142AB120-3E54-44AC-8DCF-5470A2FC8AB3@dalkescientific.com>
References: <D37A1790-EB87-4EA4-8D1F-D890199F4CA7@dalkescientific.com>
	<CAB6mnxKqWO7H-gBV2FSwMQDJje3TXYpdZDU_-wVaJcLTgtmeqA@mail.gmail.com>
	<142AB120-3E54-44AC-8DCF-5470A2FC8AB3@dalkescientific.com>
Message-ID: <CABL7CQg9iLuw0J6T6KVtdwJJWTZsfOut=wWUgjWoVpr1Nud_6g@mail.gmail.com>

On Wed, Aug 7, 2013 at 3:26 PM, Andrew Dalke <dalke at dalkescientific.com>wrote:

> On Aug 7, 2013, at 4:37 AM, Charles R Harris wrote:
> > I haven't forgotten and intend to look at it before the next release.
>
> Thanks!
>
> On a related topic, last night I looked into deferring the
> import for numpy.testing. This is the only other big place
> where numpy's import overhead might be reduced without
> breaking backwards compatibility.
>

It does break backwards compatibility though, because now you can do:

    import numpy as np
    np.testing.assert_equal(x, y)



> I made a _DeferredTester [1] and replaced the 10 __init__.py
> uses of:
>
> from .testing import Tester
> test = Tester().test
> bench = Tester().bench
>
> to use the _DeferredTester instead.
>
> With that in place the "import numpy" time (best of 5)
> goes from 0.0796 seconds to 0.0741 seconds, or 7%.
>
> That 0.0796 includes the 0.02 seconds for the exec()
> of the polynomial templates. Without that 0.02 seconds
> in the baseline would give a 10% speedup. [2]
>
> Would this sort of approach be acceptable to NumPy?
> If so, I could improve the patch to make it be acceptable.
>

I think if it's tested well, implementing np.test() with a deferred tester
is OK imho.

However, I would not be happy with breaking backwards compatibility for the
numpy.testing module. Do you have more detailed timings? I'm guessing the
bottleneck is importing nose. If so, you can still import numpy.testing
into the numpy namespace without losing your gain in import time (nose is
an optional dependency, so not imported by default inside numpy.testing).


> The outstanding code issues to be resolve before making
> a pull request are:
>
> 1) The current wrapper uses *args and **kwargs to forward
>    any test() and bench() calls to the actual function.
>    As a result, parameter introspection doesn't work.
>
> 2) The current wrapper doesn't have a __doc__
>
> 3) The only way to fix 1) and 2) is to copy the signatures
>    and docstring from the actual Tester() implementation,
>    which causes code/docstring duplication.
>

That all sounds fixable.


>
> 4) I don't know if it's appropriate to add my _DeferredTester
>    to numpy.core vs. some other place in the code base.
>

numpy.lib I'd think.


> If you want to see the patch, I followed the NumPy instructions at
>  http://docs.scipy.org/doc/numpy/dev/gitwash/git_development.html
> and made an experimental fork at
>  https://github.com/adalke/numpy/tree/no-default-tester-import
>
> I have no git/github experience beyond what I did for this
> patch, so let me know if there are problems in what I did.
>

You did everything correctly.

Cheers,
Ralf

P.S. I also see some unused imports and files. Will send a cleanup PR.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130810/48d060ad/attachment.html>

From sudheer.joseph at yahoo.com  Sat Aug 10 11:10:50 2013
From: sudheer.joseph at yahoo.com (Sudheer Joseph)
Date: Sat, 10 Aug 2013 23:10:50 +0800 (SGT)
Subject: [Numpy-discussion] solving matrix
Message-ID: <1376147450.41789.YahooMailNeo@web193402.mail.sg3.yahoo.com>

Dear Experts,
I am trying to write the below piece of matlab code to a python script. The objective of the code is to find a fit of annual harmonic to a time series and then remove it from the time series. It uses matlabs E\ts to find the coefficients of harmonic fit, I used linlag.lstsq equivalent in python but feel the result is not correct. Can any one advice on this please.

##########Matlab CODE################

function[an_fit,sam_fit,fit_both]=calcharm(t,ts,dt)
subplot 211
plot(t,ts)
legend('Orig Tser')
xlabel('TIME')
title('ORIGINAL TIMESERIES')
figure(gcf)
disp('Paused...');pause
%% Prepare Annual signal
fann = 2*pi/(365.25/dt);
E = [ones(size(t)) cos(fann*t) sin(fann*t)];
a = E\ts;
%dfit = a(1) + a(2)*cos(fann*t) + a(3)*sin(fann*t);
% % or more easily:
an_fit = E*a;

#################PYTHON CODE ##################
import numpy as np
from numpy import pi,c_,ones,size,cos,sin
t=np.arange(1,365.25*5)
dt=np.average(np.diff(t))
acyc = 2*pi/(365.25/dt)
 
365.25/4)/dt)
E1=c_[ones(size(t)),cos(acyc*t),sin(acyc*t)]
linalg.lstsq(E1,ts)
an_fit = E*a

with best regards,
Sudheer
***************************************************************
Sudheer Joseph 
Indian National Centre for Ocean Information Services
Ministry of Earth Sciences, Govt. of India
POST BOX NO: 21, IDA Jeedeemetla P.O.
Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
Web- http://oppamthadathil.tripod.com
***************************************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130810/396b33f4/attachment.html>

From dalke at dalkescientific.com  Sat Aug 10 11:21:53 2013
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Sat, 10 Aug 2013 17:21:53 +0200
Subject: [Numpy-discussion] import overhead of numpy.testing
In-Reply-To: <CABL7CQg9iLuw0J6T6KVtdwJJWTZsfOut=wWUgjWoVpr1Nud_6g@mail.gmail.com>
References: <D37A1790-EB87-4EA4-8D1F-D890199F4CA7@dalkescientific.com>
	<CAB6mnxKqWO7H-gBV2FSwMQDJje3TXYpdZDU_-wVaJcLTgtmeqA@mail.gmail.com>
	<142AB120-3E54-44AC-8DCF-5470A2FC8AB3@dalkescientific.com>
	<CABL7CQg9iLuw0J6T6KVtdwJJWTZsfOut=wWUgjWoVpr1Nud_6g@mail.gmail.com>
Message-ID: <B4E4287A-01B6-46A3-BFB5-0D19205CAFE9@dalkescientific.com>

[Short version: It doesn't look like my proposal or any
simple alternative is tenable.]

On Aug 10, 2013, at 10:28 AM, Ralf Gommers wrote:
> It does break backwards compatibility though, because now you can do:
> 
>    import numpy as np
>    np.testing.assert_equal(x, y)

Yes, it does.

I realize that a design goal in numpy was that (most?) submodules are
available without any additional imports. This is the main reason for
the "import numpy" overhead. The tension between ease-of-use for some
and overhead for others is well known. For example, Sage tickets 3577,
6494, and 11714 relate to deferring numpy import during startup.


The three relevant questions are:

1) is numpy.testing part of that promise? This can be split
   into multiple ways.

    o The design goal could be that only the numerics that people use
       for interactive/REPL computing are accessible without
       additional explicit imports, which implies that the import of
       numpy.testing is an implementation side-effect of providing
       submodule-level "test()" and "bench()" APIs

    o all NumPy modules with user-facing APIs should be accessible
      from numpy without additional imports

While I would like to believe that the import of numpy.testing
is an implementation side-effect of providing test() and bench(),
I believe that I'm unlikely to convince the majority.

For justifiable reasons, the numpy project is loath to break
backwards compatibility, and I don't think there's an existing
bright-line policy which would say that "import numpy; numpy.testing"
should be avoided.


2) If it isn't a promise that "numpy.testing" is usable after an
   "import numpy" then how many people will be affected by an
    implementation change, and at what level of severity?



I looked to see which packages might fail. A Debian code
search of "numpy.testing" showed no problems, and no one
uses "np.testing".

I did a code search at http://code.ohloh.net . Of the first
200 or so hits for "numpy.testing", nearly all of them fell
into uses like:

from numpy.testing import Tester
from numpy.testing import assert_equal, TestCase
from numpy.testing.utils import *
from numpy.testing import *

There were, however, several packages which would fail:

 test_io.py and test_image.py and test_array_bridge.py in MediPy
    (Interestingly, test_circle.py has a "import numpy.testing",
     so it's not universal practice in that package)
 calculators_test.py in  OpenQuake Engine
 ForcePlatformsExtractorTest.py in b-tk

Note that these failures are in the test programs, and not
in the main body code, so are unlikely to break end-user
programs.


HOWEVER!

The real test is for people who do "import numpy as np" then
refer to "np.testing". There are "about 454" such matches in
Ohloh.

One example is 'test_polygon.py' from scikit-image. Others are:
 test_anova.py in statsmodel
 test_graph.py in scikit-learn
 test_rmagic.py in IPython
 test_mlab.py in matplotlib

Nearly all the cases I looked at were in files starting "test",
or a handful which ended in "test.py" or "Test.py". Others use
np.test only as part of a unit test, such as:

 affine_grid.py and others in pyMor (as part of in-file unit tests)
 technical_indicators.py in QuantPy (as part of unittest.TestCase)
 coord_tools.py in NiPy-OLD (as part of in-file unit tests)
 predstd.py and others in statsmodels (as a main-line unit test)
 galsim_test_helpers.py in GalSim

These would likely not break end-user code.

Sadly, not all are that safe. For examples:
 simple_contrast.py  example program for nippy
 try_signal_lti.py in joePython
 run.py in python-seminar
 verify.py in bell_d_project (a final project for a CS class)
 ex_shrink_pickle.py in statsmodels (as an example?)
 parametric_design.py in nippy (uses assert_almost_equal to verify an example)
 model.py in pymc-devs's pymc
 model.py in duffy
 zipline in olmar
 utils.py in MNE
.... and I gave up at result 320 of 454.

Based on this, about 1% of the programs which use numpy.testing
would break. This tells me that there are enough user programs
which would fail that I don't think numpy will decide to make
this change.




And the third question is

  3) Are there other alternatives?

Or as Ralf Gommers wrote:
> Do you have more detailed timings? I'm guessing the bottleneck is importing nose.


I do have more detailed timings. "nose" is not imported
during an "import numpy". (For one, "import nose" takes
a full 0.11 seconds on my laptop and adds 199 modules
to sys.modules!)


The hit is the "import unittest" in numpy.testing, which
exists only to place "TestCase" in the numpy.testing namespace.
"numpy.testing.TestCase" is only used by the unit tests,
and not by any direct end-user code.

Here's the full hierarchical timing breakdown showing
 - module name
 - cumulative time to load
 - parent module

    testing: 0.0065 (numpy.core.numeric)
     unittest: 0.0055 (testing)
      result: 0.0011 (unittest)
       traceback: 0.0004 (result)
        linecache: 0.0000 (traceback)
       StringIO: 0.0004 (result)
        errno: 0.0000 (StringIO)
      case: 0.0021 (unittest)
       difflib: 0.0011 (case)
       pprint: 0.0004 (case)
       util: 0.0000 (case)
      suite: 0.0002 (unittest)
      loader: 0.0006 (unittest)
       fnmatch: 0.0002 (loader)
      main: 0.0010 (unittest)
       time: 0.0000 (main)
       signals: 0.0006 (main)
        signal: 0.0000 (signals)
        weakref: 0.0005 (signals)
         UserDict: 0.0000 (weakref)
         _weakref: 0.0000 (weakref)
         _weakrefset: 0.0000 (weakref)
         exceptions: 0.0000 (weakref)
      runner: 0.0000 (unittest)
     utils: 0.0005 (testing)
      nosetester: 0.0002 (utils)
       numpy.testing.utils: 0.0000 (nosetester)
     numpytest: 0.0001 (testing)

As you can see, "unittest" imports a large number of modules.

I see no good way to get rid of this unittest import.

Even if all of the tests were rewritten to use unittest.TestCase,
numpy.testing.TestCase would still need to be present so
third-party packages could derive from it, and there's no (easy?)
way to make that TestCase some sort of deferred object which
gets the correct TestCase when needed.


In conclusion, it looks like my proposal is not tenable and
there's no easy way to squeeze out that ~5% of startup overhead.

Cheers,

				Andrew
				dalke at dalkescientific.com




From sudheer.joseph at yahoo.com  Sat Aug 10 11:26:55 2013
From: sudheer.joseph at yahoo.com (Sudheer Joseph)
Date: Sat, 10 Aug 2013 23:26:55 +0800 (SGT)
Subject: [Numpy-discussion] solving matrix
In-Reply-To: <1376147450.41789.YahooMailNeo@web193402.mail.sg3.yahoo.com>
References: <1376147450.41789.YahooMailNeo@web193402.mail.sg3.yahoo.com>
Message-ID: <1376148415.55300.YahooMailNeo@web193402.mail.sg3.yahoo.com>

I thought I should add the error I get too.
From: Sudheer Joseph <sudheer.joseph at yahoo.com>

To: "numpy-discussion at scipy.org" <numpy-discussion at scipy.org> 
>Sent: Saturday, 10 August 2013 8:40 PM
>Subject: [Numpy-discussion] solving matrix
> 
>
>
>Dear Experts,
>I am trying to write the below piece of matlab code to a python script. The objective of the code is to find a fit of annual harmonic to a time series and then remove it from the time series. It uses matlabs E\ts to find the coefficients of harmonic fit, I used linlag.lstsq equivalent in python but feel the result is not correct. Can any one advice on this please.
>
>
>##########Matlab CODE################
>
>function[an_fit,sam_fit,fit_both]=calcharm(t,ts,dt)
>subplot 211
>plot(t,ts)
>legend('Orig Tser')
>xlabel('TIME')
>title('ORIGINAL TIMESERIES')
>figure(gcf)
>disp('Paused...');pause
>%% Prepare Annual signal
>fann = 2*pi/(365.25/dt);
>E = [ones(size(t)) cos(fann*t) sin(fann*t)];
>a = E\ts;
>%dfit = a(1) + a(2)*cos(fann*t) + a(3)*sin(fann*t);
>% % or more easily:
>an_fit = E*a;
>
>#################PYTHON CODE ##################
>import numpy as np
>from numpy import pi,c_,ones,size,cos,sin
>t=np.arange(1,365.25*5)
>dt=np.average(np.diff(t))
>acyc = 2*pi/(365.25/dt)
> 
>
>365.25/4)/dt)
>E1=c_[ones(size(t)),cos(acyc*t),sin(acyc*t)]
>linalg.lstsq(E1,ts)
>an_fit = E*a
>
>###############ERROR##################
>ts.shape
>Out[243]: (1828, 1)
>E1.shape
>Out[244]: (1826, 3)
>linalg.lstsq(E1,ss)
>---------------------------------------------------------------------------
>LinAlgError?????????????????????????????? Traceback (most recent call last)
>/home/sjo/work/PY_WORK/PY_TSER/<ipython-input-245-2eb3822d2564> in <module>()
>----> 1 linalg.lstsq(E1,ss)
>
>/usr/local/lib/python2.7/dist-packages/numpy-1.7.0-py2.7-linux-x86_64.egg/numpy/linalg/linalg.pyc in lstsq(a, b, rcond)
>?? 1801???? ldb = max(n, m)
>?? 1802???? if m != b.shape[0]:
>-> 1803???????? raise LinAlgError('Incompatible dimensions')
>?? 1804???? t, result_t = _commonType(a, b)
>?? 1805???? result_real_t = _realType(result_t)
>
>LinAlgError: Incompatible dimensions
>
>
>with best regards,
>Sudheer
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130810/33504034/attachment.html>

From ralf.gommers at gmail.com  Sat Aug 10 12:50:16 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sat, 10 Aug 2013 18:50:16 +0200
Subject: [Numpy-discussion] import overhead of numpy.testing
In-Reply-To: <B4E4287A-01B6-46A3-BFB5-0D19205CAFE9@dalkescientific.com>
References: <D37A1790-EB87-4EA4-8D1F-D890199F4CA7@dalkescientific.com>
	<CAB6mnxKqWO7H-gBV2FSwMQDJje3TXYpdZDU_-wVaJcLTgtmeqA@mail.gmail.com>
	<142AB120-3E54-44AC-8DCF-5470A2FC8AB3@dalkescientific.com>
	<CABL7CQg9iLuw0J6T6KVtdwJJWTZsfOut=wWUgjWoVpr1Nud_6g@mail.gmail.com>
	<B4E4287A-01B6-46A3-BFB5-0D19205CAFE9@dalkescientific.com>
Message-ID: <CABL7CQgFeOMsW-zb7AfNsAA9auQeveJymr6m8z5ezi8q9M15RA@mail.gmail.com>

On Sat, Aug 10, 2013 at 5:21 PM, Andrew Dalke <dalke at dalkescientific.com>wrote:

> [Short version: It doesn't look like my proposal or any
> simple alternative is tenable.]
>
> On Aug 10, 2013, at 10:28 AM, Ralf Gommers wrote:
> > It does break backwards compatibility though, because now you can do:
> >
> >    import numpy as np
> >    np.testing.assert_equal(x, y)
>
> Yes, it does.
>
> I realize that a design goal in numpy was that (most?) submodules are
> available without any additional imports. This is the main reason for
> the "import numpy" overhead. The tension between ease-of-use for some
> and overhead for others is well known. For example, Sage tickets 3577,
> 6494, and 11714 relate to deferring numpy import during startup.
>
>
> The three relevant questions are:
>
> 1) is numpy.testing part of that promise? This can be split
>    into multiple ways.
>
>     o The design goal could be that only the numerics that people use
>        for interactive/REPL computing are accessible without
>        additional explicit imports, which implies that the import of
>        numpy.testing is an implementation side-effect of providing
>        submodule-level "test()" and "bench()" APIs
>
>     o all NumPy modules with user-facing APIs should be accessible
>       from numpy without additional imports
>
> While I would like to believe that the import of numpy.testing
> is an implementation side-effect of providing test() and bench(),
> I believe that I'm unlikely to convince the majority.
>

It likely is a side-effect rather than intentional design, but at this
point that doesn't matter much anymore. There never was a clear distinction
between private and public modules and now, as your investigation shows,
the cost of removing the import is quite high.

For justifiable reasons, the numpy project is loath to break
> backwards compatibility, and I don't think there's an existing
> bright-line policy which would say that "import numpy; numpy.testing"
> should be avoided.
>
>
> 2) If it isn't a promise that "numpy.testing" is usable after an
>    "import numpy" then how many people will be affected by an
>     implementation change, and at what level of severity?
>
>
>
> I looked to see which packages might fail. A Debian code
> search of "numpy.testing" showed no problems, and no one
> uses "np.testing".
>
> I did a code search at http://code.ohloh.net . Of the first
> 200 or so hits for "numpy.testing", nearly all of them fell
> into uses like:
>
> from numpy.testing import Tester
> from numpy.testing import assert_equal, TestCase
> from numpy.testing.utils import *
> from numpy.testing import *
>
> There were, however, several packages which would fail:
>
>  test_io.py and test_image.py and test_array_bridge.py in MediPy
>     (Interestingly, test_circle.py has a "import numpy.testing",
>      so it's not universal practice in that package)
>  calculators_test.py in  OpenQuake Engine
>  ForcePlatformsExtractorTest.py in b-tk
>
> Note that these failures are in the test programs, and not
> in the main body code, so are unlikely to break end-user
> programs.
>
>
> HOWEVER!
>
> The real test is for people who do "import numpy as np" then
> refer to "np.testing". There are "about 454" such matches in
> Ohloh.
>
> One example is 'test_polygon.py' from scikit-image. Others are:
>  test_anova.py in statsmodel
>  test_graph.py in scikit-learn
>  test_rmagic.py in IPython
>  test_mlab.py in matplotlib
>
> Nearly all the cases I looked at were in files starting "test",
> or a handful which ended in "test.py" or "Test.py". Others use
> np.test only as part of a unit test, such as:
>
>  affine_grid.py and others in pyMor (as part of in-file unit tests)
>  technical_indicators.py in QuantPy (as part of unittest.TestCase)
>  coord_tools.py in NiPy-OLD (as part of in-file unit tests)
>  predstd.py and others in statsmodels (as a main-line unit test)
>  galsim_test_helpers.py in GalSim
>
> These would likely not break end-user code.
>
> Sadly, not all are that safe. For examples:
>  simple_contrast.py  example program for nippy
>  try_signal_lti.py in joePython
>  run.py in python-seminar
>  verify.py in bell_d_project (a final project for a CS class)
>  ex_shrink_pickle.py in statsmodels (as an example?)
>  parametric_design.py in nippy (uses assert_almost_equal to verify an
> example)
>  model.py in pymc-devs's pymc
>  model.py in duffy
>  zipline in olmar
>  utils.py in MNE
> .... and I gave up at result 320 of 454.
>
> Based on this, about 1% of the programs which use numpy.testing
> would break. This tells me that there are enough user programs
> which would fail that I don't think numpy will decide to make
> this change.
>
>
>
>
> And the third question is
>
>   3) Are there other alternatives?
>
> Or as Ralf Gommers wrote:
> > Do you have more detailed timings? I'm guessing the bottleneck is
> importing nose.
>
>
> I do have more detailed timings. "nose" is not imported
> during an "import numpy". (For one, "import nose" takes
> a full 0.11 seconds on my laptop and adds 199 modules
> to sys.modules!)
>
>
> The hit is the "import unittest" in numpy.testing, which
> exists only to place "TestCase" in the numpy.testing namespace.
> "numpy.testing.TestCase" is only used by the unit tests,
> and not by any direct end-user code.
>
> Here's the full hierarchical timing breakdown showing
>  - module name
>  - cumulative time to load
>  - parent module
>
>     testing: 0.0065 (numpy.core.numeric)
>      unittest: 0.0055 (testing)
>       result: 0.0011 (unittest)
>        traceback: 0.0004 (result)
>         linecache: 0.0000 (traceback)
>        StringIO: 0.0004 (result)
>         errno: 0.0000 (StringIO)
>       case: 0.0021 (unittest)
>        difflib: 0.0011 (case)
>        pprint: 0.0004 (case)
>        util: 0.0000 (case)
>       suite: 0.0002 (unittest)
>       loader: 0.0006 (unittest)
>        fnmatch: 0.0002 (loader)
>       main: 0.0010 (unittest)
>        time: 0.0000 (main)
>        signals: 0.0006 (main)
>         signal: 0.0000 (signals)
>         weakref: 0.0005 (signals)
>          UserDict: 0.0000 (weakref)
>          _weakref: 0.0000 (weakref)
>          _weakrefset: 0.0000 (weakref)
>          exceptions: 0.0000 (weakref)
>       runner: 0.0000 (unittest)
>      utils: 0.0005 (testing)
>       nosetester: 0.0002 (utils)
>        numpy.testing.utils: 0.0000 (nosetester)
>      numpytest: 0.0001 (testing)
>
> As you can see, "unittest" imports a large number of modules.
>
> I see no good way to get rid of this unittest import.
>

Indeed. I had a quick look at the benefit of copying TestCase into
numpy.testing so the import of unittest can be removed, but more than half
the time is spent inside case.py and result.py, which would still be needed.


> Even if all of the tests were rewritten to use unittest.TestCase,
> numpy.testing.TestCase would still need to be present so
> third-party packages could derive from it, and there's no (easy?)
> way to make that TestCase some sort of deferred object which
> gets the correct TestCase when needed.
>
>
> In conclusion, it looks like my proposal is not tenable and
> there's no easy way to squeeze out that ~5% of startup overhead.
>

It does look that way.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130810/f9db7d12/attachment.html>

From sudheer.joseph at yahoo.com  Sat Aug 10 21:23:17 2013
From: sudheer.joseph at yahoo.com (Sudheer Joseph)
Date: Sun, 11 Aug 2013 09:23:17 +0800 (SGT)
Subject: [Numpy-discussion] solving matrix
In-Reply-To: <1376148415.55300.YahooMailNeo@web193402.mail.sg3.yahoo.com>
References: <1376147450.41789.YahooMailNeo@web193402.mail.sg3.yahoo.com>
	<1376148415.55300.YahooMailNeo@web193402.mail.sg3.yahoo.com>
Message-ID: <1376184197.75639.YahooMailNeo@web193402.mail.sg3.yahoo.com>

Dear Admin,
???????????????? I found the the issue was a silly mistake of difference size of array and now would like to withdraw this post. Is there a way to delete this question.
with best regards,
Sudheer






>________________________________
> From: Sudheer Joseph <sudheer.joseph at yahoo.com>
>To: Discussion of Numerical Python <numpy-discussion at scipy.org> 
>Sent: Saturday, 10 August 2013 8:56 PM
>Subject: Re: [Numpy-discussion] solving matrix
> 
>
>
>I thought I should add the error I get too.
>
>From: Sudheer Joseph <sudheer.joseph at yahoo.com>
>
>To: "numpy-discussion at scipy.org" <numpy-discussion at scipy.org> 
>>Sent: Saturday, 10 August 2013 8:40 PM
>>Subject: [Numpy-discussion] solving matrix
>> 
>>
>>
>>Dear Experts,
>>I am trying to write the below piece of matlab code to a python script. The objective of the code is to find a fit of annual harmonic to a time series and then remove it from the time series. It uses matlabs E\ts to find the coefficients of harmonic fit, I used linlag.lstsq equivalent in python but feel the result is not correct. Can any one advice on this please.
>>
>>
>>##########Matlab CODE################
>>
>>function[an_fit,sam_fit,fit_both]=calcharm(t,ts,dt)
>>subplot 211
>>plot(t,ts)
>>legend('Orig Tser')
>>xlabel('TIME')
>>title('ORIGINAL TIMESERIES')
>>figure(gcf)
>>disp('Paused...');pause
>>%% Prepare Annual signal
>>fann = 2*pi/(365.25/dt);
>>E = [ones(size(t)) cos(fann*t) sin(fann*t)];
>>a = E\ts;
>>%dfit = a(1) + a(2)*cos(fann*t) + a(3)*sin(fann*t);
>>% % or more easily:
>>an_fit = E*a;
>>
>>#################PYTHON CODE ##################
>>import numpy as np
>>from numpy import
 pi,c_,ones,size,cos,sin
>>t=np.arange(1,365.25*5)
>>dt=np.average(np.diff(t))
>>acyc = 2*pi/(365.25/dt)
>> 
>>
>>365.25/4)/dt)
>>E1=c_[ones(size(t)),cos(acyc*t),sin(acyc*t)]
>>linalg.lstsq(E1,ts)
>>an_fit = E*a
>>
>>###############ERROR##################
>>ts.shape
>>Out[243]: (1828, 1)
>>E1.shape
>>Out[244]: (1826, 3)
>>linalg.lstsq(E1,ss)
>>---------------------------------------------------------------------------
>>LinAlgError?????????????????????????????? Traceback (most recent call last)
>>/home/sjo/work/PY_WORK/PY_TSER/<ipython-input-245-2eb3822d2564> in <module>()
>>----> 1 linalg.lstsq(E1,ss)
>>
>>/usr/local/lib/python2.7/dist-packages/numpy-1.7.0-py2.7-linux-x86_64.egg/numpy/linalg/linalg.pyc in lstsq(a, b, rcond)
>>?? 1801???? ldb = max(n,
 m)
>>?? 1802???? if m != b.shape[0]:
>>-> 1803???????? raise LinAlgError('Incompatible dimensions')
>>?? 1804???? t, result_t = _commonType(a, b)
>>?? 1805???? result_real_t = _realType(result_t)
>>
>>LinAlgError: Incompatible dimensions
>>
>>
>>with best regards,
>>Sudheer
>>
>>
>_______________________________________________
>NumPy-Discussion mailing list
>NumPy-Discussion at scipy.org
>http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130811/7ff85d86/attachment.html>

From ben.root at ou.edu  Sat Aug 10 21:35:13 2013
From: ben.root at ou.edu (Benjamin Root)
Date: Sat, 10 Aug 2013 21:35:13 -0400
Subject: [Numpy-discussion] import overhead of numpy.testing
In-Reply-To: <CABL7CQgFeOMsW-zb7AfNsAA9auQeveJymr6m8z5ezi8q9M15RA@mail.gmail.com>
References: <D37A1790-EB87-4EA4-8D1F-D890199F4CA7@dalkescientific.com>
	<CAB6mnxKqWO7H-gBV2FSwMQDJje3TXYpdZDU_-wVaJcLTgtmeqA@mail.gmail.com>
	<142AB120-3E54-44AC-8DCF-5470A2FC8AB3@dalkescientific.com>
	<CABL7CQg9iLuw0J6T6KVtdwJJWTZsfOut=wWUgjWoVpr1Nud_6g@mail.gmail.com>
	<B4E4287A-01B6-46A3-BFB5-0D19205CAFE9@dalkescientific.com>
	<CABL7CQgFeOMsW-zb7AfNsAA9auQeveJymr6m8z5ezi8q9M15RA@mail.gmail.com>
Message-ID: <CANNq6FkWJG0cUo0HsmSNDCBU3GW=+GvCwU3ufdQAuY+-KwhEeQ@mail.gmail.com>

On Aug 10, 2013 12:50 PM, "Ralf Gommers" <ralf.gommers at gmail.com> wrote:
>
>
>
>
> On Sat, Aug 10, 2013 at 5:21 PM, Andrew Dalke <dalke at dalkescientific.com>
wrote:
>>
>> [Short version: It doesn't look like my proposal or any
>> simple alternative is tenable.]
>>
>> On Aug 10, 2013, at 10:28 AM, Ralf Gommers wrote:
>> > It does break backwards compatibility though, because now you can do:
>> >
>> >    import numpy as np
>> >    np.testing.assert_equal(x, y)
>>
>> Yes, it does.
>>
>> I realize that a design goal in numpy was that (most?) submodules are
>> available without any additional imports. This is the main reason for
>> the "import numpy" overhead. The tension between ease-of-use for some
>> and overhead for others is well known. For example, Sage tickets 3577,
>> 6494, and 11714 relate to deferring numpy import during startup.
>>
>>
>> The three relevant questions are:
>>
>> 1) is numpy.testing part of that promise? This can be split
>>    into multiple ways.
>>
>>     o The design goal could be that only the numerics that people use
>>        for interactive/REPL computing are accessible without
>>        additional explicit imports, which implies that the import of
>>        numpy.testing is an implementation side-effect of providing
>>        submodule-level "test()" and "bench()" APIs
>>
>>     o all NumPy modules with user-facing APIs should be accessible
>>       from numpy without additional imports
>>
>> While I would like to believe that the import of numpy.testing
>> is an implementation side-effect of providing test() and bench(),
>> I believe that I'm unlikely to convince the majority.
>
>
> It likely is a side-effect rather than intentional design, but at this
point that doesn't matter much anymore. There never was a clear distinction
between private and public modules and now, as your investigation shows,
the cost of removing the import is quite high.
>
>> For justifiable reasons, the numpy project is loath to break
>> backwards compatibility, and I don't think there's an existing
>> bright-line policy which would say that "import numpy; numpy.testing"
>> should be avoided.
>>
>>
>> 2) If it isn't a promise that "numpy.testing" is usable after an
>>    "import numpy" then how many people will be affected by an
>>     implementation change, and at what level of severity?
>>
>>
>>
>> I looked to see which packages might fail. A Debian code
>> search of "numpy.testing" showed no problems, and no one
>> uses "np.testing".
>>
>> I did a code search at http://code.ohloh.net . Of the first
>> 200 or so hits for "numpy.testing", nearly all of them fell
>> into uses like:
>>
>> from numpy.testing import Tester
>> from numpy.testing import assert_equal, TestCase
>> from numpy.testing.utils import *
>> from numpy.testing import *
>>
>> There were, however, several packages which would fail:
>>
>>  test_io.py and test_image.py and test_array_bridge.py in MediPy
>>     (Interestingly, test_circle.py has a "import numpy.testing",
>>      so it's not universal practice in that package)
>>  calculators_test.py in  OpenQuake Engine
>>  ForcePlatformsExtractorTest.py in b-tk
>>
>> Note that these failures are in the test programs, and not
>> in the main body code, so are unlikely to break end-user
>> programs.
>>
>>
>> HOWEVER!
>>
>> The real test is for people who do "import numpy as np" then
>> refer to "np.testing". There are "about 454" such matches in
>> Ohloh.
>>
>> One example is 'test_polygon.py' from scikit-image. Others are:
>>  test_anova.py in statsmodel
>>  test_graph.py in scikit-learn
>>  test_rmagic.py in IPython
>>  test_mlab.py in matplotlib
>>
>> Nearly all the cases I looked at were in files starting "test",
>> or a handful which ended in "test.py" or "Test.py". Others use
>> np.test only as part of a unit test, such as:
>>
>>  affine_grid.py and others in pyMor (as part of in-file unit tests)
>>  technical_indicators.py in QuantPy (as part of unittest.TestCase)
>>  coord_tools.py in NiPy-OLD (as part of in-file unit tests)
>>  predstd.py and others in statsmodels (as a main-line unit test)
>>  galsim_test_helpers.py in GalSim
>>
>> These would likely not break end-user code.
>>
>> Sadly, not all are that safe. For examples:
>>  simple_contrast.py  example program for nippy
>>  try_signal_lti.py in joePython
>>  run.py in python-seminar
>>  verify.py in bell_d_project (a final project for a CS class)
>>  ex_shrink_pickle.py in statsmodels (as an example?)
>>  parametric_design.py in nippy (uses assert_almost_equal to verify an
example)
>>  model.py in pymc-devs's pymc
>>  model.py in duffy
>>  zipline in olmar
>>  utils.py in MNE
>> .... and I gave up at result 320 of 454.
>>
>> Based on this, about 1% of the programs which use numpy.testing
>> would break. This tells me that there are enough user programs
>> which would fail that I don't think numpy will decide to make
>> this change.
>>
>>
>>
>>
>> And the third question is
>>
>>   3) Are there other alternatives?
>>
>> Or as Ralf Gommers wrote:
>> > Do you have more detailed timings? I'm guessing the bottleneck is
importing nose.
>>
>>
>> I do have more detailed timings. "nose" is not imported
>> during an "import numpy". (For one, "import nose" takes
>> a full 0.11 seconds on my laptop and adds 199 modules
>> to sys.modules!)
>>
>>
>> The hit is the "import unittest" in numpy.testing, which
>> exists only to place "TestCase" in the numpy.testing namespace.
>> "numpy.testing.TestCase" is only used by the unit tests,
>> and not by any direct end-user code.
>>
>> Here's the full hierarchical timing breakdown showing
>>  - module name
>>  - cumulative time to load
>>  - parent module
>>
>>     testing: 0.0065 (numpy.core.numeric)
>>      unittest: 0.0055 (testing)
>>       result: 0.0011 (unittest)
>>        traceback: 0.0004 (result)
>>         linecache: 0.0000 (traceback)
>>        StringIO: 0.0004 (result)
>>         errno: 0.0000 (StringIO)
>>       case: 0.0021 (unittest)
>>        difflib: 0.0011 (case)
>>        pprint: 0.0004 (case)
>>        util: 0.0000 (case)
>>       suite: 0.0002 (unittest)
>>       loader: 0.0006 (unittest)
>>        fnmatch: 0.0002 (loader)
>>       main: 0.0010 (unittest)
>>        time: 0.0000 (main)
>>        signals: 0.0006 (main)
>>         signal: 0.0000 (signals)
>>         weakref: 0.0005 (signals)
>>          UserDict: 0.0000 (weakref)
>>          _weakref: 0.0000 (weakref)
>>          _weakrefset: 0.0000 (weakref)
>>          exceptions: 0.0000 (weakref)
>>       runner: 0.0000 (unittest)
>>      utils: 0.0005 (testing)
>>       nosetester: 0.0002 (utils)
>>        numpy.testing.utils: 0.0000 (nosetester)
>>      numpytest: 0.0001 (testing)
>>
>> As you can see, "unittest" imports a large number of modules.
>>
>> I see no good way to get rid of this unittest import.
>
>
> Indeed. I had a quick look at the benefit of copying TestCase into
numpy.testing so the import of unittest can be removed, but more than half
the time is spent inside case.py and result.py, which would still be needed.
>
>>
>> Even if all of the tests were rewritten to use unittest.TestCase,
>> numpy.testing.TestCase would still need to be present so
>> third-party packages could derive from it, and there's no (easy?)
>> way to make that TestCase some sort of deferred object which
>> gets the correct TestCase when needed.
>>
>>
>> In conclusion, it looks like my proposal is not tenable and
>> there's no easy way to squeeze out that ~5% of startup overhead.
>
>
> It does look that way.
>
> Ralf
>
>

Would there be some sort of way to detect that numpy.testing wasn't
explicitly imported and issue a deprecation warning? Say, move the code
into numpy._testing, import in into the namespace as testing, but then have
the testing.py file set a flag in _testing to indicate an explicit import
has occurred?

Eventually, even _testing would no longer get imported by default and all
will be well.

Of course, that might be too convoluted?

Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130810/777c11dd/attachment.html>

From ralf.gommers at gmail.com  Sun Aug 11 05:02:47 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 11 Aug 2013 11:02:47 +0200
Subject: [Numpy-discussion] import overhead of numpy.testing
In-Reply-To: <CANNq6FkWJG0cUo0HsmSNDCBU3GW=+GvCwU3ufdQAuY+-KwhEeQ@mail.gmail.com>
References: <D37A1790-EB87-4EA4-8D1F-D890199F4CA7@dalkescientific.com>
	<CAB6mnxKqWO7H-gBV2FSwMQDJje3TXYpdZDU_-wVaJcLTgtmeqA@mail.gmail.com>
	<142AB120-3E54-44AC-8DCF-5470A2FC8AB3@dalkescientific.com>
	<CABL7CQg9iLuw0J6T6KVtdwJJWTZsfOut=wWUgjWoVpr1Nud_6g@mail.gmail.com>
	<B4E4287A-01B6-46A3-BFB5-0D19205CAFE9@dalkescientific.com>
	<CABL7CQgFeOMsW-zb7AfNsAA9auQeveJymr6m8z5ezi8q9M15RA@mail.gmail.com>
	<CANNq6FkWJG0cUo0HsmSNDCBU3GW=+GvCwU3ufdQAuY+-KwhEeQ@mail.gmail.com>
Message-ID: <CABL7CQiLXajaDaRR_bx=P13RAf5_TpKszVJLepz1wOvX=s2BWQ@mail.gmail.com>

On Sun, Aug 11, 2013 at 3:35 AM, Benjamin Root <ben.root at ou.edu> wrote:

> Would there be some sort of way to detect that numpy.testing wasn't
> explicitly imported and issue a deprecation warning? Say, move the code
> into numpy._testing, import in into the namespace as testing, but then have
> the testing.py file set a flag in _testing to indicate an explicit import
> has occurred?
>
> Eventually, even _testing would no longer get imported by default and all
> will be well.
>
> Of course, that might be too convoluted?
>
I'm not sure how that would work (you didn't describe how to decide that
the import was explicit), but imho the impact would be too high.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130811/67cf59de/attachment.html>

From ralf.gommers at gmail.com  Sun Aug 11 06:27:20 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Sun, 11 Aug 2013 12:27:20 +0200
Subject: [Numpy-discussion] Releaseplan for 1.8?
In-Reply-To: <loom.20130807T155120-219@post.gmane.org>
References: <loom.20130807T155120-219@post.gmane.org>
Message-ID: <CABL7CQjHFP42s4gA5_1_qTK5UgdzeXyO086gsj-BJN+QtW7sFA@mail.gmail.com>

On Wed, Aug 7, 2013 at 3:52 PM, Till Stensitzki <mail.till at gmx.de> wrote:

> Hi,
> there already a plan to release 1.8? I would like to play around with
> gufuncs.
>

No fixed plan I think, but there aren't a lot of blockers:
https://github.com/numpy/numpy/issues?milestone=1&page=1&state=open. Main
issue is the timezone handling, but if no one picks that up I'm not sure
it's really a blocker.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130811/5846f300/attachment.html>

From charlesr.harris at gmail.com  Sun Aug 11 14:59:09 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 11 Aug 2013 12:59:09 -0600
Subject: [Numpy-discussion] Releaseplan for 1.8?
In-Reply-To: <CABL7CQjHFP42s4gA5_1_qTK5UgdzeXyO086gsj-BJN+QtW7sFA@mail.gmail.com>
References: <loom.20130807T155120-219@post.gmane.org>
	<CABL7CQjHFP42s4gA5_1_qTK5UgdzeXyO086gsj-BJN+QtW7sFA@mail.gmail.com>
Message-ID: <CAB6mnx+aZ4jeEqXtRpMR3Q7rY=9Jpb+VTh7SHROYve1Hn_Rr0w@mail.gmail.com>

On Sun, Aug 11, 2013 at 4:27 AM, Ralf Gommers <ralf.gommers at gmail.com>wrote:

>
>
>
> On Wed, Aug 7, 2013 at 3:52 PM, Till Stensitzki <mail.till at gmx.de> wrote:
>
>> Hi,
>> there already a plan to release 1.8? I would like to play around with
>> gufuncs.
>>
>
> No fixed plan I think, but there aren't a lot of blockers:
> https://github.com/numpy/numpy/issues?milestone=1&page=1&state=open. Main
> issue is the timezone handling, but if no one picks that up I'm not sure
> it's really a blocker.
>
>
I've been looking into that and my inclination at this point is to let it
slide. There are two problems with making changes in the 1.8 release:
backward compatibility and deciding on a solution. Chris mentions that
other project have wrappers to work around the current problems, those
wrappers might break if we change things. And what the change should be
hasn't been decided.

Over the long term I'm inclined to go with solution 1, no timezone. But
could we still do conversions to calender units? If not, how to manage
that. And then how do we manage times that do have timezone information?
For that perhaps we could make a class that uses a naive datetime64
internally, but that adds timezone information, although string parsing
might be a problem since I assume we'd have to check all strings for
consistency.

Related to solution 1 is the question of what lowlevel implementation might
provide the best foundation more specialized classes?  Here the thought is
that the low level implementation should make as few assumptions as
possible, but still provide a basis for specialization. For this it would
be nice to get more input from current projects that attempt to use
datetime64.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130811/c30c1331/attachment.html>

From ben.root at ou.edu  Sun Aug 11 16:24:19 2013
From: ben.root at ou.edu (Benjamin Root)
Date: Sun, 11 Aug 2013 16:24:19 -0400
Subject: [Numpy-discussion] import overhead of numpy.testing
In-Reply-To: <CABL7CQiLXajaDaRR_bx=P13RAf5_TpKszVJLepz1wOvX=s2BWQ@mail.gmail.com>
References: <D37A1790-EB87-4EA4-8D1F-D890199F4CA7@dalkescientific.com>
	<CAB6mnxKqWO7H-gBV2FSwMQDJje3TXYpdZDU_-wVaJcLTgtmeqA@mail.gmail.com>
	<142AB120-3E54-44AC-8DCF-5470A2FC8AB3@dalkescientific.com>
	<CABL7CQg9iLuw0J6T6KVtdwJJWTZsfOut=wWUgjWoVpr1Nud_6g@mail.gmail.com>
	<B4E4287A-01B6-46A3-BFB5-0D19205CAFE9@dalkescientific.com>
	<CABL7CQgFeOMsW-zb7AfNsAA9auQeveJymr6m8z5ezi8q9M15RA@mail.gmail.com>
	<CANNq6FkWJG0cUo0HsmSNDCBU3GW=+GvCwU3ufdQAuY+-KwhEeQ@mail.gmail.com>
	<CABL7CQiLXajaDaRR_bx=P13RAf5_TpKszVJLepz1wOvX=s2BWQ@mail.gmail.com>
Message-ID: <CANNq6Fn8GD5uetk0i7sa-yxqZMzS=aUmmwpJv4Wur-p=VMhCjQ@mail.gmail.com>

On Aug 11, 2013 5:02 AM, "Ralf Gommers" <ralf.gommers at gmail.com> wrote:
>
>
>
>
> On Sun, Aug 11, 2013 at 3:35 AM, Benjamin Root <ben.root at ou.edu> wrote:
>>
>> Would there be some sort of way to detect that numpy.testing wasn't
explicitly imported and issue a deprecation warning? Say, move the code
into numpy._testing, import in into the namespace as testing, but then have
the testing.py file set a flag in _testing to indicate an explicit import
has occurred?
>>
>> Eventually, even _testing would no longer get imported by default and
all will be well.
>>
>> Of course, that might be too convoluted?
>
> I'm not sure how that would work (you didn't describe how to decide that
the import was explicit), but imho the impact would be too high.
>
> Ralf
>

The idea would be that within numpy (and we should fix SciPy as well), we
would always import numpy._testing as testing, and not import testing.py
ourselves. Then, there would be a flag in _testing.py that would be set to
emit, by default, warnings about using np.testing without an explicit
import, and stating which version all code will have to be switched by
perhaps 2.0?).

testing.py would do a from _testing import *, but also set the flag in
_testing to not emit warnings, because only a non-numpy (and SciPy) module
would have imported it.

It isn't foolproof. If a project has multiple dependencies that use
np.testing, and only one of them explicitly imports np.testing, then the
warning becomes hidden for the non-compliant parts. However, if we make
sure that the core SciPy projects use np._testing, it would go a long way
to get the word out.

Again, I am just throwing it out there as an idea. The speedups we are
getting right now so far are nice, so it is entirely possible that this
kludge is just not worth the last remaining bits of extra time.

Cheers!
Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130811/677809d2/attachment.html>

From dalke at dalkescientific.com  Sun Aug 11 16:37:33 2013
From: dalke at dalkescientific.com (Andrew Dalke)
Date: Sun, 11 Aug 2013 22:37:33 +0200
Subject: [Numpy-discussion] import overhead of numpy.testing
In-Reply-To: <CANNq6Fn8GD5uetk0i7sa-yxqZMzS=aUmmwpJv4Wur-p=VMhCjQ@mail.gmail.com>
References: <D37A1790-EB87-4EA4-8D1F-D890199F4CA7@dalkescientific.com>
	<CAB6mnxKqWO7H-gBV2FSwMQDJje3TXYpdZDU_-wVaJcLTgtmeqA@mail.gmail.com>
	<142AB120-3E54-44AC-8DCF-5470A2FC8AB3@dalkescientific.com>
	<CABL7CQg9iLuw0J6T6KVtdwJJWTZsfOut=wWUgjWoVpr1Nud_6g@mail.gmail.com>
	<B4E4287A-01B6-46A3-BFB5-0D19205CAFE9@dalkescientific.com>
	<CABL7CQgFeOMsW-zb7AfNsAA9auQeveJymr6m8z5ezi8q9M15RA@mail.gmail.com>
	<CANNq6FkWJG0cUo0HsmSNDCBU3GW=+GvCwU3ufdQAuY+-KwhEeQ@mail.gmail.com>
	<CABL7CQiLXajaDaRR_bx=P13RAf5_TpKszVJLepz1wOvX=s2BWQ@mail.gmail.com>
	<CANNq6Fn8GD5uetk0i7sa-yxqZMzS=aUmmwpJv4Wur-p=VMhCjQ@mail.gmail.com>
Message-ID: <37CC8D40-97A0-4961-A3CA-E9D96041ECF2@dalkescientific.com>

On Aug 11, 2013, at 10:24 PM, Benjamin Root wrote:
> The idea would be that within numpy (and we should fix SciPy as well), we would always import numpy._testing as testing, and not import testing.py ourselves.

The problem is the existing code out there which does:

import numpy as np
 ...
np.testing.utils.assert_almost_equal(x, y)

(That is, without an additional import), and other code which does

from numpy.testing import *


There's no way to make these both give a warning without some
horrible hack, like interposing our own __import__ or sticking
some non-module object into sys.modules.

Down that path lies madness, and the corpses of many earlier
attempts.

Perhaps the latest Python import hooks might have a solution,
but it's not something I want to spend time solving.


				Andrew
				dalke at dalkescientific.com




From charlesr.harris at gmail.com  Sun Aug 11 16:44:00 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 11 Aug 2013 14:44:00 -0600
Subject: [Numpy-discussion] import overhead of numpy.testing
In-Reply-To: <CANNq6Fn8GD5uetk0i7sa-yxqZMzS=aUmmwpJv4Wur-p=VMhCjQ@mail.gmail.com>
References: <D37A1790-EB87-4EA4-8D1F-D890199F4CA7@dalkescientific.com>
	<CAB6mnxKqWO7H-gBV2FSwMQDJje3TXYpdZDU_-wVaJcLTgtmeqA@mail.gmail.com>
	<142AB120-3E54-44AC-8DCF-5470A2FC8AB3@dalkescientific.com>
	<CABL7CQg9iLuw0J6T6KVtdwJJWTZsfOut=wWUgjWoVpr1Nud_6g@mail.gmail.com>
	<B4E4287A-01B6-46A3-BFB5-0D19205CAFE9@dalkescientific.com>
	<CABL7CQgFeOMsW-zb7AfNsAA9auQeveJymr6m8z5ezi8q9M15RA@mail.gmail.com>
	<CANNq6FkWJG0cUo0HsmSNDCBU3GW=+GvCwU3ufdQAuY+-KwhEeQ@mail.gmail.com>
	<CABL7CQiLXajaDaRR_bx=P13RAf5_TpKszVJLepz1wOvX=s2BWQ@mail.gmail.com>
	<CANNq6Fn8GD5uetk0i7sa-yxqZMzS=aUmmwpJv4Wur-p=VMhCjQ@mail.gmail.com>
Message-ID: <CAB6mnxJ_4Ozv=bE=bd4cpkQ2sUFw5AtmifGuM6q7PiTNQDLq7A@mail.gmail.com>

On Sun, Aug 11, 2013 at 2:24 PM, Benjamin Root <ben.root at ou.edu> wrote:

>
> On Aug 11, 2013 5:02 AM, "Ralf Gommers" <ralf.gommers at gmail.com> wrote:
> >
> >
> >
> >
> > On Sun, Aug 11, 2013 at 3:35 AM, Benjamin Root <ben.root at ou.edu> wrote:
> >>
> >> Would there be some sort of way to detect that numpy.testing wasn't
> explicitly imported and issue a deprecation warning? Say, move the code
> into numpy._testing, import in into the namespace as testing, but then have
> the testing.py file set a flag in _testing to indicate an explicit import
> has occurred?
> >>
> >> Eventually, even _testing would no longer get imported by default and
> all will be well.
> >>
> >> Of course, that might be too convoluted?
> >
> > I'm not sure how that would work (you didn't describe how to decide that
> the import was explicit), but imho the impact would be too high.
> >
> > Ralf
> >
>
> The idea would be that within numpy (and we should fix SciPy as well), we
> would always import numpy._testing as testing, and not import testing.py
> ourselves. Then, there would be a flag in _testing.py that would be set to
> emit, by default, warnings about using np.testing without an explicit
> import, and stating which version all code will have to be switched by
> perhaps 2.0?).
>
> testing.py would do a from _testing import *, but also set the flag in
> _testing to not emit warnings, because only a non-numpy (and SciPy) module
> would have imported it.
>
> It isn't foolproof. If a project has multiple dependencies that use
> np.testing, and only one of them explicitly imports np.testing, then the
> warning becomes hidden for the non-compliant parts. However, if we make
> sure that the core SciPy projects use np._testing, it would go a long way
> to get the word out.
>
> Again, I am just throwing it out there as an idea. The speedups we are
> getting right now so far are nice, so it is entirely possible that this
> kludge is just not worth the last remaining bits of extra time.
>
>
OT: Benjamin, would you take a look at PR #3534 <http://The number of nests
varies a lot here year to year. Some years the yellow jackets have nests in
every little overhang, cranny, and all over the eaves. Other years there
are very few. I usually leave them alone and watch their comings and
goings. One year I think there was a big fight or plague because dead and
dying wasps and larvae were falling onto the ground from a nest behind some
siding. Honeybees can be worse, I once saw an old house where the bees had
taken over the inside of a whole wall. The honey was just oozing through
and dripping down the wall. That said, if I get stung I clean them buggers
out. Darwinian selection in action.>. It is the continuation of your
nanmean, nanvar, and nanstd work.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130811/82c73f1c/attachment.html>

From arnaldorusso at gmail.com  Sun Aug 11 19:10:16 2013
From: arnaldorusso at gmail.com (Arnaldo Russo)
Date: Sun, 11 Aug 2013 20:10:16 -0300
Subject: [Numpy-discussion] Nanmean review
In-Reply-To: <CABL7CQjTuzvueb-73RrBmXVahvUBRkiJeXShQcGS2HjRg9EFNQ@mail.gmail.com>
References: <91C3FC67-3D37-4674-BEB1-0E086604E232@gmail.com>
	<CABL7CQjTuzvueb-73RrBmXVahvUBRkiJeXShQcGS2HjRg9EFNQ@mail.gmail.com>
Message-ID: <CAKUv9OPY8Mq9jELONaiFbYjM=k_KrMLU-X-X6txzH_0+t+6tWQ@mail.gmail.com>

Congratulations!!

I have been waiting for a long time to see a nanmean function inside numpy.
I never looked inside this function, but I'm happy seeing this discussion
on github!

Cheers,
Arnaldo.






---
*Arnaldo D'Amaral Pereira Granja Russo*
Lab. de Estudos dos Oceanos e Clima
Instituto de Oceanografia - FURG




2013/8/10 Ralf Gommers <ralf.gommers at gmail.com>

>
>
>
> On Sat, Aug 10, 2013 at 12:48 AM, David Reed <david.reed.c at gmail.com>wrote:
>
>>
>> Hello,
>>
>> This is my first time contributing and I was hoping to get a review of a
>> change I made.  Here is the comparison link.
>>
>> https://github.com/dvreed77/numpy/compare/nanmean
>>
>
> Hi David, your contribution looks good and adding nanmean is a good idea.
> However, your code happens to overlap for almost 100% with an already
> submitted pull request: https://github.com/numpy/numpy/pull/3534. I
> suggest that you compare your implementation with the one in PR-3534, and
> maybe you can add an example there or fix some inconsistency.
>
> This doesn't seem all that consistent by the way. The PR doesn't have an
> example, so I don't know if it behaves the same:
>
>     >>> np.nanmean([1, np.nan, np.inf])       inf
> >>> np.nanmean([1, np.nan, np.inf, np.NINF])       nan
> Cheers,
> Ralf
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130811/bd27059f/attachment.html>

From ben.root at ou.edu  Sun Aug 11 23:27:40 2013
From: ben.root at ou.edu (Benjamin Root)
Date: Sun, 11 Aug 2013 23:27:40 -0400
Subject: [Numpy-discussion] import overhead of numpy.testing
In-Reply-To: <37CC8D40-97A0-4961-A3CA-E9D96041ECF2@dalkescientific.com>
References: <D37A1790-EB87-4EA4-8D1F-D890199F4CA7@dalkescientific.com>
	<CAB6mnxKqWO7H-gBV2FSwMQDJje3TXYpdZDU_-wVaJcLTgtmeqA@mail.gmail.com>
	<142AB120-3E54-44AC-8DCF-5470A2FC8AB3@dalkescientific.com>
	<CABL7CQg9iLuw0J6T6KVtdwJJWTZsfOut=wWUgjWoVpr1Nud_6g@mail.gmail.com>
	<B4E4287A-01B6-46A3-BFB5-0D19205CAFE9@dalkescientific.com>
	<CABL7CQgFeOMsW-zb7AfNsAA9auQeveJymr6m8z5ezi8q9M15RA@mail.gmail.com>
	<CANNq6FkWJG0cUo0HsmSNDCBU3GW=+GvCwU3ufdQAuY+-KwhEeQ@mail.gmail.com>
	<CABL7CQiLXajaDaRR_bx=P13RAf5_TpKszVJLepz1wOvX=s2BWQ@mail.gmail.com>
	<CANNq6Fn8GD5uetk0i7sa-yxqZMzS=aUmmwpJv4Wur-p=VMhCjQ@mail.gmail.com>
	<37CC8D40-97A0-4961-A3CA-E9D96041ECF2@dalkescientific.com>
Message-ID: <CANNq6Fkxy7Y2EmFBTWX2OzFeWEYz1Zr4tTUHxeTYZAeCq+VfsQ@mail.gmail.com>

On Aug 11, 2013 4:37 PM, "Andrew Dalke" <dalke at dalkescientific.com> wrote:
>
> On Aug 11, 2013, at 10:24 PM, Benjamin Root wrote:
> > The idea would be that within numpy (and we should fix SciPy as well),
we would always import numpy._testing as testing, and not import testing.py
ourselves.
>
> The problem is the existing code out there which does:
>
> import numpy as np
>  ...
> np.testing.utils.assert_almost_equal(x, y)
>
> (That is, without an additional import), and other code which does
>
> from numpy.testing import *
>

I wouldn't consider having then both emit a warning. The latter one is an
explicit import (albeit horrible). Iirc, that should import the testing.py,
and deactivate the warnings.

However, "from numpy import testing" would be a problem... Drat...

Forget I said anything. The idea wouldn't work.

Ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130811/06d72d29/attachment.html>

From jtaylor.debian at googlemail.com  Mon Aug 12 10:23:34 2013
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Mon, 12 Aug 2013 16:23:34 +0200
Subject: [Numpy-discussion] quickselect via np.partition available in 1.8.dev
In-Reply-To: <51B7A595.2090306@googlemail.com>
References: <CAB6mnxJtcm_VdqgLtMtxszQ1+MUW6kvf2d82noLW=fuxOe8EsA@mail.gmail.com>
	<CAMMTP+ArWn1hVswH6GXfJSbBtsAg3j9RZd2R52GzcjAeMjynEQ@mail.gmail.com>
	<51A62C0F.6040109@googlemail.com>
	<CAMMTP+D_hjjotGHcgG1PxFrS-yjEB2z5LXL=qYasi-E226NLZg@mail.gmail.com>
	<CAMMTP+A5YttfZQ2iXv_VckDZH+FRgUp4D+CCE6AhF3CS-C4Hmg@mail.gmail.com>
	<51B49EEC.6000400@googlemail.com>
	<E22CE11B-A88C-495B-957D-76E79CE40B74@gmail.com>
	<51B7A595.2090306@googlemail.com>
Message-ID: <5208EFE6.4040507@googlemail.com>

Hi,
a selection algorithm [0] has now landed in the numpy development branch 
[1].
The function exposing it is:
numpy.partition(data, kth=int/array, axis=-1, kind="introselect", 
order=None)
Please see the docstrings on what it actually does (and report if they 
are confusing).

Thanks to the numpy developers for the review and thanks to all who 
provided comments.

If you have a program which might benefit from using selection instead 
of sorting please try it out and report if you are happy with its result 
and the api.

As first function in numpy median has been converted to use partition so 
it now scales linear with the datasize instead of linearithmic. You can 
probably expect a five times speedup for array sizes around 1e6.

But this also involves a slight change in behavior for the case where 
you use overwrite_input=True
In the past this sorted the full array, now it will only be partially 
sorted. That the array is sorted was always documented as an 
implementation detail so hopefully that won't break any code.

The next function to be adapted will most likely be numpy.percentile.

[0] http://en.wikipedia.org/wiki/Selection_algorithm
[1] https://github.com/numpy/numpy/pull/3360


From stefan at sun.ac.za  Mon Aug 12 12:30:36 2013
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Mon, 12 Aug 2013 18:30:36 +0200
Subject: [Numpy-discussion] quickselect via np.partition available in
	1.8.dev
In-Reply-To: <5208EFE6.4040507@googlemail.com>
References: <CAB6mnxJtcm_VdqgLtMtxszQ1+MUW6kvf2d82noLW=fuxOe8EsA@mail.gmail.com>
	<CAMMTP+ArWn1hVswH6GXfJSbBtsAg3j9RZd2R52GzcjAeMjynEQ@mail.gmail.com>
	<51A62C0F.6040109@googlemail.com>
	<CAMMTP+D_hjjotGHcgG1PxFrS-yjEB2z5LXL=qYasi-E226NLZg@mail.gmail.com>
	<CAMMTP+A5YttfZQ2iXv_VckDZH+FRgUp4D+CCE6AhF3CS-C4Hmg@mail.gmail.com>
	<51B49EEC.6000400@googlemail.com>
	<E22CE11B-A88C-495B-957D-76E79CE40B74@gmail.com>
	<51B7A595.2090306@googlemail.com> <5208EFE6.4040507@googlemail.com>
Message-ID: <CABDkGQnZnWuUjCa3r5T=Dd43Q3=vQ8ZK+h3O2ey1vGvxAQy-nw@mail.gmail.com>

Hi Julian

On Mon, Aug 12, 2013 at 4:23 PM, Julian Taylor
<jtaylor.debian at googlemail.com> wrote:
> The function exposing it is:
> numpy.partition(data, kth=int/array, axis=-1, kind="introselect",
> order=None)

This looks great, thanks very much!

A minor bug was introduced into the Bento build:

https://github.com/numpy/numpy/pull/3606
https://github.com/numpy/numpy/pull/3607

St?fan


From Nicolas.Rougier at inria.fr  Mon Aug 12 17:01:26 2013
From: Nicolas.Rougier at inria.fr (Nicolas Rougier)
Date: Mon, 12 Aug 2013 23:01:26 +0200
Subject: [Numpy-discussion] Using as_strided to avoid copy on repeat (with
	2-dimensional array)
Message-ID: <E9D40DEC-6C3D-4EBF-97F0-4C9B08F958CD@inria.fr>


Hi,

I have a (n,2) shaped array representing points and I would like to double each point as in:

A = np.arange(10*2).reshape(10,2)
B = np.repeat( A, 2, axis=0 )

Is it possible to do the same using 'as_strided' to avoid copy (and still get the same output shape for B) ?


I found this reference:
http://stackoverflow.com/questions/5564098/repeat-numpy-array-without-replicating-data

but did not manage to adapt it to my case.


Bonus question (if previous 'as_strided' trick is possible), would this work or do I need a different construct ?

A = np.arange(10*2).reshape(10,2)
B = np.repeat( A[::2], 2, axis=0 )



Nicolas

From robert.kern at gmail.com  Mon Aug 12 17:23:29 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 12 Aug 2013 22:23:29 +0100
Subject: [Numpy-discussion] Using as_strided to avoid copy on repeat
 (with 2-dimensional array)
In-Reply-To: <E9D40DEC-6C3D-4EBF-97F0-4C9B08F958CD@inria.fr>
References: <E9D40DEC-6C3D-4EBF-97F0-4C9B08F958CD@inria.fr>
Message-ID: <CAF6FJitHfxCBaLzcTYuAt4VgoPdY3xTKJ8LFxxbWMojgEkVXUQ@mail.gmail.com>

On Mon, Aug 12, 2013 at 10:01 PM, Nicolas Rougier <Nicolas.Rougier at inria.fr>
wrote:
>
>
> Hi,
>
> I have a (n,2) shaped array representing points and I would like to
double each point as in:
>
> A = np.arange(10*2).reshape(10,2)
> B = np.repeat( A, 2, axis=0 )
>
> Is it possible to do the same using 'as_strided' to avoid copy (and still
get the same output shape for B) ?

No, this would not be uniformly strided in the 0 axis.

--
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130812/42009266/attachment.html>

From charlesr.harris at gmail.com  Mon Aug 12 17:41:17 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 12 Aug 2013 15:41:17 -0600
Subject: [Numpy-discussion] About ready to start 1.8 release process.
Message-ID: <CAB6mnxJi8LF2_1CamBzR0yV_ea07ioyNdbmzx+sDKxiWAgerHw@mail.gmail.com>

Hi All,

I think we are about ready to start on the 1.8 release. There are a few
things left to do, but there are PR's for most of them. There are a couple
of issues that I think need discussion.

   1. Should diagonal return a view in this release or the next?
   2. Should multiple field selection return a view in this release of the
   next?
   3. Datetime64 will not be modified in this release.

I'd also like to do a major whitespace clean up before the release, a lot
has crept in. A style clean up, spaces after commas, etc., might also be
desirable but I think it can wait and be done piecemeal.

If there is anything that you think* has to be done*, please comment.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130812/40af3007/attachment.html>

From chris.barker at noaa.gov  Mon Aug 12 18:05:02 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Mon, 12 Aug 2013 15:05:02 -0700
Subject: [Numpy-discussion] About ready to start 1.8 release process.
In-Reply-To: <CAB6mnxJi8LF2_1CamBzR0yV_ea07ioyNdbmzx+sDKxiWAgerHw@mail.gmail.com>
References: <CAB6mnxJi8LF2_1CamBzR0yV_ea07ioyNdbmzx+sDKxiWAgerHw@mail.gmail.com>
Message-ID: <CALGmxE+ug-gwrZEaTEyUFX3vko-4VhVS+34vc=yp2KYzB17FAQ@mail.gmail.com>

On Mon, Aug 12, 2013 at 2:41 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:

> Datetime64 will not be modified in this release.

I now there is neither the time nor the will for all that it needs,
but please, please, please, can we yank out the broken timezone
handling at least?

https://github.com/numpy/numpy/issues/3290

-Chris



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From charlesr.harris at gmail.com  Mon Aug 12 18:14:25 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 12 Aug 2013 16:14:25 -0600
Subject: [Numpy-discussion] About ready to start 1.8 release process.
In-Reply-To: <CALGmxE+ug-gwrZEaTEyUFX3vko-4VhVS+34vc=yp2KYzB17FAQ@mail.gmail.com>
References: <CAB6mnxJi8LF2_1CamBzR0yV_ea07ioyNdbmzx+sDKxiWAgerHw@mail.gmail.com>
	<CALGmxE+ug-gwrZEaTEyUFX3vko-4VhVS+34vc=yp2KYzB17FAQ@mail.gmail.com>
Message-ID: <CAB6mnxKS5Rg7As+fy4cfMduHsh7fuNQ=-__fFcbzKRdDwgq1rA@mail.gmail.com>

On Mon, Aug 12, 2013 at 4:05 PM, Chris Barker - NOAA Federal <
chris.barker at noaa.gov> wrote:

> On Mon, Aug 12, 2013 at 2:41 PM, Charles R Harris
> <charlesr.harris at gmail.com> wrote:
>
> > Datetime64 will not be modified in this release.
>
> I now there is neither the time nor the will for all that it needs,
> but please, please, please, can we yank out the broken timezone
> handling at least?
>
> https://github.com/numpy/numpy/issues/3290
>

You need to be a bit more specific, what parts should be yanked out? I'm
also worried about breaking peoples' work arounds this late in the game.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130812/cea2617c/attachment.html>

From chris.barker at noaa.gov  Mon Aug 12 19:16:21 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Mon, 12 Aug 2013 16:16:21 -0700
Subject: [Numpy-discussion] About ready to start 1.8 release process.
In-Reply-To: <CAB6mnxKS5Rg7As+fy4cfMduHsh7fuNQ=-__fFcbzKRdDwgq1rA@mail.gmail.com>
References: <CAB6mnxJi8LF2_1CamBzR0yV_ea07ioyNdbmzx+sDKxiWAgerHw@mail.gmail.com>
	<CALGmxE+ug-gwrZEaTEyUFX3vko-4VhVS+34vc=yp2KYzB17FAQ@mail.gmail.com>
	<CAB6mnxKS5Rg7As+fy4cfMduHsh7fuNQ=-__fFcbzKRdDwgq1rA@mail.gmail.com>
Message-ID: <CALGmxE+h1yQtMm+7CJLJtvnkCMWVHQu4PfLfWesNuH08n5+qnw@mail.gmail.com>

On Mon, Aug 12, 2013 at 3:14 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:

>> > Datetime64 will not be modified in this release.
>>
>> I now there is neither the time nor the will for all that it needs,
>> but please, please, please, can we yank out the broken timezone
>> handling at least?
>>
>> https://github.com/numpy/numpy/issues/3290
>
> You need to be a bit more specific, what parts should be yanked out?

It's pretty well discussed in issue 3290 -- but what I'm suggesting is
essentially to ignore time zones completely -- i.e. make the
datetime64 "naive" with respect to time zones.

In fact, it already is -- the only timezone handling it has is that if
it parses an ISO string and no timezone is specified, it applies the
Locale time zone -- this is pretty broken behavior. At the moment, I
can't recall what it does with a datetime.datetime instance, but it's
not quite consitent with what it does parsing string.

I _think_ the only point of contention in that issue 3290 discussion
is how datetime64 should parse and ISO string that provides an offset:

1) ignore it -- probably a bad idea
2) raise an error -- you can't do it.
3) apply the offset so that the resulting datetime64 is assumed to be UTC.

Personally, I think (2) is the way to go, with the possible addition
of allowing zero offset, 'cause, why not?

> I'm
> also worried about breaking peoples' work arounds this late in the game.

well, that is an issue -- though I think most work-arounds will not be
broken, and the discussion about this didn't reveal a single user that
actually used the "use the Locale time zone" feature -- granted,
people contributing to the discussion is a pretty small subset of
users.

But in 1.7 datetime64 is officially experimental, and keeping a broken
implementation around longer will only make for more broken code
later.

-Chris



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From Thomas.Goebel at th-nuernberg.de  Tue Aug 13 02:50:46 2013
From: Thomas.Goebel at th-nuernberg.de (Thomas Goebel)
Date: Tue, 13 Aug 2013 08:50:46 +0200
Subject: [Numpy-discussion] Optimize removing nan-values of dataset
Message-ID: <5209D746.3000807@th-nuernberg.de>

Hi,

i am trying to remove nan-values from an array of shape(40, 6).
These nan-values at point data[x] should be replaced by the mean
of data[x-1] and data[x+1] if both values at x-1 and x+1 are not
nan. The function nan_to_mean (see below) is working but i wonder
if i could optimize the code.

I thought about something like
  1. Find all nan values in array:
     nans = np.isnan(dataarray)
  2. Check if values before, after nan indice are not nan
  3. Calculate mean

While using this script for my original dataset of
shape(63856, 6) it takes 139.343 seconds to run it. And some
datasets are even bigger. I attached the example_dataset.txt and
the example.py script.

Thanks for any help,
Tom

def nan_to_mean(arr):
    for cnt, value in enumerate(arr):
        # Check if first value is nan, if so continue
        if cnt == 0 and np.isnan(value):
            continue
        # Check if last value is nan:
        #     If x-1 value is nan dont do anything!
        #     If x-1 is float, last value will be value of x-1
        elif cnt == (len(arr)-1):
            if np.isnan(value) and not np.isnan(arr[cnt-1]):
                arr[cnt] = arr[cnt-1]
        # If the first values of file are nan ignore them all
        elif np.isnan(value) and np.isnan(arr[cnt-1]):
            continue
        # Found nan value and x-1 value is of type float
        elif np.isnan(value) and not np.isnan(arr[cnt-1]):
            # Check if x+1 value is not nan
            if not np.isnan(arr[cnt+1]):
                arr[cnt] = '%.1f' % np.mean((
                        arr[cnt-1],arr[cnt+1]))
            # If x+1 value is nan, go to next value
            else:
                for N in xrange(2, 30):
                    if cnt+N == (len(arr)):
                        break
                    elif not np.isnan(arr[cnt+N]):
                        arr[cnt] = '%.1f' % np.mean(
                                (arr[cnt-1], arr[cnt+N]))
    return arr
-------------- next part --------------
16.04.2013;18:09:13;nan;nan;nan;nan;nan;nan
16.04.2013;18:09:18;nan;nan;nan;nan;nan;nan
16.04.2013;18:09:23;nan;nan;nan;nan;nan;nan
16.04.2013;18:09:28;nan;nan;nan;nan;nan;nan
16.04.2013;18:09:33;nan;nan;nan;nan;nan;nan
16.04.2013;18:09:38;nan;nan;nan;nan;nan;nan
16.04.2013;18:09:43;23.9;38.4;nan;nan;nan;nan
16.04.2013;18:09:48;23.9;38.7;nan;nan;nan;nan
16.04.2013;18:09:53;23.9;38.8;23.9;38.3;nan;nan
16.04.2013;18:09:58;23.9;38.8;23.9;38.3;nan;nan
16.04.2013;18:10:03;nan;nan;nan;nan;24.7;37
16.04.2013;18:10:08;24.1;38.6;23.9;38.2;nan;nan
16.04.2013;18:10:13;nan;nan;23.9;38.3;24.6;37.3
16.04.2013;18:10:18;24.1;38.6;23.9;38.3;24.6;37.3
16.04.2013;18:10:23;24.1;38.3;23.9;38.4;24.5;37
16.04.2013;18:10:28;24.2;38.6;23.9;38.1;nan;nan
16.04.2013;18:10:33;24.2;38.6;23.8;38.2;24.5;37.3
16.04.2013;18:10:38;nan;nan;23.8;38.2;nan;nan
16.04.2013;18:10:43;24.2;38.5;23.8;38.1;24.4;37.9
16.04.2013;18:10:48;24.2;38.4;23.8;38.1;24.4;37.5
16.04.2013;18:10:53;24.3;38.3;nan;nan;nan;nan
16.04.2013;18:10:58;nan;nan;23.9;38.1;24.3;37.6
16.04.2013;18:11:03;24.4;38.2;23.9;38.2;24.4;37.6
16.04.2013;18:11:08;24.4;38.2;23.9;38.5;24.3;38.2
16.04.2013;18:11:13;24.4;38.2;23.9;38.8;24.3;38.3
16.04.2013;18:11:18;24.4;39.1;23.9;39;24.3;38.4
16.04.2013;18:11:23;24.4;39.8;23.9;38.5;24.2;38.4
16.04.2013;18:11:28;24.4;41.8;nan;nan;24.1;38.5
16.04.2013;18:12:13;24.6;34.5;nan;nan;23.4;37.2
16.04.2013;18:12:18;24.6;34.4;23.4;36.5;nan;nan
16.04.2013;18:12:23;24.6;34.6;23.3;36.6;nan;nan
16.04.2013;18:12:28;nan;nan;23.2;36.6;23.2;37.4
16.04.2013;18:12:33;nan;nan;23.2;36.9;nan;nan
16.04.2013;18:12:38;nan;nan;nan;nan;23.1;37.8
16.04.2013;18:12:43;24.6;35.1;nan;nan;23.1;37.9
16.04.2013;18:12:48;24.6;35.1;23;37.4;nan;nan
16.04.2013;18:12:53;nan;nan;23;38;nan;nan
16.04.2013;18:12:58;nan;nan;nan;nan;23;37.9
16.04.2013;18:13:03;nan;nan;23;37.9;23;37.7
16.04.2013;18:13:08;nan;nan;23;37.7;23;37.8
-------------- next part --------------
A non-text attachment was scrubbed...
Name: example.py
Type: text/x-python
Size: 1590 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130813/dce44e61/attachment.py>

From stefan at sun.ac.za  Tue Aug 13 05:54:52 2013
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Tue, 13 Aug 2013 11:54:52 +0200
Subject: [Numpy-discussion] Using as_strided to avoid copy on repeat
 (with 2-dimensional array)
In-Reply-To: <CAF6FJitHfxCBaLzcTYuAt4VgoPdY3xTKJ8LFxxbWMojgEkVXUQ@mail.gmail.com>
References: <E9D40DEC-6C3D-4EBF-97F0-4C9B08F958CD@inria.fr>
	<CAF6FJitHfxCBaLzcTYuAt4VgoPdY3xTKJ8LFxxbWMojgEkVXUQ@mail.gmail.com>
Message-ID: <CABDkGQnnxZG_9yHFk05iPtrUZotAFd75H3ybfXTaPKvcm=6Nfw@mail.gmail.com>

On Mon, Aug 12, 2013 at 11:23 PM, Robert Kern <robert.kern at gmail.com> wrote:
>> Is it possible to do the same using 'as_strided' to avoid copy (and still
>> get the same output shape for B) ?
>
> No, this would not be uniformly strided in the 0 axis.

Now, if only we supported simple fraction strides...

St?fan


From tmp50 at ukr.net  Tue Aug 13 06:30:40 2013
From: tmp50 at ukr.net (Dmitrey)
Date: Tue, 13 Aug 2013 13:30:40 +0300
Subject: [Numpy-discussion] numpy bug with negative int pow
Message-ID: <1376389644.102368986.jjc8hhxg2fmst-1.ukr.net>

Python 3.3.1 (default, Apr 17 2013, 22:32:14) 
[GCC 4.7.3] on linux 

>>> import numpy 
>>> numpy.__version__ 
'1.8.0.dev-d62f11d' 

>>> numpy.array((1,2,3)) / 2 
array([ 0.5,? 1. ,? 1.5]) 

#ok, but since division of integer arrays has been converted to float, pow is expected as well, but it's not: 

>>> numpy.array((1,2,3)) ** -1 
array([1, 0, 0], dtype=int32) 

>>> numpy.array((1,2,3)) ** -2 
array([1, 0, 0], dtype=int32) 

>>> 3**-1 
0.3333333333333333 

D. 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130813/df7d0710/attachment.html>

From l.resmi at gmail.com  Tue Aug 13 08:20:05 2013
From: l.resmi at gmail.com (Resmi)
Date: Tue, 13 Aug 2013 17:50:05 +0530
Subject: [Numpy-discussion] Reading footer lines
Message-ID: <CA+0FijsLkXnUgN84m4+m1v6g=Bqt7b4oX9sEgWXusuRdB8irhg@mail.gmail.com>

Hi,

I've a list of long files of numerical data ending with footer lines
(beginning with #). I am using numpy.loadtxt to read the numbers, and
loadtxt ignores these footer lines. I want the numpy code to read one of
the footer lines and extract words from it. Is there a way to use loadtxt
for this? If there weren't many files I could have used the line
number (which keep varying between the files) of the footer line along with
*linecache.*
Nevertheless there should be a generic way to do this in numpy?

As a workaround, I've tried using os.system along with grep. And I get the
following output :

*>>> os.system("grep -e 'tx' 'data.dat' ")*
* ## tx =    2023.06 *
*0*

Why is there a 0 in the output? The file has no blank lines.

Since I want the value 2023.06 in the numpy code for later use I tried to
pipe the output to a variable:-
*test = os.system(command)*
But that isn't working as *test* is getting assigned the value 0.
Tried *subprocess.call(['grep','-e','tx','data.dat'])* which is also ending
up in the same fashion.

It'll be great if I can get to know (i) a way to read the footer lines (ii)
to sort out the operation of os.system and subprocess.call output
processing.

Thanks,
Resmi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130813/c4bd3aa7/attachment.html>

From p.j.a.cock at googlemail.com  Tue Aug 13 08:52:48 2013
From: p.j.a.cock at googlemail.com (Peter Cock)
Date: Tue, 13 Aug 2013 13:52:48 +0100
Subject: [Numpy-discussion] Reading footer lines
In-Reply-To: <CA+0FijsLkXnUgN84m4+m1v6g=Bqt7b4oX9sEgWXusuRdB8irhg@mail.gmail.com>
References: <CA+0FijsLkXnUgN84m4+m1v6g=Bqt7b4oX9sEgWXusuRdB8irhg@mail.gmail.com>
Message-ID: <CAKVJ-_5_GXh_8fHtkG_rYSPFxf43GXJ3p3Ez32FOcR6N+fr83Q@mail.gmail.com>

On Tue, Aug 13, 2013 at 1:20 PM, Resmi <l.resmi at gmail.com> wrote:
> Hi,
>
> I've a list of long files of numerical data ending with footer lines
> (beginning with #). I am using numpy.loadtxt to read the numbers, and
> loadtxt ignores these footer lines. I want the numpy code to read one of the
> footer lines and extract words from it. Is there a way to use loadtxt for
> this? If there weren't many files I could have used the line number (which
> keep varying between the files) of the footer line along with linecache.
> Nevertheless there should be a generic way to do this in numpy?
>
> As a workaround, I've tried using os.system along with grep. And I get the
> following output :
>
>>>> os.system("grep -e 'tx' 'data.dat' ")
>  ## tx =    2023.06
> 0
>
> Why is there a 0 in the output? The file has no blank lines.

The os.system function call returns the integer return value
(error level) of the command invoked, by convention zero
for success but the value is set by the tool (here grep).

> Since I want the value 2023.06 in the numpy code for later use I tried to
> pipe the output to a variable:-
> test = os.system(command)
> But that isn't working as test is getting assigned the value 0.
> Tried subprocess.call(['grep','-e','tx','data.dat']) which is also ending up
> in the same fashion.
>
> It'll be great if I can get to know (i) a way to read the footer lines (ii)
> to sort out the operation of os.system and subprocess.call output
> processing.

Don't use os.system, instead you should use subprocess:
http://docs.python.org/2/library/subprocess.html

The standard library module commands would also work
but is not cross platform, and is also deprecated in favour
of subprocess: http://docs.python.org/2/library/commands.html

Peter


From stefan at sun.ac.za  Tue Aug 13 09:00:36 2013
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Tue, 13 Aug 2013 15:00:36 +0200
Subject: [Numpy-discussion] Reading footer lines
In-Reply-To: <CA+0FijsLkXnUgN84m4+m1v6g=Bqt7b4oX9sEgWXusuRdB8irhg@mail.gmail.com>
References: <CA+0FijsLkXnUgN84m4+m1v6g=Bqt7b4oX9sEgWXusuRdB8irhg@mail.gmail.com>
Message-ID: <CABDkGQ=B6cU-emweTUvY=aQjku9ng+-DrqJMr1bhXhbDE6Wf+A@mail.gmail.com>

Hi Resmi

On Tue, Aug 13, 2013 at 2:20 PM, Resmi <l.resmi at gmail.com> wrote:
> I've a list of long files of numerical data ending with footer lines
> (beginning with #). I am using numpy.loadtxt to read the numbers, and
> loadtxt ignores these footer lines. I want the numpy code to read one of the
> footer lines and extract words from it. Is there a way to use loadtxt for
> this? If there weren't many files I could have used the line number (which
> keep varying between the files) of the footer line along with linecache.
> Nevertheless there should be a generic way to do this in numpy?

If the data-set is not too big, you can read it into memory with
"f.readlines()" and then do a bit of preparsing on the resulting array
before sending the rest of it to np.loadtxt (which allows an array of
strings as input).

St?fan


From davidmenhur at gmail.com  Tue Aug 13 09:01:34 2013
From: davidmenhur at gmail.com (=?UTF-8?B?RGHPgGlk?=)
Date: Tue, 13 Aug 2013 15:01:34 +0200
Subject: [Numpy-discussion] Reading footer lines
In-Reply-To: <CA+0FijsLkXnUgN84m4+m1v6g=Bqt7b4oX9sEgWXusuRdB8irhg@mail.gmail.com>
References: <CA+0FijsLkXnUgN84m4+m1v6g=Bqt7b4oX9sEgWXusuRdB8irhg@mail.gmail.com>
Message-ID: <CAJhcF=3i0svvq9yfT-bXDb4MC=ANDuVPdgqSFcwD-zRHa5_49A@mail.gmail.com>

On 13 August 2013 14:20, Resmi <l.resmi at gmail.com> wrote:
> As a workaround, I've tried using os.system along with grep. And I get the
> following output :
>
>>>> os.system("grep -e 'tx' 'data.dat' ")
>  ## tx =    2023.06
> 0
>
> Why is there a 0 in the output? The file has no blank lines.

That 0 corresponds to the exit status of os.system, as there were no
errors, it returns a 0.

My idea would be to read the file and store it into a StringIO. That
can be fed to np.loadtxt to extract the numbers, and look for your
number. My (untested) attempt:


import StringIO

with datafile open as f:
... raw_data = StringIO.StringIO()
... raw_data.write(f.read())

data = np.loadfromtxt(raw_data)
numbers = [float(line.split['='][1].strip()) for line in
raw_data.readlines() if line[0] == '#']

If there is only one number there and it is after all the data, you
could skip them all. From the shape of data you know there are N lines
of data:

for _ in range(N):
... raw_data.readline()
while True:
... line = raw_data.readline()
... if line[0] == '#':
...... number = float(line.split['='][1].strip())
...... break


Alternatively, you could also use seek to put the pointer a certain
distance from the end of the file and start from there, but this could
cause problems in Windows.




David.


From chris.barker at noaa.gov  Tue Aug 13 11:50:28 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Tue, 13 Aug 2013 08:50:28 -0700
Subject: [Numpy-discussion] Reading footer lines
In-Reply-To: <CAJhcF=3i0svvq9yfT-bXDb4MC=ANDuVPdgqSFcwD-zRHa5_49A@mail.gmail.com>
References: <CA+0FijsLkXnUgN84m4+m1v6g=Bqt7b4oX9sEgWXusuRdB8irhg@mail.gmail.com>
	<CAJhcF=3i0svvq9yfT-bXDb4MC=ANDuVPdgqSFcwD-zRHa5_49A@mail.gmail.com>
Message-ID: <CALGmxE+W6JSTxGhjE6is5LSVvzq4Z+_wAQ27f8feSM0XOBWHnQ@mail.gmail.com>

On Tue, Aug 13, 2013 at 6:01 AM, Da?id <davidmenhur at gmail.com> wrote:

> Alternatively, you could also use seek to put the pointer a certain
> distance from the end of the file and start from there,

That's what I'd do if the file(s) may be too large to simply dump into memory.

> but this could cause problems in Windows.

what problems? this is pretty straightforward stuff!

-Chris


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From pengyu.ut at gmail.com  Tue Aug 13 12:10:55 2013
From: pengyu.ut at gmail.com (Peng Yu)
Date: Tue, 13 Aug 2013 11:10:55 -0500
Subject: [Numpy-discussion] Do the prebuilt packages work with the python
	version that I compile myself?
Message-ID: <CABrM6w=Wdom=MpnHL2kjiV6v01d+NMTPQUmjCoxZ9qMSa7b8CQ@mail.gmail.com>

Hi,

I compile the latest python myself on Ubuntu. In this case, I think
the packages listed on the following webpage does not work with the
python version that I compiled. Am I correct? Thanks.

http://www.scipy.org/install.html

--
Regards,
Peng


From pengyu.ut at gmail.com  Tue Aug 13 13:03:30 2013
From: pengyu.ut at gmail.com (Peng Yu)
Date: Tue, 13 Aug 2013 12:03:30 -0500
Subject: [Numpy-discussion] How to compile numpy on ubuntu
Message-ID: <CABrM6wkSOHPBC5U0n3RLV4xPvDbuKfzQaoEfqkr78AurBwPn2A@mail.gmail.com>

Hi,

The instructions for compiling numpy on ubuntu seems to be very
outdated. Could anybody provide an updated instruction on installing
numpy? Thanks.

http://docs.scipy.org/doc/numpy/user/install.html

-- 
Regards,
Peng


From pmhobson at gmail.com  Tue Aug 13 14:38:28 2013
From: pmhobson at gmail.com (Paul Hobson)
Date: Tue, 13 Aug 2013 11:38:28 -0700
Subject: [Numpy-discussion] How to compile numpy on ubuntu
In-Reply-To: <CABrM6wkSOHPBC5U0n3RLV4xPvDbuKfzQaoEfqkr78AurBwPn2A@mail.gmail.com>
References: <CABrM6wkSOHPBC5U0n3RLV4xPvDbuKfzQaoEfqkr78AurBwPn2A@mail.gmail.com>
Message-ID: <CADT3MEC3K1U=RCpe_vA+r5CE8+_+wUsWXCUU-5PY=Zoufh+_9A@mail.gmail.com>

On Tue, Aug 13, 2013 at 10:03 AM, Peng Yu <pengyu.ut at gmail.com> wrote:

> Hi,
>
> The instructions for compiling numpy on ubuntu seems to be very
> outdated. Could anybody provide an updated instruction on installing
> numpy? Thanks.
>
> http://docs.scipy.org/doc/numpy/user/install.html


The latest instructions should still work fine. I didn't have any problems
on Ubuntu 12.04
-paul
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130813/e3bd284e/attachment.html>

From ralf.gommers at gmail.com  Tue Aug 13 14:49:02 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Tue, 13 Aug 2013 20:49:02 +0200
Subject: [Numpy-discussion] Do the prebuilt packages work with the
 python version that I compile myself?
In-Reply-To: <CABrM6w=Wdom=MpnHL2kjiV6v01d+NMTPQUmjCoxZ9qMSa7b8CQ@mail.gmail.com>
References: <CABrM6w=Wdom=MpnHL2kjiV6v01d+NMTPQUmjCoxZ9qMSa7b8CQ@mail.gmail.com>
Message-ID: <CABL7CQj=JS=EN=_xWQy94nm9JYB2ooxmMndeGYs_psyuZiYAgw@mail.gmail.com>

On Tue, Aug 13, 2013 at 6:10 PM, Peng Yu <pengyu.ut at gmail.com> wrote:

> Hi,
>
> I compile the latest python myself on Ubuntu. In this case, I think
> the packages listed on the following webpage does not work with the
> python version that I compiled. Am I correct? Thanks.
>
> http://www.scipy.org/install.html
>

If you're talking about the listed distributions, they ship including
Python itself. If you compile Python, you should compile all packages you
want to use with it.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130813/dd23d2b2/attachment.html>

From david.reed.c at gmail.com  Tue Aug 13 17:32:47 2013
From: david.reed.c at gmail.com (David Reed)
Date: Tue, 13 Aug 2013 17:32:47 -0400
Subject: [Numpy-discussion] Optimize removing nan-values of dataset
In-Reply-To: <5209D746.3000807@th-nuernberg.de>
References: <5209D746.3000807@th-nuernberg.de>
Message-ID: <CAM6XA7nis8FQMC7ehr9tK2wb4PFYA4ryCrOKYYPTP6_UFbipvQ@mail.gmail.com>

Hi Thomas,

Your array is Nx6 do you want the nan values replace by the mean of the 2
adjacent elemets by row or by column?


On Tue, Aug 13, 2013 at 2:50 AM, Thomas Goebel <
Thomas.Goebel at th-nuernberg.de> wrote:

> Hi,
>
> i am trying to remove nan-values from an array of shape(40, 6).
> These nan-values at point data[x] should be replaced by the mean
> of data[x-1] and data[x+1] if both values at x-1 and x+1 are not
> nan. The function nan_to_mean (see below) is working but i wonder
> if i could optimize the code.
>
> I thought about something like
>   1. Find all nan values in array:
>      nans = np.isnan(dataarray)
>   2. Check if values before, after nan indice are not nan
>   3. Calculate mean
>
> While using this script for my original dataset of
> shape(63856, 6) it takes 139.343 seconds to run it. And some
> datasets are even bigger. I attached the example_dataset.txt and
> the example.py script.
>
> Thanks for any help,
> Tom
>
> def nan_to_mean(arr):
>     for cnt, value in enumerate(arr):
>         # Check if first value is nan, if so continue
>         if cnt == 0 and np.isnan(value):
>             continue
>         # Check if last value is nan:
>         #     If x-1 value is nan dont do anything!
>         #     If x-1 is float, last value will be value of x-1
>         elif cnt == (len(arr)-1):
>             if np.isnan(value) and not np.isnan(arr[cnt-1]):
>                 arr[cnt] = arr[cnt-1]
>         # If the first values of file are nan ignore them all
>         elif np.isnan(value) and np.isnan(arr[cnt-1]):
>             continue
>         # Found nan value and x-1 value is of type float
>         elif np.isnan(value) and not np.isnan(arr[cnt-1]):
>             # Check if x+1 value is not nan
>             if not np.isnan(arr[cnt+1]):
>                 arr[cnt] = '%.1f' % np.mean((
>                         arr[cnt-1],arr[cnt+1]))
>             # If x+1 value is nan, go to next value
>             else:
>                 for N in xrange(2, 30):
>                     if cnt+N == (len(arr)):
>                         break
>                     elif not np.isnan(arr[cnt+N]):
>                         arr[cnt] = '%.1f' % np.mean(
>                                 (arr[cnt-1], arr[cnt+N]))
>     return arr
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130813/785dd606/attachment.html>

From matthew.brett at gmail.com  Tue Aug 13 20:45:51 2013
From: matthew.brett at gmail.com (Matthew Brett)
Date: Tue, 13 Aug 2013 17:45:51 -0700
Subject: [Numpy-discussion] About ready to start 1.8 release process.
In-Reply-To: <CALGmxE+h1yQtMm+7CJLJtvnkCMWVHQu4PfLfWesNuH08n5+qnw@mail.gmail.com>
References: <CAB6mnxJi8LF2_1CamBzR0yV_ea07ioyNdbmzx+sDKxiWAgerHw@mail.gmail.com>
	<CALGmxE+ug-gwrZEaTEyUFX3vko-4VhVS+34vc=yp2KYzB17FAQ@mail.gmail.com>
	<CAB6mnxKS5Rg7As+fy4cfMduHsh7fuNQ=-__fFcbzKRdDwgq1rA@mail.gmail.com>
	<CALGmxE+h1yQtMm+7CJLJtvnkCMWVHQu4PfLfWesNuH08n5+qnw@mail.gmail.com>
Message-ID: <CAH6Pt5rvrrHUNwXx4+H9a1N8aKBP-zevj4toW3sTB+ZKKbW3pg@mail.gmail.com>

Hi,

On Mon, Aug 12, 2013 at 4:16 PM, Chris Barker - NOAA Federal
<chris.barker at noaa.gov> wrote:
> On Mon, Aug 12, 2013 at 3:14 PM, Charles R Harris
> <charlesr.harris at gmail.com> wrote:
>
>>> > Datetime64 will not be modified in this release.
>>>
>>> I now there is neither the time nor the will for all that it needs,
>>> but please, please, please, can we yank out the broken timezone
>>> handling at least?
>>>
>>> https://github.com/numpy/numpy/issues/3290
>>
>> You need to be a bit more specific, what parts should be yanked out?
>
> It's pretty well discussed in issue 3290 -- but what I'm suggesting is
> essentially to ignore time zones completely -- i.e. make the
> datetime64 "naive" with respect to time zones.
>
> In fact, it already is -- the only timezone handling it has is that if
> it parses an ISO string and no timezone is specified, it applies the
> Locale time zone -- this is pretty broken behavior. At the moment, I
> can't recall what it does with a datetime.datetime instance, but it's
> not quite consitent with what it does parsing string.
>
> I _think_ the only point of contention in that issue 3290 discussion
> is how datetime64 should parse and ISO string that provides an offset:
>
> 1) ignore it -- probably a bad idea
> 2) raise an error -- you can't do it.
> 3) apply the offset so that the resulting datetime64 is assumed to be UTC.
>
> Personally, I think (2) is the way to go, with the possible addition
> of allowing zero offset, 'cause, why not?
>
>> I'm
>> also worried about breaking peoples' work arounds this late in the game.
>
> well, that is an issue -- though I think most work-arounds will not be
> broken, and the discussion about this didn't reveal a single user that
> actually used the "use the Locale time zone" feature -- granted,
> people contributing to the discussion is a pretty small subset of
> users.
>
> But in 1.7 datetime64 is officially experimental, and keeping a broken
> implementation around longer will only make for more broken code
> later.

Is it easy to identify who is currently developing against datetime64 in numpy?

Do you think it is possible to get rough consensus within this group?

Cheers,

Matthew


From charlesr.harris at gmail.com  Tue Aug 13 20:54:54 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Tue, 13 Aug 2013 18:54:54 -0600
Subject: [Numpy-discussion] About ready to start 1.8 release process.
In-Reply-To: <CAH6Pt5rvrrHUNwXx4+H9a1N8aKBP-zevj4toW3sTB+ZKKbW3pg@mail.gmail.com>
References: <CAB6mnxJi8LF2_1CamBzR0yV_ea07ioyNdbmzx+sDKxiWAgerHw@mail.gmail.com>
	<CALGmxE+ug-gwrZEaTEyUFX3vko-4VhVS+34vc=yp2KYzB17FAQ@mail.gmail.com>
	<CAB6mnxKS5Rg7As+fy4cfMduHsh7fuNQ=-__fFcbzKRdDwgq1rA@mail.gmail.com>
	<CALGmxE+h1yQtMm+7CJLJtvnkCMWVHQu4PfLfWesNuH08n5+qnw@mail.gmail.com>
	<CAH6Pt5rvrrHUNwXx4+H9a1N8aKBP-zevj4toW3sTB+ZKKbW3pg@mail.gmail.com>
Message-ID: <CAB6mnx+YmZbuqwMxJ9DYNVEWn1fiM03Du8WC5X429Vkj+bRz1Q@mail.gmail.com>

I wish it were. It seems unreasonably difficult to get constructive
feedback. Chris is pretty much the only one making the attempt and that
discussion petered out. I don't use datetime64 much myself, probably what
we need is a developer who does use that facility. Any volunteers?

Chuck


On Tue, Aug 13, 2013 at 6:45 PM, Matthew Brett <matthew.brett at gmail.com>wrote:

> Hi,
>
> On Mon, Aug 12, 2013 at 4:16 PM, Chris Barker - NOAA Federal
> <chris.barker at noaa.gov> wrote:
> > On Mon, Aug 12, 2013 at 3:14 PM, Charles R Harris
> > <charlesr.harris at gmail.com> wrote:
> >
> >>> > Datetime64 will not be modified in this release.
> >>>
> >>> I now there is neither the time nor the will for all that it needs,
> >>> but please, please, please, can we yank out the broken timezone
> >>> handling at least?
> >>>
> >>> https://github.com/numpy/numpy/issues/3290
> >>
> >> You need to be a bit more specific, what parts should be yanked out?
> >
> > It's pretty well discussed in issue 3290 -- but what I'm suggesting is
> > essentially to ignore time zones completely -- i.e. make the
> > datetime64 "naive" with respect to time zones.
> >
> > In fact, it already is -- the only timezone handling it has is that if
> > it parses an ISO string and no timezone is specified, it applies the
> > Locale time zone -- this is pretty broken behavior. At the moment, I
> > can't recall what it does with a datetime.datetime instance, but it's
> > not quite consitent with what it does parsing string.
> >
> > I _think_ the only point of contention in that issue 3290 discussion
> > is how datetime64 should parse and ISO string that provides an offset:
> >
> > 1) ignore it -- probably a bad idea
> > 2) raise an error -- you can't do it.
> > 3) apply the offset so that the resulting datetime64 is assumed to be
> UTC.
> >
> > Personally, I think (2) is the way to go, with the possible addition
> > of allowing zero offset, 'cause, why not?
> >
> >> I'm
> >> also worried about breaking peoples' work arounds this late in the game.
> >
> > well, that is an issue -- though I think most work-arounds will not be
> > broken, and the discussion about this didn't reveal a single user that
> > actually used the "use the Locale time zone" feature -- granted,
> > people contributing to the discussion is a pretty small subset of
> > users.
> >
> > But in 1.7 datetime64 is officially experimental, and keeping a broken
> > implementation around longer will only make for more broken code
> > later.
>
> Is it easy to identify who is currently developing against datetime64 in
> numpy?
>
> Do you think it is possible to get rough consensus within this group?
>
> Cheers,
>
> Matthew
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130813/0599a7e2/attachment.html>

From Thomas.Goebel at th-nuernberg.de  Wed Aug 14 04:38:38 2013
From: Thomas.Goebel at th-nuernberg.de (Thomas Goebel)
Date: Wed, 14 Aug 2013 10:38:38 +0200
Subject: [Numpy-discussion] Optimize removing nan-values of dataset
In-Reply-To: <CAM6XA7nis8FQMC7ehr9tK2wb4PFYA4ryCrOKYYPTP6_UFbipvQ@mail.gmail.com>
References: <5209D746.3000807@th-nuernberg.de>
	<CAM6XA7nis8FQMC7ehr9tK2wb4PFYA4ryCrOKYYPTP6_UFbipvQ@mail.gmail.com>
Message-ID: <520B420E.3060405@th-nuernberg.de>

* On 13/08/2013 23:32, David Reed wrote:
> Hi Thomas,
> 
> Your array is Nx6 do you want the nan values replace by the
> mean of the 2 adjacent elemets by row or by column?

Hi David,

i want it to be replaced by column.


I also found numpy.interp but this function replaces all nan
values at the beginning/end of array which should be omitted.

As an example:
y = np.array([nan, nan, 1, 2, 3, nan, nan, 4, nan, 5, nan, nan])
nans = np.isnan(y)

Only the values y[5:7] and y[8] should be replaced. Is it
possible to set nans[0:2] and nans[-2:] to False with something
like nans.startswith nans.endswith?


From opensource.numpy at user.fastmail.fm  Wed Aug 14 09:34:56 2013
From: opensource.numpy at user.fastmail.fm (Hugo Gagnon)
Date: Wed, 14 Aug 2013 09:34:56 -0400
Subject: [Numpy-discussion] Setter on array
Message-ID: <1376487296.28630.9716979.36ED8A61@webmail.messagingengine.com>

Hi,

What is the best way, if any, to "do something" whenever array elements
are changed in-place?  For example, if I have a = arange(10), then
setting a[3] = 1 would, say, call a function automatically.

Thanks,
-- 
  Hugo Gagnon


From brett.olsen at gmail.com  Wed Aug 14 11:48:50 2013
From: brett.olsen at gmail.com (Brett Olsen)
Date: Wed, 14 Aug 2013 10:48:50 -0500
Subject: [Numpy-discussion] Optimize removing nan-values of dataset
In-Reply-To: <5209D746.3000807@th-nuernberg.de>
References: <5209D746.3000807@th-nuernberg.de>
Message-ID: <CAFq1z2WucLgi1vLRoAG8=EbRgF-NAcFsizw0oCspyYe2d2qDaQ@mail.gmail.com>

The example data/method you've provided doesn't do what you describe.
 E.g., in your example data you have several 2x2 blocks of NaNs.  According
to your description, these should not be replaced (as they all have a
neighbor that is also a NaN).  Your example method, however, replaces them
- in fact, replaces any NaN values that are not in the first or last row or
contiguous with NaNs in the first or last row.

Here's a replacement method that does do what you've described:
def nan_to_mean(data):
        data[1:-1][np.isnan(data[1:-1])] = ((data[:-2] + data[2:]) /
2)[np.isnan(data[1:-1])]
        return data

~Brett


On Tue, Aug 13, 2013 at 1:50 AM, Thomas Goebel <
Thomas.Goebel at th-nuernberg.de> wrote:

> Hi,
>
> i am trying to remove nan-values from an array of shape(40, 6).
> These nan-values at point data[x] should be replaced by the mean
> of data[x-1] and data[x+1] if both values at x-1 and x+1 are not
> nan. The function nan_to_mean (see below) is working but i wonder
> if i could optimize the code.
>
> I thought about something like
>   1. Find all nan values in array:
>      nans = np.isnan(dataarray)
>   2. Check if values before, after nan indice are not nan
>   3. Calculate mean
>
> While using this script for my original dataset of
> shape(63856, 6) it takes 139.343 seconds to run it. And some
> datasets are even bigger. I attached the example_dataset.txt and
> the example.py script.
>
> Thanks for any help,
> Tom
>
> def nan_to_mean(arr):
>     for cnt, value in enumerate(arr):
>         # Check if first value is nan, if so continue
>         if cnt == 0 and np.isnan(value):
>             continue
>         # Check if last value is nan:
>         #     If x-1 value is nan dont do anything!
>         #     If x-1 is float, last value will be value of x-1
>         elif cnt == (len(arr)-1):
>             if np.isnan(value) and not np.isnan(arr[cnt-1]):
>                 arr[cnt] = arr[cnt-1]
>         # If the first values of file are nan ignore them all
>         elif np.isnan(value) and np.isnan(arr[cnt-1]):
>             continue
>         # Found nan value and x-1 value is of type float
>         elif np.isnan(value) and not np.isnan(arr[cnt-1]):
>             # Check if x+1 value is not nan
>             if not np.isnan(arr[cnt+1]):
>                 arr[cnt] = '%.1f' % np.mean((
>                         arr[cnt-1],arr[cnt+1]))
>             # If x+1 value is nan, go to next value
>             else:
>                 for N in xrange(2, 30):
>                     if cnt+N == (len(arr)):
>                         break
>                     elif not np.isnan(arr[cnt+N]):
>                         arr[cnt] = '%.1f' % np.mean(
>                                 (arr[cnt-1], arr[cnt+N]))
>     return arr
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130814/d88e4bf5/attachment.html>

From david.reed.c at gmail.com  Wed Aug 14 11:49:32 2013
From: david.reed.c at gmail.com (David Reed)
Date: Wed, 14 Aug 2013 11:49:32 -0400
Subject: [Numpy-discussion] Optimize removing nan-values of dataset
In-Reply-To: <520B420E.3060405@th-nuernberg.de>
References: <5209D746.3000807@th-nuernberg.de>
	<CAM6XA7nis8FQMC7ehr9tK2wb4PFYA4ryCrOKYYPTP6_UFbipvQ@mail.gmail.com>
	<520B420E.3060405@th-nuernberg.de>
Message-ID: <CAM6XA7n-JetZ3VBq9UsRKny0iU2afG0DCw5wCQ0eQYig3F5Eyg@mail.gmail.com>

Yeah, interp is what you want.  What you want to do with the end values is
up to you, but could be done like this:

ind = where(logical_not(np.isnan(y)))[0]
y1 = interp(range(len(y)), ind, y[ind])
y1 = y1[ind[0]:ind[-1]]


On Wed, Aug 14, 2013 at 4:38 AM, Thomas Goebel <
Thomas.Goebel at th-nuernberg.de> wrote:

> * On 13/08/2013 23:32, David Reed wrote:
> > Hi Thomas,
> >
> > Your array is Nx6 do you want the nan values replace by the
> > mean of the 2 adjacent elemets by row or by column?
>
> Hi David,
>
> i want it to be replaced by column.
>
>
> I also found numpy.interp but this function replaces all nan
> values at the beginning/end of array which should be omitted.
>
> As an example:
> y = np.array([nan, nan, 1, 2, 3, nan, nan, 4, nan, 5, nan, nan])
> nans = np.isnan(y)
>
> Only the values y[5:7] and y[8] should be replaced. Is it
> possible to set nans[0:2] and nans[-2:] to False with something
> like nans.startswith nans.endswith?
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130814/c684b336/attachment.html>

From chris.barker at noaa.gov  Wed Aug 14 12:00:45 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Wed, 14 Aug 2013 09:00:45 -0700
Subject: [Numpy-discussion] About ready to start 1.8 release process.
In-Reply-To: <CAB6mnx+YmZbuqwMxJ9DYNVEWn1fiM03Du8WC5X429Vkj+bRz1Q@mail.gmail.com>
References: <CAB6mnxJi8LF2_1CamBzR0yV_ea07ioyNdbmzx+sDKxiWAgerHw@mail.gmail.com>
	<CALGmxE+ug-gwrZEaTEyUFX3vko-4VhVS+34vc=yp2KYzB17FAQ@mail.gmail.com>
	<CAB6mnxKS5Rg7As+fy4cfMduHsh7fuNQ=-__fFcbzKRdDwgq1rA@mail.gmail.com>
	<CALGmxE+h1yQtMm+7CJLJtvnkCMWVHQu4PfLfWesNuH08n5+qnw@mail.gmail.com>
	<CAH6Pt5rvrrHUNwXx4+H9a1N8aKBP-zevj4toW3sTB+ZKKbW3pg@mail.gmail.com>
	<CAB6mnx+YmZbuqwMxJ9DYNVEWn1fiM03Du8WC5X429Vkj+bRz1Q@mail.gmail.com>
Message-ID: <CALGmxEJ1yUxpir-2UuxjvuBA9A+bCxjX_sSvtfDC751bRsELqg@mail.gmail.com>

On Tue, Aug 13, 2013 at 5:54 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
> I wish it were. It seems unreasonably difficult to get constructive
> feedback. Chris is pretty much the only one making the attempt and that
> discussion petered out.

well, Nathaniel Smith chimed in, and Mark Weibe commented a bit (as he
wrote the code in the first place). Wes McKinney also chimed in,
essentially saying that Pandas mostly has to do its own thing, and I'm
pretty sure would prefer that the current TZ converting was ripped out
as I suggest.

In the thread on the mialing list I started:

http://numpy-discussion.10968.n7.nabble.com/timezones-and-datetime64-tt33407.html#none

A bunch of core people chimed in. I just re-read that thread and
provide a summary below:

 - No one said they liked it as it is, and there were a number of +1
(and even a +10) comment to cleaning up the current conversion issues.

Some folks would like "real" improvements:
  - proper, full time zone handling
  - proper (or maybe just better?) handling of leap-seconds, and all that.

 - Marc Weibe had a suggestion or two that were "not a trivial
addition" -- so off the table for this release, and would need some
real discussion, a NEP, and developer time...

- Marc also pointed out that handling the Locale time zone is helpful
for the interactive use case, which is a major numpy use-case.
However, doing it half-way isn't really helpful anyway.

- Marc also commented that making datetime64 time-zone naive would be
"the easy way"

- There was also discussion about having a settable epoch -- I think
that would be really useful, but not for this release, and would
require a NEP, etc...

- From Travis O.:"It seems to me that the biggest issue is just the
automatic conversion that is occurring on string or date-time input.
We should stop using the local time-zone (explicit is better than
implicit strikes again) and not use any time-zone unless time-zone
information is provided in the string. I am definitely +1 on that."

My conclusions:

The current code is not really usable for ANY use without careful
checking of inputs and/or work-arounds.

No one involved in the numpy list would mind removing the time zone
translation on I/O. A number of people would find it a real
improvement. (granted most users don't get involved with these
discussion on the list....)

There are some great ideas for how to improve it that would require
NEPs, discussion and real developer time -- that's not happening any
time soon.

Ripping out the translation on I/O would be fairly easy (at least for
someone familiar with the code)

So I say let's do it!

But who? Travis offered up himself and Mark as possibilities, but who
knows what their schedules allow at this point -- are either of you
following this thread?

It's probably not too big a deal for anyone familiar with the numpy
code base -- if no one else steps up, I can give it a try, though I'm
sure I wouldn't have even found what code to edit in the time it would
take Mark to fix it.

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From wbuckner at beatsmusic.com  Wed Aug 14 12:31:00 2013
From: wbuckner at beatsmusic.com (Will Buckner)
Date: Wed, 14 Aug 2013 09:31:00 -0700
Subject: [Numpy-discussion] Do I Still Have to Edit intelcompiler.py For ICC?
Message-ID: <CAL_nGS7GcFmX_sGJPp9LUHkij0Qe=YCDrFFFoBZVb6Rvg-E1yA@mail.gmail.com>

Hey guys,

I normally build and install packages with pip, as I like to keep my
toolchain up to date with new releases, etc. I was happy to find out both
numpy and scipy will honor a ~/.numpy-site.cfg, but, since I need to build
with the Intel compilers, I have to supply --compiler and --fcompiler. Is
there any way to just do this in site config?

Thanks!
-Will
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130814/10955829/attachment.html>

From mwwiebe at gmail.com  Wed Aug 14 13:35:47 2013
From: mwwiebe at gmail.com (Mark Wiebe)
Date: Wed, 14 Aug 2013 10:35:47 -0700
Subject: [Numpy-discussion] About ready to start 1.8 release process.
In-Reply-To: <CALGmxEJ1yUxpir-2UuxjvuBA9A+bCxjX_sSvtfDC751bRsELqg@mail.gmail.com>
References: <CAB6mnxJi8LF2_1CamBzR0yV_ea07ioyNdbmzx+sDKxiWAgerHw@mail.gmail.com>
	<CALGmxE+ug-gwrZEaTEyUFX3vko-4VhVS+34vc=yp2KYzB17FAQ@mail.gmail.com>
	<CAB6mnxKS5Rg7As+fy4cfMduHsh7fuNQ=-__fFcbzKRdDwgq1rA@mail.gmail.com>
	<CALGmxE+h1yQtMm+7CJLJtvnkCMWVHQu4PfLfWesNuH08n5+qnw@mail.gmail.com>
	<CAH6Pt5rvrrHUNwXx4+H9a1N8aKBP-zevj4toW3sTB+ZKKbW3pg@mail.gmail.com>
	<CAB6mnx+YmZbuqwMxJ9DYNVEWn1fiM03Du8WC5X429Vkj+bRz1Q@mail.gmail.com>
	<CALGmxEJ1yUxpir-2UuxjvuBA9A+bCxjX_sSvtfDC751bRsELqg@mail.gmail.com>
Message-ID: <CAMRnEmrjA3Mq+T4-013meez0nk+E8zSgSoK-uW+T5o-8Z3=RCw@mail.gmail.com>

On Wed, Aug 14, 2013 at 9:00 AM, Chris Barker - NOAA Federal <
chris.barker at noaa.gov> wrote:

> On Tue, Aug 13, 2013 at 5:54 PM, Charles R Harris
> <charlesr.harris at gmail.com> wrote:
> > I wish it were. It seems unreasonably difficult to get constructive
> > feedback. Chris is pretty much the only one making the attempt and that
> > discussion petered out.
>
> well, Nathaniel Smith chimed in, and Mark Weibe commented a bit (as he
> wrote the code in the first place). Wes McKinney also chimed in,
> essentially saying that Pandas mostly has to do its own thing, and I'm
> pretty sure would prefer that the current TZ converting was ripped out
> as I suggest.
>
> In the thread on the mialing list I started:
>
>
> http://numpy-discussion.10968.n7.nabble.com/timezones-and-datetime64-tt33407.html#none
>
> A bunch of core people chimed in. I just re-read that thread and
> provide a summary below:
>
>  - No one said they liked it as it is, and there were a number of +1
> (and even a +10) comment to cleaning up the current conversion issues.
>
> Some folks would like "real" improvements:
>   - proper, full time zone handling
>   - proper (or maybe just better?) handling of leap-seconds, and all that.
>
>  - Marc Weibe had a suggestion or two that were "not a trivial
> addition" -- so off the table for this release, and would need some
> real discussion, a NEP, and developer time...
>
> - Marc also pointed out that handling the Locale time zone is helpful
> for the interactive use case, which is a major numpy use-case.
> However, doing it half-way isn't really helpful anyway.
>
> - Marc also commented that making datetime64 time-zone naive would be
> "the easy way"
>

I've experimented a little bit with one possibility in this direction
within dynd. Basically, adding a time zone to the metadata which initially
only can be UTC or abstract. (I'm calling it "abstract" because I don't
think "naive" communicates the right idea about it.) I'm -1 on simply
removing a distinction between UTC and abstract ISO 8601 strings, I think
equivocating the two would be as bad as the current situation. Adding a
limited timezone with just these two options might be a way to improve the
situation without too much development effort, while maintaining a clear
path to proper timezone support.

In this formulation, strings without a timezone are parsed as abstract
datetimes, and strings with a timezone as UTC datetimes, without allowing
implicit conversions:

>>> from dynd import nd, ndt
>>> nd.array('2011-03-22T12:00', dtype='datetime("min")')
nd.array(2011-03-22T12:00, datetime<unit=minute,tz=abstract>)
>>> nd.array('2011-03-22T12:00Z', dtype='datetime("min", "UTC")')
nd.array(2011-03-22T12:00Z, datetime<unit=minute,tz=utc>)
>>> nd.array('2011-03-22T12:00Z', dtype='datetime("min")')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "_pydynd.pyx", line 892, in _pydynd.w_array.__cinit__
(_pydynd.cxx:5774)
RuntimeError: cannot parse "2011-03-22T12:00Z" as an abstract datetime
using rule "strict", because a timezone was present in the string
>>> nd.array('2011-03-22T12:00', dtype='datetime("min", "UTC")')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "_pydynd.pyx", line 892, in _pydynd.w_array.__cinit__
(_pydynd.cxx:5774)
RuntimeError: cannot parse "2011-03-22T12:00" as a datetime with timezone
using
rule "strict", because no timezone was present in the string


I've also implemented one way of getting a more relaxed parsing mode, with
the parameter that controls casting strictness. Note that in this case it
is throwing away the timezone information, so the "-0400" doesn't get
incorporated in the third value.

>>> nd.array(['2011-03-22T12:00', '2012-04-23T01:00Z',
'2012-05-01T13:00-0400'])
nd.array([2011-03-22T12:00, 2012-04-23T01:00, 2012-05-01T13:00],
strided_dim<convert<to=datetime<unit=minute,tz=abstract>, from=string,
errmode=none>>)

This stuff is all experimental/work in progress, but if you want to try
these examples you can install Anaconda and use its conda package manager
to update the package dynd-python. (
https://github.com/ContinuumIO/dynd-python/blob/master/README.md#trying-out-dynd
).

Thanks,
Mark



> - There was also discussion about having a settable epoch -- I think

that would be really useful, but not for this release, and would

require a NEP, etc...


> - From Travis O.:"It seems to me that the biggest issue is just the

automatic conversion that is occurring on string or date-time input.

We should stop using the local time-zone (explicit is better than

implicit strikes again) and not use any time-zone unless time-zone

information is provided in the string. I am definitely +1 on that."


> My conclusions:


> The current code is not really usable for ANY use without careful

checking of inputs and/or work-arounds.


> No one involved in the numpy list would mind removing the time zone

translation on I/O. A number of people would find it a real

improvement. (granted most users don't get involved with these

discussion on the list....)


> There are some great ideas for how to improve it that would require

NEPs, discussion and real developer time -- that's not happening any

time soon.


> Ripping out the translation on I/O would be fairly easy (at least for

someone familiar with the code)


> So I say let's do it!


> But who? Travis offered up himself and Mark as possibilities, but who

knows what their schedules allow at this point -- are either of you

following this thread?


> It's probably not too big a deal for anyone familiar with the numpy

code base -- if no one else steps up, I can give it a try, though I'm

sure I wouldn't have even found what code to edit in the time it would

take Mark to fix it.


> -Chris
>

> --
>

> Christopher Barker, Ph.D.
>
Oceanographer
>

> Emergency Response Division
>
NOAA/NOS/OR&R            (206) 526-6959   voice
>
7600 Sand Point Way NE   (206) 526-6329   fax
>
Seattle, WA  98115       (206) 526-6317   main reception
>

> Chris.Barker at noaa.gov
>
_______________________________________________
>
NumPy-Discussion mailing list
>
NumPy-Discussion at scipy.org
>
http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130814/f95d55e4/attachment.html>

From chris.barker at noaa.gov  Wed Aug 14 14:58:46 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Wed, 14 Aug 2013 11:58:46 -0700
Subject: [Numpy-discussion] About ready to start 1.8 release process.
In-Reply-To: <CAMRnEmrjA3Mq+T4-013meez0nk+E8zSgSoK-uW+T5o-8Z3=RCw@mail.gmail.com>
References: <CAB6mnxJi8LF2_1CamBzR0yV_ea07ioyNdbmzx+sDKxiWAgerHw@mail.gmail.com>
	<CALGmxE+ug-gwrZEaTEyUFX3vko-4VhVS+34vc=yp2KYzB17FAQ@mail.gmail.com>
	<CAB6mnxKS5Rg7As+fy4cfMduHsh7fuNQ=-__fFcbzKRdDwgq1rA@mail.gmail.com>
	<CALGmxE+h1yQtMm+7CJLJtvnkCMWVHQu4PfLfWesNuH08n5+qnw@mail.gmail.com>
	<CAH6Pt5rvrrHUNwXx4+H9a1N8aKBP-zevj4toW3sTB+ZKKbW3pg@mail.gmail.com>
	<CAB6mnx+YmZbuqwMxJ9DYNVEWn1fiM03Du8WC5X429Vkj+bRz1Q@mail.gmail.com>
	<CALGmxEJ1yUxpir-2UuxjvuBA9A+bCxjX_sSvtfDC751bRsELqg@mail.gmail.com>
	<CAMRnEmrjA3Mq+T4-013meez0nk+E8zSgSoK-uW+T5o-8Z3=RCw@mail.gmail.com>
Message-ID: <CALGmxEJcKnCsC+HQGxK3UFoAreY22b-8pNkvQmb4t3G0JiT0NQ@mail.gmail.com>

On Wed, Aug 14, 2013 at 10:35 AM, Mark Wiebe <mwwiebe at gmail.com> wrote:

Hi Mark, great to have you thinking (and coding) about this!

>> - Marc also commented that making datetime64 time-zone naive would be
>> "the easy way"
>
> I've experimented a little bit with one possibility in this direction within
> dynd. Basically, adding a time zone to the metadata which initially only can
> be UTC or abstract. (I'm calling it "abstract" because I don't think "naive"
> communicates the right idea about it.)

I like that term better, too -- I only borrowed "naive" from the
stdlib datetime docs.

And having these two is good -- saves us the argument of which to do


>  I'm -1 on simply removing a
> distinction between UTC and abstract ISO 8601 strings, I think equivocating
> the two would be as bad as the current situation.

well, I wouldn't equate them (or did you mean equate, thought that
would be equivocating...), I'd say one or the other if you couldnt' do
both. though I"m still confused about the difference if you don't
provide any other time zone capability.

> Adding a limited timezone
> with just these two options might be a way to improve the situation without
> too much development effort, while maintaining a clear path to proper
> timezone support.

yup -- though my concern is that calling a datetime64 "UTC" may imply
that there is some other time zone functionality in there -- which
there isn't (at least not yet) i.e. it's really jsut a hint ot the
user as to how it should be thought of (and makes it clear what to do
when you parse an ISO sting with an offset...)

> In this formulation, strings without a timezone are parsed as abstract
> datetimes, and strings with a timezone as UTC datetimes, without allowing
> implicit conversions:

>>>> from dynd import nd, ndt
>>>> nd.array('2011-03-22T12:00', dtype='datetime("min")')
> nd.array(2011-03-22T12:00, datetime<unit=minute,tz=abstract>)
>>>> nd.array('2011-03-22T12:00Z', dtype='datetime("min", "UTC")')
> nd.array(2011-03-22T12:00Z, datetime<unit=minute,tz=utc>)
>>>> nd.array('2011-03-22T12:00Z', dtype='datetime("min")')
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "_pydynd.pyx", line 892, in _pydynd.w_array.__cinit__
> (_pydynd.cxx:5774)
> RuntimeError: cannot parse "2011-03-22T12:00Z" as an abstract datetime using
> rule "strict", because a timezone was present in the string
>>>> nd.array('2011-03-22T12:00', dtype='datetime("min", "UTC")')
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "_pydynd.pyx", line 892, in _pydynd.w_array.__cinit__
> (_pydynd.cxx:5774)
> RuntimeError: cannot parse "2011-03-22T12:00" as a datetime with timezone
> using
> rule "strict", because no timezone was present in the string

This is looking good to me.

> I've also implemented one way of getting a more relaxed parsing mode, with
> the parameter that controls casting strictness. Note that in this case it is
> throwing away the timezone information, so the "-0400" doesn't get
> incorporated in the third value.
>
>>>> nd.array(['2011-03-22T12:00', '2012-04-23T01:00Z',
>>>> '2012-05-01T13:00-0400'])
> nd.array([2011-03-22T12:00, 2012-04-23T01:00, 2012-05-01T13:00],
> strided_dim<convert<to=datetime<unit=minute,tz=abstract>, from=string,
> errmode=none>>)

hmm -- it is explicite, but still seems fragile to me -- one coudl
test with "Z" and 0, and then  later get a different offset and things
would go tot heck -- but buyer beware with a non-default setting, I
guess.

> This stuff is all experimental/work in progress, but if you want to try
> these examples you can install Anaconda and use its conda package manager to
> update the package dynd-python.
> (https://github.com/ContinuumIO/dynd-python/blob/master/README.md#trying-out-dynd).

Great, thanks!

Any prospects of something similar in numpy relatively soon?

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From ralf.gommers at gmail.com  Wed Aug 14 17:12:48 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Wed, 14 Aug 2013 23:12:48 +0200
Subject: [Numpy-discussion] Do I Still Have to Edit intelcompiler.py For
	ICC?
In-Reply-To: <CAL_nGS7GcFmX_sGJPp9LUHkij0Qe=YCDrFFFoBZVb6Rvg-E1yA@mail.gmail.com>
References: <CAL_nGS7GcFmX_sGJPp9LUHkij0Qe=YCDrFFFoBZVb6Rvg-E1yA@mail.gmail.com>
Message-ID: <CABL7CQjMjj8JOriv+G1vOsJk-nYqvDRYmvUuQ3zRVVenFxzNKw@mail.gmail.com>

On Wed, Aug 14, 2013 at 6:31 PM, Will Buckner <wbuckner at beatsmusic.com>wrote:

>
> Hey guys,
>
> I normally build and install packages with pip, as I like to keep my
> toolchain up to date with new releases, etc. I was happy to find out both
> numpy and scipy will honor a ~/.numpy-site.cfg, but, since I need to build
> with the Intel compilers, I have to supply --compiler and --fcompiler. Is
> there any way to just do this in site config?
>

Hi Will, I don't think you can do that in site.cfg. All sections that are
recognized in site.cfg are listed in get_info() in
numpy/distutils/system_info.py. Compilers have to be specified on the
command line. Maybe pip has a way to supply that info but I've never
bothered to look for it - python setup.py .... works just as well.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130814/93ff3588/attachment.html>

From chris.barker at noaa.gov  Wed Aug 14 17:31:05 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Wed, 14 Aug 2013 14:31:05 -0700
Subject: [Numpy-discussion] Do I Still Have to Edit intelcompiler.py For
	ICC?
In-Reply-To: <CABL7CQjMjj8JOriv+G1vOsJk-nYqvDRYmvUuQ3zRVVenFxzNKw@mail.gmail.com>
References: <CAL_nGS7GcFmX_sGJPp9LUHkij0Qe=YCDrFFFoBZVb6Rvg-E1yA@mail.gmail.com>
	<CABL7CQjMjj8JOriv+G1vOsJk-nYqvDRYmvUuQ3zRVVenFxzNKw@mail.gmail.com>
Message-ID: <CALGmxEJSVW4BdAhRa7_E_6LO9ZzuxVDd6yPkkJxwS+A1OJhfwg@mail.gmail.com>

On Wed, Aug 14, 2013 at 2:12 PM, Ralf Gommers <ralf.gommers at gmail.com> wrote:

>> with the Intel compilers, I have to supply --compiler and --fcompiler. Is
>> there any way to just do this in site config?

>  Maybe pip has a way to supply that info but I've never
> bothered to look for it - python setup.py .... works just as well.

It look slike you can pass distuitls commands through pip:

"""
--install-option <options>

Extra arguments to be supplied to the setup.py install command (use
like ?install-option=??install-scripts=/usr/local/bin?). Use multiple
?install-option options to pass multiple options to setup.py install.
If you are using an option with a directory path, be sure to use
absolute path.
"""

There is also a way to specify default arguments for pip install, so
that it woudl get used every time.

http://www.pip-installer.org/en/latest/usage.html#pip-install

-HTH, Chris


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From charlesr.harris at gmail.com  Thu Aug 15 09:42:37 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Thu, 15 Aug 2013 07:42:37 -0600
Subject: [Numpy-discussion] Upcoming 1.8 release.
Message-ID: <CAB6mnxJoUzEonhRT18JkQWQmDPusxdyWqbRiZwpRGwDhGbAYag@mail.gmail.com>

Hi All,

I'm thinking of making the 1.8.x branch next Sunday. Any complaints,
thoughts?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130815/081ee567/attachment.html>

From stefan at sun.ac.za  Thu Aug 15 09:51:12 2013
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Thu, 15 Aug 2013 15:51:12 +0200
Subject: [Numpy-discussion] Upcoming 1.8 release.
In-Reply-To: <CAB6mnxJoUzEonhRT18JkQWQmDPusxdyWqbRiZwpRGwDhGbAYag@mail.gmail.com>
References: <CAB6mnxJoUzEonhRT18JkQWQmDPusxdyWqbRiZwpRGwDhGbAYag@mail.gmail.com>
Message-ID: <CABDkGQmJPGN3Gk8tR11CbDvhbRf78QbAPDYrjnxyhiKBJ=Tmqw@mail.gmail.com>

On Thu, Aug 15, 2013 at 3:42 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
>
> I'm thinking of making the 1.8.x branch next Sunday. Any complaints, thoughts?

Thanks, Chuck.  Are there any specific PRs up for review that should
be incorporated into 1.8?

St?fan


From charlesr.harris at gmail.com  Thu Aug 15 10:21:47 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Thu, 15 Aug 2013 08:21:47 -0600
Subject: [Numpy-discussion] Upcoming 1.8 release.
In-Reply-To: <CABDkGQmJPGN3Gk8tR11CbDvhbRf78QbAPDYrjnxyhiKBJ=Tmqw@mail.gmail.com>
References: <CAB6mnxJoUzEonhRT18JkQWQmDPusxdyWqbRiZwpRGwDhGbAYag@mail.gmail.com>
	<CABDkGQmJPGN3Gk8tR11CbDvhbRf78QbAPDYrjnxyhiKBJ=Tmqw@mail.gmail.com>
Message-ID: <CAB6mnxJRgcX=15PSrmL5YHBpASNTng33=iGaLVe-XF-KP9ok-g@mail.gmail.com>

I don't see any that *have* to go in, but there are a few that could be
included. The most significant is probably the inplace fancy indexing if it
is ready. The nanmean etc. functions are not committed yet, but I think
they are ready. If the Polynomial import fixes show up, they can go in.
There are the usual janitorial things,  the release notes need some clean
up, the docs need merging, and the HOWTO_RELEASE document needs updating.

For datetime64, I think a comment should be added to the release notes that
it is still experimental and that changes are expected in 1.9. Hopefully
the next release will come out next spring.

I think we are also about ready for a 1.7.2 release.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130815/17ba86e3/attachment.html>

From blake.a.griffith at gmail.com  Thu Aug 15 12:48:31 2013
From: blake.a.griffith at gmail.com (Blake Griffith)
Date: Thu, 15 Aug 2013 11:48:31 -0500
Subject: [Numpy-discussion] Upcoming 1.8 release.
In-Reply-To: <CAB6mnxJRgcX=15PSrmL5YHBpASNTng33=iGaLVe-XF-KP9ok-g@mail.gmail.com>
References: <CAB6mnxJoUzEonhRT18JkQWQmDPusxdyWqbRiZwpRGwDhGbAYag@mail.gmail.com>
	<CABDkGQmJPGN3Gk8tR11CbDvhbRf78QbAPDYrjnxyhiKBJ=Tmqw@mail.gmail.com>
	<CAB6mnxJRgcX=15PSrmL5YHBpASNTng33=iGaLVe-XF-KP9ok-g@mail.gmail.com>
Message-ID: <CAOiFMpwvfpDkFH7XjtL-8Tq68YGc73J3Wx+3nXWa5X1ip5cykA@mail.gmail.com>

I would like to have the ufunc overrides in 1.8 if it is possible.


On Thu, Aug 15, 2013 at 9:21 AM, Charles R Harris <charlesr.harris at gmail.com
> wrote:

> I don't see any that *have* to go in, but there are a few that could be
> included. The most significant is probably the inplace fancy indexing if it
> is ready. The nanmean etc. functions are not committed yet, but I think
> they are ready. If the Polynomial import fixes show up, they can go in.
> There are the usual janitorial things,  the release notes need some clean
> up, the docs need merging, and the HOWTO_RELEASE document needs updating.
>
> For datetime64, I think a comment should be added to the release notes
> that it is still experimental and that changes are expected in 1.9.
> Hopefully the next release will come out next spring.
>
> I think we are also about ready for a 1.7.2 release.
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130815/0ae18643/attachment.html>

From charlesr.harris at gmail.com  Thu Aug 15 12:52:01 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Thu, 15 Aug 2013 10:52:01 -0600
Subject: [Numpy-discussion] Upcoming 1.8 release.
In-Reply-To: <CAOiFMpwvfpDkFH7XjtL-8Tq68YGc73J3Wx+3nXWa5X1ip5cykA@mail.gmail.com>
References: <CAB6mnxJoUzEonhRT18JkQWQmDPusxdyWqbRiZwpRGwDhGbAYag@mail.gmail.com>
	<CABDkGQmJPGN3Gk8tR11CbDvhbRf78QbAPDYrjnxyhiKBJ=Tmqw@mail.gmail.com>
	<CAB6mnxJRgcX=15PSrmL5YHBpASNTng33=iGaLVe-XF-KP9ok-g@mail.gmail.com>
	<CAOiFMpwvfpDkFH7XjtL-8Tq68YGc73J3Wx+3nXWa5X1ip5cykA@mail.gmail.com>
Message-ID: <CAB6mnxLhNkAuPT3uHGsC7+MWq2Su8YtuhkgMX-f+EByWaAQejg@mail.gmail.com>

On Thu, Aug 15, 2013 at 10:48 AM, Blake Griffith <blake.a.griffith at gmail.com
> wrote:

> I would like to have the ufunc overrides in 1.8 if it is possible.
>
>
> On Thu, Aug 15, 2013 at 9:21 AM, Charles R Harris <
> charlesr.harris at gmail.com> wrote:
>
>> I don't see any that *have* to go in, but there are a few that could be
>> included. The most significant is probably the inplace fancy indexing if it
>> is ready. The nanmean etc. functions are not committed yet, but I think
>> they are ready. If the Polynomial import fixes show up, they can go in.
>> There are the usual janitorial things,  the release notes need some clean
>> up, the docs need merging, and the HOWTO_RELEASE document needs updating.
>>
>> For datetime64, I think a comment should be added to the release notes
>> that it is still experimental and that changes are expected in 1.9.
>> Hopefully the next release will come out next spring.
>>
>> I think we are also about ready for a 1.7.2 release.
>>
>>
>>
What is the status of that? I've been leaving that commit up the Pauli.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130815/632e815c/attachment.html>

From blake.a.griffith at gmail.com  Thu Aug 15 14:21:08 2013
From: blake.a.griffith at gmail.com (Blake Griffith)
Date: Thu, 15 Aug 2013 13:21:08 -0500
Subject: [Numpy-discussion] Upcoming 1.8 release.
In-Reply-To: <CAB6mnxLhNkAuPT3uHGsC7+MWq2Su8YtuhkgMX-f+EByWaAQejg@mail.gmail.com>
References: <CAB6mnxJoUzEonhRT18JkQWQmDPusxdyWqbRiZwpRGwDhGbAYag@mail.gmail.com>
	<CABDkGQmJPGN3Gk8tR11CbDvhbRf78QbAPDYrjnxyhiKBJ=Tmqw@mail.gmail.com>
	<CAB6mnxJRgcX=15PSrmL5YHBpASNTng33=iGaLVe-XF-KP9ok-g@mail.gmail.com>
	<CAOiFMpwvfpDkFH7XjtL-8Tq68YGc73J3Wx+3nXWa5X1ip5cykA@mail.gmail.com>
	<CAB6mnxLhNkAuPT3uHGsC7+MWq2Su8YtuhkgMX-f+EByWaAQejg@mail.gmail.com>
Message-ID: <CAOiFMpydc2R3=Bzthpv2P5W0wN_EL8n1mDo1-7j_dSGM60jLnQ@mail.gmail.com>

I think it is nearly complete. Although there are some recent changes that
need review.

I still need to go back and make changes to the original NEP noting the
differences in final implementation.


On Thu, Aug 15, 2013 at 11:52 AM, Charles R Harris <
charlesr.harris at gmail.com> wrote:

>
> On Thu, Aug 15, 2013 at 10:48 AM, Blake Griffith <
> blake.a.griffith at gmail.com> wrote:
>
>> I would like to have the ufunc overrides in 1.8 if it is possible.
>>
>>
>> On Thu, Aug 15, 2013 at 9:21 AM, Charles R Harris <
>> charlesr.harris at gmail.com> wrote:
>>
>>> I don't see any that *have* to go in, but there are a few that could be
>>> included. The most significant is probably the inplace fancy indexing if it
>>> is ready. The nanmean etc. functions are not committed yet, but I think
>>> they are ready. If the Polynomial import fixes show up, they can go in.
>>> There are the usual janitorial things,  the release notes need some clean
>>> up, the docs need merging, and the HOWTO_RELEASE document needs updating.
>>>
>>> For datetime64, I think a comment should be added to the release notes
>>> that it is still experimental and that changes are expected in 1.9.
>>> Hopefully the next release will come out next spring.
>>>
>>> I think we are also about ready for a 1.7.2 release.
>>>
>>>
>>>
> What is the status of that? I've been leaving that commit up the Pauli.
>
> Chuck
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130815/4fc7b752/attachment.html>

From ralf.gommers at gmail.com  Thu Aug 15 15:10:51 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Thu, 15 Aug 2013 21:10:51 +0200
Subject: [Numpy-discussion] Upcoming 1.8 release.
In-Reply-To: <CAB6mnxJoUzEonhRT18JkQWQmDPusxdyWqbRiZwpRGwDhGbAYag@mail.gmail.com>
References: <CAB6mnxJoUzEonhRT18JkQWQmDPusxdyWqbRiZwpRGwDhGbAYag@mail.gmail.com>
Message-ID: <CABL7CQji5VZEY+Nd2VavQ9Kx+EoQ4ReX=kOGzH=X=KpuxhHcRQ@mail.gmail.com>

On Thu, Aug 15, 2013 at 3:42 PM, Charles R Harris <charlesr.harris at gmail.com
> wrote:

> Hi All,
>
> I'm thinking of making the 1.8.x branch next Sunday. Any complaints,
> thoughts?
>

First thought: thanks a lot for doing this.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130815/ba761733/attachment.html>

From matthew.brett at gmail.com  Thu Aug 15 15:22:36 2013
From: matthew.brett at gmail.com (Matthew Brett)
Date: Thu, 15 Aug 2013 12:22:36 -0700
Subject: [Numpy-discussion] Upcoming 1.8 release.
In-Reply-To: <CABL7CQji5VZEY+Nd2VavQ9Kx+EoQ4ReX=kOGzH=X=KpuxhHcRQ@mail.gmail.com>
References: <CAB6mnxJoUzEonhRT18JkQWQmDPusxdyWqbRiZwpRGwDhGbAYag@mail.gmail.com>
	<CABL7CQji5VZEY+Nd2VavQ9Kx+EoQ4ReX=kOGzH=X=KpuxhHcRQ@mail.gmail.com>
Message-ID: <CAH6Pt5q=smLvDCDUy=PsS4__v4YmR305VcB1Y1M2bnAMJ=HBuw@mail.gmail.com>

Hi,

On Thu, Aug 15, 2013 at 12:10 PM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>
>
> On Thu, Aug 15, 2013 at 3:42 PM, Charles R Harris
> <charlesr.harris at gmail.com> wrote:
>>
>> Hi All,
>>
>> I'm thinking of making the 1.8.x branch next Sunday. Any complaints,
>> thoughts?
>
>
> First thought: thanks a lot for doing this.

I'm afraid I don't understand the discussion on timezones in
datetime64, but I have the impression that those who do think it needs
an urgent decision and some action for the short term.  Is that right,
datetimers?

If that's so, and there are worthwhile changes that are practical in
the next few weeks, it seems reasonable to wait.

Best,

Matthew


From charlesr.harris at gmail.com  Thu Aug 15 15:28:34 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Thu, 15 Aug 2013 13:28:34 -0600
Subject: [Numpy-discussion] Upcoming 1.8 release.
In-Reply-To: <CAH6Pt5q=smLvDCDUy=PsS4__v4YmR305VcB1Y1M2bnAMJ=HBuw@mail.gmail.com>
References: <CAB6mnxJoUzEonhRT18JkQWQmDPusxdyWqbRiZwpRGwDhGbAYag@mail.gmail.com>
	<CABL7CQji5VZEY+Nd2VavQ9Kx+EoQ4ReX=kOGzH=X=KpuxhHcRQ@mail.gmail.com>
	<CAH6Pt5q=smLvDCDUy=PsS4__v4YmR305VcB1Y1M2bnAMJ=HBuw@mail.gmail.com>
Message-ID: <CAB6mnx+oFzfvaPGwRgeH-krHRn5MSv+Fv4HfTSC9ozJhOLnd4Q@mail.gmail.com>

On Thu, Aug 15, 2013 at 1:22 PM, Matthew Brett <matthew.brett at gmail.com>wrote:

> Hi,
>
> On Thu, Aug 15, 2013 at 12:10 PM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> >
> >
> >
> > On Thu, Aug 15, 2013 at 3:42 PM, Charles R Harris
> > <charlesr.harris at gmail.com> wrote:
> >>
> >> Hi All,
> >>
> >> I'm thinking of making the 1.8.x branch next Sunday. Any complaints,
> >> thoughts?
> >
> >
> > First thought: thanks a lot for doing this.
>
> I'm afraid I don't understand the discussion on timezones in
> datetime64, but I have the impression that those who do think it needs
> an urgent decision and some action for the short term.  Is that right,
> datetimers?
>
> If that's so, and there are worthwhile changes that are practical in
> the next few weeks, it seems reasonable to wait.
>
>
My impression is that we will have something for 1.9. If it comes in for
1.8, fine. But I think it is still under development. Hopefully the 1.9
release will come out next spring.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130815/37a71aa8/attachment.html>

From matthew.brett at gmail.com  Thu Aug 15 15:31:48 2013
From: matthew.brett at gmail.com (Matthew Brett)
Date: Thu, 15 Aug 2013 12:31:48 -0700
Subject: [Numpy-discussion] Upcoming 1.8 release.
In-Reply-To: <CAB6mnx+oFzfvaPGwRgeH-krHRn5MSv+Fv4HfTSC9ozJhOLnd4Q@mail.gmail.com>
References: <CAB6mnxJoUzEonhRT18JkQWQmDPusxdyWqbRiZwpRGwDhGbAYag@mail.gmail.com>
	<CABL7CQji5VZEY+Nd2VavQ9Kx+EoQ4ReX=kOGzH=X=KpuxhHcRQ@mail.gmail.com>
	<CAH6Pt5q=smLvDCDUy=PsS4__v4YmR305VcB1Y1M2bnAMJ=HBuw@mail.gmail.com>
	<CAB6mnx+oFzfvaPGwRgeH-krHRn5MSv+Fv4HfTSC9ozJhOLnd4Q@mail.gmail.com>
Message-ID: <CAH6Pt5oBy3D0tQmgS87rMYX4Vq-3sYJ+pAuwz-z9ufgqH+5Qqw@mail.gmail.com>

Hi,

On Thu, Aug 15, 2013 at 12:28 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
>
>
>
> On Thu, Aug 15, 2013 at 1:22 PM, Matthew Brett <matthew.brett at gmail.com>
> wrote:
>>
>> Hi,
>>
>> On Thu, Aug 15, 2013 at 12:10 PM, Ralf Gommers <ralf.gommers at gmail.com>
>> wrote:
>> >
>> >
>> >
>> > On Thu, Aug 15, 2013 at 3:42 PM, Charles R Harris
>> > <charlesr.harris at gmail.com> wrote:
>> >>
>> >> Hi All,
>> >>
>> >> I'm thinking of making the 1.8.x branch next Sunday. Any complaints,
>> >> thoughts?
>> >
>> >
>> > First thought: thanks a lot for doing this.
>>
>> I'm afraid I don't understand the discussion on timezones in
>> datetime64, but I have the impression that those who do think it needs
>> an urgent decision and some action for the short term.  Is that right,
>> datetimers?
>>
>> If that's so, and there are worthwhile changes that are practical in
>> the next few weeks, it seems reasonable to wait.
>>
>
> My impression is that we will have something for 1.9. If it comes in for
> 1.8, fine. But I think it is still under development. Hopefully the 1.9
> release will come out next spring.

OK - then I guess you are saying it is up you, our datetimer friends,
to make a proposal and timetable and implementation, if y'all think it
can be done in the next few weeks,

Best,

Matthew


From ralf.gommers at gmail.com  Thu Aug 15 16:04:39 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Thu, 15 Aug 2013 22:04:39 +0200
Subject: [Numpy-discussion] Upcoming 1.8 release.
In-Reply-To: <CAH6Pt5oBy3D0tQmgS87rMYX4Vq-3sYJ+pAuwz-z9ufgqH+5Qqw@mail.gmail.com>
References: <CAB6mnxJoUzEonhRT18JkQWQmDPusxdyWqbRiZwpRGwDhGbAYag@mail.gmail.com>
	<CABL7CQji5VZEY+Nd2VavQ9Kx+EoQ4ReX=kOGzH=X=KpuxhHcRQ@mail.gmail.com>
	<CAH6Pt5q=smLvDCDUy=PsS4__v4YmR305VcB1Y1M2bnAMJ=HBuw@mail.gmail.com>
	<CAB6mnx+oFzfvaPGwRgeH-krHRn5MSv+Fv4HfTSC9ozJhOLnd4Q@mail.gmail.com>
	<CAH6Pt5oBy3D0tQmgS87rMYX4Vq-3sYJ+pAuwz-z9ufgqH+5Qqw@mail.gmail.com>
Message-ID: <CABL7CQhBLjb9ZZxSHTTc1L=3t1G_=ALz=o0yh_CE_=3jdsC8VA@mail.gmail.com>

On Thu, Aug 15, 2013 at 9:31 PM, Matthew Brett <matthew.brett at gmail.com>wrote:

> Hi,
>
> On Thu, Aug 15, 2013 at 12:28 PM, Charles R Harris
> <charlesr.harris at gmail.com> wrote:
> >
> >
> >
> > On Thu, Aug 15, 2013 at 1:22 PM, Matthew Brett <matthew.brett at gmail.com>
> > wrote:
> >>
> >> Hi,
> >>
> >> On Thu, Aug 15, 2013 at 12:10 PM, Ralf Gommers <ralf.gommers at gmail.com>
> >> wrote:
> >> >
> >> >
> >> >
> >> > On Thu, Aug 15, 2013 at 3:42 PM, Charles R Harris
> >> > <charlesr.harris at gmail.com> wrote:
> >> >>
> >> >> Hi All,
> >> >>
> >> >> I'm thinking of making the 1.8.x branch next Sunday. Any complaints,
> >> >> thoughts?
> >> >
> >> >
> >> > First thought: thanks a lot for doing this.
> >>
> >> I'm afraid I don't understand the discussion on timezones in
> >> datetime64, but I have the impression that those who do think it needs
> >> an urgent decision and some action for the short term.  Is that right,
> >> datetimers?
> >>
> >> If that's so, and there are worthwhile changes that are practical in
> >> the next few weeks, it seems reasonable to wait.
> >>
> >
> > My impression is that we will have something for 1.9. If it comes in for
> > 1.8, fine. But I think it is still under development. Hopefully the 1.9
> > release will come out next spring.
>
> OK - then I guess you are saying it is up you, our datetimer friends,
> to make a proposal and timetable and implementation, if y'all think it
> can be done in the next few weeks,
>

My impression: there's a reasonable amount of agreement on what has to be
done, but no one has stepped up to do the work. It doesn't look like
something that should block a release, because there's not a huge amount of
interest and the API is already labeled 'experimental'. So I don't really
see an issue in releasing 1.8 with the same behavior as 1.7.

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130815/02542eb6/attachment.html>

From matthew.brett at gmail.com  Thu Aug 15 18:16:41 2013
From: matthew.brett at gmail.com (Matthew Brett)
Date: Thu, 15 Aug 2013 15:16:41 -0700
Subject: [Numpy-discussion] Upcoming 1.8 release.
In-Reply-To: <CABL7CQhBLjb9ZZxSHTTc1L=3t1G_=ALz=o0yh_CE_=3jdsC8VA@mail.gmail.com>
References: <CAB6mnxJoUzEonhRT18JkQWQmDPusxdyWqbRiZwpRGwDhGbAYag@mail.gmail.com>
	<CABL7CQji5VZEY+Nd2VavQ9Kx+EoQ4ReX=kOGzH=X=KpuxhHcRQ@mail.gmail.com>
	<CAH6Pt5q=smLvDCDUy=PsS4__v4YmR305VcB1Y1M2bnAMJ=HBuw@mail.gmail.com>
	<CAB6mnx+oFzfvaPGwRgeH-krHRn5MSv+Fv4HfTSC9ozJhOLnd4Q@mail.gmail.com>
	<CAH6Pt5oBy3D0tQmgS87rMYX4Vq-3sYJ+pAuwz-z9ufgqH+5Qqw@mail.gmail.com>
	<CABL7CQhBLjb9ZZxSHTTc1L=3t1G_=ALz=o0yh_CE_=3jdsC8VA@mail.gmail.com>
Message-ID: <CAH6Pt5rmAvZiox40a1aWD6+o8aE_jFvLmfGsOnjFTw=Cc+kbmg@mail.gmail.com>

Hi,

On Thu, Aug 15, 2013 at 1:04 PM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>
>
> On Thu, Aug 15, 2013 at 9:31 PM, Matthew Brett <matthew.brett at gmail.com>
> wrote:
>>
>> Hi,
>>
>> On Thu, Aug 15, 2013 at 12:28 PM, Charles R Harris
>> <charlesr.harris at gmail.com> wrote:
>> >
>> >
>> >
>> > On Thu, Aug 15, 2013 at 1:22 PM, Matthew Brett <matthew.brett at gmail.com>
>> > wrote:
>> >>
>> >> Hi,
>> >>
>> >> On Thu, Aug 15, 2013 at 12:10 PM, Ralf Gommers <ralf.gommers at gmail.com>
>> >> wrote:
>> >> >
>> >> >
>> >> >
>> >> > On Thu, Aug 15, 2013 at 3:42 PM, Charles R Harris
>> >> > <charlesr.harris at gmail.com> wrote:
>> >> >>
>> >> >> Hi All,
>> >> >>
>> >> >> I'm thinking of making the 1.8.x branch next Sunday. Any complaints,
>> >> >> thoughts?
>> >> >
>> >> >
>> >> > First thought: thanks a lot for doing this.
>> >>
>> >> I'm afraid I don't understand the discussion on timezones in
>> >> datetime64, but I have the impression that those who do think it needs
>> >> an urgent decision and some action for the short term.  Is that right,
>> >> datetimers?
>> >>
>> >> If that's so, and there are worthwhile changes that are practical in
>> >> the next few weeks, it seems reasonable to wait.
>> >>
>> >
>> > My impression is that we will have something for 1.9. If it comes in for
>> > 1.8, fine. But I think it is still under development. Hopefully the 1.9
>> > release will come out next spring.
>>
>> OK - then I guess you are saying it is up you, our datetimer friends,
>> to make a proposal and timetable and implementation, if y'all think it
>> can be done in the next few weeks,
>
>
> My impression: there's a reasonable amount of agreement on what has to be
> done, but no one has stepped up to do the work. It doesn't look like
> something that should block a release, because there's not a huge amount of
> interest and the API is already labeled 'experimental'. So I don't really
> see an issue in releasing 1.8 with the same behavior as 1.7.

Chris B - are you the point man on this one?  What do you think?

Cheers,

Matthew


From chris.barker at noaa.gov  Thu Aug 15 18:26:38 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Thu, 15 Aug 2013 15:26:38 -0700
Subject: [Numpy-discussion] Upcoming 1.8 release.
In-Reply-To: <CAH6Pt5oBy3D0tQmgS87rMYX4Vq-3sYJ+pAuwz-z9ufgqH+5Qqw@mail.gmail.com>
References: <CAB6mnxJoUzEonhRT18JkQWQmDPusxdyWqbRiZwpRGwDhGbAYag@mail.gmail.com>
	<CABL7CQji5VZEY+Nd2VavQ9Kx+EoQ4ReX=kOGzH=X=KpuxhHcRQ@mail.gmail.com>
	<CAH6Pt5q=smLvDCDUy=PsS4__v4YmR305VcB1Y1M2bnAMJ=HBuw@mail.gmail.com>
	<CAB6mnx+oFzfvaPGwRgeH-krHRn5MSv+Fv4HfTSC9ozJhOLnd4Q@mail.gmail.com>
	<CAH6Pt5oBy3D0tQmgS87rMYX4Vq-3sYJ+pAuwz-z9ufgqH+5Qqw@mail.gmail.com>
Message-ID: <CALGmxEK97960fBhc8eq2-DSULZbTiL41f53PQFcUcTxpQJBENw@mail.gmail.com>

On Thu, Aug 15, 2013 at 12:31 PM, Matthew Brett <matthew.brett at gmail.com> wrote:

>>> I'm afraid I don't understand the discussion on timezones in
>>> datetime64, but I have the impression that those who do think it needs
>>> an urgent decision and some action for the short term.  Is that right,
>>> datetimers?
>>>
>>> If that's so, and there are worthwhile changes that are practical in
>>> the next few weeks, it seems reasonable to wait.

Well, it's only "urgent" in the sense that there are indeed a couple
small changes that would really help, and if we don't use a release to
motivate us, when will we it ever get done?

But it'll still take someone to do it -- I'm afraid it's out of my
depth to do so.

There is a chance that Mark W. or Travis O. could do it, but it does
seem unlikely that they'll find the time in the next week or two, so I
guess we'll put it off, and keep the "experimental" label on there.

>> My impression is that we will have something for 1.9. If it comes in for
>> 1.8, fine.

I sure hope we can at least get the "rip out the ugly default I/O TZ
behavior" fix in time for 1.9. Whether something more ambitious can be
done, we'll have to see.

> OK - then I guess you are saying it is up you, our datetimer friends,
> to make a proposal and timetable and implementation, if y'all think it
> can be done in the next few weeks,

Yup -- if anyone wants to pipe up and offer to do it, speak now or
forever hold your piece.

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From chris.barker at noaa.gov  Thu Aug 15 18:28:01 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Thu, 15 Aug 2013 15:28:01 -0700
Subject: [Numpy-discussion] Upcoming 1.8 release.
In-Reply-To: <CAH6Pt5rmAvZiox40a1aWD6+o8aE_jFvLmfGsOnjFTw=Cc+kbmg@mail.gmail.com>
References: <CAB6mnxJoUzEonhRT18JkQWQmDPusxdyWqbRiZwpRGwDhGbAYag@mail.gmail.com>
	<CABL7CQji5VZEY+Nd2VavQ9Kx+EoQ4ReX=kOGzH=X=KpuxhHcRQ@mail.gmail.com>
	<CAH6Pt5q=smLvDCDUy=PsS4__v4YmR305VcB1Y1M2bnAMJ=HBuw@mail.gmail.com>
	<CAB6mnx+oFzfvaPGwRgeH-krHRn5MSv+Fv4HfTSC9ozJhOLnd4Q@mail.gmail.com>
	<CAH6Pt5oBy3D0tQmgS87rMYX4Vq-3sYJ+pAuwz-z9ufgqH+5Qqw@mail.gmail.com>
	<CABL7CQhBLjb9ZZxSHTTc1L=3t1G_=ALz=o0yh_CE_=3jdsC8VA@mail.gmail.com>
	<CAH6Pt5rmAvZiox40a1aWD6+o8aE_jFvLmfGsOnjFTw=Cc+kbmg@mail.gmail.com>
Message-ID: <CALGmxEKpu3ysUhfr5So3wmGWaZ+5Z1V+w05FU0LNHg=Vpnij0g@mail.gmail.com>

On Thu, Aug 15, 2013 at 3:16 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
> Chris B - are you the point man on this one?  What do you think?

Only the point man in the sense that I'm poking at people to try to
get what I want ;-)

But see my other note.

-Chris


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From charlesr.harris at gmail.com  Thu Aug 15 19:53:18 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Thu, 15 Aug 2013 17:53:18 -0600
Subject: [Numpy-discussion] Upcoming 1.8 release.
In-Reply-To: <CALGmxEK97960fBhc8eq2-DSULZbTiL41f53PQFcUcTxpQJBENw@mail.gmail.com>
References: <CAB6mnxJoUzEonhRT18JkQWQmDPusxdyWqbRiZwpRGwDhGbAYag@mail.gmail.com>
	<CABL7CQji5VZEY+Nd2VavQ9Kx+EoQ4ReX=kOGzH=X=KpuxhHcRQ@mail.gmail.com>
	<CAH6Pt5q=smLvDCDUy=PsS4__v4YmR305VcB1Y1M2bnAMJ=HBuw@mail.gmail.com>
	<CAB6mnx+oFzfvaPGwRgeH-krHRn5MSv+Fv4HfTSC9ozJhOLnd4Q@mail.gmail.com>
	<CAH6Pt5oBy3D0tQmgS87rMYX4Vq-3sYJ+pAuwz-z9ufgqH+5Qqw@mail.gmail.com>
	<CALGmxEK97960fBhc8eq2-DSULZbTiL41f53PQFcUcTxpQJBENw@mail.gmail.com>
Message-ID: <CAB6mnx+o2JbUoLRknFrpjPhQua4rHsL=61mdQ20b5QQEZj-9NQ@mail.gmail.com>

On Thu, Aug 15, 2013 at 4:26 PM, Chris Barker - NOAA Federal <
chris.barker at noaa.gov> wrote:

> On Thu, Aug 15, 2013 at 12:31 PM, Matthew Brett <matthew.brett at gmail.com>
> wrote:
>
> >>> I'm afraid I don't understand the discussion on timezones in
> >>> datetime64, but I have the impression that those who do think it needs
> >>> an urgent decision and some action for the short term.  Is that right,
> >>> datetimers?
> >>>
> >>> If that's so, and there are worthwhile changes that are practical in
> >>> the next few weeks, it seems reasonable to wait.
>
> Well, it's only "urgent" in the sense that there are indeed a couple
> small changes that would really help, and if we don't use a release to
> motivate us, when will we it ever get done?
>
> But it'll still take someone to do it -- I'm afraid it's out of my
> depth to do so.
>
> There is a chance that Mark W. or Travis O. could do it, but it does
> seem unlikely that they'll find the time in the next week or two, so I
> guess we'll put it off, and keep the "experimental" label on there.
>
>
I think your best bet might be to cultivate Mark by testing and reviewing
his current work in dynd ;)

<snip>

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130815/37e19d1e/attachment.html>

From pierre.haessig at crans.org  Fri Aug 16 05:24:21 2013
From: pierre.haessig at crans.org (Pierre Haessig)
Date: Fri, 16 Aug 2013 11:24:21 +0200
Subject: [Numpy-discussion] Setter on array
In-Reply-To: <1376487296.28630.9716979.36ED8A61@webmail.messagingengine.com>
References: <1376487296.28630.9716979.36ED8A61@webmail.messagingengine.com>
Message-ID: <520DEFC5.5030902@crans.org>

Hi Hugo,

Le 14/08/2013 15:34, Hugo Gagnon a ?crit :
> What is the best way, if any, to "do something" whenever array elements
> are changed in-place?  For example, if I have a = arange(10), then
> setting a[3] = 1 would, say, call a function automatically.
I've never seen such a signal mechanism specifically for an array *element*.

The only thing that comes close in my mind are the "traits" from
Enthought which are somehow more general.
(http://docs.enthought.com/traits/traits_user_manual/intro.html for an
introduction)
And if I remember, there is an Array type of traits. Then you would get
a signal for any change in such an array, not just a specific element...
May this help ?

best,
Pierre


From njs at pobox.com  Fri Aug 16 05:50:15 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 16 Aug 2013 10:50:15 +0100
Subject: [Numpy-discussion] Setter on array
In-Reply-To: <1376487296.28630.9716979.36ED8A61@webmail.messagingengine.com>
References: <1376487296.28630.9716979.36ED8A61@webmail.messagingengine.com>
Message-ID: <CAPJVwBkO2RSn+do+DQUP1Vr3KaeWNcb0fzNNs_MWTCN23rvguw@mail.gmail.com>

On 14 Aug 2013 14:37, "Hugo Gagnon" <opensource.numpy at user.fastmail.fm>
wrote:
>
> Hi,
>
> What is the best way, if any, to "do something" whenever array elements
> are changed in-place?  For example, if I have a = arange(10), then
> setting a[3] = 1 would, say, call a function automatically.

There isn't really any reliable way to do this.

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130816/7330e869/attachment.html>

From ckkart at hoc.net  Fri Aug 16 08:33:38 2013
From: ckkart at hoc.net (Christian K.)
Date: Fri, 16 Aug 2013 09:33:38 -0300
Subject: [Numpy-discussion] Setter on array
In-Reply-To: <1376487296.28630.9716979.36ED8A61@webmail.messagingengine.com>
References: <1376487296.28630.9716979.36ED8A61@webmail.messagingengine.com>
Message-ID: <kul66r$3sl$1@ger.gmane.org>

Hi Hugo,

Am 14.08.13 10:34, schrieb Hugo Gagnon:
> What is the best way, if any, to "do something" whenever array elements
> are changed in-place?  For example, if I have a = arange(10), then
> setting a[3] = 1 would, say, call a function automatically.

a one made a simple subclass of ndarray which takes a callback argument 
and which should do exactly what you are asking for. I am not an expert 
so you might want to try to understand the concerns mentioned in the 
other answers.

class cbarray(N.ndarray):
     def __new__(subtype, data, cb=None, dtype=None, copy=False):
         subtype.__defaultcb = cb

         if copy:
             data = N.array(data,dtype=dtype)
         else:
             data = N.asarray(data,dtype=dtype)

         data = data.view(subtype)
         return data

     def _notify(self):
         if self.cb is not None:
             self.cb()

     def _get_shape(self):
         return super(cbarray, self).shape
     shape = property(_get_shape)

     def __setitem__(self, item, val):
         N.ndarray.__setitem__(self, item, val)
         self._notify()

     def __array_finalize__(self,obj):
         if not hasattr(self, "cb"):
             # The object does not yet have a `.cb` attribute
             self.cb = getattr(obj,'cb',self.__defaultcb)

     def __reduce__(self):
         object_state = list(N.ndarray.__reduce__(self))
         subclass_state = (self.cb,)
         object_state[2] = (object_state[2],subclass_state)
         return tuple(object_state)

     def __setstate__(self,state):
         nd_state, own_state = state
         N.ndarray.__setstate__(self,nd_state)

         cb, = own_state
         self.cb = cb


Regards, Christian




From alan.isaac at gmail.com  Fri Aug 16 11:20:32 2013
From: alan.isaac at gmail.com (Alan G Isaac)
Date: Fri, 16 Aug 2013 11:20:32 -0400
Subject: [Numpy-discussion] PEP 450 (stats module for standard library)
Message-ID: <520E4340.5010108@gmail.com>

http://www.python.org/dev/peps/pep-0450/
https://groups.google.com/forum/#!topic/comp.lang.python/IV-3mobU7L0

Alan Isaac


From chris.barker at noaa.gov  Fri Aug 16 12:35:19 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Fri, 16 Aug 2013 09:35:19 -0700
Subject: [Numpy-discussion] PEP 450 (stats module for standard library)
In-Reply-To: <520E4340.5010108@gmail.com>
References: <520E4340.5010108@gmail.com>
Message-ID: <CALGmxELnLgogGOY3hJX7vTMBRHiyGAE7H-HYbp9st6aKk6RsMQ@mail.gmail.com>

On Fri, Aug 16, 2013 at 8:20 AM, Alan G Isaac <alan.isaac at gmail.com> wrote:
> http://www.python.org/dev/peps/pep-0450/
> https://groups.google.com/forum/#!topic/comp.lang.python/IV-3mobU7L0

as numpy is the "right" way to do this sort of stuff, I think this is
a better argument for a numpy-lite in the stdlib than anything else.

But I doubt that will happen. Oh well.

-Chris


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From valentin at haenel.co  Fri Aug 16 12:44:45 2013
From: valentin at haenel.co (Valentin Haenel)
Date: Fri, 16 Aug 2013 18:44:45 +0200
Subject: [Numpy-discussion] PEP 450 (stats module for standard library)
In-Reply-To: <CALGmxELnLgogGOY3hJX7vTMBRHiyGAE7H-HYbp9st6aKk6RsMQ@mail.gmail.com>
References: <520E4340.5010108@gmail.com>
	<CALGmxELnLgogGOY3hJX7vTMBRHiyGAE7H-HYbp9st6aKk6RsMQ@mail.gmail.com>
Message-ID: <20130816164445.GA26581@kudu.in-berlin.de>

* Chris Barker - NOAA Federal <chris.barker at noaa.gov> [2013-08-16]:
> On Fri, Aug 16, 2013 at 8:20 AM, Alan G Isaac <alan.isaac at gmail.com> wrote:
> > http://www.python.org/dev/peps/pep-0450/
> > https://groups.google.com/forum/#!topic/comp.lang.python/IV-3mobU7L0
> 
> as numpy is the "right" way to do this sort of stuff, I think this is
> a better argument for a numpy-lite in the stdlib than anything else.
> 
> But I doubt that will happen. Oh well.

The PEP also says something about numpy being a ?nuclear reactor?..
While I do see the point of a simple stats library being an included
battery and numpy being something more powerful -- I wouldn't take the
analogy too far. Since this would imply, amongst other things, Numpy
could potentially suffer a meltdown and be hazardous to environment. ;)

best,

V-


From cjwilliams43 at gmail.com  Fri Aug 16 17:02:06 2013
From: cjwilliams43 at gmail.com (Colin J. Williams)
Date: Fri, 16 Aug 2013 17:02:06 -0400
Subject: [Numpy-discussion] Python PEP 450
Message-ID: <520E934E.2030206@gmail.com>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130816/72e7d497/attachment.html>

From alan.isaac at gmail.com  Fri Aug 16 17:24:47 2013
From: alan.isaac at gmail.com (Alan G Isaac)
Date: Fri, 16 Aug 2013 17:24:47 -0400
Subject: [Numpy-discussion] Python PEP 450
In-Reply-To: <520E934E.2030206@gmail.com>
References: <520E934E.2030206@gmail.com>
Message-ID: <520E989F.20306@gmail.com>

Hi Colin,

I'm just the messenger.  I thought this list might be interested.
Feedback should go to Steven D'Aprano on comp.lang.python

I think the PEP makes it clear that the target is not secondary
students but rather users who want numerically robust versions
of some basic statistical procedures.

Imo, a question arises: is there any way that NumPy (or the
statsmodels gang, for that matter) should influence or
participate in this PEP?

Alan


From arinkverma at iitrpr.ac.in  Sat Aug 17 10:27:52 2013
From: arinkverma at iitrpr.ac.in (Arink Verma)
Date: Sat, 17 Aug 2013 19:57:52 +0530
Subject: [Numpy-discussion] Test cases for integer behvaiour
Message-ID: <CAJAs4rS5sstno3_1XjA2NiCuCef-nAaVLEdqhZy+ce=sXpjk2Q@mail.gmail.com>

Hi All,

For scalar operations Numpy first try to extract the underlying C value
from a Python Integers. It causes bottleneck because it first converts the
Python scalar into its matching NumPy scalar (e.g. PyLong -> int32) and
then it extracts the C value from the NumPy scalar.

Its quicker to just extract the value directly from the Python scalar. Raul
did it for float but for integers case is different as it get handled by
different OS differently. There are basically two standards for long on 64
bit os, Microsoft uses long = int (32 bits), linux uses long = long long
(64 bits).

Hence, before getting into above mentioned modifications, there is need to
have much test case to ensure behaviour of integer remains same. I have
written fews at pr #3592 <https://github.com/numpy/numpy/pull/3592>

-
Arink
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130817/dfea3bb3/attachment.html>

From pav at iki.fi  Sat Aug 17 13:43:02 2013
From: pav at iki.fi (Pauli Virtanen)
Date: Sat, 17 Aug 2013 20:43:02 +0300
Subject: [Numpy-discussion] Upcoming 1.8 release.
In-Reply-To: <CAB6mnxLhNkAuPT3uHGsC7+MWq2Su8YtuhkgMX-f+EByWaAQejg@mail.gmail.com>
References: <CAB6mnxJoUzEonhRT18JkQWQmDPusxdyWqbRiZwpRGwDhGbAYag@mail.gmail.com>
	<CABDkGQmJPGN3Gk8tR11CbDvhbRf78QbAPDYrjnxyhiKBJ=Tmqw@mail.gmail.com>
	<CAB6mnxJRgcX=15PSrmL5YHBpASNTng33=iGaLVe-XF-KP9ok-g@mail.gmail.com>
	<CAOiFMpwvfpDkFH7XjtL-8Tq68YGc73J3Wx+3nXWa5X1ip5cykA@mail.gmail.com>
	<CAB6mnxLhNkAuPT3uHGsC7+MWq2Su8YtuhkgMX-f+EByWaAQejg@mail.gmail.com>
Message-ID: <kuocmu$vh2$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

15.08.2013 19:52, Charles R Harris kirjoitti:
> On Thu, Aug 15, 2013 at 10:48 AM, Blake Griffith
> <blake.a.griffith at gmail.com
>> wrote: I would like to have the ufunc overrides in 1.8 if it is
>> possible.
[clip]
> What is the status of that? I've been leaving that commit up the
> Pauli.

I think the PR itself is getting quite complete and does what the spec
promises.

However, I think the fact that the change comes this late in the
release cycle is sort of a problem. There has not been very much time
to test it out "in the wild", so we don't know for sure if we'd like
still to tweak something in the API.

So I'd either suggest merging the PR labelling this as an experimental
API in the docs, or if you think this is still too hasty, leave it for
1.9. The Numpy C code changes themselves are quite simple and
localized, so it seems unlikely that they could cause any breakage.

Getting it in now would have the advantage that we could also manage
to squeeze in the corresponding user-side part inside scipy.sparse for
Scipy 0.13.0.

- -- 
Pauli Virtanen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlIPtiYACgkQ6BQxb7O0pWAllACgiGdzoR6uZK4Y8GcqU1A3p7Dg
SoAAnjlFg2NVYS3qzy7KPQ5TSJ+ZUdPU
=Xdl7
-----END PGP SIGNATURE-----



From charlesr.harris at gmail.com  Sat Aug 17 14:20:50 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 17 Aug 2013 12:20:50 -0600
Subject: [Numpy-discussion] Upcoming 1.8 release.
In-Reply-To: <kuocmu$vh2$1@ger.gmane.org>
References: <CAB6mnxJoUzEonhRT18JkQWQmDPusxdyWqbRiZwpRGwDhGbAYag@mail.gmail.com>
	<CABDkGQmJPGN3Gk8tR11CbDvhbRf78QbAPDYrjnxyhiKBJ=Tmqw@mail.gmail.com>
	<CAB6mnxJRgcX=15PSrmL5YHBpASNTng33=iGaLVe-XF-KP9ok-g@mail.gmail.com>
	<CAOiFMpwvfpDkFH7XjtL-8Tq68YGc73J3Wx+3nXWa5X1ip5cykA@mail.gmail.com>
	<CAB6mnxLhNkAuPT3uHGsC7+MWq2Su8YtuhkgMX-f+EByWaAQejg@mail.gmail.com>
	<kuocmu$vh2$1@ger.gmane.org>
Message-ID: <CAB6mnx+1Eu3haT34qGPACt8U10GCZG=t3vLAbcKrhgn7VkG9Dw@mail.gmail.com>

On Sat, Aug 17, 2013 at 11:43 AM, Pauli Virtanen <pav at iki.fi> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> 15.08.2013 19:52, Charles R Harris kirjoitti:
> > On Thu, Aug 15, 2013 at 10:48 AM, Blake Griffith
> > <blake.a.griffith at gmail.com
> >> wrote: I would like to have the ufunc overrides in 1.8 if it is
> >> possible.
> [clip]
> > What is the status of that? I've been leaving that commit up the
> > Pauli.
>
> I think the PR itself is getting quite complete and does what the spec
> promises.
>
> However, I think the fact that the change comes this late in the
> release cycle is sort of a problem. There has not been very much time
> to test it out "in the wild", so we don't know for sure if we'd like
> still to tweak something in the API.
>
> So I'd either suggest merging the PR labelling this as an experimental
> API in the docs, or if you think this is still too hasty, leave it for
> 1.9. The Numpy C code changes themselves are quite simple and
> localized, so it seems unlikely that they could cause any breakage.
>
> Getting it in now would have the advantage that we could also manage
> to squeeze in the corresponding user-side part inside scipy.sparse for
> Scipy 0.13.0.
>
>
Experimental would be OK if it would help you with Scipy 0.13.0. But if it
does go in and is used in 0.13, won't that effectively lock it in until the
next scipy/numpy release? That seems a bit dangerous if you think some
changes might be warranted. OTOH, the testing might be worth the risk...

My hope is that the next numpy release take much less time than 1.8, which
is almost three releases worth of changes.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130817/4ceeb87d/attachment.html>

From pav at iki.fi  Sat Aug 17 14:42:42 2013
From: pav at iki.fi (Pauli Virtanen)
Date: Sat, 17 Aug 2013 21:42:42 +0300
Subject: [Numpy-discussion] Upcoming 1.8 release.
In-Reply-To: <CAB6mnx+1Eu3haT34qGPACt8U10GCZG=t3vLAbcKrhgn7VkG9Dw@mail.gmail.com>
References: <CAB6mnxJoUzEonhRT18JkQWQmDPusxdyWqbRiZwpRGwDhGbAYag@mail.gmail.com>
	<CABDkGQmJPGN3Gk8tR11CbDvhbRf78QbAPDYrjnxyhiKBJ=Tmqw@mail.gmail.com>
	<CAB6mnxJRgcX=15PSrmL5YHBpASNTng33=iGaLVe-XF-KP9ok-g@mail.gmail.com>
	<CAOiFMpwvfpDkFH7XjtL-8Tq68YGc73J3Wx+3nXWa5X1ip5cykA@mail.gmail.com>
	<CAB6mnxLhNkAuPT3uHGsC7+MWq2Su8YtuhkgMX-f+EByWaAQejg@mail.gmail.com>
	<kuocmu$vh2$1@ger.gmane.org>
	<CAB6mnx+1Eu3haT34qGPACt8U10GCZG=t3vLAbcKrhgn7VkG9Dw@mail.gmail.com>
Message-ID: <kuog6r$vn$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

17.08.2013 21:20, Charles R Harris kirjoitti:
[clip]
> Experimental would be OK if it would help you with Scipy 0.13.0.
> But if it does go in and is used in 0.13, won't that effectively
> lock it in until the next scipy/numpy release? That seems a bit
> dangerous if you think some changes might be warranted. OTOH, the
> testing might be worth the risk...
> 
> My hope is that the next numpy release take much less time than
> 1.8, which is almost three releases worth of changes.

Ok, it starts to seem that it's a bit late also in the Scipy release
schedule to get this in 0.13.0, as Ralf is planning to branch that on
next Tuesday.

The Numpy interactions may reveal some surprises, so I'd prefer this
to cook for a while in the dev version.

The best route is probably to postpone until 1.9 and 0.14.0, I think.

- -- 
Pauli Virtanen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlIPxCIACgkQ6BQxb7O0pWBU3ACgy4NXXXyQJthIcGPD3GTDKljg
G+wAoLVhEiH9bVjxfBswTSAP5fb+Tcsb
=rKWa
-----END PGP SIGNATURE-----



From sudheer.joseph at yahoo.com  Sun Aug 18 03:49:09 2013
From: sudheer.joseph at yahoo.com (Sudheer Joseph)
Date: Sun, 18 Aug 2013 15:49:09 +0800 (SGT)
Subject: [Numpy-discussion] strange behavior of variable
Message-ID: <1376812149.31789.YahooMailNeo@web193406.mail.sg3.yahoo.com>

Hi,
???????? I have defined a small function to find the n maximum values of an array as below. With in it I assign the input array to a second array and temporarily make the array location after first iteration as nan. I expected this temporary change to be limited to the second variable. However my initial variable gets modified. Can any one through some light to what is happening here?. In case of matlab this logic works.

######
#FUNCTION maxn
######

import numpy as np
def max_n(a,n):
???? b=a
???? result=[]
???? for i in np.arange(1,n+1):
???????? mxidx=np.where(b==max(b))
???????? result.append(mxidx)
???????? b[mxidx]=np.nan
???? result=np.ravel(result)??? 
???? return(result)


### TEST

In [8]: x=np.arange(float(0),10)

In [9]: max
max??? max_n? 

In [9]: max_n(x,2)
Out[9]: array([9, 8])

In [10]: x
Out[10]: array([? 0.,?? 1.,?? 2.,?? 3.,?? 4.,?? 5.,?? 6.,?? 7.,? nan,? nan])
?
***************************************************************
Sudheer Joseph 
Indian National Centre for Ocean Information Services
Ministry of Earth Sciences, Govt. of India
POST BOX NO: 21, IDA Jeedeemetla P.O.
Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
Web- http://oppamthadathil.tripod.com
***************************************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130818/141f13fc/attachment.html>

From efiring at hawaii.edu  Sun Aug 18 04:27:00 2013
From: efiring at hawaii.edu (Eric Firing)
Date: Sat, 17 Aug 2013 22:27:00 -1000
Subject: [Numpy-discussion] strange behavior of variable
In-Reply-To: <1376812149.31789.YahooMailNeo@web193406.mail.sg3.yahoo.com>
References: <1376812149.31789.YahooMailNeo@web193406.mail.sg3.yahoo.com>
Message-ID: <52108554.8040204@hawaii.edu>

On 2013/08/17 9:49 PM, Sudheer Joseph wrote:
> Hi,
>           I have defined a small function to find the n maximum values
> of an array as below. With in it I assign the input array to a second
> array and temporarily make the array location after first iteration as
> nan. I expected this temporary change to be limited to the second
> variable. However my initial variable gets modified. Can any one through
> some light to what is happening here?. In case of matlab this logic works.
>
> ######
> #FUNCTION maxn
> ######
> import numpy as np
> def max_n(a,n):
>       b=a

This is not making "b" a copy of "a", it is simply making it an alias 
for it.  To make it a copy you could use "b = a[:]", or "b = a.copy()"

It sounds like you don't really need a function, however.  Try this:

# test data:
a = np.random.randn(10)
n = 2

# One-line solution:
biggest_n = np.sort(a)[-n:]

print a
print biggest_n

If you want them ordered from largest to smallest, just reverse the list:

biggest_n = biggest_n[::-1]

Eric


>       result=[]
>       for i in np.arange(1,n+1):
>           mxidx=np.where(b==max(b))
>           result.append(mxidx)
>           b[mxidx]=np.nan
>       result=np.ravel(result)
>       return(result)
>
> ### TEST
> In [8]: x=np.arange(float(0),10)
>
> In [9]: max
> max    max_n
>
> In [9]: max_n(x,2)
> Out[9]: array([9, 8])
>
> In [10]: x
> Out[10]: array([  0.,   1.,   2.,   3.,   4.,   5.,   6.,   7.,  nan,  nan])
> ***************************************************************
> Sudheer Joseph
> Indian National Centre for Ocean Information Services
> Ministry of Earth Sciences, Govt. of India
> POST BOX NO: 21, IDA Jeedeemetla P.O.
> Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
> Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
> Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
> E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
> Web- http://oppamthadathil.tripod.com
> ***************************************************************
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>



From sudheer.joseph at yahoo.com  Sun Aug 18 04:50:45 2013
From: sudheer.joseph at yahoo.com (Sudheer Joseph)
Date: Sun, 18 Aug 2013 16:50:45 +0800 (SGT)
Subject: [Numpy-discussion] strange behavior of variable
In-Reply-To: <52108554.8040204@hawaii.edu>
References: <1376812149.31789.YahooMailNeo@web193406.mail.sg3.yahoo.com>
	<52108554.8040204@hawaii.edu>
Message-ID: <1376815845.90059.YahooMailNeo@web193402.mail.sg3.yahoo.com>

Thanks a lot Eric,
??????????????????? Here comes the smartness of python!!, I am just climbing the bottom steps...
with best regards,
Sudheer


?
***************************************************************
Sudheer Joseph 
Indian National Centre for Ocean Information Services
Ministry of Earth Sciences, Govt. of India
POST BOX NO: 21, IDA Jeedeemetla P.O.
Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
Web- http://oppamthadathil.tripod.com
***************************************************************



>________________________________
> From: Eric Firing <efiring at hawaii.edu>
>To: numpy-discussion at scipy.org 
>Sent: Sunday, 18 August 2013 1:57 PM
>Subject: Re: [Numpy-discussion] strange behavior of variable
> 
>
>On 2013/08/17 9:49 PM, Sudheer Joseph wrote:
>> Hi,
>>? ? ? ? ?  I have defined a small function to find the n maximum values
>> of an array as below. With in it I assign the input array to a second
>> array and temporarily make the array location after first iteration as
>> nan. I expected this temporary change to be limited to the second
>> variable. However my initial variable gets modified. Can any one through
>> some light to what is happening here?. In case of matlab this logic works.
>>
>> ######
>> #FUNCTION maxn
>> ######
>> import numpy as np
>> def max_n(a,n):
>>? ? ?  b=a
>
>This is not making "b" a copy of "a", it is simply making it an alias 
>for it.? To make it a copy you could use "b = a[:]", or "b = a.copy()"
>
>It sounds like you don't really need a function, however.? Try this:
>
># test data:
>a = np.random.randn(10)
>n = 2
>
># One-line solution:
>biggest_n = np.sort(a)[-n:]
>
>print a
>print biggest_n
>
>If you want them ordered from largest to smallest, just reverse the list:
>
>biggest_n = biggest_n[::-1]
>
>Eric
>
>
>>? ? ?  result=[]
>>? ? ?  for i in np.arange(1,n+1):
>>? ? ? ? ?  mxidx=np.where(b==max(b))
>>? ? ? ? ?  result.append(mxidx)
>>? ? ? ? ?  b[mxidx]=np.nan
>>? ? ?  result=np.ravel(result)
>>? ? ?  return(result)
>>
>> ### TEST
>> In [8]: x=np.arange(float(0),10)
>>
>> In [9]: max
>> max? ? max_n
>>
>> In [9]: max_n(x,2)
>> Out[9]: array([9, 8])
>>
>> In [10]: x
>> Out[10]: array([? 0.,?  1.,?  2.,?  3.,?  4.,?  5.,?  6.,?  7.,? nan,? nan])
>> ***************************************************************
>> Sudheer Joseph
>> Indian National Centre for Ocean Information Services
>> Ministry of Earth Sciences, Govt. of India
>> POST BOX NO: 21, IDA Jeedeemetla P.O.
>> Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
>> Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
>> Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
>> E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
>> Web- http://oppamthadathil.tripod.com
>> ***************************************************************
>>
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>
>_______________________________________________
>NumPy-Discussion mailing list
>NumPy-Discussion at scipy.org
>http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130818/6168e39f/attachment.html>

From sudheer.joseph at yahoo.com  Sun Aug 18 05:06:19 2013
From: sudheer.joseph at yahoo.com (Sudheer Joseph)
Date: Sun, 18 Aug 2013 17:06:19 +0800 (SGT)
Subject: [Numpy-discussion] strange behavior of variable
In-Reply-To: <1376815845.90059.YahooMailNeo@web193402.mail.sg3.yahoo.com>
References: <1376812149.31789.YahooMailNeo@web193406.mail.sg3.yahoo.com>	<52108554.8040204@hawaii.edu>
	<1376815845.90059.YahooMailNeo@web193402.mail.sg3.yahoo.com>
Message-ID: <1376816779.68121.YahooMailNeo@web193406.mail.sg3.yahoo.com>



?
However, I was looking for the indices of the biggest 3 numbers which can be used to find corresponding values in another vector, which made me to write the function. Is there a smarter way for getting indices?
with best regards,
Sudheer
Thanks a lot Eric,

??????????????????? Here comes the smartness of python!!, I am just climbing the bottom steps...
>with best regards,
>Sudheer
>
>
>
>?
>***************************************************************
>Sudheer Joseph 
>Indian National Centre for Ocean Information Services
>Ministry of Earth Sciences, Govt. of India
>POST BOX NO: 21, IDA Jeedeemetla P.O.
>Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
>Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
>Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
>E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
>Web-
 http://oppamthadathil.tripod.com
>***************************************************************
>
>
>
>>________________________________
>> From: Eric Firing <efiring at hawaii.edu>
>>To: numpy-discussion at scipy.org 
>>Sent: Sunday, 18 August 2013 1:57 PM
>>Subject: Re: [Numpy-discussion] strange behavior of variable
>> 
>>
>>On 2013/08/17 9:49 PM, Sudheer Joseph wrote:
>>>
 Hi,
>>>? ? ? ? ?  I have defined a small function to find the n maximum values
>>> of an array as below. With in it I assign the input array to a second
>>> array and temporarily make the array location after first iteration as
>>> nan. I expected this temporary change to be limited to the second
>>> variable. However my initial variable gets modified. Can any one through
>>> some light to what is happening here?. In case of matlab this logic works.
>>>
>>> ######
>>> #FUNCTION maxn
>>> ######
>>> import numpy as np
>>> def max_n(a,n):
>>>? ? ?  b=a
>>
>>This is not making "b" a copy of "a", it is simply making it an alias 
>>for it.? To make it a copy you could use "b = a[:]", or "b = a.copy()"
>>
>>It sounds like you don't really need a function, however.? Try this:
>>
>># test data:
>>a = np.random.randn(10)
>>n = 2
>>
>># One-line
 solution:
>>biggest_n = np.sort(a)[-n:]
>>
>>print a
>>print biggest_n
>>
>>If you want them ordered from largest to smallest, just reverse the list:
>>
>>biggest_n = biggest_n[::-1]
>>
>>Eric
>>
>>
>>>? ? ?  result=[]
>>>? ? ?  for i in np.arange(1,n+1):
>>>? ? ? ? ?  mxidx=np.where(b==max(b))
>>>? ? ? ? ?  result.append(mxidx)
>>>? ? ? ? ?  b[mxidx]=np.nan
>>>? ? ?  result=np.ravel(result)
>>>? ? ?  return(result)
>>>
>>> ### TEST
>>> In [8]: x=np.arange(float(0),10)
>>>
>>> In [9]: max
>>> max? ? max_n
>>>
>>> In [9]: max_n(x,2)
>>> Out[9]: array([9, 8])
>>>
>>> In [10]: x
>>> Out[10]: array([? 0.,?  1.,?  2.,?  3.,?  4.,?  5.,?  6.,?  7.,? nan,? nan])
>>>
 ***************************************************************
>>> Sudheer Joseph
>>> Indian National Centre for Ocean Information Services
>>> Ministry of Earth Sciences, Govt. of India
>>> POST BOX NO: 21, IDA Jeedeemetla P.O.
>>> Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
>>> Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
>>> Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
>>> E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
>>> Web- http://oppamthadathil.tripod.com
>>> ***************************************************************
>>>
>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>
>>_______________________________________________
>>NumPy-Discussion mailing list
>>NumPy-Discussion at scipy.org
>>http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>>
>_______________________________________________
>NumPy-Discussion mailing list
>NumPy-Discussion at scipy.org
>http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130818/3bd70256/attachment.html>

From njs at pobox.com  Sun Aug 18 05:31:53 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Sun, 18 Aug 2013 10:31:53 +0100
Subject: [Numpy-discussion] strange behavior of variable
In-Reply-To: <1376816779.68121.YahooMailNeo@web193406.mail.sg3.yahoo.com>
References: <1376812149.31789.YahooMailNeo@web193406.mail.sg3.yahoo.com>
	<52108554.8040204@hawaii.edu>
	<1376815845.90059.YahooMailNeo@web193402.mail.sg3.yahoo.com>
	<1376816779.68121.YahooMailNeo@web193406.mail.sg3.yahoo.com>
Message-ID: <CAPJVwBnoPhOVBOOf898R3nkEdntH2q83-SV=nsL5jk2aT_QT7Q@mail.gmail.com>

Try:

np.argsort(a)[-n:]

-n
On 18 Aug 2013 10:10, "Sudheer Joseph" <sudheer.joseph at yahoo.com> wrote:

>
>
>
> However, I was looking for the indices of the biggest 3 numbers which can
> be used to find corresponding values in another vector, which made me to
> write the function. Is there a smarter way for getting indices?
> with best regards,
> Sudheer
> Thanks a lot Eric,
>
>                     Here comes the smartness of python!!, I am just
> climbing the bottom steps...
> with best regards,
> Sudheer
>
>
> ***************************************************************
> Sudheer Joseph
> Indian National Centre for Ocean Information Services
> Ministry of Earth Sciences, Govt. of India
> POST BOX NO: 21, IDA Jeedeemetla P.O.
> Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
> Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
> Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
> E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
> Web- http://oppamthadathil.tripod.com
> ***************************************************************
>
>   ------------------------------
>  *From:* Eric Firing <efiring at hawaii.edu>
> *To:* numpy-discussion at scipy.org
> *Sent:* Sunday, 18 August 2013 1:57 PM
> *Subject:* Re: [Numpy-discussion] strange behavior of variable
>
> On 2013/08/17 9:49 PM, Sudheer Joseph wrote:
> > Hi,
> >          I have defined a small function to find the n maximum values
> > of an array as below. With in it I assign the input array to a second
> > array and temporarily make the array location after first iteration as
> > nan. I expected this temporary change to be limited to the second
> > variable. However my initial variable gets modified. Can any one through
> > some light to what is happening here?. In case of matlab this logic
> works.
> >
> > ######
> > #FUNCTION maxn
> > ######
> > import numpy as np
> > def max_n(a,n):
> >      b=a
>
> This is not making "b" a copy of "a", it is simply making it an alias
> for it.  To make it a copy you could use "b = a[:]", or "b = a.copy()"
>
> It sounds like you don't really need a function, however.  Try this:
>
> # test data:
> a = np.random.randn(10)
> n = 2
>
> # One-line solution:
> biggest_n = np.sort(a)[-n:]
>
> print a
> print biggest_n
>
> If you want them ordered from largest to smallest, just reverse the list:
>
> biggest_n = biggest_n[::-1]
>
> Eric
>
>
> >      result=[]
> >      for i in np.arange(1,n+1):
> >          mxidx=np.where(b==max(b))
> >          result.append(mxidx)
> >          b[mxidx]=np.nan
> >      result=np.ravel(result)
> >      return(result)
> >
> > ### TEST
> > In [8]: x=np.arange(float(0),10)
> >
> > In [9]: max
> > max    max_n
> >
> > In [9]: max_n(x,2)
> > Out[9]: array([9, 8])
> >
> > In [10]: x
> > Out[10]: array([  0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.,  nan,  nan])
> > ***************************************************************
> > Sudheer Joseph
> > Indian National Centre for Ocean Information Services
> > Ministry of Earth Sciences, Govt. of India
> > POST BOX NO: 21, IDA Jeedeemetla P.O.
> > Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
> > Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
> > Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
> > E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
> > Web- http://oppamthadathil.tripod.com
> > ***************************************************************
> >
> >
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130818/4f9ef388/attachment.html>

From sudheer.joseph at yahoo.com  Sun Aug 18 06:05:33 2013
From: sudheer.joseph at yahoo.com (Sudheer Joseph)
Date: Sun, 18 Aug 2013 18:05:33 +0800 (SGT)
Subject: [Numpy-discussion] strange behavior of variable
In-Reply-To: <CAPJVwBnoPhOVBOOf898R3nkEdntH2q83-SV=nsL5jk2aT_QT7Q@mail.gmail.com>
References: <1376812149.31789.YahooMailNeo@web193406.mail.sg3.yahoo.com>	<52108554.8040204@hawaii.edu>	<1376815845.90059.YahooMailNeo@web193402.mail.sg3.yahoo.com>	<1376816779.68121.YahooMailNeo@web193406.mail.sg3.yahoo.com>
	<CAPJVwBnoPhOVBOOf898R3nkEdntH2q83-SV=nsL5jk2aT_QT7Q@mail.gmail.com>
Message-ID: <1376820333.15139.YahooMailNeo@web193406.mail.sg3.yahoo.com>

Tank you,
It worked fine,
with best regards,
Sudheer

From: Nathaniel Smith <njs at pobox.com>

To: Discussion of Numerical Python <numpy-discussion at scipy.org> 
>Sent: Sunday, 18 August 2013 3:01 PM
>Subject: Re: [Numpy-discussion] strange behavior of variable
> 
>
>
>Try:
>np.argsort(a)[-n:]
>-n
>On 18 Aug 2013 10:10, "Sudheer Joseph" <sudheer.joseph at yahoo.com> wrote:
>
>
>>
>>?
>>However, I was looking for the indices of the biggest 3 numbers which can be used to find corresponding values in another vector, which made me to write the function. Is there a smarter way for getting indices?
>>with best regards,
>>Sudheer
>>Thanks a lot Eric,
>>
>>??????????????????? Here comes the smartness of python!!, I am just climbing the bottom steps...
>>>with best regards,
>>>Sudheer
>>>
>>>
>>>
>>>?
>>>***************************************************************
>>>Sudheer Joseph 
>>>Indian National Centre for Ocean Information Services
>>>Ministry of Earth Sciences, Govt. of India
>>>POST BOX NO: 21, IDA Jeedeemetla P.O.
>>>Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
>>>Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
>>>Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
>>>E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
>>>Web- http://oppamthadathil.tripod.com
>>>***************************************************************
>>>
>>>
>>>
>>>>________________________________
>>>> From: Eric Firing <efiring at hawaii.edu>
>>>>To: numpy-discussion at scipy.org 
>>>>Sent: Sunday, 18 August 2013 1:57 PM
>>>>Subject: Re: [Numpy-discussion] strange behavior of variable
>>>> 
>>>>
>>>>On 2013/08/17 9:49 PM, Sudheer Joseph wrote:
>>>>>
 Hi,
>>>>>? ? ? ? ?  I have defined a small function to find the n maximum values
>>>>> of an array as below. With in it I assign the input array to a second
>>>>> array and temporarily make the array location after first iteration as
>>>>> nan. I expected this temporary change to be limited to the second
>>>>> variable. However my initial variable gets modified. Can any one through
>>>>> some light to what is happening here?. In case of matlab this logic works.
>>>>>
>>>>> ######
>>>>> #FUNCTION maxn
>>>>> ######
>>>>> import numpy as np
>>>>> def max_n(a,n):
>>>>>? ? ?  b=a
>>>>
>>>>This is not making "b" a copy of "a", it is simply making it an alias 
>>>>for it.? To make it a copy you could use "b = a[:]", or "b = a.copy()"
>>>>
>>>>It sounds like you don't really need a function, however.? Try this:
>>>>
>>>># test data:
>>>>a = np.random.randn(10)
>>>>n = 2
>>>>
>>>># One-line
 solution:
>>>>biggest_n = np.sort(a)[-n:]
>>>>
>>>>print a
>>>>print biggest_n
>>>>
>>>>If you want them ordered from largest to smallest, just reverse the list:
>>>>
>>>>biggest_n = biggest_n[::-1]
>>>>
>>>>Eric
>>>>
>>>>
>>>>>? ? ?  result=[]
>>>>>? ? ?  for i in np.arange(1,n+1):
>>>>>? ? ? ? ?  mxidx=np.where(b==max(b))
>>>>>? ? ? ? ?  result.append(mxidx)
>>>>>? ? ? ? ?  b[mxidx]=np.nan
>>>>>? ? ?  result=np.ravel(result)
>>>>>? ? ?  return(result)
>>>>>
>>>>> ### TEST
>>>>> In [8]: x=np.arange(float(0),10)
>>>>>
>>>>> In [9]: max
>>>>> max? ? max_n
>>>>>
>>>>> In [9]: max_n(x,2)
>>>>> Out[9]: array([9, 8])
>>>>>
>>>>> In [10]: x
>>>>> Out[10]: array([? 0.,?  1.,?  2.,?  3.,?  4.,?  5.,?  6.,?  7.,? nan,? nan])
>>>>>
 ***************************************************************
>>>>> Sudheer Joseph
>>>>> Indian National Centre for Ocean Information Services
>>>>> Ministry of Earth Sciences, Govt. of India
>>>>> POST BOX NO: 21, IDA Jeedeemetla P.O.
>>>>> Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55
>>>>> Tel:+91-40-23886047(O),Fax:+91-40-23895011(O),
>>>>> Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile)
>>>>> E-mail:sjo.India at gmail.com;sudheer.joseph at yahoo.com
>>>>> Web- http://oppamthadathil.tripod.com
>>>>> ***************************************************************
>>>>>
>>>>>
>>>>>
 _______________________________________________
>>>>> NumPy-Discussion mailing list
>>>>> NumPy-Discussion at scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>>>
>>>>
>>>>_______________________________________________
>>>>NumPy-Discussion mailing list
>>>>NumPy-Discussion at scipy.org
>>>>http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>>
>>>>
>>>>
>>>_______________________________________________
>>>NumPy-Discussion
 mailing list
>>>NumPy-Discussion at scipy.org
>>>http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>>
>>>
>>_______________________________________________
>>NumPy-Discussion mailing list
>>NumPy-Discussion at scipy.org
>>http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>_______________________________________________
>NumPy-Discussion mailing list
>NumPy-Discussion at scipy.org
>http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130818/521f736f/attachment.html>

From charlesr.harris at gmail.com  Sun Aug 18 14:17:42 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 18 Aug 2013 12:17:42 -0600
Subject: [Numpy-discussion] 1.8.0 branch reminder
Message-ID: <CAB6mnx+G5JY-szhVH__-sFv=YF6bJ1JvCFQqHm=Nih_-9ACMjg@mail.gmail.com>

Just a reminder that 1.8.0 will be branched tonight. I've put up a big STY:
PR <https://github.com/numpy/numpy/pull/3635> that removes trailing
whitespace and fixes spacing after commas. I would like to apply before the
branch, but it may cause merge difficulties down the line. I'd like
feedback on that option.

Thoughts?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130818/c96aea6a/attachment.html>

From charlesr.harris at gmail.com  Sun Aug 18 14:36:28 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 18 Aug 2013 12:36:28 -0600
Subject: [Numpy-discussion] 1.8.0 branch reminder
In-Reply-To: <CAB6mnx+G5JY-szhVH__-sFv=YF6bJ1JvCFQqHm=Nih_-9ACMjg@mail.gmail.com>
References: <CAB6mnx+G5JY-szhVH__-sFv=YF6bJ1JvCFQqHm=Nih_-9ACMjg@mail.gmail.com>
Message-ID: <CAB6mnxLrzFM=2Rt4u137URt6gbRo1wnBXxLvNO0mZK7GP106vA@mail.gmail.com>

On Sun, Aug 18, 2013 at 12:17 PM, Charles R Harris <
charlesr.harris at gmail.com> wrote:

> Just a reminder that 1.8.0 will be branched tonight. I've put up a big STY:
> PR <https://github.com/numpy/numpy/pull/3635> that removes trailing
> whitespace and fixes spacing after commas. I would like to apply before the
> branch, but it may cause merge difficulties down the line. I'd like
> feedback on that option.
>
>
I've also run autopep8 on the code and it does a nice job of cleaning up
things. It gets a little lost in deeply nested lists, but there aren't too
many of those. By default it doesn't fix spaces about operator (it seems).
I can apply that also if there is interest in doing so.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130818/f8f400bd/attachment.html>

From joferkington at gmail.com  Sun Aug 18 19:14:55 2013
From: joferkington at gmail.com (Joe Kington)
Date: Sun, 18 Aug 2013 18:14:55 -0500
Subject: [Numpy-discussion] Adding an axis argument to numpy.unique
Message-ID: <CACe1pJf3ta4UD_Dd20_NUi2HAfPFGPZBQRGDXBO7EeF_yLB7tw@mail.gmail.com>

Hi everyone,

I've recently put together a pull request that adds an `axis` kwarg to
`numpy.unique` so that `unique`can easily be used to find unique
rows/columns/sub-arrays/etc of a larger array.

https://github.com/numpy/numpy/pull/3584

Currently, this works as a warpper around `unique`. If `axis` is specified,
it reshapes the input to a 2D contiguous array, views each row as a single
item, then passes it on to `unique`.  For int and string dtypes, each row
is viewed as a void dtype and therefore bitwise-equality is used for
comparisons.  For all other dtypes, the each row is viewed as a structured
array.

The current implementation has two main drawbacks:

   1. For anything other than ints and strings, it's relatively slow.
   2. It doesn't work with object arrays of any sort.

I'd appreciate any thoughts/feedback folks might have on both the general
idea and this specific implementation.  It think it's a worthwhile
addition, but I'm biased.

Thanks!
-Joe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130818/7d888994/attachment.html>

From charlesr.harris at gmail.com  Sun Aug 18 20:17:27 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 18 Aug 2013 18:17:27 -0600
Subject: [Numpy-discussion] maintenance/1.8.x has been branched.
Message-ID: <CAB6mnx+sK1ak1Wp65VWY9bwZsfhVxUNLWF=3A6dsxx_Mt8tVug@mail.gmail.com>

Master is open for 1.9.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130818/8acec97a/attachment.html>

From charlesr.harris at gmail.com  Sun Aug 18 23:38:55 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 18 Aug 2013 21:38:55 -0600
Subject: [Numpy-discussion] Removal of numarray and oldnumeric modules.
Message-ID: <CAB6mnx+DhE6M9ay7zNcZNK39N60r04msxU09H+X8LhHajCOjSA@mail.gmail.com>

Hi All,

I've put up a PR <https://github.com/numpy/numpy/pull/3638> that removes
the numarray and oldnumeric modules from Numpy 1.9. The hope is that 1.9
will be released in five to six months. If you have trouble with removal on
that schedule, now is the time to complain.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130818/f9151111/attachment.html>

From juanlu001 at gmail.com  Mon Aug 19 02:37:59 2013
From: juanlu001 at gmail.com (Juan Luis Cano)
Date: Mon, 19 Aug 2013 08:37:59 +0200
Subject: [Numpy-discussion] Deprecation of financial routines
Message-ID: <5211BD47.1040407@gmail.com>

As now master is open for 1.9, following the discussion opened here

https://github.com/numpy/numpy/issues/2880

it was suggested that we deprecate and eventually remove the financial 
functions in NumPy, because they pollute the main namespace and some are 
unimplemented. We could put them in a separate package, in case it 
doesn't exist yet. Nathaniel Smith and Ralf Gommers already gave +1, and 
Charles Harris suggested bringing this up in the mailing list.

Thoughts?


From cournape at gmail.com  Mon Aug 19 02:46:01 2013
From: cournape at gmail.com (David Cournapeau)
Date: Mon, 19 Aug 2013 07:46:01 +0100
Subject: [Numpy-discussion] Deprecation of financial routines
In-Reply-To: <5211BD47.1040407@gmail.com>
References: <5211BD47.1040407@gmail.com>
Message-ID: <CAGY4rcXN=eafE64RU3XkuZoEgGRbjcdazXk0Q1Bzmzz87PQOdw@mail.gmail.com>

I am +1 as well, I don't think they should have been included in the first
place.

The deprecation should happen after a separate package has been made
available, in case some people depend on it.


On Mon, Aug 19, 2013 at 7:37 AM, Juan Luis Cano <juanlu001 at gmail.com> wrote:

> As now master is open for 1.9, following the discussion opened here
>
> https://github.com/numpy/numpy/issues/2880
>
> it was suggested that we deprecate and eventually remove the financial
> functions in NumPy, because they pollute the main namespace and some are
> unimplemented. We could put them in a separate package, in case it
> doesn't exist yet. Nathaniel Smith and Ralf Gommers already gave +1, and
> Charles Harris suggested bringing this up in the mailing list.
>
> Thoughts?
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130819/b1622730/attachment.html>

From alan.isaac at gmail.com  Mon Aug 19 07:34:36 2013
From: alan.isaac at gmail.com (Alan G Isaac)
Date: Mon, 19 Aug 2013 07:34:36 -0400
Subject: [Numpy-discussion] Deprecation of financial routines
In-Reply-To: <5211BD47.1040407@gmail.com>
References: <5211BD47.1040407@gmail.com>
Message-ID: <521202CC.7010708@gmail.com>

On 8/19/2013 2:37 AM, Juan Luis Cano wrote:
> https://github.com/numpy/numpy/issues/2880
>
> it was suggested that we deprecate and eventually remove the financial
> functions in NumPy


It seems that this summary is a bit one-sided.  There was also
a suggestion to move these into numpy.financial, which is much
less disruptive.

I'm not arguing for either of those choices, but it seems to
me that the right way to bring this forward to to recall the
original motivation for having them and ask if that motivation
has fallen apart.  I believe that the original motivation for
providing the financial functions was to complete the sense
that Python users could turn to NumPy whenever they might
turn to a calculator:
http://mail.scipy.org/pipermail/numpy-discussion/2008-April/032422.html
Behind this, I believe, was a desire to draw new users to NumPy.

The request for simple financial functions does arise,
and people do get pointed to NumPy for this.  This seems to me
to be a benefit to the NumPy community, although a small one.

I'm getting the feeling that some of the opposition to the financial
functions is purely aesthetic. ("What are *these* doing here??")
It seems to me that a request for comments would be more useful if
it tried to lay out the perceived costs and benefits to this change.

One cost that has been clearly stated is namespace pollution.
That seems pretty small to me, but in any case would be fully
addressed by making these available as numpy.financial.

Alan Isaac



From josef.pktd at gmail.com  Mon Aug 19 08:39:09 2013
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Mon, 19 Aug 2013 08:39:09 -0400
Subject: [Numpy-discussion] Adding an axis argument to numpy.unique
In-Reply-To: <CACe1pJf3ta4UD_Dd20_NUi2HAfPFGPZBQRGDXBO7EeF_yLB7tw@mail.gmail.com>
References: <CACe1pJf3ta4UD_Dd20_NUi2HAfPFGPZBQRGDXBO7EeF_yLB7tw@mail.gmail.com>
Message-ID: <CAMMTP+B7KOvWtZY+gNgKygB07og9JC_PhkmWgv9oxCznew5NvA@mail.gmail.com>

On Sun, Aug 18, 2013 at 7:14 PM, Joe Kington <joferkington at gmail.com> wrote:
> Hi everyone,
>
> I've recently put together a pull request that adds an `axis` kwarg to
> `numpy.unique` so that `unique`can easily be used to find unique
> rows/columns/sub-arrays/etc of a larger array.
>
> https://github.com/numpy/numpy/pull/3584
>
> Currently, this works as a warpper around `unique`. If `axis` is specified,
> it reshapes the input to a 2D contiguous array, views each row as a single
> item, then passes it on to `unique`.  For int and string dtypes, each row is
> viewed as a void dtype and therefore bitwise-equality is used for
> comparisons.  For all other dtypes, the each row is viewed as a structured
> array.
>
> The current implementation has two main drawbacks:
>
> For anything other than ints and strings, it's relatively slow.
> It doesn't work with object arrays of any sort.
>
> I'd appreciate any thoughts/feedback folks might have on both the general
> idea and this specific implementation.  It think it's a worthwhile addition,
> but I'm biased.


just a general comment

I have been missing a `unique_rows` or something like that, which
seems to be the target of this change.

However, my first interpretation of an axis argument in unique would
be that it treats each column (or whatever along axis) separately.
Analogously to max, argmax and similar.

On second thought:
unique with axis working on each column separately wouldn't create a
nice return array, because it won't be rectangular (in general)

Josef

>
> Thanks!
> -Joe
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>


From juanlu001 at gmail.com  Mon Aug 19 08:45:53 2013
From: juanlu001 at gmail.com (Juan Luis Cano)
Date: Mon, 19 Aug 2013 14:45:53 +0200
Subject: [Numpy-discussion] Deprecation of financial routines
In-Reply-To: <521202CC.7010708@gmail.com>
References: <5211BD47.1040407@gmail.com> <521202CC.7010708@gmail.com>
Message-ID: <52121381.7000207@gmail.com>

[clip]
On 08/19/2013 01:34 PM, Alan G Isaac wrote:
> On 8/19/2013 2:37 AM, Juan Luis Cano wrote:
>> https://github.com/numpy/numpy/issues/2880
>>
>> it was suggested that we deprecate and eventually remove the financial
>> functions in NumPy
>
> It seems that this summary is a bit one-sided.  There was also
> a suggestion to move these into numpy.financial, which is much
> less disruptive.

Sorry for that, it's true that the summary was one-sided. I wanted to be 
brief, and ended supressing some information.

I was not aware of the original motivation for having those functions 
there, and I see those points are valid too. Nevertheless my motivations 
for bringing this up on the mailing list go no further than raising 
discussion and help closing and old issue, so apart from getting them 
out from the main namespace (either numpy.financial or complete removal, 
and yes, for aesthetic reasons) I am +-0 on either proposal.

Juan Luis Cano


From tim at cerazone.net  Mon Aug 19 09:38:19 2013
From: tim at cerazone.net (Cera, Tim)
Date: Mon, 19 Aug 2013 09:38:19 -0400
Subject: [Numpy-discussion] Deprecation of financial routines
In-Reply-To: <5211BD47.1040407@gmail.com>
References: <5211BD47.1040407@gmail.com>
Message-ID: <CAO5s+D_28kN-Hrh14wrWUfMNZ_p6q2NqF6TF9wcHxDkKEQ56+w@mail.gmail.com>

On Mon, Aug 19, 2013 at 2:37 AM, Juan Luis Cano <juanlu001 at gmail.com> wrote:
>
> As now master is open for 1.9, following the discussion opened here
>
> https://github.com/numpy/numpy/issues/2880
>
> it was suggested that we deprecate and eventually remove the financial
> functions in NumPy, because they pollute the main namespace and some are
> unimplemented. We could put them in a separate package, in case it
> doesn't exist yet. Nathaniel Smith and Ralf Gommers already gave +1, and
> Charles Harris suggested bringing this up in the mailing list.

When I was initially working with the docs it galled me to find
documented, but unimplemented financial functions.  I spent the time
to implement all of the unimplemented functions. As an example see
https://github.com/numpy/numpy/pull/190.  I just glanced through the
code and all functions are there, implemented and documented, so I
don't know where that comment came from in
https://github.com/numpy/numpy/issues/2880.  Definitely in 1.7.

About whether they should stay or go, I vote 0.  I see financial
functions as an absolute requirement in engineering (though often
engineers allow financial optimization and decisions to default to
others, IMO a big mistake).  Financial analysis in science?  Probably
not so much, which is my guess as to why the discussion was brought
up.

Since I wrote a couple of the financial functions, you would think I
might vote -1 to deprecation, but the reason I wrote the functions was
to remove the NotImplemented errors.  They really bothered me.  I
thought that if the functions were already included in numpy, they
must be useful to someone.  For me, typically I do any financial work
in a spreadsheet - that is why I vote 0.

Kindest regards,
Tim


From jsseabold at gmail.com  Mon Aug 19 09:47:35 2013
From: jsseabold at gmail.com (Skipper Seabold)
Date: Mon, 19 Aug 2013 09:47:35 -0400
Subject: [Numpy-discussion] Deprecation of financial routines
In-Reply-To: <CAO5s+D_28kN-Hrh14wrWUfMNZ_p6q2NqF6TF9wcHxDkKEQ56+w@mail.gmail.com>
References: <5211BD47.1040407@gmail.com>
	<CAO5s+D_28kN-Hrh14wrWUfMNZ_p6q2NqF6TF9wcHxDkKEQ56+w@mail.gmail.com>
Message-ID: <CAKF=Djst+swB9iAS_3Xq_GFQRiWXPUstnEzas04L2zgdTuzfdQ@mail.gmail.com>

On Mon, Aug 19, 2013 at 9:38 AM, Cera, Tim <tim at cerazone.net> wrote:

> On Mon, Aug 19, 2013 at 2:37 AM, Juan Luis Cano <juanlu001 at gmail.com>
> wrote:
> >
> > As now master is open for 1.9, following the discussion opened here
> >
> > https://github.com/numpy/numpy/issues/2880
> >
> > it was suggested that we deprecate and eventually remove the financial
> > functions in NumPy, because they pollute the main namespace and some are
> > unimplemented. We could put them in a separate package, in case it
> > doesn't exist yet. Nathaniel Smith and Ralf Gommers already gave +1, and
> > Charles Harris suggested bringing this up in the mailing list.
>

+1 on scipy.finance / scipy.financial (or even numpy.finance /
numpy.financial)

Skipper
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130819/34ed4679/attachment.html>

From josef.pktd at gmail.com  Mon Aug 19 10:04:19 2013
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Mon, 19 Aug 2013 10:04:19 -0400
Subject: [Numpy-discussion] Deprecation of financial routines
In-Reply-To: <CAKF=Djst+swB9iAS_3Xq_GFQRiWXPUstnEzas04L2zgdTuzfdQ@mail.gmail.com>
References: <5211BD47.1040407@gmail.com>
	<CAO5s+D_28kN-Hrh14wrWUfMNZ_p6q2NqF6TF9wcHxDkKEQ56+w@mail.gmail.com>
	<CAKF=Djst+swB9iAS_3Xq_GFQRiWXPUstnEzas04L2zgdTuzfdQ@mail.gmail.com>
Message-ID: <CAMMTP+DU_1rXjA5wSpmoebJNhhOsQZMzn=WVhrW-PbcHOYgYfQ@mail.gmail.com>

On Mon, Aug 19, 2013 at 9:47 AM, Skipper Seabold <jsseabold at gmail.com> wrote:
> On Mon, Aug 19, 2013 at 9:38 AM, Cera, Tim <tim at cerazone.net> wrote:
>>
>> On Mon, Aug 19, 2013 at 2:37 AM, Juan Luis Cano <juanlu001 at gmail.com>
>> wrote:
>> >
>> > As now master is open for 1.9, following the discussion opened here
>> >
>> > https://github.com/numpy/numpy/issues/2880
>> >
>> > it was suggested that we deprecate and eventually remove the financial
>> > functions in NumPy, because they pollute the main namespace and some are
>> > unimplemented. We could put them in a separate package, in case it
>> > doesn't exist yet. Nathaniel Smith and Ralf Gommers already gave +1, and
>> > Charles Harris suggested bringing this up in the mailing list.
>
>
> +1 on scipy.finance / scipy.financial (or even numpy.finance /
> numpy.financial)

I think now scipy.finance is overkill. There isn't enough to warrant a
subpackage. And for serious code, users should use other packages
(like pandas, ...).

numpy.financial like matplotlib.finance
just to satisfy some basic spread sheet use cases
http://stackoverflow.com/questions/2259379/basic-financial-library-for-python

If the python standard lib can get statistics, then numpy can keep
some basic financial calculations.

+0.5 on numpy.finance / numpy.financial
with explicit import (?)

(proposal for new function `numpy.spreadsheet_calculation` :)

Josef

>
> Skipper
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>


From jh at physics.ucf.edu  Mon Aug 19 11:06:52 2013
From: jh at physics.ucf.edu (Joe Harrington)
Date: Mon, 19 Aug 2013 11:06:52 -0400
Subject: [Numpy-discussion] Deprecation of financial routines
In-Reply-To: <mailman.4319.1376916047.973.numpy-discussion@scipy.org>
	(numpy-discussion-request@scipy.org)
Message-ID: <wl8d2p9afeb.fsf@physics.ucf.edu>

> On 8/19/2013 2:37 AM, Juan Luis Cano wrote:
>> https://github.com/numpy/numpy/issues/2880
>>
>> it was suggested that we deprecate and eventually remove the financial
>> functions in NumPy

IDL has financial functions.  Matlab has financial functions.  Financial
functions are something that a subset of potential customers of
numerical packages look for.  There is a strong tradition of people
going from math/science/engineering into the finanical world, and people
start on that route by playing with strategies and seeing what they can
do relative to the market.  To see how well they do, they need to
calculate an internal rate of return, and so forth.

The fin routines are tiny and don't require much maintenance once
written.  If we made an effort (putting up pages with examples of common
financial calculations and collecting those under a topical web page,
then linking to that page from various places and talking it up), I
would think they could attract users looking for a free way to play with
financial scenarios.  Python is a perfect language for newbies to
programming, which many who want to play with finances are.

So, I would say we keep them.  If ours are not the best, we should bring
them up to snuff.

I think we can endlessly debate namespace pollution.  Developers hate
it, but most normal users just don't mind a large number of routines in
the top level, and many actively dislike the code litter of lots of
long, dotted routine names.  I've had a hard time getting my colleagues
not to "from numpy import *".  Some of the good ones even teach that in
their seminars to convert IDL and IRAF users, and they chew me out when
I suggest to them that it's a bad habit!

A reorg that would bring us to a very heirarchical structure would be
disruptive to existing code.  Yet, there are maintenance and startup
speed arguments in favor of a heirarchy.  However we resolve it, I don't
know that singling out the financial routines is the right short-term
approach, and it wouldn't make much of a dent.  So, I'd suggest we take
it as two issues to solve separately 1) keeping or tossing the fin
routines and 2) moving toward a heirarchy of routines or not.

I'm -1 for deprecating the routines.

I'll hold off on the second question until someone makes a clear
proposal that resolves namespace pollution, and we discuss it.

--jh--


From joseluismietta at yahoo.com.ar  Mon Aug 19 12:33:10 2013
From: joseluismietta at yahoo.com.ar (=?iso-8859-1?Q?Jos=E8_Luis_Mietta?=)
Date: Mon, 19 Aug 2013 09:33:10 -0700 (PDT)
Subject: [Numpy-discussion] At.: question about RAM problem in numpy array
	code
In-Reply-To: <1376929625.20253.YahooMailNeo@web142301.mail.bf1.yahoo.com>
References: <1372287629.89197.YahooMailNeo@web142306.mail.bf1.yahoo.com>
	<1376929522.8308.YahooMailNeo@web142302.mail.bf1.yahoo.com>
	<1376929625.20253.YahooMailNeo@web142301.mail.bf1.yahoo.com>
Message-ID: <1376929990.72906.YahooMailNeo@web142306.mail.bf1.yahoo.com>

?Hi experts!

I have a core i3 3GB RAM Toshiba laptop. Im a newby Ubuntu 13.04, python and sage user. I note that RAM memory becomes full while running script (starts in 50% of RAM ocupation and becomes to 100% (full)). This generate that operating system become slower... In the code few numpy arrays are gereated (each one with ~700 elements or more).
The script looks like this:
? OUTPUT=[]
? number of elements=[n1,n2,.....]
? numbers of executions =N
? forj innumber of elements:
? ? ? ? lalala=[]
? ? ? ? fori insrange(N):
? ? ? ? ? ? ?Algorithmisexecuted 'N'times an append one value inarray 'lalala'each time.Ineach execution three numpy matrix of 300x 300(=90000)elements (each one)participate.Thismatrix are called 'M','M_v'and'M_h'andare re-generated foreach 'i'index inthe for-cycle.
? ? ? ? Doingmath on all 'N'elements in'lalala'a 'pipipi'element isgenerated an thenappend intoOUTPUT array.

When execution ends the OUTPUT array have the same lenght that 'number of elements' array.
In each 'for' cycle, other arrayrs participate.
What can I do for throubleshooting that? IS possible that the algorithm and old arrays are saving in RAM memory? If tahts possible, how can I di for deleting that?
Waiting for your answers.
Thanks a lot!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130819/0a9ccd6c/attachment.html>

From joferkington at gmail.com  Mon Aug 19 20:39:03 2013
From: joferkington at gmail.com (Joe Kington)
Date: Mon, 19 Aug 2013 19:39:03 -0500
Subject: [Numpy-discussion] Adding an axis argument to numpy.unique
In-Reply-To: <CAMMTP+B7KOvWtZY+gNgKygB07og9JC_PhkmWgv9oxCznew5NvA@mail.gmail.com>
References: <CACe1pJf3ta4UD_Dd20_NUi2HAfPFGPZBQRGDXBO7EeF_yLB7tw@mail.gmail.com>
	<CAMMTP+B7KOvWtZY+gNgKygB07og9JC_PhkmWgv9oxCznew5NvA@mail.gmail.com>
Message-ID: <CACe1pJfsLrvLG+cV_VLbpvLm1hCPxSoVOa_=RAdZpLc=1C1QTg@mail.gmail.com>

...<snip>

>
> However, my first interpretation of an axis argument in unique would
> be that it treats each column (or whatever along axis) separately.
> Analogously to max, argmax and similar.
>

Good point!

That's certainly a potential source of confusion.  However, I can't seem to
come up with a better name for the kwarg. Matlab's "unique" function has a
"rows" option, which is probably a more intuitive name, but doesn't imply
the expansion to N-dimensions.

"axis" is still fairly idiomatic, despite the confusion over "unique
rows/columns/etc" vs "unique items within each row/column/etc".

Any thoughts on a better name for the argument?


> On second thought:
> unique with axis working on each column separately wouldn't create a
> nice return array, because it won't be rectangular (in general)
>
Josef
>

Yeah, and "unique items within each row/column/etc" would be best
implemented as a one-line list comprehension for that reason, rather than
an addition to unique, i.m.o.

Thanks for the feedback!
-Joe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130819/dd25c1b6/attachment.html>

From stefan at sun.ac.za  Tue Aug 20 04:51:48 2013
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Tue, 20 Aug 2013 10:51:48 +0200
Subject: [Numpy-discussion] Adding an axis argument to numpy.unique
In-Reply-To: <CACe1pJfsLrvLG+cV_VLbpvLm1hCPxSoVOa_=RAdZpLc=1C1QTg@mail.gmail.com>
References: <CACe1pJf3ta4UD_Dd20_NUi2HAfPFGPZBQRGDXBO7EeF_yLB7tw@mail.gmail.com>
	<CAMMTP+B7KOvWtZY+gNgKygB07og9JC_PhkmWgv9oxCznew5NvA@mail.gmail.com>
	<CACe1pJfsLrvLG+cV_VLbpvLm1hCPxSoVOa_=RAdZpLc=1C1QTg@mail.gmail.com>
Message-ID: <CABDkGQ=tb0w_UVi1Biitcee2fCAF=PoM0BNmoTmUL_pb+Jub-g@mail.gmail.com>

On Tue, Aug 20, 2013 at 2:39 AM, Joe Kington <joferkington at gmail.com> wrote:
> That's certainly a potential source of confusion.  However, I can't seem to
> come up with a better name for the kwarg. Matlab's "unique" function has a
> "rows" option, which is probably a more intuitive name, but doesn't imply
> the expansion to N-dimensions.

To me, `unique_rows` sounds perfect.  To go along columns: unique_rows(A.T)

St?fan


From njs at pobox.com  Tue Aug 20 05:04:25 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 20 Aug 2013 10:04:25 +0100
Subject: [Numpy-discussion] Adding an axis argument to numpy.unique
In-Reply-To: <CACe1pJfsLrvLG+cV_VLbpvLm1hCPxSoVOa_=RAdZpLc=1C1QTg@mail.gmail.com>
References: <CACe1pJf3ta4UD_Dd20_NUi2HAfPFGPZBQRGDXBO7EeF_yLB7tw@mail.gmail.com>
	<CAMMTP+B7KOvWtZY+gNgKygB07og9JC_PhkmWgv9oxCznew5NvA@mail.gmail.com>
	<CACe1pJfsLrvLG+cV_VLbpvLm1hCPxSoVOa_=RAdZpLc=1C1QTg@mail.gmail.com>
Message-ID: <CAPJVwBk5-4GqC9FD5Hf=zLMZ04=QaYaNoeLNpti2pkyoppaeEQ@mail.gmail.com>

On 20 Aug 2013 01:39, "Joe Kington" <joferkington at gmail.com> wrote:
>
>
>
>
> ...<snip>
>>
>>
>> However, my first interpretation of an axis argument in unique would
>> be that it treats each column (or whatever along axis) separately.
>> Analogously to max, argmax and similar.
>
>
> Good point!
>
> That's certainly a potential source of confusion.  However, I can't seem
to come up with a better name for the kwarg. Matlab's "unique" function has
a "rows" option, which is probably a more intuitive name, but doesn't imply
the expansion to N-dimensions.
>
> "axis" is still fairly idiomatic, despite the confusion over "unique
rows/columns/etc" vs "unique items within each row/column/etc".
>
> Any thoughts on a better name for the argument?

I also found this pretty confusing when first looking at the PR.

One option might be to invert the sense of the argument to emphasize that
it's treating subarrays as units, so instead of specifying the iteration
axis you specify the axes of the subarray. compare_axis= or something?

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130820/536cef3a/attachment.html>

From stefan at sun.ac.za  Tue Aug 20 05:30:10 2013
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Tue, 20 Aug 2013 11:30:10 +0200
Subject: [Numpy-discussion] Deprecation of financial routines
In-Reply-To: <CAKF=Djst+swB9iAS_3Xq_GFQRiWXPUstnEzas04L2zgdTuzfdQ@mail.gmail.com>
References: <5211BD47.1040407@gmail.com>
	<CAO5s+D_28kN-Hrh14wrWUfMNZ_p6q2NqF6TF9wcHxDkKEQ56+w@mail.gmail.com>
	<CAKF=Djst+swB9iAS_3Xq_GFQRiWXPUstnEzas04L2zgdTuzfdQ@mail.gmail.com>
Message-ID: <CABDkGQkCm27P10aQDQmRQ5BkFaZtSDSi43O+2P_3ftuknw94bw@mail.gmail.com>

On Mon, Aug 19, 2013 at 3:47 PM, Skipper Seabold <jsseabold at gmail.com> wrote:
>
> +1 on scipy.finance / scipy.financial (or even numpy.finance /
> numpy.financial)

Are there no external libraries that deal with these things?  If they
exist, we can deprecate with two releases pointing to that external
source.

Factoring the code out to a package only makes sense if someone is
willing to take ownership, otherwise we're essentially deprecating it
out to die.

St?fan


From josef.pktd at gmail.com  Tue Aug 20 07:08:46 2013
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Tue, 20 Aug 2013 07:08:46 -0400
Subject: [Numpy-discussion] Adding an axis argument to numpy.unique
In-Reply-To: <CAPJVwBk5-4GqC9FD5Hf=zLMZ04=QaYaNoeLNpti2pkyoppaeEQ@mail.gmail.com>
References: <CACe1pJf3ta4UD_Dd20_NUi2HAfPFGPZBQRGDXBO7EeF_yLB7tw@mail.gmail.com>
	<CAMMTP+B7KOvWtZY+gNgKygB07og9JC_PhkmWgv9oxCznew5NvA@mail.gmail.com>
	<CACe1pJfsLrvLG+cV_VLbpvLm1hCPxSoVOa_=RAdZpLc=1C1QTg@mail.gmail.com>
	<CAPJVwBk5-4GqC9FD5Hf=zLMZ04=QaYaNoeLNpti2pkyoppaeEQ@mail.gmail.com>
Message-ID: <CAMMTP+AZ6G8LK-rx7=YnHMAR=n5L-9YZ=y2tHxZj_nooOJ915g@mail.gmail.com>

On Tue, Aug 20, 2013 at 5:04 AM, Nathaniel Smith <njs at pobox.com> wrote:
> On 20 Aug 2013 01:39, "Joe Kington" <joferkington at gmail.com> wrote:
>>
>>
>>
>>
>> ...<snip>
>>>
>>>
>>> However, my first interpretation of an axis argument in unique would
>>> be that it treats each column (or whatever along axis) separately.
>>> Analogously to max, argmax and similar.
>>
>>
>> Good point!
>>
>> That's certainly a potential source of confusion.  However, I can't seem
>> to come up with a better name for the kwarg. Matlab's "unique" function has
>> a "rows" option, which is probably a more intuitive name, but doesn't imply
>> the expansion to N-dimensions.
>>
>> "axis" is still fairly idiomatic, despite the confusion over "unique
>> rows/columns/etc" vs "unique items within each row/column/etc".
>>
>> Any thoughts on a better name for the argument?
>
> I also found this pretty confusing when first looking at the PR.
>
> One option might be to invert the sense of the argument to emphasize that
> it's treating subarrays as units, so instead of specifying the iteration
> axis you specify the axes of the subarray. compare_axis= or something?

you would need compare_axes (plural for ndim>2) and have to specify
all but one axis, AFAICS.

Josef

>
> -n
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>


From njs at pobox.com  Tue Aug 20 07:34:48 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 20 Aug 2013 12:34:48 +0100
Subject: [Numpy-discussion] Adding an axis argument to numpy.unique
In-Reply-To: <CAMMTP+AZ6G8LK-rx7=YnHMAR=n5L-9YZ=y2tHxZj_nooOJ915g@mail.gmail.com>
References: <CACe1pJf3ta4UD_Dd20_NUi2HAfPFGPZBQRGDXBO7EeF_yLB7tw@mail.gmail.com>
	<CAMMTP+B7KOvWtZY+gNgKygB07og9JC_PhkmWgv9oxCznew5NvA@mail.gmail.com>
	<CACe1pJfsLrvLG+cV_VLbpvLm1hCPxSoVOa_=RAdZpLc=1C1QTg@mail.gmail.com>
	<CAPJVwBk5-4GqC9FD5Hf=zLMZ04=QaYaNoeLNpti2pkyoppaeEQ@mail.gmail.com>
	<CAMMTP+AZ6G8LK-rx7=YnHMAR=n5L-9YZ=y2tHxZj_nooOJ915g@mail.gmail.com>
Message-ID: <CAPJVwB=qGX2bdoPDFoGHfqDfy0RWbP0b4QQJMNVsAoYqqxWChA@mail.gmail.com>

On 20 Aug 2013 12:09, <josef.pktd at gmail.com> wrote:
>
> On Tue, Aug 20, 2013 at 5:04 AM, Nathaniel Smith <njs at pobox.com> wrote:
> > On 20 Aug 2013 01:39, "Joe Kington" <joferkington at gmail.com> wrote:
> >>
> >>
> >>
> >>
> >> ...<snip>
> >>>
> >>>
> >>> However, my first interpretation of an axis argument in unique would
> >>> be that it treats each column (or whatever along axis) separately.
> >>> Analogously to max, argmax and similar.
> >>
> >>
> >> Good point!
> >>
> >> That's certainly a potential source of confusion.  However, I can't
seem
> >> to come up with a better name for the kwarg. Matlab's "unique"
function has
> >> a "rows" option, which is probably a more intuitive name, but doesn't
imply
> >> the expansion to N-dimensions.
> >>
> >> "axis" is still fairly idiomatic, despite the confusion over "unique
> >> rows/columns/etc" vs "unique items within each row/column/etc".
> >>
> >> Any thoughts on a better name for the argument?
> >
> > I also found this pretty confusing when first looking at the PR.
> >
> > One option might be to invert the sense of the argument to emphasize
that
> > it's treating subarrays as units, so instead of specifying the iteration
> > axis you specify the axes of the subarray. compare_axis= or something?
>
> you would need compare_axes (plural for ndim>2) and have to specify
> all but one axis, AFAICS.

Well, it makes sense to specify any arbitrary subset of axes, whether or
not that's currently implemented.

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130820/07317780/attachment.html>

From josef.pktd at gmail.com  Tue Aug 20 07:47:53 2013
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Tue, 20 Aug 2013 07:47:53 -0400
Subject: [Numpy-discussion] Adding an axis argument to numpy.unique
In-Reply-To: <CAPJVwB=qGX2bdoPDFoGHfqDfy0RWbP0b4QQJMNVsAoYqqxWChA@mail.gmail.com>
References: <CACe1pJf3ta4UD_Dd20_NUi2HAfPFGPZBQRGDXBO7EeF_yLB7tw@mail.gmail.com>
	<CAMMTP+B7KOvWtZY+gNgKygB07og9JC_PhkmWgv9oxCznew5NvA@mail.gmail.com>
	<CACe1pJfsLrvLG+cV_VLbpvLm1hCPxSoVOa_=RAdZpLc=1C1QTg@mail.gmail.com>
	<CAPJVwBk5-4GqC9FD5Hf=zLMZ04=QaYaNoeLNpti2pkyoppaeEQ@mail.gmail.com>
	<CAMMTP+AZ6G8LK-rx7=YnHMAR=n5L-9YZ=y2tHxZj_nooOJ915g@mail.gmail.com>
	<CAPJVwB=qGX2bdoPDFoGHfqDfy0RWbP0b4QQJMNVsAoYqqxWChA@mail.gmail.com>
Message-ID: <CAMMTP+CLxipzmOEV3YL6_MctRN_iTyj-4YorvR6UgzjM1gQg1w@mail.gmail.com>

On Tue, Aug 20, 2013 at 7:34 AM, Nathaniel Smith <njs at pobox.com> wrote:
> On 20 Aug 2013 12:09, <josef.pktd at gmail.com> wrote:
>>
>> On Tue, Aug 20, 2013 at 5:04 AM, Nathaniel Smith <njs at pobox.com> wrote:
>> > On 20 Aug 2013 01:39, "Joe Kington" <joferkington at gmail.com> wrote:
>> >>
>> >>
>> >>
>> >>
>> >> ...<snip>
>> >>>
>> >>>
>> >>> However, my first interpretation of an axis argument in unique would
>> >>> be that it treats each column (or whatever along axis) separately.
>> >>> Analogously to max, argmax and similar.
>> >>
>> >>
>> >> Good point!
>> >>
>> >> That's certainly a potential source of confusion.  However, I can't
>> >> seem
>> >> to come up with a better name for the kwarg. Matlab's "unique" function
>> >> has
>> >> a "rows" option, which is probably a more intuitive name, but doesn't
>> >> imply
>> >> the expansion to N-dimensions.
>> >>
>> >> "axis" is still fairly idiomatic, despite the confusion over "unique
>> >> rows/columns/etc" vs "unique items within each row/column/etc".
>> >>
>> >> Any thoughts on a better name for the argument?
>> >
>> > I also found this pretty confusing when first looking at the PR.
>> >
>> > One option might be to invert the sense of the argument to emphasize
>> > that
>> > it's treating subarrays as units, so instead of specifying the iteration
>> > axis you specify the axes of the subarray. compare_axis= or something?
>>
>> you would need compare_axes (plural for ndim>2) and have to specify
>> all but one axis, AFAICS.
>
> Well, it makes sense to specify any arbitrary subset of axes, whether or not
> that's currently implemented.

not AFAICS, if you want to return a rectangular array without
nans/missing values.

Josef

>
> -n
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>


From josef.pktd at gmail.com  Tue Aug 20 07:49:56 2013
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Tue, 20 Aug 2013 07:49:56 -0400
Subject: [Numpy-discussion] Adding an axis argument to numpy.unique
In-Reply-To: <CAMMTP+CLxipzmOEV3YL6_MctRN_iTyj-4YorvR6UgzjM1gQg1w@mail.gmail.com>
References: <CACe1pJf3ta4UD_Dd20_NUi2HAfPFGPZBQRGDXBO7EeF_yLB7tw@mail.gmail.com>
	<CAMMTP+B7KOvWtZY+gNgKygB07og9JC_PhkmWgv9oxCznew5NvA@mail.gmail.com>
	<CACe1pJfsLrvLG+cV_VLbpvLm1hCPxSoVOa_=RAdZpLc=1C1QTg@mail.gmail.com>
	<CAPJVwBk5-4GqC9FD5Hf=zLMZ04=QaYaNoeLNpti2pkyoppaeEQ@mail.gmail.com>
	<CAMMTP+AZ6G8LK-rx7=YnHMAR=n5L-9YZ=y2tHxZj_nooOJ915g@mail.gmail.com>
	<CAPJVwB=qGX2bdoPDFoGHfqDfy0RWbP0b4QQJMNVsAoYqqxWChA@mail.gmail.com>
	<CAMMTP+CLxipzmOEV3YL6_MctRN_iTyj-4YorvR6UgzjM1gQg1w@mail.gmail.com>
Message-ID: <CAMMTP+AY+e+-=YVECT+xsh6LxW15Gj4saD+Rq0xXW36Yd+fPAw@mail.gmail.com>

On Tue, Aug 20, 2013 at 7:47 AM,  <josef.pktd at gmail.com> wrote:
> On Tue, Aug 20, 2013 at 7:34 AM, Nathaniel Smith <njs at pobox.com> wrote:
>> On 20 Aug 2013 12:09, <josef.pktd at gmail.com> wrote:
>>>
>>> On Tue, Aug 20, 2013 at 5:04 AM, Nathaniel Smith <njs at pobox.com> wrote:
>>> > On 20 Aug 2013 01:39, "Joe Kington" <joferkington at gmail.com> wrote:
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> ...<snip>
>>> >>>
>>> >>>
>>> >>> However, my first interpretation of an axis argument in unique would
>>> >>> be that it treats each column (or whatever along axis) separately.
>>> >>> Analogously to max, argmax and similar.
>>> >>
>>> >>
>>> >> Good point!
>>> >>
>>> >> That's certainly a potential source of confusion.  However, I can't
>>> >> seem
>>> >> to come up with a better name for the kwarg. Matlab's "unique" function
>>> >> has
>>> >> a "rows" option, which is probably a more intuitive name, but doesn't
>>> >> imply
>>> >> the expansion to N-dimensions.
>>> >>
>>> >> "axis" is still fairly idiomatic, despite the confusion over "unique
>>> >> rows/columns/etc" vs "unique items within each row/column/etc".
>>> >>
>>> >> Any thoughts on a better name for the argument?
>>> >
>>> > I also found this pretty confusing when first looking at the PR.
>>> >
>>> > One option might be to invert the sense of the argument to emphasize
>>> > that
>>> > it's treating subarrays as units, so instead of specifying the iteration
>>> > axis you specify the axes of the subarray. compare_axis= or something?
>>>
>>> you would need compare_axes (plural for ndim>2) and have to specify
>>> all but one axis, AFAICS.
>>
>> Well, it makes sense to specify any arbitrary subset of axes, whether or not
>> that's currently implemented.
>
> not AFAICS, if you want to return a rectangular array without
> nans/missing values.

and unless you want to ravel() the remaining axis, which is also weird
(I think).

Josef

>
> Josef
>
>>
>> -n
>>
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>


From njs at pobox.com  Tue Aug 20 07:55:47 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Tue, 20 Aug 2013 12:55:47 +0100
Subject: [Numpy-discussion] Adding an axis argument to numpy.unique
In-Reply-To: <CAMMTP+AY+e+-=YVECT+xsh6LxW15Gj4saD+Rq0xXW36Yd+fPAw@mail.gmail.com>
References: <CACe1pJf3ta4UD_Dd20_NUi2HAfPFGPZBQRGDXBO7EeF_yLB7tw@mail.gmail.com>
	<CAMMTP+B7KOvWtZY+gNgKygB07og9JC_PhkmWgv9oxCznew5NvA@mail.gmail.com>
	<CACe1pJfsLrvLG+cV_VLbpvLm1hCPxSoVOa_=RAdZpLc=1C1QTg@mail.gmail.com>
	<CAPJVwBk5-4GqC9FD5Hf=zLMZ04=QaYaNoeLNpti2pkyoppaeEQ@mail.gmail.com>
	<CAMMTP+AZ6G8LK-rx7=YnHMAR=n5L-9YZ=y2tHxZj_nooOJ915g@mail.gmail.com>
	<CAPJVwB=qGX2bdoPDFoGHfqDfy0RWbP0b4QQJMNVsAoYqqxWChA@mail.gmail.com>
	<CAMMTP+CLxipzmOEV3YL6_MctRN_iTyj-4YorvR6UgzjM1gQg1w@mail.gmail.com>
	<CAMMTP+AY+e+-=YVECT+xsh6LxW15Gj4saD+Rq0xXW36Yd+fPAw@mail.gmail.com>
Message-ID: <CAPJVwBm7=cAKeDGS=GkVmkmfY10AN-eogkvPfWxXe9M6S5+mCQ@mail.gmail.com>

On 20 Aug 2013 12:53, <josef.pktd at gmail.com> wrote:
>
> On Tue, Aug 20, 2013 at 7:47 AM,  <josef.pktd at gmail.com> wrote:
> > On Tue, Aug 20, 2013 at 7:34 AM, Nathaniel Smith <njs at pobox.com> wrote:
> >> On 20 Aug 2013 12:09, <josef.pktd at gmail.com> wrote:
> >>>
> >>> On Tue, Aug 20, 2013 at 5:04 AM, Nathaniel Smith <njs at pobox.com>
wrote:
> >>> > On 20 Aug 2013 01:39, "Joe Kington" <joferkington at gmail.com> wrote:
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >> ...<snip>
> >>> >>>
> >>> >>>
> >>> >>> However, my first interpretation of an axis argument in unique
would
> >>> >>> be that it treats each column (or whatever along axis) separately.
> >>> >>> Analogously to max, argmax and similar.
> >>> >>
> >>> >>
> >>> >> Good point!
> >>> >>
> >>> >> That's certainly a potential source of confusion.  However, I can't
> >>> >> seem
> >>> >> to come up with a better name for the kwarg. Matlab's "unique"
function
> >>> >> has
> >>> >> a "rows" option, which is probably a more intuitive name, but
doesn't
> >>> >> imply
> >>> >> the expansion to N-dimensions.
> >>> >>
> >>> >> "axis" is still fairly idiomatic, despite the confusion over
"unique
> >>> >> rows/columns/etc" vs "unique items within each row/column/etc".
> >>> >>
> >>> >> Any thoughts on a better name for the argument?
> >>> >
> >>> > I also found this pretty confusing when first looking at the PR.
> >>> >
> >>> > One option might be to invert the sense of the argument to emphasize
> >>> > that
> >>> > it's treating subarrays as units, so instead of specifying the
iteration
> >>> > axis you specify the axes of the subarray. compare_axis= or
something?
> >>>
> >>> you would need compare_axes (plural for ndim>2) and have to specify
> >>> all but one axis, AFAICS.
> >>
> >> Well, it makes sense to specify any arbitrary subset of axes, whether
or not
> >> that's currently implemented.
> >
> > not AFAICS, if you want to return a rectangular array without
> > nans/missing values.
>
> and unless you want to ravel() the remaining axis, which is also weird
> (I think).

The default (and until this patch, only) behaviour is to ravel all axes, so
it'd be consistent.

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130820/64ae4678/attachment.html>

From ralf.gommers at gmail.com  Tue Aug 20 16:48:51 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Tue, 20 Aug 2013 22:48:51 +0200
Subject: [Numpy-discussion] OS X binaries for releases
Message-ID: <CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ@mail.gmail.com>

Hi all,

Building binaries for releases is currently quite complex and
time-consuming. For OS X we need two different machines, because we still
provide binaries for OS X 10.5 and PPC machines. I propose to not do this
anymore. It doesn't mean we completely drop support for 10.5 and PPC, just
that we don't produce binaries. PPC was phased out in 2006 and OS X 10.6
came out in 2009, so there can't be a lot of demand for it (and the
download stats at
http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/confirm this).

Furthermore I propose to not provide 2.6 binaries anymore. Downloads of 2.6
OS X binaries were <5% of the 2.7 ones. We did the same with 2.4 for a long
time - support it but no binaries.

So what we'd have left at the moment is only the 64-bit/32-bit universal
binary for 10.6 and up. What we finally need to add is 3.x OS X binaries.
We can make an attempt to build these on 10.8 - since we have access to a
hosted 10.8 Mac Mini it would allow all devs to easily do a release
(leaving aside the Windows issue). If anyone has tried the 10.6 SDK on 10.8
and knows if it actually works, that would be helpful.

Any concerns, objections?

Cheers,
Ralf

P.S. the same proposal applies of course also to scipy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130820/4d74e9d0/attachment.html>

From cournape at gmail.com  Tue Aug 20 18:17:19 2013
From: cournape at gmail.com (David Cournapeau)
Date: Tue, 20 Aug 2013 23:17:19 +0100
Subject: [Numpy-discussion] OS X binaries for releases
In-Reply-To: <CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ@mail.gmail.com>
References: <CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ@mail.gmail.com>
Message-ID: <CAGY4rcXt+=09qP4TJdCzpXkyWStBKfeXODzuMo9MQe9XYy5jwA@mail.gmail.com>

On Tue, Aug 20, 2013 at 9:48 PM, Ralf Gommers <ralf.gommers at gmail.com>wrote:

> Hi all,
>
> Building binaries for releases is currently quite complex and
> time-consuming. For OS X we need two different machines, because we still
> provide binaries for OS X 10.5 and PPC machines. I propose to not do this
> anymore. It doesn't mean we completely drop support for 10.5 and PPC, just
> that we don't produce binaries. PPC was phased out in 2006 and OS X 10.6
> came out in 2009, so there can't be a lot of demand for it (and the
> download stats at http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/confirm this).
>
> Furthermore I propose to not provide 2.6 binaries anymore. Downloads of
> 2.6 OS X binaries were <5% of the 2.7 ones. We did the same with 2.4 for a
> long time - support it but no binaries.
>
> So what we'd have left at the moment is only the 64-bit/32-bit universal
> binary for 10.6 and up. What we finally need to add is 3.x OS X binaries.
> We can make an attempt to build these on 10.8 - since we have access to a
> hosted 10.8 Mac Mini it would allow all devs to easily do a release
> (leaving aside the Windows issue). If anyone has tried the 10.6 SDK on 10.8
> and knows if it actually works, that would be helpful.
>

I am not sure one can use 10.6 SDK on 10.8 ? I am actually looking into
those issues for our mac support at Enthought

Are you here for euroscipy ?

David

>
> Any concerns, objections?
>
> Cheers,
> Ralf
>
> P.S. the same proposal applies of course also to scipy
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130820/b3b69f9c/attachment.html>

From Tom.KACVINSKY at 3ds.com  Tue Aug 20 18:20:21 2013
From: Tom.KACVINSKY at 3ds.com (KACVINSKY Tom)
Date: Tue, 20 Aug 2013 22:20:21 +0000
Subject: [Numpy-discussion] OS X binaries for releases
In-Reply-To: <CAGY4rcXt+=09qP4TJdCzpXkyWStBKfeXODzuMo9MQe9XYy5jwA@mail.gmail.com>
References: <CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ@mail.gmail.com>,
	<CAGY4rcXt+=09qP4TJdCzpXkyWStBKfeXODzuMo9MQe9XYy5jwA@mail.gmail.com>
Message-ID: <35E5002C-FDEA-45FD-A43A-762FCCDEA8F7@3ds.com>

You can use the 10.6 SDK on 10.8.  At least we do.

Tom

On Aug 20, 2013, at 18:17, "David Cournapeau" <cournape at gmail.com<mailto:cournape at gmail.com>> wrote:




On Tue, Aug 20, 2013 at 9:48 PM, Ralf Gommers <ralf.gommers at gmail.com<mailto:ralf.gommers at gmail.com>> wrote:
Hi all,

Building binaries for releases is currently quite complex and time-consuming. For OS X we need two different machines, because we still provide binaries for OS X 10.5 and PPC machines. I propose to not do this anymore. It doesn't mean we completely drop support for 10.5 and PPC, just that we don't produce binaries. PPC was phased out in 2006 and OS X 10.6 came out in 2009, so there can't be a lot of demand for it (and the download stats at http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/ confirm this).

Furthermore I propose to not provide 2.6 binaries anymore. Downloads of 2.6 OS X binaries were <5% of the 2.7 ones. We did the same with 2.4 for a long time - support it but no binaries.

So what we'd have left at the moment is only the 64-bit/32-bit universal binary for 10.6 and up. What we finally need to add is 3.x OS X binaries. We can make an attempt to build these on 10.8 - since we have access to a hosted 10.8 Mac Mini it would allow all devs to easily do a release (leaving aside the Windows issue). If anyone has tried the 10.6 SDK on 10.8 and knows if it actually works, that would be helpful.

I am not sure one can use 10.6 SDK on 10.8 ? I am actually looking into those issues for our mac support at Enthought

Are you here for euroscipy ?

David

Any concerns, objections?

Cheers,
Ralf

P.S. the same proposal applies of course also to scipy



_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at scipy.org<mailto:NumPy-Discussion at scipy.org>
http://mail.scipy.org/mailman/listinfo/numpy-discussion


_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at scipy.org<mailto:NumPy-Discussion at scipy.org>
http://mail.scipy.org/mailman/listinfo/numpy-discussion

This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.

For other languages, go to http://www.3ds.com/terms/email-disclaimer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130820/baca79c9/attachment.html>

From cournape at gmail.com  Tue Aug 20 18:31:11 2013
From: cournape at gmail.com (David Cournapeau)
Date: Tue, 20 Aug 2013 23:31:11 +0100
Subject: [Numpy-discussion] OS X binaries for releases
In-Reply-To: <35E5002C-FDEA-45FD-A43A-762FCCDEA8F7@3ds.com>
References: <CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ@mail.gmail.com>
	<CAGY4rcXt+=09qP4TJdCzpXkyWStBKfeXODzuMo9MQe9XYy5jwA@mail.gmail.com>
	<35E5002C-FDEA-45FD-A43A-762FCCDEA8F7@3ds.com>
Message-ID: <CAGY4rcUR=cty1asAp1fwhAY6hStbiEU_R+DZs86nJ=VKO+9zXQ@mail.gmail.com>

On Tue, Aug 20, 2013 at 11:20 PM, KACVINSKY Tom <Tom.KACVINSKY at 3ds.com>wrote:

>  You can use the 10.6 SDK on 10.8.  At least we do.
>

With which compiler ?

David

>
>  Tom
>
> On Aug 20, 2013, at 18:17, "David Cournapeau" <cournape at gmail.com> wrote:
>
>
>
>
> On Tue, Aug 20, 2013 at 9:48 PM, Ralf Gommers <ralf.gommers at gmail.com>wrote:
>
>>  Hi all,
>>
>>  Building binaries for releases is currently quite complex and
>> time-consuming. For OS X we need two different machines, because we still
>> provide binaries for OS X 10.5 and PPC machines. I propose to not do this
>> anymore. It doesn't mean we completely drop support for 10.5 and PPC, just
>> that we don't produce binaries. PPC was phased out in 2006 and OS X 10.6
>> came out in 2009, so there can't be a lot of demand for it (and the
>> download stats at
>> http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/ confirm this).
>>
>>  Furthermore I propose to not provide 2.6 binaries anymore. Downloads of
>> 2.6 OS X binaries were <5% of the 2.7 ones. We did the same with 2.4 for a
>> long time - support it but no binaries.
>>
>>  So what we'd have left at the moment is only the 64-bit/32-bit
>> universal binary for 10.6 and up. What we finally need to add is 3.x OS X
>> binaries. We can make an attempt to build these on 10.8 - since we have
>> access to a hosted 10.8 Mac Mini it would allow all devs to easily do a
>> release (leaving aside the Windows issue). If anyone has tried the 10.6 SDK
>> on 10.8 and knows if it actually works, that would be helpful.
>>
>
>  I am not sure one can use 10.6 SDK on 10.8 ? I am actually looking into
> those issues for our mac support at Enthought
>
>  Are you here for euroscipy ?
>
>  David
>
>>
>>  Any concerns, objections?
>>
>>  Cheers,
>> Ralf
>>
>>  P.S. the same proposal applies of course also to scipy
>>
>>
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>    _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>  This email and any attachments are intended solely for the use of the
> individual or entity to whom it is addressed and may be confidential and/or
> privileged.
>
> If you are not one of the named recipients or have received this email in
> error,
>
> (i) you should not read, disclose, or copy it,
>
> (ii) please notify sender of your receipt by reply email and delete this
> email and all attachments,
>
> (iii) Dassault Systemes does not accept or assume any liability or
> responsibility for any use of or reliance on this email.
>
>  For other languages, go to http://www.3ds.com/terms/email-disclaimer
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130820/e939bfa6/attachment.html>

From chris.barker at noaa.gov  Tue Aug 20 18:32:33 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Tue, 20 Aug 2013 15:32:33 -0700
Subject: [Numpy-discussion] OS X binaries for releases
In-Reply-To: <CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ@mail.gmail.com>
References: <CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ@mail.gmail.com>
Message-ID: <CALGmxEJ+UYCaO7kyJp1X3gqFT7qPpprPhw7hP+tn2XufizDgAA@mail.gmail.com>

Ralf,

Thanks for doing all this!

> Building binaries for releases is currently quite complex and
> time-consuming.

It sure would be nice to clean that up.

 For OS X we need two different machines, because we still
> provide binaries for OS X 10.5 and PPC machines. I propose to not do this
> anymore. It doesn't mean we completely drop support for 10.5 and PPC, just
> that we don't produce binaries. PPC was phased out in 2006 and OS X 10.6
> came out in 2009, so there can't be a lot of demand for it (and the download
> stats at http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/ confirm
> this).
>
> Furthermore I propose to not provide 2.6 binaries anymore. Downloads of 2.6
> OS X binaries were <5% of the 2.7 ones. We did the same with 2.4 for a long
> time - support it but no binaries.

Adding 2.6 is not a huge deal, but I agree -- what's the point? And it
really is time to drop PPC an 10.5, though I'd be interested to see if
anyone IS still using it.

> So what we'd have left at the moment is only the 64-bit/32-bit universal
> binary for 10.6 and up. What we finally need to add is 3.x OS X binaries. We
> can make an attempt to build these on 10.8 - since we have access to a
> hosted 10.8 Mac Mini it would allow all devs to easily do a release (leaving
> aside the Windows issue). If anyone has tried the 10.6 SDK on 10.8 and knows
> if it actually works, that would be helpful.

haven't tried it yet, but it should be doable -- if a pain and a laudable goal.

> Any concerns, objections?

nope -- good plan.

Note that I am trying to unify some of this binary building -- the
same techniques used for matplotlib and useful for other packages, too
(numpy/scipy for sure), but also PIL, netCDF, who knows what?

I'll post here when I get a little further and there is something for
others to look at and/or contribute to.

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From Tom.KACVINSKY at 3ds.com  Tue Aug 20 18:35:15 2013
From: Tom.KACVINSKY at 3ds.com (KACVINSKY Tom)
Date: Tue, 20 Aug 2013 22:35:15 +0000
Subject: [Numpy-discussion] OS X binaries for releases
In-Reply-To: <CAGY4rcUR=cty1asAp1fwhAY6hStbiEU_R+DZs86nJ=VKO+9zXQ@mail.gmail.com>
References: <CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ@mail.gmail.com>
	<CAGY4rcXt+=09qP4TJdCzpXkyWStBKfeXODzuMo9MQe9XYy5jwA@mail.gmail.com>
	<35E5002C-FDEA-45FD-A43A-762FCCDEA8F7@3ds.com>,
	<CAGY4rcUR=cty1asAp1fwhAY6hStbiEU_R+DZs86nJ=VKO+9zXQ@mail.gmail.com>
Message-ID: <48CF3C84-BBEC-4DAF-99C0-F28A0CC42016@3ds.com>

llvm-gcc.  You have to specify the right options which I can look up tomorrow when I'm back in the office.  We don't invoke gcc directly, we use xcrun.

On Aug 20, 2013, at 18:31, "David Cournapeau" <cournape at gmail.com<mailto:cournape at gmail.com>> wrote:




On Tue, Aug 20, 2013 at 11:20 PM, KACVINSKY Tom <Tom.KACVINSKY at 3ds.com<mailto:Tom.KACVINSKY at 3ds.com>> wrote:
You can use the 10.6 SDK on 10.8.  At least we do.

With which compiler ?

David

Tom

On Aug 20, 2013, at 18:17, "David Cournapeau" <cournape at gmail.com<mailto:cournape at gmail.com>> wrote:




On Tue, Aug 20, 2013 at 9:48 PM, Ralf Gommers <ralf.gommers at gmail.com<mailto:ralf.gommers at gmail.com>> wrote:
Hi all,

Building binaries for releases is currently quite complex and time-consuming. For OS X we need two different machines, because we still provide binaries for OS X 10.5 and PPC machines. I propose to not do this anymore. It doesn't mean we completely drop support for 10.5 and PPC, just that we don't produce binaries. PPC was phased out in 2006 and OS X 10.6 came out in 2009, so there can't be a lot of demand for it (and the download stats at http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/ confirm this).

Furthermore I propose to not provide 2.6 binaries anymore. Downloads of 2.6 OS X binaries were <5% of the 2.7 ones. We did the same with 2.4 for a long time - support it but no binaries.

So what we'd have left at the moment is only the 64-bit/32-bit universal binary for 10.6 and up. What we finally need to add is 3.x OS X binaries. We can make an attempt to build these on 10.8 - since we have access to a hosted 10.8 Mac Mini it would allow all devs to easily do a release (leaving aside the Windows issue). If anyone has tried the 10.6 SDK on 10.8 and knows if it actually works, that would be helpful.

I am not sure one can use 10.6 SDK on 10.8 ? I am actually looking into those issues for our mac support at Enthought

Are you here for euroscipy ?

David

Any concerns, objections?

Cheers,
Ralf

P.S. the same proposal applies of course also to scipy



_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at scipy.org<mailto:NumPy-Discussion at scipy.org>
http://mail.scipy.org/mailman/listinfo/numpy-discussion


_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at scipy.org<mailto:NumPy-Discussion at scipy.org>
http://mail.scipy.org/mailman/listinfo/numpy-discussion

This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.

For other languages, go to http://www.3ds.com/terms/email-disclaimer

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at scipy.org<mailto:NumPy-Discussion at scipy.org>
http://mail.scipy.org/mailman/listinfo/numpy-discussion


_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at scipy.org<mailto:NumPy-Discussion at scipy.org>
http://mail.scipy.org/mailman/listinfo/numpy-discussion

This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.

For other languages, go to http://www.3ds.com/terms/email-disclaimer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130820/ab2cf3e4/attachment.html>

From rodrigokoblitz at gmail.com  Tue Aug 20 20:17:50 2013
From: rodrigokoblitz at gmail.com (rodrigo koblitz)
Date: Tue, 20 Aug 2013 20:17:50 -0400
Subject: [Numpy-discussion] NumPy-Discussion Digest, Vol 83, Issue 33
In-Reply-To: <mailman.4531.1377037550.973.numpy-discussion@scipy.org>
References: <mailman.4531.1377037550.973.numpy-discussion@scipy.org>
Message-ID: <CAAZkdU9uo_wBjqtRMiSSMy=Xmvh11xSG1mJkOg_LEYxnTbSXhA@mail.gmail.com>

Hi,
How I can do this:
int(scipy.comb(20314,117))
...
OverflowError: cannot convert float infinity to integer

abs,
Koblitz


2013/8/20 <numpy-discussion-request at scipy.org>

> Send NumPy-Discussion mailing list submissions to
>         numpy-discussion at scipy.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         http://mail.scipy.org/mailman/listinfo/numpy-discussion
> or, via email, send a message with subject or body 'help' to
>         numpy-discussion-request at scipy.org
>
> You can reach the person managing the list at
>         numpy-discussion-owner at scipy.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of NumPy-Discussion digest..."
>
>
> Today's Topics:
>
>    1. OS X binaries for releases (Ralf Gommers)
>    2. Re: OS X binaries for releases (David Cournapeau)
>    3. Re: OS X binaries for releases (KACVINSKY Tom)
>    4. Re: OS X binaries for releases (David Cournapeau)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 20 Aug 2013 22:48:51 +0200
> From: Ralf Gommers <ralf.gommers at gmail.com>
> Subject: [Numpy-discussion] OS X binaries for releases
> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
> Message-ID:
>         <
> CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi all,
>
> Building binaries for releases is currently quite complex and
> time-consuming. For OS X we need two different machines, because we still
> provide binaries for OS X 10.5 and PPC machines. I propose to not do this
> anymore. It doesn't mean we completely drop support for 10.5 and PPC, just
> that we don't produce binaries. PPC was phased out in 2006 and OS X 10.6
> came out in 2009, so there can't be a lot of demand for it (and the
> download stats at
> http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/confirm this).
>
> Furthermore I propose to not provide 2.6 binaries anymore. Downloads of 2.6
> OS X binaries were <5% of the 2.7 ones. We did the same with 2.4 for a long
> time - support it but no binaries.
>
> So what we'd have left at the moment is only the 64-bit/32-bit universal
> binary for 10.6 and up. What we finally need to add is 3.x OS X binaries.
> We can make an attempt to build these on 10.8 - since we have access to a
> hosted 10.8 Mac Mini it would allow all devs to easily do a release
> (leaving aside the Windows issue). If anyone has tried the 10.6 SDK on 10.8
> and knows if it actually works, that would be helpful.
>
> Any concerns, objections?
>
> Cheers,
> Ralf
>
> P.S. the same proposal applies of course also to scipy
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mail.scipy.org/pipermail/numpy-discussion/attachments/20130820/4d74e9d0/attachment-0001.html
>
> ------------------------------
>
> Message: 2
> Date: Tue, 20 Aug 2013 23:17:19 +0100
> From: David Cournapeau <cournape at gmail.com>
> Subject: Re: [Numpy-discussion] OS X binaries for releases
> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
> Message-ID:
>         <CAGY4rcXt+=
> 09qP4TJdCzpXkyWStBKfeXODzuMo9MQe9XYy5jwA at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> On Tue, Aug 20, 2013 at 9:48 PM, Ralf Gommers <ralf.gommers at gmail.com
> >wrote:
>
> > Hi all,
> >
> > Building binaries for releases is currently quite complex and
> > time-consuming. For OS X we need two different machines, because we still
> > provide binaries for OS X 10.5 and PPC machines. I propose to not do this
> > anymore. It doesn't mean we completely drop support for 10.5 and PPC,
> just
> > that we don't produce binaries. PPC was phased out in 2006 and OS X 10.6
> > came out in 2009, so there can't be a lot of demand for it (and the
> > download stats at
> http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/confirm this).
> >
> > Furthermore I propose to not provide 2.6 binaries anymore. Downloads of
> > 2.6 OS X binaries were <5% of the 2.7 ones. We did the same with 2.4 for
> a
> > long time - support it but no binaries.
> >
> > So what we'd have left at the moment is only the 64-bit/32-bit universal
> > binary for 10.6 and up. What we finally need to add is 3.x OS X binaries.
> > We can make an attempt to build these on 10.8 - since we have access to a
> > hosted 10.8 Mac Mini it would allow all devs to easily do a release
> > (leaving aside the Windows issue). If anyone has tried the 10.6 SDK on
> 10.8
> > and knows if it actually works, that would be helpful.
> >
>
> I am not sure one can use 10.6 SDK on 10.8 ? I am actually looking into
> those issues for our mac support at Enthought
>
> Are you here for euroscipy ?
>
> David
>
> >
> > Any concerns, objections?
> >
> > Cheers,
> > Ralf
> >
> > P.S. the same proposal applies of course also to scipy
> >
> >
> >
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mail.scipy.org/pipermail/numpy-discussion/attachments/20130820/b3b69f9c/attachment-0001.html
>
> ------------------------------
>
> Message: 3
> Date: Tue, 20 Aug 2013 22:20:21 +0000
> From: KACVINSKY Tom <Tom.KACVINSKY at 3ds.com>
> Subject: Re: [Numpy-discussion] OS X binaries for releases
> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
> Message-ID: <35E5002C-FDEA-45FD-A43A-762FCCDEA8F7 at 3ds.com>
> Content-Type: text/plain; charset="us-ascii"
>
> You can use the 10.6 SDK on 10.8.  At least we do.
>
> Tom
>
> On Aug 20, 2013, at 18:17, "David Cournapeau" <cournape at gmail.com<mailto:
> cournape at gmail.com>> wrote:
>
>
>
>
> On Tue, Aug 20, 2013 at 9:48 PM, Ralf Gommers <ralf.gommers at gmail.com
> <mailto:ralf.gommers at gmail.com>> wrote:
> Hi all,
>
> Building binaries for releases is currently quite complex and
> time-consuming. For OS X we need two different machines, because we still
> provide binaries for OS X 10.5 and PPC machines. I propose to not do this
> anymore. It doesn't mean we completely drop support for 10.5 and PPC, just
> that we don't produce binaries. PPC was phased out in 2006 and OS X 10.6
> came out in 2009, so there can't be a lot of demand for it (and the
> download stats at http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/confirm this).
>
> Furthermore I propose to not provide 2.6 binaries anymore. Downloads of
> 2.6 OS X binaries were <5% of the 2.7 ones. We did the same with 2.4 for a
> long time - support it but no binaries.
>
> So what we'd have left at the moment is only the 64-bit/32-bit universal
> binary for 10.6 and up. What we finally need to add is 3.x OS X binaries.
> We can make an attempt to build these on 10.8 - since we have access to a
> hosted 10.8 Mac Mini it would allow all devs to easily do a release
> (leaving aside the Windows issue). If anyone has tried the 10.6 SDK on 10.8
> and knows if it actually works, that would be helpful.
>
> I am not sure one can use 10.6 SDK on 10.8 ? I am actually looking into
> those issues for our mac support at Enthought
>
> Are you here for euroscipy ?
>
> David
>
> Any concerns, objections?
>
> Cheers,
> Ralf
>
> P.S. the same proposal applies of course also to scipy
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org<mailto:NumPy-Discussion at scipy.org>
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org<mailto:NumPy-Discussion at scipy.org>
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
> This email and any attachments are intended solely for the use of the
> individual or entity to whom it is addressed and may be confidential and/or
> privileged.
>
> If you are not one of the named recipients or have received this email in
> error,
>
> (i) you should not read, disclose, or copy it,
>
> (ii) please notify sender of your receipt by reply email and delete this
> email and all attachments,
>
> (iii) Dassault Systemes does not accept or assume any liability or
> responsibility for any use of or reliance on this email.
>
> For other languages, go to http://www.3ds.com/terms/email-disclaimer
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mail.scipy.org/pipermail/numpy-discussion/attachments/20130820/baca79c9/attachment-0001.html
>
> ------------------------------
>
> Message: 4
> Date: Tue, 20 Aug 2013 23:31:11 +0100
> From: David Cournapeau <cournape at gmail.com>
> Subject: Re: [Numpy-discussion] OS X binaries for releases
> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
> Message-ID:
>         <CAGY4rcUR=cty1asAp1fwhAY6hStbiEU_R+DZs86nJ=
> VKO+9zXQ at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> On Tue, Aug 20, 2013 at 11:20 PM, KACVINSKY Tom <Tom.KACVINSKY at 3ds.com
> >wrote:
>
> >  You can use the 10.6 SDK on 10.8.  At least we do.
> >
>
> With which compiler ?
>
> David
>
> >
> >  Tom
> >
> > On Aug 20, 2013, at 18:17, "David Cournapeau" <cournape at gmail.com>
> wrote:
> >
> >
> >
> >
> > On Tue, Aug 20, 2013 at 9:48 PM, Ralf Gommers <ralf.gommers at gmail.com
> >wrote:
> >
> >>  Hi all,
> >>
> >>  Building binaries for releases is currently quite complex and
> >> time-consuming. For OS X we need two different machines, because we
> still
> >> provide binaries for OS X 10.5 and PPC machines. I propose to not do
> this
> >> anymore. It doesn't mean we completely drop support for 10.5 and PPC,
> just
> >> that we don't produce binaries. PPC was phased out in 2006 and OS X 10.6
> >> came out in 2009, so there can't be a lot of demand for it (and the
> >> download stats at
> >> http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/ confirm this).
> >>
> >>  Furthermore I propose to not provide 2.6 binaries anymore. Downloads of
> >> 2.6 OS X binaries were <5% of the 2.7 ones. We did the same with 2.4
> for a
> >> long time - support it but no binaries.
> >>
> >>  So what we'd have left at the moment is only the 64-bit/32-bit
> >> universal binary for 10.6 and up. What we finally need to add is 3.x OS
> X
> >> binaries. We can make an attempt to build these on 10.8 - since we have
> >> access to a hosted 10.8 Mac Mini it would allow all devs to easily do a
> >> release (leaving aside the Windows issue). If anyone has tried the 10.6
> SDK
> >> on 10.8 and knows if it actually works, that would be helpful.
> >>
> >
> >  I am not sure one can use 10.6 SDK on 10.8 ? I am actually looking into
> > those issues for our mac support at Enthought
> >
> >  Are you here for euroscipy ?
> >
> >  David
> >
> >>
> >>  Any concerns, objections?
> >>
> >>  Cheers,
> >> Ralf
> >>
> >>  P.S. the same proposal applies of course also to scipy
> >>
> >>
> >>
> >> _______________________________________________
> >> NumPy-Discussion mailing list
> >> NumPy-Discussion at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >>
> >>
> >    _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
> >  This email and any attachments are intended solely for the use of the
> > individual or entity to whom it is addressed and may be confidential
> and/or
> > privileged.
> >
> > If you are not one of the named recipients or have received this email in
> > error,
> >
> > (i) you should not read, disclose, or copy it,
> >
> > (ii) please notify sender of your receipt by reply email and delete this
> > email and all attachments,
> >
> > (iii) Dassault Systemes does not accept or assume any liability or
> > responsibility for any use of or reliance on this email.
> >
> >  For other languages, go to http://www.3ds.com/terms/email-disclaimer
> >
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mail.scipy.org/pipermail/numpy-discussion/attachments/20130820/e939bfa6/attachment.html
>
> ------------------------------
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
> End of NumPy-Discussion Digest, Vol 83, Issue 33
> ************************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130820/59852169/attachment.html>

From warren.weckesser at gmail.com  Tue Aug 20 21:24:42 2013
From: warren.weckesser at gmail.com (Warren Weckesser)
Date: Tue, 20 Aug 2013 21:24:42 -0400
Subject: [Numpy-discussion] NumPy-Discussion Digest, Vol 83, Issue 33
In-Reply-To: <CAAZkdU9uo_wBjqtRMiSSMy=Xmvh11xSG1mJkOg_LEYxnTbSXhA@mail.gmail.com>
References: <mailman.4531.1377037550.973.numpy-discussion@scipy.org>
	<CAAZkdU9uo_wBjqtRMiSSMy=Xmvh11xSG1mJkOg_LEYxnTbSXhA@mail.gmail.com>
Message-ID: <CAGzF1uc59-7CBv+NCn1Lnc69KB7Z8xFHky=MaQHbeyTYgnpoRg@mail.gmail.com>

On 8/20/13, rodrigo koblitz <rodrigokoblitz at gmail.com> wrote:
> Hi,
> How I can do this:
> int(scipy.comb(20314,117))
> ...
> OverflowError: cannot convert float infinity to integer
>


I assume you mean `scipy.misc.comb`.  If you give `comb` the argument
`exact=True`, it will give the exact result as a Python long integer:

>>> comb(20314, 117, exact=True)
185322125435964088726782059829379016108985668708943661610691107922797953622540795237396216566443123945647349065209794249915331405960154995668715672694861752881279482861934217563789733636501993781318815128638676831358831027763891670979664077780116887804168965068781398413964827499948504201570364142600031022445600L

By the way, scipy questions like this should be asked on the
scipy-user mailing list: mail.scipy.org/mailman/listinfo/scipy-user
(although I can't seem to connect to that site at the moment).

Warren


> abs,
> Koblitz
>
>
> 2013/8/20 <numpy-discussion-request at scipy.org>
>
>> Send NumPy-Discussion mailing list submissions to
>>         numpy-discussion at scipy.org
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>>         http://mail.scipy.org/mailman/listinfo/numpy-discussion
>> or, via email, send a message with subject or body 'help' to
>>         numpy-discussion-request at scipy.org
>>
>> You can reach the person managing the list at
>>         numpy-discussion-owner at scipy.org
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of NumPy-Discussion digest..."
>>
>>
>> Today's Topics:
>>
>>    1. OS X binaries for releases (Ralf Gommers)
>>    2. Re: OS X binaries for releases (David Cournapeau)
>>    3. Re: OS X binaries for releases (KACVINSKY Tom)
>>    4. Re: OS X binaries for releases (David Cournapeau)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Tue, 20 Aug 2013 22:48:51 +0200
>> From: Ralf Gommers <ralf.gommers at gmail.com>
>> Subject: [Numpy-discussion] OS X binaries for releases
>> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
>> Message-ID:
>>         <
>> CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ at mail.gmail.com>
>> Content-Type: text/plain; charset="iso-8859-1"
>>
>> Hi all,
>>
>> Building binaries for releases is currently quite complex and
>> time-consuming. For OS X we need two different machines, because we still
>> provide binaries for OS X 10.5 and PPC machines. I propose to not do this
>> anymore. It doesn't mean we completely drop support for 10.5 and PPC,
>> just
>> that we don't produce binaries. PPC was phased out in 2006 and OS X 10.6
>> came out in 2009, so there can't be a lot of demand for it (and the
>> download stats at
>> http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/confirm this).
>>
>> Furthermore I propose to not provide 2.6 binaries anymore. Downloads of
>> 2.6
>> OS X binaries were <5% of the 2.7 ones. We did the same with 2.4 for a
>> long
>> time - support it but no binaries.
>>
>> So what we'd have left at the moment is only the 64-bit/32-bit universal
>> binary for 10.6 and up. What we finally need to add is 3.x OS X binaries.
>> We can make an attempt to build these on 10.8 - since we have access to a
>> hosted 10.8 Mac Mini it would allow all devs to easily do a release
>> (leaving aside the Windows issue). If anyone has tried the 10.6 SDK on
>> 10.8
>> and knows if it actually works, that would be helpful.
>>
>> Any concerns, objections?
>>
>> Cheers,
>> Ralf
>>
>> P.S. the same proposal applies of course also to scipy
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL:
>> http://mail.scipy.org/pipermail/numpy-discussion/attachments/20130820/4d74e9d0/attachment-0001.html
>>
>> ------------------------------
>>
>> Message: 2
>> Date: Tue, 20 Aug 2013 23:17:19 +0100
>> From: David Cournapeau <cournape at gmail.com>
>> Subject: Re: [Numpy-discussion] OS X binaries for releases
>> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
>> Message-ID:
>>         <CAGY4rcXt+=
>> 09qP4TJdCzpXkyWStBKfeXODzuMo9MQe9XYy5jwA at mail.gmail.com>
>> Content-Type: text/plain; charset="utf-8"
>>
>> On Tue, Aug 20, 2013 at 9:48 PM, Ralf Gommers <ralf.gommers at gmail.com
>> >wrote:
>>
>> > Hi all,
>> >
>> > Building binaries for releases is currently quite complex and
>> > time-consuming. For OS X we need two different machines, because we
>> > still
>> > provide binaries for OS X 10.5 and PPC machines. I propose to not do
>> > this
>> > anymore. It doesn't mean we completely drop support for 10.5 and PPC,
>> just
>> > that we don't produce binaries. PPC was phased out in 2006 and OS X
>> > 10.6
>> > came out in 2009, so there can't be a lot of demand for it (and the
>> > download stats at
>> http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/confirm this).
>> >
>> > Furthermore I propose to not provide 2.6 binaries anymore. Downloads of
>> > 2.6 OS X binaries were <5% of the 2.7 ones. We did the same with 2.4
>> > for
>> a
>> > long time - support it but no binaries.
>> >
>> > So what we'd have left at the moment is only the 64-bit/32-bit
>> > universal
>> > binary for 10.6 and up. What we finally need to add is 3.x OS X
>> > binaries.
>> > We can make an attempt to build these on 10.8 - since we have access to
>> > a
>> > hosted 10.8 Mac Mini it would allow all devs to easily do a release
>> > (leaving aside the Windows issue). If anyone has tried the 10.6 SDK on
>> 10.8
>> > and knows if it actually works, that would be helpful.
>> >
>>
>> I am not sure one can use 10.6 SDK on 10.8 ? I am actually looking into
>> those issues for our mac support at Enthought
>>
>> Are you here for euroscipy ?
>>
>> David
>>
>> >
>> > Any concerns, objections?
>> >
>> > Cheers,
>> > Ralf
>> >
>> > P.S. the same proposal applies of course also to scipy
>> >
>> >
>> >
>> > _______________________________________________
>> > NumPy-Discussion mailing list
>> > NumPy-Discussion at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
>> >
>> >
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL:
>> http://mail.scipy.org/pipermail/numpy-discussion/attachments/20130820/b3b69f9c/attachment-0001.html
>>
>> ------------------------------
>>
>> Message: 3
>> Date: Tue, 20 Aug 2013 22:20:21 +0000
>> From: KACVINSKY Tom <Tom.KACVINSKY at 3ds.com>
>> Subject: Re: [Numpy-discussion] OS X binaries for releases
>> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
>> Message-ID: <35E5002C-FDEA-45FD-A43A-762FCCDEA8F7 at 3ds.com>
>> Content-Type: text/plain; charset="us-ascii"
>>
>> You can use the 10.6 SDK on 10.8.  At least we do.
>>
>> Tom
>>
>> On Aug 20, 2013, at 18:17, "David Cournapeau" <cournape at gmail.com<mailto:
>> cournape at gmail.com>> wrote:
>>
>>
>>
>>
>> On Tue, Aug 20, 2013 at 9:48 PM, Ralf Gommers <ralf.gommers at gmail.com
>> <mailto:ralf.gommers at gmail.com>> wrote:
>> Hi all,
>>
>> Building binaries for releases is currently quite complex and
>> time-consuming. For OS X we need two different machines, because we still
>> provide binaries for OS X 10.5 and PPC machines. I propose to not do this
>> anymore. It doesn't mean we completely drop support for 10.5 and PPC,
>> just
>> that we don't produce binaries. PPC was phased out in 2006 and OS X 10.6
>> came out in 2009, so there can't be a lot of demand for it (and the
>> download stats at
>> http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/confirm this).
>>
>> Furthermore I propose to not provide 2.6 binaries anymore. Downloads of
>> 2.6 OS X binaries were <5% of the 2.7 ones. We did the same with 2.4 for
>> a
>> long time - support it but no binaries.
>>
>> So what we'd have left at the moment is only the 64-bit/32-bit universal
>> binary for 10.6 and up. What we finally need to add is 3.x OS X binaries.
>> We can make an attempt to build these on 10.8 - since we have access to a
>> hosted 10.8 Mac Mini it would allow all devs to easily do a release
>> (leaving aside the Windows issue). If anyone has tried the 10.6 SDK on
>> 10.8
>> and knows if it actually works, that would be helpful.
>>
>> I am not sure one can use 10.6 SDK on 10.8 ? I am actually looking into
>> those issues for our mac support at Enthought
>>
>> Are you here for euroscipy ?
>>
>> David
>>
>> Any concerns, objections?
>>
>> Cheers,
>> Ralf
>>
>> P.S. the same proposal applies of course also to scipy
>>
>>
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org<mailto:NumPy-Discussion at scipy.org>
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org<mailto:NumPy-Discussion at scipy.org>
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>> This email and any attachments are intended solely for the use of the
>> individual or entity to whom it is addressed and may be confidential
>> and/or
>> privileged.
>>
>> If you are not one of the named recipients or have received this email in
>> error,
>>
>> (i) you should not read, disclose, or copy it,
>>
>> (ii) please notify sender of your receipt by reply email and delete this
>> email and all attachments,
>>
>> (iii) Dassault Systemes does not accept or assume any liability or
>> responsibility for any use of or reliance on this email.
>>
>> For other languages, go to http://www.3ds.com/terms/email-disclaimer
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL:
>> http://mail.scipy.org/pipermail/numpy-discussion/attachments/20130820/baca79c9/attachment-0001.html
>>
>> ------------------------------
>>
>> Message: 4
>> Date: Tue, 20 Aug 2013 23:31:11 +0100
>> From: David Cournapeau <cournape at gmail.com>
>> Subject: Re: [Numpy-discussion] OS X binaries for releases
>> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
>> Message-ID:
>>         <CAGY4rcUR=cty1asAp1fwhAY6hStbiEU_R+DZs86nJ=
>> VKO+9zXQ at mail.gmail.com>
>> Content-Type: text/plain; charset="utf-8"
>>
>> On Tue, Aug 20, 2013 at 11:20 PM, KACVINSKY Tom <Tom.KACVINSKY at 3ds.com
>> >wrote:
>>
>> >  You can use the 10.6 SDK on 10.8.  At least we do.
>> >
>>
>> With which compiler ?
>>
>> David
>>
>> >
>> >  Tom
>> >
>> > On Aug 20, 2013, at 18:17, "David Cournapeau" <cournape at gmail.com>
>> wrote:
>> >
>> >
>> >
>> >
>> > On Tue, Aug 20, 2013 at 9:48 PM, Ralf Gommers <ralf.gommers at gmail.com
>> >wrote:
>> >
>> >>  Hi all,
>> >>
>> >>  Building binaries for releases is currently quite complex and
>> >> time-consuming. For OS X we need two different machines, because we
>> still
>> >> provide binaries for OS X 10.5 and PPC machines. I propose to not do
>> this
>> >> anymore. It doesn't mean we completely drop support for 10.5 and PPC,
>> just
>> >> that we don't produce binaries. PPC was phased out in 2006 and OS X
>> >> 10.6
>> >> came out in 2009, so there can't be a lot of demand for it (and the
>> >> download stats at
>> >> http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/ confirm
>> >> this).
>> >>
>> >>  Furthermore I propose to not provide 2.6 binaries anymore. Downloads
>> >> of
>> >> 2.6 OS X binaries were <5% of the 2.7 ones. We did the same with 2.4
>> for a
>> >> long time - support it but no binaries.
>> >>
>> >>  So what we'd have left at the moment is only the 64-bit/32-bit
>> >> universal binary for 10.6 and up. What we finally need to add is 3.x
>> >> OS
>> X
>> >> binaries. We can make an attempt to build these on 10.8 - since we
>> >> have
>> >> access to a hosted 10.8 Mac Mini it would allow all devs to easily do
>> >> a
>> >> release (leaving aside the Windows issue). If anyone has tried the
>> >> 10.6
>> SDK
>> >> on 10.8 and knows if it actually works, that would be helpful.
>> >>
>> >
>> >  I am not sure one can use 10.6 SDK on 10.8 ? I am actually looking
>> > into
>> > those issues for our mac support at Enthought
>> >
>> >  Are you here for euroscipy ?
>> >
>> >  David
>> >
>> >>
>> >>  Any concerns, objections?
>> >>
>> >>  Cheers,
>> >> Ralf
>> >>
>> >>  P.S. the same proposal applies of course also to scipy
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> NumPy-Discussion mailing list
>> >> NumPy-Discussion at scipy.org
>> >> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>> >>
>> >>
>> >    _______________________________________________
>> > NumPy-Discussion mailing list
>> > NumPy-Discussion at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
>> >
>> >  This email and any attachments are intended solely for the use of the
>> > individual or entity to whom it is addressed and may be confidential
>> and/or
>> > privileged.
>> >
>> > If you are not one of the named recipients or have received this email
>> > in
>> > error,
>> >
>> > (i) you should not read, disclose, or copy it,
>> >
>> > (ii) please notify sender of your receipt by reply email and delete
>> > this
>> > email and all attachments,
>> >
>> > (iii) Dassault Systemes does not accept or assume any liability or
>> > responsibility for any use of or reliance on this email.
>> >
>> >  For other languages, go to http://www.3ds.com/terms/email-disclaimer
>> >
>> > _______________________________________________
>> > NumPy-Discussion mailing list
>> > NumPy-Discussion at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
>> >
>> >
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL:
>> http://mail.scipy.org/pipermail/numpy-discussion/attachments/20130820/e939bfa6/attachment.html
>>
>> ------------------------------
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>> End of NumPy-Discussion Digest, Vol 83, Issue 33
>> ************************************************
>>
>


From joferkington at gmail.com  Tue Aug 20 21:45:13 2013
From: joferkington at gmail.com (Joe Kington)
Date: Tue, 20 Aug 2013 20:45:13 -0500
Subject: [Numpy-discussion] Adding an axis argument to numpy.unique
In-Reply-To: <CABDkGQ=tb0w_UVi1Biitcee2fCAF=PoM0BNmoTmUL_pb+Jub-g@mail.gmail.com>
References: <CACe1pJf3ta4UD_Dd20_NUi2HAfPFGPZBQRGDXBO7EeF_yLB7tw@mail.gmail.com>
	<CAMMTP+B7KOvWtZY+gNgKygB07og9JC_PhkmWgv9oxCznew5NvA@mail.gmail.com>
	<CACe1pJfsLrvLG+cV_VLbpvLm1hCPxSoVOa_=RAdZpLc=1C1QTg@mail.gmail.com>
	<CABDkGQ=tb0w_UVi1Biitcee2fCAF=PoM0BNmoTmUL_pb+Jub-g@mail.gmail.com>
Message-ID: <CACe1pJeWk2kyScj70DtDEXrAm+AzA8e9EZtuDSbu5_JBsY9VXA@mail.gmail.com>

>
> To me, `unique_rows` sounds perfect.  To go along columns: unique_rows(A.T)
>

St?fan


Personally, I like this idea as well.  A separate `unique_rows` function,
which potentially takes an `axis` argument.  (Alternately,
`unique_sequences` wouldn't imply a particular axis.)

Of course, the obvious downside to this is namespace pollution.  The upside
would be discoverability.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130820/71c2ab53/attachment.html>

From kyle.mandli at gmail.com  Tue Aug 20 22:10:18 2013
From: kyle.mandli at gmail.com (Kyle Mandli)
Date: Tue, 20 Aug 2013 21:10:18 -0500
Subject: [Numpy-discussion] OS X binaries for releases
In-Reply-To: <48CF3C84-BBEC-4DAF-99C0-F28A0CC42016@3ds.com>
References: <CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ@mail.gmail.com>
	<CAGY4rcXt+=09qP4TJdCzpXkyWStBKfeXODzuMo9MQe9XYy5jwA@mail.gmail.com>
	<35E5002C-FDEA-45FD-A43A-762FCCDEA8F7@3ds.com>
	<CAGY4rcUR=cty1asAp1fwhAY6hStbiEU_R+DZs86nJ=VKO+9zXQ@mail.gmail.com>
	<48CF3C84-BBEC-4DAF-99C0-F28A0CC42016@3ds.com>
Message-ID: <CAO1YWcYMpQDP-BbtmH6m_ZWX0LHrTT5=DO1=B3HS4qPiv9Gahg@mail.gmail.com>

On Tue, Aug 20, 2013 at 5:35 PM, KACVINSKY Tom <Tom.KACVINSKY at 3ds.com>wrote:

>  llvm-gcc.  You have to specify the right options which I can look up
> tomorrow when I'm back in the office.  We don't invoke gcc directly, we use
> xcrun.
>
> On Aug 20, 2013, at 18:31, "David Cournapeau" <cournape at gmail.com> wrote:
>
>
>
>
> On Tue, Aug 20, 2013 at 11:20 PM, KACVINSKY Tom <Tom.KACVINSKY at 3ds.com>wrote:
>
>>  You can use the 10.6 SDK on 10.8.  At least we do.
>>
>
>  With which compiler ?
>
>  David
>
>>
>>  Tom
>>
>> On Aug 20, 2013, at 18:17, "David Cournapeau" <cournape at gmail.com> wrote:
>>
>>
>>
>>
>> On Tue, Aug 20, 2013 at 9:48 PM, Ralf Gommers <ralf.gommers at gmail.com>wrote:
>>
>>>  Hi all,
>>>
>>>  Building binaries for releases is currently quite complex and
>>> time-consuming. For OS X we need two different machines, because we still
>>> provide binaries for OS X 10.5 and PPC machines. I propose to not do this
>>> anymore. It doesn't mean we completely drop support for 10.5 and PPC, just
>>> that we don't produce binaries. PPC was phased out in 2006 and OS X 10.6
>>> came out in 2009, so there can't be a lot of demand for it (and the
>>> download stats at
>>> http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/ confirm this).
>>>
>>>  Furthermore I propose to not provide 2.6 binaries anymore. Downloads
>>> of 2.6 OS X binaries were <5% of the 2.7 ones. We did the same with 2.4 for
>>> a long time - support it but no binaries.
>>>
>>>  So what we'd have left at the moment is only the 64-bit/32-bit
>>> universal binary for 10.6 and up. What we finally need to add is 3.x OS X
>>> binaries. We can make an attempt to build these on 10.8 - since we have
>>> access to a hosted 10.8 Mac Mini it would allow all devs to easily do a
>>> release (leaving aside the Windows issue). If anyone has tried the 10.6 SDK
>>> on 10.8 and knows if it actually works, that would be helpful.
>>>
>>
>>  I am not sure one can use 10.6 SDK on 10.8 ? I am actually looking into
>> those issues for our mac support at Enthought
>>
>>  Are you here for euroscipy ?
>>
>>  David
>>
>>>
>>>  Any concerns, objections?
>>>
>>>  Cheers,
>>> Ralf
>>>
>>>  P.S. the same proposal applies of course also to scipy
>>>
>>>
>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>>
>>    _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>   This email and any attachments are intended solely for the use of the
>> individual or entity to whom it is addressed and may be confidential and/or
>> privileged.
>>
>> If you are not one of the named recipients or have received this email in
>> error,
>>
>> (i) you should not read, disclose, or copy it,
>>
>> (ii) please notify sender of your receipt by reply email and delete this
>> email and all attachments,
>>
>> (iii) Dassault Systemes does not accept or assume any liability or
>> responsibility for any use of or reliance on this email.
>>
>>  For other languages, go to http://www.3ds.com/terms/email-disclaimer
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>    _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>  This email and any attachments are intended solely for the use of the
> individual or entity to whom it is addressed and may be confidential and/or
> privileged.
>
> If you are not one of the named recipients or have received this email in
> error,
>
> (i) you should not read, disclose, or copy it,
>
> (ii) please notify sender of your receipt by reply email and delete this
> email and all attachments,
>
> (iii) Dassault Systemes does not accept or assume any liability or
> responsibility for any use of or reliance on this email.
>
>  For other languages, go to http://www.3ds.com/terms/email-disclaimer
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
This would be an appropriate time I suppose to say I am attempting to build
numpy, scipy and matplotlib on 10.9.  NDA of course prohibits me
(unfortunately) from really discussing things but safe to say I would
support "moving on".  There are a multitude of issues that would be
resolved by dropping support for PPC (build complexity) and supporting SDKs
versions > 10.6.

Kyle
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130820/2f5054ee/attachment.html>

From rodrigokoblitz at gmail.com  Tue Aug 20 23:20:20 2013
From: rodrigokoblitz at gmail.com (rodrigo koblitz)
Date: Tue, 20 Aug 2013 23:20:20 -0400
Subject: [Numpy-discussion] NumPy-Discussion Digest, Vol 83, Issue 35
In-Reply-To: <mailman.4575.1377050699.973.numpy-discussion@scipy.org>
References: <mailman.4575.1377050699.973.numpy-discussion@scipy.org>
Message-ID: <CAAZkdU9NA6m+KWvGZWR-02VVS_oQDLf_USYR9JwBn+7TxrqsdQ@mail.gmail.com>

Thanks Warren,
ok,
I will pay more atention in the next time and address correctly.
abs,
Koblitz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130820/14ee09fb/attachment.html>

From ralf.gommers at gmail.com  Wed Aug 21 02:35:17 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Wed, 21 Aug 2013 08:35:17 +0200
Subject: [Numpy-discussion] OS X binaries for releases
In-Reply-To: <CAGY4rcXt+=09qP4TJdCzpXkyWStBKfeXODzuMo9MQe9XYy5jwA@mail.gmail.com>
References: <CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ@mail.gmail.com>
	<CAGY4rcXt+=09qP4TJdCzpXkyWStBKfeXODzuMo9MQe9XYy5jwA@mail.gmail.com>
Message-ID: <CABL7CQj_F7tEwq=i3qoJ8qa7izSV9zaUhC5LyUJQtNq6hfEkhA@mail.gmail.com>

On Wed, Aug 21, 2013 at 12:17 AM, David Cournapeau <cournape at gmail.com>wrote:

>
>
>
> On Tue, Aug 20, 2013 at 9:48 PM, Ralf Gommers <ralf.gommers at gmail.com>wrote:
>
>> Hi all,
>>
>> Building binaries for releases is currently quite complex and
>> time-consuming. For OS X we need two different machines, because we still
>> provide binaries for OS X 10.5 and PPC machines. I propose to not do this
>> anymore. It doesn't mean we completely drop support for 10.5 and PPC, just
>> that we don't produce binaries. PPC was phased out in 2006 and OS X 10.6
>> came out in 2009, so there can't be a lot of demand for it (and the
>> download stats at
>> http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/ confirm this).
>>
>> Furthermore I propose to not provide 2.6 binaries anymore. Downloads of
>> 2.6 OS X binaries were <5% of the 2.7 ones. We did the same with 2.4 for a
>> long time - support it but no binaries.
>>
>> So what we'd have left at the moment is only the 64-bit/32-bit universal
>> binary for 10.6 and up. What we finally need to add is 3.x OS X binaries.
>> We can make an attempt to build these on 10.8 - since we have access to a
>> hosted 10.8 Mac Mini it would allow all devs to easily do a release
>> (leaving aside the Windows issue). If anyone has tried the 10.6 SDK on 10.8
>> and knows if it actually works, that would be helpful.
>>
>
> I am not sure one can use 10.6 SDK on 10.8 ? I am actually looking into
> those issues for our mac support at Enthought
>
> Are you here for euroscipy ?
>

Yes, I'll be there from tomorrow till Sunday.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130821/b221edfa/attachment.html>

From ralf.gommers at gmail.com  Wed Aug 21 02:41:57 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Wed, 21 Aug 2013 08:41:57 +0200
Subject: [Numpy-discussion] OS X binaries for releases
In-Reply-To: <CAO1YWcYMpQDP-BbtmH6m_ZWX0LHrTT5=DO1=B3HS4qPiv9Gahg@mail.gmail.com>
References: <CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ@mail.gmail.com>
	<CAGY4rcXt+=09qP4TJdCzpXkyWStBKfeXODzuMo9MQe9XYy5jwA@mail.gmail.com>
	<35E5002C-FDEA-45FD-A43A-762FCCDEA8F7@3ds.com>
	<CAGY4rcUR=cty1asAp1fwhAY6hStbiEU_R+DZs86nJ=VKO+9zXQ@mail.gmail.com>
	<48CF3C84-BBEC-4DAF-99C0-F28A0CC42016@3ds.com>
	<CAO1YWcYMpQDP-BbtmH6m_ZWX0LHrTT5=DO1=B3HS4qPiv9Gahg@mail.gmail.com>
Message-ID: <CABL7CQhHmV3NFH-mTjL7-zxKsPirm-x4m2gHDxceL3NSW4OAbw@mail.gmail.com>

On Wed, Aug 21, 2013 at 4:10 AM, Kyle Mandli <kyle.mandli at gmail.com> wrote:

>
> This would be an appropriate time I suppose to say I am attempting to
> build numpy, scipy and matplotlib on 10.9.  NDA of course prohibits me
> (unfortunately) from really discussing things but safe to say I would
> support "moving on".  There are a multitude of issues that would be
> resolved by dropping support for PPC (build complexity) and supporting SDKs
> versions > 10.6.
>

I don't think dropping anything will help you with building on 10.9. Unless
you're trying to replicate the numpy binaries? There's now two builds:
1. 10.5+, universal binary, 32bit Intel + PPC
2. 10.6+, universal binary, 32bit Intel + 64bit Intel

You need the latter, so dropping the former doesn't really make a
difference. You also don't need to worry about getting your binary to run
on 10.6-10.8 I guess, so you're left with:
3. 10.9 universal binary, 32bit Intel + 64bit Intel

We're not going to drop the 32-bit part of that build, so nothing changes
for building for yourself on recent OS X's.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130821/81864783/attachment.html>

From Tom.KACVINSKY at 3ds.com  Wed Aug 21 09:31:10 2013
From: Tom.KACVINSKY at 3ds.com (KACVINSKY Tom)
Date: Wed, 21 Aug 2013 13:31:10 +0000
Subject: [Numpy-discussion] OS X binaries for releases
In-Reply-To: <CAO1YWcYMpQDP-BbtmH6m_ZWX0LHrTT5=DO1=B3HS4qPiv9Gahg@mail.gmail.com>
References: <CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ@mail.gmail.com>
	<CAGY4rcXt+=09qP4TJdCzpXkyWStBKfeXODzuMo9MQe9XYy5jwA@mail.gmail.com>
	<35E5002C-FDEA-45FD-A43A-762FCCDEA8F7@3ds.com>
	<CAGY4rcUR=cty1asAp1fwhAY6hStbiEU_R+DZs86nJ=VKO+9zXQ@mail.gmail.com>
	<48CF3C84-BBEC-4DAF-99C0-F28A0CC42016@3ds.com>
	<CAO1YWcYMpQDP-BbtmH6m_ZWX0LHrTT5=DO1=B3HS4qPiv9Gahg@mail.gmail.com>
Message-ID: <B8F5CDE7FC952A4580ABD28F488EF27930E84BE5@AG-DCC-MBX11.dsone.3ds.com>



From: numpy-discussion-bounces at scipy.org [mailto:numpy-discussion-bounces at scipy.org] On Behalf Of Kyle Mandli
Sent: Tuesday, August 20, 2013 10:10 PM
To: Discussion of Numerical Python
Subject: Re: [Numpy-discussion] OS X binaries for releases

On Tue, Aug 20, 2013 at 5:35 PM, KACVINSKY Tom <Tom.KACVINSKY at 3ds.com<mailto:Tom.KACVINSKY at 3ds.com>> wrote:
llvm-gcc.  You have to specify the right options which I can look up tomorrow when I'm back in the office.  We don't invoke gcc directly, we use xcrun.

On Aug 20, 2013, at 18:31, "David Cournapeau" <cournape at gmail.com<mailto:cournape at gmail.com>> wrote:


On Tue, Aug 20, 2013 at 11:20 PM, KACVINSKY Tom <Tom.KACVINSKY at 3ds.com<mailto:Tom.KACVINSKY at 3ds.com>> wrote:
You can use the 10.6 SDK on 10.8.  At least we do.

With which compiler ?

David

Tom

On Aug 20, 2013, at 18:17, "David Cournapeau" <cournape at gmail.com<mailto:cournape at gmail.com>> wrote:


On Tue, Aug 20, 2013 at 9:48 PM, Ralf Gommers <ralf.gommers at gmail.com<mailto:ralf.gommers at gmail.com>> wrote:
Hi all,
Building binaries for releases is currently quite complex and time-consuming. For OS X we need two different machines, because we still provide binaries for OS X 10.5 and PPC machines. I propose to not do this anymore. It doesn't mean we completely drop support for 10.5 and PPC, just that we don't produce binaries. PPC was phased out in 2006 and OS X 10.6 came out in 2009, so there can't be a lot of demand for it (and the download stats at http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/ confirm this).
Furthermore I propose to not provide 2.6 binaries anymore. Downloads of 2.6 OS X binaries were <5% of the 2.7 ones. We did the same with 2.4 for a long time - support it but no binaries.
So what we'd have left at the moment is only the 64-bit/32-bit universal binary for 10.6 and up. What we finally need to add is 3.x OS X binaries. We can make an attempt to build these on 10.8 - since we have access to a hosted 10.8 Mac Mini it would allow all devs to easily do a release (leaving aside the Windows issue). If anyone has tried the 10.6 SDK on 10.8 and knows if it actually works, that would be helpful.

I am not sure one can use 10.6 SDK on 10.8 ? I am actually looking into those issues for our mac support at Enthought

Are you here for euroscipy ?

David

Any concerns, objections?
Cheers,
Ralf
P.S. the same proposal applies of course also to scipy


_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at scipy.org<mailto:NumPy-Discussion at scipy.org>
http://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at scipy.org<mailto:NumPy-Discussion at scipy.org>
http://mail.scipy.org/mailman/listinfo/numpy-discussion

This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.

For other languages, go to http://www.3ds.com/terms/email-disclaimer

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at scipy.org<mailto:NumPy-Discussion at scipy.org>
http://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at scipy.org<mailto:NumPy-Discussion at scipy.org>
http://mail.scipy.org/mailman/listinfo/numpy-discussion

This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.

For other languages, go to http://www.3ds.com/terms/email-disclaimer

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at scipy.org<mailto:NumPy-Discussion at scipy.org>
http://mail.scipy.org/mailman/listinfo/numpy-discussion

This would be an appropriate time I suppose to say I am attempting to build numpy, scipy and matplotlib on 10.9.  NDA of course prohibits me (unfortunately) from really discussing things but safe to say I would support "moving on".  There are a multitude of issues that would be resolved by dropping support for PPC (build complexity) and supporting SDKs versions > 10.6.

Kyle

My bad.  I am using Xcode 4.5.2, and the only SDKs installed are the 10.7 and 10.8 SDKs.  So I don't know if it is possible to build with the 10.6 SDK (unless there is an option ot install the 10.6 SDK from within Xcode).

Anyway,  this is what we use to get the 10.7 SDK:

For compiling:

/usr/bin/xcrun --sdk macosx10.7 --run llvm-gcc -c -m64 -fPIC -- this gives 64 bit builds, I am guessing adding -m32 will give you 32 bit builds.

For Linking:

/usr/bin/xcrun --sdk macosx10.7 --run llvm-gcc -flat_namespace -dynamiclib -headerpad_max_install_names -install_name <library name here> -m64 -mmacosx-version-min=10.6

I think the -mmacos-version option is crucial, as there were some change in the Mach-O format starting with 10.5/6.

Tom





This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.

For other languages, go to http://www.3ds.com/terms/email-disclaimer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130821/88fa8ffe/attachment.html>

From pierre.raybaut at gmail.com  Wed Aug 21 14:56:43 2013
From: pierre.raybaut at gmail.com (Pierre Raybaut)
Date: Wed, 21 Aug 2013 20:56:43 +0200
Subject: [Numpy-discussion] ANN: Spyder v2.2.3 released
Message-ID: <CAKegKuBbKzdjv8vGiimWOPp8Mu172qAZEvf00PRHkodFQg2=2Q@mail.gmail.com>

Hi all,



On the behalf of Spyder's development team (
http://code.google.com/p/spyderlib/people/list), I'm pleased to announce
that Spyder v2.2.3 has been released and is available for Windows
XP/Vista/7/8, GNU/Linux and MacOS X: http://code.google.com/p/spyderlib/.

This is a maintenance release of the v2.2 branch which is the last release
to support Python 2.5:

* Spyder 2.2 supports Python 2.5 to 2.7

* Spyder 2.3 will support Python 2.7 to 3.3

* (Spyder 2.3.0dev6 is also available as of today: this is an experimental
release but quite stable which already supports Python 3)

See also https://code.google.com/p/spyderlib/downloads/list.

Since v2.2.2:
* Several bug have been fixed (see changelog for further details:
https://code.google.com/p/spyderlib/wiki/ChangeLog)
* New features have been added like full support for a MATLAB-like cell
mode (see "Run" menu) and the Optional Dependencies dialog box (see menu
"?") which gives the user a status of Spyder's dependencies


Spyder is a free, open-source (MIT license) interactive development
environment for the Python language with advanced editing, interactive
testing, debugging and introspection features. Originally designed to
provide MATLAB-like features (integrated help, interactive console,
variable explorer with GUI-based editors for dictionaries, NumPy arrays,
...), it is strongly oriented towards scientific computing and software
development. Thanks to the `spyderlib` library, Spyder also provides
powerful ready-to-use widgets: embedded Python console (example:
http://packages.python.org/guiqwt/_images/sift3.png), NumPy array editor
(example: http://packages.python.org/guiqwt/_images/sift2.png), dictionary
editor, source code editor, etc.


Description of key features with tasty screenshots can be found at:

http://code.google.com/p/spyderlib/wiki/Features



Don't forget to follow Spyder updates/news:

* on the project website: http://code.google.com/p/spyderlib/

* and on our official blog: http://spyder-ide.blogspot.com/



Last, but not least, we welcome any contribution that helps making Spyder
an efficient scientific development/computing environment. Join us to help
creating your favourite environment!

(http://code.google.com/p/spyderlib/wiki/NoteForContributors)



Enjoy!

-Pierre
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130821/080c2441/attachment.html>

From ralf.gommers at gmail.com  Thu Aug 22 08:26:08 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Thu, 22 Aug 2013 14:26:08 +0200
Subject: [Numpy-discussion] Numpy/Scipy sprint this Sunday
Message-ID: <CABL7CQgJvgoW86zttVGjh74K0TJC=o1Ak6BArZHUkpQqnBFaLw@mail.gmail.com>

Hi all,

Here's a reminder that we have a sprint this Sunday (Aug 25) at EuroScipy.
There are already 12 people who indicated they'd join; even more would be
even better.

Location: https://www.euroscipy.org/venue/ (room K3.601)
Time: 9am - 7pm
More details:
https://github.com/rgommers/scipy/wiki/EuroSciPy%2713-numpy-scipy-sprint
No previous experience contributing to numpy and scipy required (although
you should know how to use python/numpy/scipy/git to a reasonable extent).
Please try to get your development environment set up beforehand - see
developer guides at http://docs.scipy.org/doc/ for explanation of how to do
that.

If you can't join in Brussels, we'll also be on IRC and mailing lists. Hope
to see a large turnout!

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130822/9acc0851/attachment.html>

From ralf.gommers at gmail.com  Thu Aug 22 09:12:21 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Thu, 22 Aug 2013 15:12:21 +0200
Subject: [Numpy-discussion] ANN: Scipy 0.13.0 beta 1 release
Message-ID: <CABL7CQga-Qkq8_hG1E+_KfpmVjSE-duyRePVcZw_-s-PXay75w@mail.gmail.com>

Hi all,

I'm happy to announce the availability of the first beta release of Scipy
0.13.0. Please try this beta and report any issues on the scipy-dev mailing
list.

Source tarballs and release notes can be found at
https://sourceforge.net/projects/scipy/files/scipy/0.13.0b1/. Windows and
OS X installers will follow later (we have a minor infrastructure issue to
solve, and I'm at EuroScipy now).

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130822/60ea1dc9/attachment.html>

From rowen at uw.edu  Thu Aug 22 15:14:11 2013
From: rowen at uw.edu (Russell E. Owen)
Date: Thu, 22 Aug 2013 12:14:11 -0700
Subject: [Numpy-discussion] OS X binaries for releases
References: <CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ@mail.gmail.com>
Message-ID: <rowen-5A7B80.12141122082013@news.gmane.org>

In article 
<CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ at mail.gmail.com>,
 Ralf Gommers <ralf.gommers at gmail.com> wrote:

> Hi all,
> 
> Building binaries for releases is currently quite complex and
> time-consuming. For OS X we need two different machines, because we still
> provide binaries for OS X 10.5 and PPC machines. I propose to not do this
> anymore. It doesn't mean we completely drop support for 10.5 and PPC, just
> that we don't produce binaries. PPC was phased out in 2006 and OS X 10.6
> came out in 2009, so there can't be a lot of demand for it (and the
> download stats at
> http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/confirm this).
> 
> Furthermore I propose to not provide 2.6 binaries anymore. Downloads of 2.6
> OS X binaries were <5% of the 2.7 ones. We did the same with 2.4 for a long
> time - support it but no binaries.
> 
> So what we'd have left at the moment is only the 64-bit/32-bit universal
> binary for 10.6 and up. What we finally need to add is 3.x OS X binaries.
> We can make an attempt to build these on 10.8 - since we have access to a
> hosted 10.8 Mac Mini it would allow all devs to easily do a release
> (leaving aside the Windows issue). If anyone has tried the 10.6 SDK on 10.8
> and knows if it actually works, that would be helpful.
> 
> Any concerns, objections?

I am in strong agreement.

I'll be interested to learn how you make binary installers for python 
3.x because the standard version of bdist_mpkg will not do it. I have 
heard of two other projects (forks or variants of bdist_mpkg) that will, 
but I have no idea of either is supported.

I have been able to building packages on 10.8 using 
MACOSX_DEPLOYMENT_TARGET=10.6 that will run on 10.6, so it will probably 
work. However I have run into several odd problems over the years 
building a binary installer on a newer system only to find it won't work 
on older systems for various reasons. Thus my personal recommendation is 
that you build on 10.6 if you want an installer that reliably works for 
10.6 and later. I keep an older computer around for this reason. In fact 
that is one good reason to drop support for ancient operating systems 
and PPC.

-- Russell



From chris.barker at noaa.gov  Thu Aug 22 16:05:54 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Thu, 22 Aug 2013 13:05:54 -0700
Subject: [Numpy-discussion] OS X binaries for releases
In-Reply-To: <rowen-5A7B80.12141122082013@news.gmane.org>
References: <CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ@mail.gmail.com>
	<rowen-5A7B80.12141122082013@news.gmane.org>
Message-ID: <CALGmxE+NUEr8zbTSF_xKYxyCp+D4M9HuiUu2Wb0VOt5Z5N12UQ@mail.gmail.com>

On Thu, Aug 22, 2013 at 12:14 PM, Russell E. Owen <rowen at uw.edu> wrote:
> I'll be interested to learn how you make binary installers for python
> 3.x because the standard version of bdist_mpkg will not do it. I have
> heard of two other projects (forks or variants of bdist_mpkg) that will,
> but I have no idea of either is supported.

Ideally -- we go to binary wheels -- but don't know if the world is
ready for that.

> I have been able to building packages on 10.8 using
> MACOSX_DEPLOYMENT_TARGET=10.6 that will run on 10.6, so it will probably
> work. However I have run into several odd problems over the years
> building a binary installer on a newer system only to find it won't work
> on older systems for various reasons. Thus my personal recommendation is
> that you build on 10.6 if you want an installer that reliably works for
> 10.6 and later.

That is the easiest and most robust, for sure. And you can also keep
that system "clean" -- i.e. no libfreetypes around to acidentally link
to...

-Chris



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From matthew.brett at gmail.com  Thu Aug 22 16:19:07 2013
From: matthew.brett at gmail.com (Matthew Brett)
Date: Thu, 22 Aug 2013 13:19:07 -0700
Subject: [Numpy-discussion] OS X binaries for releases
In-Reply-To: <rowen-5A7B80.12141122082013@news.gmane.org>
References: <CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ@mail.gmail.com>
	<rowen-5A7B80.12141122082013@news.gmane.org>
Message-ID: <CAH6Pt5o32Otdhk2Ms5Cy5Zo=Mn48H8X2WbsWK92etUb4MmrooQ@mail.gmail.com>

Hi,

On Thu, Aug 22, 2013 at 12:14 PM, Russell E. Owen <rowen at uw.edu> wrote:
> In article
> <CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ at mail.gmail.com>,
>  Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>> Hi all,
>>
>> Building binaries for releases is currently quite complex and
>> time-consuming. For OS X we need two different machines, because we still
>> provide binaries for OS X 10.5 and PPC machines. I propose to not do this
>> anymore. It doesn't mean we completely drop support for 10.5 and PPC, just
>> that we don't produce binaries. PPC was phased out in 2006 and OS X 10.6
>> came out in 2009, so there can't be a lot of demand for it (and the
>> download stats at
>> http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/confirm this).
>>
>> Furthermore I propose to not provide 2.6 binaries anymore. Downloads of 2.6
>> OS X binaries were <5% of the 2.7 ones. We did the same with 2.4 for a long
>> time - support it but no binaries.
>>
>> So what we'd have left at the moment is only the 64-bit/32-bit universal
>> binary for 10.6 and up. What we finally need to add is 3.x OS X binaries.
>> We can make an attempt to build these on 10.8 - since we have access to a
>> hosted 10.8 Mac Mini it would allow all devs to easily do a release
>> (leaving aside the Windows issue). If anyone has tried the 10.6 SDK on 10.8
>> and knows if it actually works, that would be helpful.
>>
>> Any concerns, objections?
>
> I am in strong agreement.
>
> I'll be interested to learn how you make binary installers for python
> 3.x because the standard version of bdist_mpkg will not do it. I have
> heard of two other projects (forks or variants of bdist_mpkg) that will,
> but I have no idea of either is supported.

I think I'm the owner of one of the forks; I supporting it, but I
should certainly make a release soon too.

> I have been able to building packages on 10.8 using
> MACOSX_DEPLOYMENT_TARGET=10.6 that will run on 10.6, so it will probably
> work. However I have run into several odd problems over the years
> building a binary installer on a newer system only to find it won't work
> on older systems for various reasons. Thus my personal recommendation is
> that you build on 10.6 if you want an installer that reliably works for
> 10.6 and later. I keep an older computer around for this reason. In fact
> that is one good reason to drop support for ancient operating systems
> and PPC.

I'm sitting next to a 10.6 machine you are welcome to use; just let me
know, I'll give you login access.

Cheers,

Matthew


From warren.weckesser at gmail.com  Thu Aug 22 22:32:16 2013
From: warren.weckesser at gmail.com (Warren Weckesser)
Date: Thu, 22 Aug 2013 22:32:16 -0400
Subject: [Numpy-discussion] Warnings not raised by np.log in 32 bit build on
	Windows
Message-ID: <CAGzF1ucMD_GBV9gqL975MSDG3nUcCV+aS3qKoq_uD3hLoEJM6g@mail.gmail.com>

I'm investigating a test error in scipy 0.13.0 beta 1 that was
reported by Christoph Gohlke.  The scipy issue is here:
https://github.com/scipy/scipy/issues/2771

I don't have a Windows environment to test it myself, but Christoph
reported that this code:

```
import numpy as np

data = np.array([-0.375, -0.25, 0.0])
s = np.log(data)
```

does not generate two RuntimeWarnings when it is run with numpy 1.7.1
in a 32 bit Windows 8 environment (numpy 1.7.1 compiled with Visual
Studio compilers and Intel's MKL).  In 64 bit Windows, and in 64 bit
linux, it generates two RuntimeWarnings.

The inconsistency seems like a bug, possibly this one:
https://github.com/numpy/numpy/issues/1958.

Can anyone check if this also occurs in the development branch?

Warren


From chris.barker at noaa.gov  Fri Aug 23 01:02:00 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Thu, 22 Aug 2013 22:02:00 -0700
Subject: [Numpy-discussion] What is a numpy.long type?
Message-ID: <CALGmxELkBR3+J8o20MF6_Jj9Sep7hCFDONVTETa4PytVMsvkUA@mail.gmail.com>

Hi folks,

I had thought that maybe a numpy.long dtype was a system
(compiler)-native C long.

But on both 32 and 64 bit python on OS-X, it seems to be 64 bit. I'm
pretty sure that on OS-X 32 bit, a C long is 32 bits. (gdd, of course)

I don't have other machines to test on , but as the MS compilers and
gcc do different things with a C long on 64 bit platforms, I"m curious
how numpy defines it.

In general, I prefer the "explicit is better than implicit" approach
and use, e.g. int32 and int64. However, in this case, we are using
Cython, and calling C code that used "long" -- so what I want is
"whatever the compiler thinks is a long" -- is there  a way to do that
without my own kludgy platform-dependent code?

I note that the Cython numpy.pxd assigns:

ctypedef signed long      npy_long

Is that a bug? (or maybe npy_long is not supposed to match np.long....

-Chris




-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From cournape at gmail.com  Fri Aug 23 02:57:16 2013
From: cournape at gmail.com (David Cournapeau)
Date: Fri, 23 Aug 2013 07:57:16 +0100
Subject: [Numpy-discussion] What is a numpy.long type?
In-Reply-To: <CALGmxELkBR3+J8o20MF6_Jj9Sep7hCFDONVTETa4PytVMsvkUA@mail.gmail.com>
References: <CALGmxELkBR3+J8o20MF6_Jj9Sep7hCFDONVTETa4PytVMsvkUA@mail.gmail.com>
Message-ID: <CAGY4rcWmjrKQVEWeZPge8tzmG=5CXLoQAgKk0jNZLWSkoXh_bA@mail.gmail.com>

On Fri, Aug 23, 2013 at 6:02 AM, Chris Barker - NOAA Federal <
chris.barker at noaa.gov> wrote:

> Hi folks,
>
> I had thought that maybe a numpy.long dtype was a system
> (compiler)-native C long.
>
> But on both 32 and 64 bit python on OS-X, it seems to be 64 bit. I'm
> pretty sure that on OS-X 32 bit, a C long is 32 bits. (gdd, of course)
>
> I don't have other machines to test on , but as the MS compilers and
> gcc do different things with a C long on 64 bit platforms, I"m curious
> how numpy defines it.


> In general, I prefer the "explicit is better than implicit" approach
> and use, e.g. int32 and int64. However, in this case, we are using
> Cython, and calling C code that used "long" -- so what I want is
> "whatever the compiler thinks is a long" -- is there  a way to do that
> without my own kludgy platform-dependent code?
>
> I note that the Cython numpy.pxd assigns:
>
> ctypedef signed long      npy_long
>
> Is that a bug? (or maybe npy_long is not supposed to match np.long....
>

npy_long is indeed just an alias to C long, np.long is an alias to python's
long:

arch -32 python -c "import numpy as np; print np.dtype(np.int); print
np.dtype(np.long)"
int32
int64

arch -64 python -c "import numpy as np; print np.dtype(np.int); print
np.dtype(np.long)"
int64
int64

and python -c "import numpy as np; print np.long is long" will print True

All this is on python 2.7, I am not sure how/if that changes on python 3
(that consolidated python int/long).

David

>
> -Chris
>
>
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chris.Barker at noaa.gov
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130823/e699b72e/attachment.html>

From charlesr.harris at gmail.com  Fri Aug 23 08:11:44 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Fri, 23 Aug 2013 06:11:44 -0600
Subject: [Numpy-discussion] Warnings not raised by np.log in 32 bit
 build on Windows
In-Reply-To: <CAGzF1ucMD_GBV9gqL975MSDG3nUcCV+aS3qKoq_uD3hLoEJM6g@mail.gmail.com>
References: <CAGzF1ucMD_GBV9gqL975MSDG3nUcCV+aS3qKoq_uD3hLoEJM6g@mail.gmail.com>
Message-ID: <CAB6mnx+LKyK_pz8aBmDVsi4+iPHd4U21nnjD2AneT9D9LHbFvQ@mail.gmail.com>

These things may depend on how the compiler implements various calls. Some
errors went the other way with Julian's SIMD work, i.e., errors getting set
that were not set before. I'm not sure what can be done about it.


On Thu, Aug 22, 2013 at 8:32 PM, Warren Weckesser <
warren.weckesser at gmail.com> wrote:

> I'm investigating a test error in scipy 0.13.0 beta 1 that was
> reported by Christoph Gohlke.  The scipy issue is here:
> https://github.com/scipy/scipy/issues/2771
>
> I don't have a Windows environment to test it myself, but Christoph
> reported that this code:
>
> ```
> import numpy as np
>
> data = np.array([-0.375, -0.25, 0.0])
> s = np.log(data)
> ```
>
> does not generate two RuntimeWarnings when it is run with numpy 1.7.1
> in a 32 bit Windows 8 environment (numpy 1.7.1 compiled with Visual
> Studio compilers and Intel's MKL).  In 64 bit Windows, and in 64 bit
> linux, it generates two RuntimeWarnings.
>
> The inconsistency seems like a bug, possibly this one:
> https://github.com/numpy/numpy/issues/1958.
>
> Can anyone check if this also occurs in the development branch?
>
> Warren
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130823/ef9cba40/attachment.html>

From alan.isaac at gmail.com  Fri Aug 23 08:23:01 2013
From: alan.isaac at gmail.com (Alan G Isaac)
Date: Fri, 23 Aug 2013 08:23:01 -0400
Subject: [Numpy-discussion] Warnings not raised by np.log in 32 bit
 build on Windows
In-Reply-To: <CAGzF1ucMD_GBV9gqL975MSDG3nUcCV+aS3qKoq_uD3hLoEJM6g@mail.gmail.com>
References: <CAGzF1ucMD_GBV9gqL975MSDG3nUcCV+aS3qKoq_uD3hLoEJM6g@mail.gmail.com>
Message-ID: <52175425.6030009@gmail.com>

On 8/22/2013 10:32 PM, Warren Weckesser wrote:
> Christoph
> reported that this code:
>
> ```
> import numpy as np
>
> data = np.array([-0.375, -0.25, 0.0])
> s = np.log(data)
> ```
>
> does not generate two RuntimeWarnings when it is run with numpy 1.7.1
> in a 32 bit Windows 8 environment (numpy 1.7.1 compiled with Visual
> Studio compilers and Intel's MKL).


Not sure if you want other (not Win 8) reports related to this,
but ... I'm appending (no runtime errors) below.  The OS is
Win 7 (64bit).

Alan Isaac


Enthought Python Distribution -- www.enthought.com
Version: 7.3-2 (32-bit)

Python 2.7.3 |EPD 7.3-2 (32-bit)| (default, Apr 12 2012, 14:30:37) [MSC v.1500 3 2 bit (Intel)] on win32
Type "credits", "demo" or "enthought" for more information.
 >>> import numpy as np
 >>> np.__version__
'1.7.1'
 >>> data = np.array([-0.375, -0.25, 0.0])
 >>> s = np.log(data)
 >>>




From njs at pobox.com  Fri Aug 23 08:29:13 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 23 Aug 2013 13:29:13 +0100
Subject: [Numpy-discussion] Warnings not raised by np.log in 32 bit
 build on Windows
In-Reply-To: <CAB6mnx+LKyK_pz8aBmDVsi4+iPHd4U21nnjD2AneT9D9LHbFvQ@mail.gmail.com>
References: <CAGzF1ucMD_GBV9gqL975MSDG3nUcCV+aS3qKoq_uD3hLoEJM6g@mail.gmail.com>
	<CAB6mnx+LKyK_pz8aBmDVsi4+iPHd4U21nnjD2AneT9D9LHbFvQ@mail.gmail.com>
Message-ID: <CAPJVwBmoy8wnwsc212cu=OKNmyc8JTa9ZgmtpB8cAG1BSDG9ag@mail.gmail.com>

Probably the thing to do for reliable behaviour is to decide on the
behaviour we want and then implement it "by hand". I.e., either clear the
FP flags inside the ufunc loop (if we decide that log shouldn't raise a
warning), or else check for nan and set the invalid flag ourselves.
(Checking for nan should be much cheaper than computing log, I think, so
this should be okay speed-wise?)
On 23 Aug 2013 13:16, "Charles R Harris" <charlesr.harris at gmail.com> wrote:

> These things may depend on how the compiler implements various calls. Some
> errors went the other way with Julian's SIMD work, i.e., errors getting set
> that were not set before. I'm not sure what can be done about it.
>
>
> On Thu, Aug 22, 2013 at 8:32 PM, Warren Weckesser <
> warren.weckesser at gmail.com> wrote:
>
>> I'm investigating a test error in scipy 0.13.0 beta 1 that was
>> reported by Christoph Gohlke.  The scipy issue is here:
>> https://github.com/scipy/scipy/issues/2771
>>
>> I don't have a Windows environment to test it myself, but Christoph
>> reported that this code:
>>
>> ```
>> import numpy as np
>>
>> data = np.array([-0.375, -0.25, 0.0])
>> s = np.log(data)
>> ```
>>
>> does not generate two RuntimeWarnings when it is run with numpy 1.7.1
>> in a 32 bit Windows 8 environment (numpy 1.7.1 compiled with Visual
>> Studio compilers and Intel's MKL).  In 64 bit Windows, and in 64 bit
>> linux, it generates two RuntimeWarnings.
>>
>> The inconsistency seems like a bug, possibly this one:
>> https://github.com/numpy/numpy/issues/1958.
>>
>> Can anyone check if this also occurs in the development branch?
>>
>> Warren
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130823/2e633a2c/attachment.html>

From charlesr.harris at gmail.com  Fri Aug 23 08:45:34 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Fri, 23 Aug 2013 06:45:34 -0600
Subject: [Numpy-discussion] Warnings not raised by np.log in 32 bit
 build on Windows
In-Reply-To: <CAPJVwBmoy8wnwsc212cu=OKNmyc8JTa9ZgmtpB8cAG1BSDG9ag@mail.gmail.com>
References: <CAGzF1ucMD_GBV9gqL975MSDG3nUcCV+aS3qKoq_uD3hLoEJM6g@mail.gmail.com>
	<CAB6mnx+LKyK_pz8aBmDVsi4+iPHd4U21nnjD2AneT9D9LHbFvQ@mail.gmail.com>
	<CAPJVwBmoy8wnwsc212cu=OKNmyc8JTa9ZgmtpB8cAG1BSDG9ag@mail.gmail.com>
Message-ID: <CAB6mnxLeRqKmg+1mXiv51_YHNhz7QTO3P-o2wuKm3tDGZ_34+Q@mail.gmail.com>

On Fri, Aug 23, 2013 at 6:29 AM, Nathaniel Smith <njs at pobox.com> wrote:

> Probably the thing to do for reliable behaviour is to decide on the
> behaviour we want and then implement it "by hand". I.e., either clear the
> FP flags inside the ufunc loop (if we decide that log shouldn't raise a
> warning), or else check for nan and set the invalid flag ourselves.
> (Checking for nan should be much cheaper than computing log, I think, so
> this should be okay speed-wise?)
> On 23 Aug 2013 13:16, "Charles R Harris" <charlesr.harris at gmail.com>
> wrote:
>
>> These things may depend on how the compiler implements various calls.
>> Some errors went the other way with Julian's SIMD work, i.e., errors
>> getting set that were not set before. I'm not sure what can be done about
>> it.
>>
>>
>> On Thu, Aug 22, 2013 at 8:32 PM, Warren Weckesser <
>> warren.weckesser at gmail.com> wrote:
>>
>>> I'm investigating a test error in scipy 0.13.0 beta 1 that was
>>> reported by Christoph Gohlke.  The scipy issue is here:
>>> https://github.com/scipy/scipy/issues/2771
>>>
>>> I don't have a Windows environment to test it myself, but Christoph
>>> reported that this code:
>>>
>>> ```
>>> import numpy as np
>>>
>>> data = np.array([-0.375, -0.25, 0.0])
>>> s = np.log(data)
>>> ```
>>>
>>> does not generate two RuntimeWarnings when it is run with numpy 1.7.1
>>> in a 32 bit Windows 8 environment (numpy 1.7.1 compiled with Visual
>>> Studio compilers and Intel's MKL).  In 64 bit Windows, and in 64 bit
>>> linux, it generates two RuntimeWarnings.
>>>
>>> The inconsistency seems like a bug, possibly this one:
>>> https://github.com/numpy/numpy/issues/1958.
>>>
>>> Can anyone check if this also occurs in the development branch?
>>>
>>>
ISTR that is can also depend on instruction reordering in the hardware.
There was a bug opened on gcc for this a couple of years ago, but I did not
have the impression that it was going to get fixed any time soon.
Explicitly checking may add noticeable overhead and probably needs to be
profiled if we go that way.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130823/5379517e/attachment.html>

From joseluismietta at yahoo.com.ar  Fri Aug 23 09:58:01 2013
From: joseluismietta at yahoo.com.ar (=?iso-8859-1?Q?Jos=E8_Luis_Mietta?=)
Date: Fri, 23 Aug 2013 06:58:01 -0700 (PDT)
Subject: [Numpy-discussion] RAM problem during code execution - Numpya arrays
Message-ID: <1377266281.65599.YahooMailNeo@web142304.mail.bf1.yahoo.com>

Hi ecperts. I need your help with a RAM porblem during execution of my script.
I wrote the next code. I use SAGE. In 1-2 hours of execution time the RAM of my laptop (8gb) is filled and the sistem crash:


fromscipy.stats importuniform importnumpy asnp cant_de_cadenas =[700,800,900]cantidad_de_cadenas=np.array([])forkkkkk incant_de_cadenas:cantidad_de_cadenas=np.append(cantidad_de_cadenas,kkkkk)cantidad_de_cadenas=np.transpose(cantidad_de_cadenas)b=10h=b Longitud=1numero_experimentos=150densidad_de_cadenas =cantidad_de_cadenas/(b**2)prob_perc=np.array([])tiempos=np.array([])S_int=np.array([])S_medio=np.array([])desviacion_standard=np.array([])desviacion_standard_nuevo=np.array([])anisotropia_macroscopica_porcentual=np.array([])componente_y=np.array([])componente_x=np.array([])importtime forN incant_de_cadenas:empieza=time.clock()PERCOLACION=np.array([])size_medio_intuitivo =np.array([])size_medio_nuevo =np.array([])std_dev_size_medio_intuitivo =np.array([])std_dev_size_medio_nuevo =np.array([])comp_y =np.array([])comp_x =np.array([])foru inxrange(numero_experimentos):perco
 =Falsearray_x1=uniform.rvs(loc=-b/2,scale=b,size=N)array_y1=uniform.rvs(loc=-h/2,scale=h,size=N)array_angle=uniform.rvs(loc=-0.5*(np.pi),scale=np.pi,size=N)array_pendiente_x=1./np.tan(array_angle)random=uniform.rvs(loc=-1,scale=2,size=N)lambda_sign=np.zeros([N])fort inxrange(N):ifrandom[t]<0:lambda_sign[t]=-1else:lambda_sign[t]=1array_lambdas=(lambda_sign*Longitud)/np.sqrt(1+array_pendiente_x**2)array_x2=array_x1 +array_lambdas*array_pendiente_x array_y2=array_y1 +array_lambdas*1array_x1 =np.append(array_x1,[-b/2,b/2,-b/2,-b/2])array_y1 =np.append(array_y1,[-h/2,-h/2,-h/2,h/2])array_x2 =np.append(array_x2,[-b/2,b/2,b/2,b/2])array_y2 =np.append(array_y2,[h/2,h/2,-h/2,h/2])M =np.zeros([N+4,N+4])forj inxrange(N+4):ifj>0:x_A1B1 =array_x2[j]-array_x1[j]y_A1B1 =array_y2[j]-array_y1[j]x_A1A2 =array_x1[0:j]-array_x1[j]y_A1A2 =array_y1[0:j]-array_y1[j]x_A2A1 =-1*x_A1A2 y_A2A1 =-1*y_A1A2 x_A2B2 =array_x2[0:j]-array_x1[0:j]y_A2B2 =array_y2[0:j]-array_y1[0:j]x_A1B2
 =array_x2[0:j]-array_x1[j]y_A1B2 =array_y2[0:j]-array_y1[j]x_A2B1 =array_x2[j]-array_x1[0:j]y_A2B1 =array_y2[j]-array_y1[0:j]p1 =x_A1B1*y_A1A2 -y_A1B1*x_A1A2 p2 =x_A1B1*y_A1B2 -y_A1B1*x_A1B2 p3 =x_A2B2*y_A2B1 -y_A2B2*x_A2B1 p4 =x_A2B2*y_A2A1 -y_A2B2*x_A2A1 condicion_1=p1*p2 condicion_2=p3*p4 fork inxrange (j):ifcondicion_1[k]<=0andcondicion_2[k]<=0:M[j,k]=1delcondicion_1 delcondicion_2 ifj+1<N+4:x_A1B1 =array_x2[j]-array_x1[j]y_A1B1 =array_y2[j]-array_y1[j]x_A1A2 =array_x1[j+1:]-array_x1[j]y_A1A2 =array_y1[j+1:]-array_y1[j]x_A2A1 =-1*x_A1A2 y_A2A1 =-1*y_A1A2 x_A2B2 =array_x2[j+1:]-array_x1[j+1:]y_A2B2 =array_y2[j+1:]-array_y1[j+1:]x_A1B2 =array_x2[j+1:]-array_x1[j]y_A1B2 =array_y2[j+1:]-array_y1[j]x_A2B1 =array_x2[j]-array_x1[j+1:]y_A2B1 =array_y2[j]-array_y1[j+1:]p1 =x_A1B1*y_A1A2 -y_A1B1*x_A1A2 p2 =x_A1B1*y_A1B2 -y_A1B1*x_A1B2 p3 =x_A2B2*y_A2B1 -y_A2B2*x_A2B1 p4 =x_A2B2*y_A2A1 -y_A2B2*x_A2A1 condicion_1=p1*p2 condicion_2=p3*p4 fork inxrange
 ((N+4)-j-1):ifcondicion_1[k]<=0andcondicion_2[k]<=0:M[j,k+j+1]=1delcondicion_1 delcondicion_2 M[N,N+2]=0M[N,N+3]=0M[N+1,N+2]=0M[N+1,N+3]=0M[N+2,N]=0M[N+2,N+1]=0M[N+3,N]=0M[N+3,N+1]=0CD=np.array([])POPOPO=[]forg inxrange(N):lala=0r=Falsewhilelala<=len(POPOPO)-1:esta=g inPOPOPO[lala]ifesta isTrue:lala=len(POPOPO)r=Trueelse:lala=lala+1ifr isFalse:L=np.array([g])fors inxrange(N):ifM[g,s]!=0:L=np.append(L,s)x=0whilex<=N:forl inxrange(N):z=l inL               d=L[x]ifz isFalseandM[d,l]!=0:L=np.append(L,l)ifx+1<len(L):x+=1else:x=N+1.q=len (L)CD=np.append(CD,q)POPOPO.append(L)M_horizontal=M.copy()M_horizontal[:,N+2]=np.zeros(N+4)M_horizontal[:,N+3]=np.zeros(N+4)M_horizontal[N+2]=np.zeros(N+4)M_horizontal[N+3]=np.zeros(N+4)L=np.array([N])fors inxrange(N+4):ifM_horizontal[N,s]!=0:L=np.append(L,s)x=0whilex<=N+4:forl inxrange(N+4):z=l inL                d=L[x]ifz isFalseandM_horizontal[d,l]!=0:L=np.append(L,l)ifx+1<len(L):x+=1else:x=(N+4)+1.LV1_in_L =N inL 
 LV2_in_L=(N+1)inL ifLV1_in_L isTrueandLV2_in_L isTrue:perc_horiz=Trueelse:perc_horiz=FalseM_vertical=M.copy()M_vertical[:,N]=np.zeros(N+4)M_vertical[:,N+1]=np.zeros(N+4)M_vertical[N]=np.zeros(N+4)M_vertical[N+1]=np.zeros(N+4)L=np.array([N+2])fors inxrange(N+4):ifM_vertical[N+2,s]!=0:L=np.append(L,s)x=0whilex<=N+4:forl inxrange(N+4):z=l inL                d=L[x]ifz isFalseandM_vertical[d,l]!=0:L=np.append(L,l)ifx+1<len(L):x+=1else:x=(N+4)+1.LH1_in_L =(N+2)inL   LH2_in_L=(N+3)inL ifLH1_in_L isTrueandLH2_in_L isTrue:perc_ver =Trueelse:perc_ver =Falseifperc_ver isTrueorperc_horiz isTrue:PERCOLACION=np.append(PERCOLACION,1)perco=TrueD =np.array([])W =np.array([])forc inxrange (int(min(CD)),int(max(CD)+1),1):D=np.append(D,c)frec =sum (CD ==c)W =np.append(W,frec)ifperco isTrue:posicion=np.argmax(D)D=np.delete(D,posicion)W=np.delete(W,posicion)iflen(D)==0andlen(W)==0:S_medio_intuitivo_exp_u=0S_medio_nuevo_exp_u =0std_dev_exp_u =0std_dev_nuevo_exp_u
 =0else:S_medio_intuitivo_exp_u =np.average (D,weights=W)peso_nuevo=D*W S_medio_nuevo_exp_u =np.average (D,weights=peso_nuevo)tipos=sum(W)X=W*((D-S_medio_intuitivo_exp_u)**2)S=sum(X)std_dev_exp_u =np.sqrt(S/(tipos-1.))tipos_nuevo=sum(peso_nuevo)X_nuevo=peso_nuevo*((D-S_medio_nuevo_exp_u)**2)S_nuevo=sum(X_nuevo)std_dev_nuevo_exp_u =np.sqrt(S_nuevo/(tipos_nuevo-1.))componente_longitudinal=Longitud*np.abs(np.cos(array_angle))comp_y=np.append(comp_y,sum(componente_longitudinal)/N)componente_transversal=Longitud*np.abs(np.sin(array_angle))comp_x=np.append(comp_x,sum(componente_transversal)/N)std_dev_size_medio_intuitivo=np.append(std_dev_size_medio_intuitivo,std_dev_exp_u)std_dev_size_medio_nuevo=np.append(std_dev_size_medio_nuevo,std_dev_nuevo_exp_u)size_medio_intuitivo=np.append(size_medio_intuitivo,S_medio_intuitivo_exp_u)size_medio_nuevo=np.append(size_medio_nuevo,S_medio_nuevo_exp_u)percolation_probability=sum(PERCOLACION)/numero_experimentos
 prob_perc=np.append(prob_perc,percolation_probability)S_int =np.append (S_int,sum(size_medio_intuitivo)/numero_experimentos)S_medio=np.append (S_medio,sum(size_medio_nuevo)/numero_experimentos)desviacion_standard =np.append (desviacion_standard,sum(std_dev_size_medio_intuitivo)/numero_experimentos)desviacion_standard_nuevo=np.append (desviacion_standard_nuevo,sum(std_dev_size_medio_nuevo)/numero_experimentos)tiempos=np.append(tiempos,time.clock()-empieza)componente_y=np.append(componente_y,sum(comp_y)/numero_experimentos)componente_x=np.append(componente_x,sum(comp_x)/numero_experimentos)anisotropia_macroscopica_porcentual=100*(1-(componente_y/componente_x))


I tryed with gc and gc.collect() and 'del'command for deleting arrays after his use and nothing work! 


What am I doing wrong? Why the memory becomes full while running 
(starts with 10% of RAM used and in 1-2hour is totally full used)?

Please help me, I'm totally stuck!
Thanks a lot!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130823/ceec6d24/attachment.html>

From francesc at continuum.io  Fri Aug 23 10:34:06 2013
From: francesc at continuum.io (Francesc Alted)
Date: Fri, 23 Aug 2013 16:34:06 +0200
Subject: [Numpy-discussion] RAM problem during code execution - Numpya
	arrays
In-Reply-To: <1377266281.65599.YahooMailNeo@web142304.mail.bf1.yahoo.com>
References: <1377266281.65599.YahooMailNeo@web142304.mail.bf1.yahoo.com>
Message-ID: <CANF6gwp2DSf-NHnUvprVgSahKBMg+d8O9qTavid7YHaNqaw+OA@mail.gmail.com>

Hi Jos?,

The code is somewhat longish for a pure visual inspection, but my advice is
that you install memory profiler (
https://pypi.python.org/pypi/memory_profiler).  This will help you
determine which line or lines are hugging the memory the most.

Saludos,
Francesc

On Fri, Aug 23, 2013 at 3:58 PM, Jos? Luis Mietta <
joseluismietta at yahoo.com.ar> wrote:

> Hi ecperts. I need your help with a RAM porblem during execution of my
> script.
> I wrote the next code. I use SAGE. In 1-2 hours of execution time the RAM
> of my laptop (8gb) is filled and the sistem crash:
>
> from scipy.stats import uniform    import numpy as np
>
> cant_de_cadenas =[700,800,900]
>
> cantidad_de_cadenas=np.array([])
> for kkkkk in cant_de_cadenas:
>     cantidad_de_cadenas=np.append(cantidad_de_cadenas,kkkkk)
>
> cantidad_de_cadenas=np.transpose(cantidad_de_cadenas)
>
> b=10
> h=bLongitud=1
> numero_experimentos=150
>
> densidad_de_cadenas =cantidad_de_cadenas/(b**2)
>
> prob_perc=np.array([])
>
> tiempos=np.array([])
>
> S_int=np.array([])
>
> S_medio=np.array([])
>
> desviacion_standard=np.array([])
>
> desviacion_standard_nuevo=np.array([])
>
> anisotropia_macroscopica_porcentual=np.array([])
>
> componente_y=np.array([])
>
> componente_x=np.array([])
> import time
> for N in cant_de_cadenas:
>
>     empieza=time.clock()
>
>     PERCOLACION=np.array([])
>
>     size_medio_intuitivo = np.array([])
>     size_medio_nuevo = np.array([])
>     std_dev_size_medio_intuitivo = np.array([])
>     std_dev_size_medio_nuevo = np.array([])
>     comp_y = np.array([])
>     comp_x = np.array([])
>
>
>     for u in xrange(numero_experimentos):
>
>
>         perco = False
>
>         array_x1=uniform.rvs(loc=-b/2, scale=b, size=N)
>         array_y1=uniform.rvs(loc=-h/2, scale=h, size=N)
>         array_angle=uniform.rvs(loc=-0.5*(np.pi), scale=np.pi, size=N)
>
>
>         array_pendiente_x=1./np.tan(array_angle)
>
>         random=uniform.rvs(loc=-1, scale=2, size=N)
>         lambda_sign=np.zeros([N])
>         for t in xrange(N):
>             if random[t]<0:
>                 lambda_sign[t]=-1
>             else:
>                 lambda_sign[t]=1
>         array_lambdas=(lambda_sign*Longitud)/np.sqrt(1+array_pendiente_x**2)
>
>
>         array_x2= array_x1 + array_lambdas*array_pendiente_x
>         array_y2= array_y1 + array_lambdas*1
>
>         array_x1 = np.append(array_x1, [-b/2, b/2, -b/2, -b/2])
>         array_y1 = np.append(array_y1, [-h/2, -h/2, -h/2, h/2])
>         array_x2 = np.append(array_x2, [-b/2, b/2, b/2, b/2])
>         array_y2 = np.append(array_y2, [h/2, h/2, -h/2, h/2])
>
>         M = np.zeros([N+4,N+4])
>
>         for j in xrange(N+4):
>             if j>0:
>                 x_A1B1 = array_x2[j]-array_x1[j]
>                 y_A1B1 = array_y2[j]-array_y1[j]
>                 x_A1A2 = array_x1[0:j]-array_x1[j]
>                 y_A1A2 = array_y1[0:j]-array_y1[j]
>                 x_A2A1 = -1*x_A1A2
>                 y_A2A1 = -1*y_A1A2
>                 x_A2B2 = array_x2[0:j]-array_x1[0:j]
>                 y_A2B2 = array_y2[0:j]-array_y1[0:j]
>                 x_A1B2 = array_x2[0:j]-array_x1[j]
>                 y_A1B2 = array_y2[0:j]-array_y1[j]
>                 x_A2B1 = array_x2[j]-array_x1[0:j]
>                 y_A2B1 = array_y2[j]-array_y1[0:j]
>
>                 p1 = x_A1B1*y_A1A2 - y_A1B1*x_A1A2
>                 p2 = x_A1B1*y_A1B2 - y_A1B1*x_A1B2
>                 p3 = x_A2B2*y_A2B1 - y_A2B2*x_A2B1
>                 p4 = x_A2B2*y_A2A1 - y_A2B2*x_A2A1
>
>                 condicion_1=p1*p2
>                 condicion_2=p3*p4
>
>                 for k in xrange (j):
>                     if condicion_1[k]<=0 and condicion_2[k]<=0:
>                         M[j,k]=1
>                 del condicion_1
>                 del condicion_2
>
>
>             if j+1<N+4:
>                 x_A1B1 = array_x2[j]-array_x1[j]
>                 y_A1B1 = array_y2[j]-array_y1[j]
>                 x_A1A2 = array_x1[j+1:]-array_x1[j]
>                 y_A1A2 = array_y1[j+1:]-array_y1[j]
>                 x_A2A1 = -1*x_A1A2
>                 y_A2A1 = -1*y_A1A2
>                 x_A2B2 = array_x2[j+1:]-array_x1[j+1:]
>                 y_A2B2 = array_y2[j+1:]-array_y1[j+1:]
>                 x_A1B2 = array_x2[j+1:]-array_x1[j]
>                 y_A1B2 = array_y2[j+1:]-array_y1[j]
>                 x_A2B1 = array_x2[j]-array_x1[j+1:]
>                 y_A2B1 = array_y2[j]-array_y1[j+1:]
>
>                 p1 = x_A1B1*y_A1A2 - y_A1B1*x_A1A2
>                 p2 = x_A1B1*y_A1B2 - y_A1B1*x_A1B2
>                 p3 = x_A2B2*y_A2B1 - y_A2B2*x_A2B1
>                 p4 = x_A2B2*y_A2A1 - y_A2B2*x_A2A1
>
>                 condicion_1=p1*p2
>                 condicion_2=p3*p4
>
>                 for k in xrange ((N+4)-j-1):
>                     if condicion_1[k]<=0 and condicion_2[k]<=0:
>                         M[j,k+j+1]=1
>                 del condicion_1
>                 del condicion_2
>
>         M[N,N+2]=0
>         M[N,N+3]=0
>         M[N+1,N+2]=0
>         M[N+1,N+3]=0
>         M[N+2,N]=0
>         M[N+2,N+1]=0
>         M[N+3,N]=0
>         M[N+3,N+1]=0
>
>
>         CD=np.array([])
>
>         POPOPO=[]
>         for g in xrange(N):
>
>             lala=0
>             r=False
>             while lala<=len(POPOPO)-1:
>                 esta= g in POPOPO[lala]
>
>                 if esta is True:
>                     lala=len(POPOPO)
>                     r=True
>                 else:
>                     lala=lala+1
>             if r is False:
>
>
>
>                 L=np.array([g])
>                 for s in xrange(N):
>                     if M[g,s] != 0:
>                         L=np.append(L,s)
>
>                 x=0
>                 while x<= N:
>                     for l in xrange(N):
>                         z= l in L
>                         d=L[x]
>                         if z is False and M[d,l] != 0:
>                             L=np.append(L,l)
>                     if x+1<len(L):
>                         x+=1
>                     else:
>                         x=N+1.
>
>                 q= len (L)
>                 CD=np.append(CD, q)
>                 POPOPO.append(L)
>
>
>         M_horizontal=M.copy()
>         M_horizontal[:,N+2] = np.zeros(N+4)
>         M_horizontal[:,N+3] = np.zeros(N+4)
>         M_horizontal[N+2] = np.zeros(N+4)
>         M_horizontal[N+3] = np.zeros(N+4)
>
>
>
>         L=np.array([N])
>         for s in xrange(N+4):
>             if M_horizontal[N,s] != 0:
>                 L=np.append(L,s)
>
>         x=0
>         while x<= N+4:
>             for l in xrange(N+4):
>                 z= l in L
>                 d=L[x]
>                 if z is False and M_horizontal[d,l] != 0:
>                     L=np.append(L,l)
>             if x+1<len(L):
>                 x+=1
>             else:
>                 x=(N+4)+1.
>
>         LV1_in_L = N in L
>         LV2_in_L= (N+1) in L
>
>         if LV1_in_L is True and LV2_in_L is True:
>             perc_horiz=True
>         else:
>             perc_horiz=False
>
>
>         M_vertical=M.copy()
>
>         M_vertical[:,N] = np.zeros(N+4)
>         M_vertical[:,N+1] = np.zeros(N+4)
>         M_vertical[N] = np.zeros(N+4)
>         M_vertical[N+1] = np.zeros(N+4)
>
>
>
>         L=np.array([N+2])
>         for s in xrange(N+4):
>             if M_vertical[N+2,s] != 0:
>                 L=np.append(L,s)
>
>         x=0
>         while x<= N+4:
>             for l in xrange(N+4):
>                 z= l in L
>                 d=L[x]
>                 if z is False and M_vertical[d,l] != 0:
>                     L=np.append(L,l)
>             if x+1<len(L):
>                 x+=1
>             else:
>                 x=(N+4)+1.
>
>         LH1_in_L = (N+2) in L
>         LH2_in_L= (N+3) in L
>
>         if LH1_in_L is True and LH2_in_L is True:
>             perc_ver = True
>         else:
>             perc_ver = False
>
>
>
>         if perc_ver is True or perc_horiz is True:
>             PERCOLACION=np.append(PERCOLACION,1)
>             perco=True
>
>
>         D = np.array([])
>         W = np.array([])
>
>         for c in xrange (int(min(CD)), int(max(CD)+1),1):
>             D=np.append(D,c)
>             frec = sum (CD == c)
>             W = np.append(W,frec)
>
>         if perco is True:
>             posicion=np.argmax(D)
>             D=np.delete(D,posicion)
>             W=np.delete(W,posicion)
>
>         if len(D) == 0 and len(W)==0:
>             S_medio_intuitivo_exp_u=0
>             S_medio_nuevo_exp_u = 0
>             std_dev_exp_u = 0
>             std_dev_nuevo_exp_u = 0
>         else:
>
>             S_medio_intuitivo_exp_u = np.average (D,weights=W)
>
>             peso_nuevo=D*W
>
>             S_medio_nuevo_exp_u = np.average (D,weights=peso_nuevo)
>
>
>             tipos=sum(W)
>             X=W*((D-S_medio_intuitivo_exp_u)**2)
>             S=sum(X)
>
>             std_dev_exp_u = np.sqrt(S/(tipos-1.))
>
>             tipos_nuevo=sum(peso_nuevo)
>             X_nuevo=peso_nuevo*((D-S_medio_nuevo_exp_u)**2)
>             S_nuevo=sum(X_nuevo)
>
>             std_dev_nuevo_exp_u = np.sqrt(S_nuevo/(tipos_nuevo-1.))
>
>
>         componente_longitudinal=Longitud*np.abs(np.cos(array_angle))
>         comp_y=np.append(comp_y, sum(componente_longitudinal)/N)
>
>         componente_transversal=Longitud*np.abs(np.sin(array_angle))
>         comp_x=np.append(comp_x, sum(componente_transversal)/N)
>
>         std_dev_size_medio_intuitivo=np.append(std_dev_size_medio_intuitivo, std_dev_exp_u)
>
>         std_dev_size_medio_nuevo=np.append(std_dev_size_medio_nuevo, std_dev_nuevo_exp_u)
>
>         size_medio_intuitivo=np.append(size_medio_intuitivo, S_medio_intuitivo_exp_u)
>
>         size_medio_nuevo=np.append(size_medio_nuevo, S_medio_nuevo_exp_u)
>
>
>     percolation_probability=sum(PERCOLACION)/numero_experimentos
>
>     prob_perc=np.append(prob_perc, percolation_probability)
>
>     S_int = np.append (S_int, sum(size_medio_intuitivo)/numero_experimentos)
>
>     S_medio=np.append (S_medio, sum(size_medio_nuevo)/numero_experimentos)
>
>     desviacion_standard = np.append (desviacion_standard, sum(std_dev_size_medio_intuitivo)/numero_experimentos)
>
>     desviacion_standard_nuevo=np.append (desviacion_standard_nuevo, sum(std_dev_size_medio_nuevo)/numero_experimentos)
>
>     tiempos=np.append(tiempos, time.clock()-empieza)
>
>     componente_y=np.append(componente_y, sum(comp_y)/numero_experimentos)
>     componente_x=np.append(componente_x, sum(comp_x)/numero_experimentos)
>
>     anisotropia_macroscopica_porcentual=100*(1-(componente_y/componente_x))
>
> I tryed with gc and gc.collect() and 'del'command for deleting arrays
> after his use and nothing work!
>
> What am I doing wrong? Why the memory becomes full while running (starts
> with 10% of RAM used and in 1-2hour is totally full used)?
>
> Please help me, I'm totally stuck!
> Thanks a lot!
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>


-- 
Francesc Alted
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130823/63c94c96/attachment.html>

From ben.root at ou.edu  Fri Aug 23 10:59:10 2013
From: ben.root at ou.edu (Benjamin Root)
Date: Fri, 23 Aug 2013 10:59:10 -0400
Subject: [Numpy-discussion] RAM problem during code execution - Numpya
	arrays
In-Reply-To: <CANF6gwp2DSf-NHnUvprVgSahKBMg+d8O9qTavid7YHaNqaw+OA@mail.gmail.com>
References: <1377266281.65599.YahooMailNeo@web142304.mail.bf1.yahoo.com>
	<CANF6gwp2DSf-NHnUvprVgSahKBMg+d8O9qTavid7YHaNqaw+OA@mail.gmail.com>
Message-ID: <CANNq6F=q5i1naS+=MrGZ-+98qeJqk6RS3_D+=wx4KtPGXAq+oA@mail.gmail.com>

On Fri, Aug 23, 2013 at 10:34 AM, Francesc Alted <francesc at continuum.io>wrote:

> Hi Jos?,
>
> The code is somewhat longish for a pure visual inspection, but my advice
> is that you install memory profiler (
> https://pypi.python.org/pypi/memory_profiler).  This will help you
> determine which line or lines are hugging the memory the most.
>
> Saludos,
> Francesc
>
> On Fri, Aug 23, 2013 at 3:58 PM, Jos? Luis Mietta <
> joseluismietta at yahoo.com.ar> wrote:
>
>> Hi ecperts. I need your help with a RAM porblem during execution of my
>> script.
>> I wrote the next code. I use SAGE. In 1-2 hours of execution time the RAM
>> of my laptop (8gb) is filled and the sistem crash:
>>
>> from scipy.stats import uniform    import numpy as np
>>
>> cant_de_cadenas =[700,800,900]
>>
>> cantidad_de_cadenas=np.array([])
>> for kkkkk in cant_de_cadenas:
>>     cantidad_de_cadenas=np.append(cantidad_de_cadenas,kkkkk)
>>
>> cantidad_de_cadenas=np.transpose(cantidad_de_cadenas)
>>
>> b=10
>> h=bLongitud=1
>> numero_experimentos=150
>>
>> densidad_de_cadenas =cantidad_de_cadenas/(b**2)
>>
>> prob_perc=np.array([])
>>
>> tiempos=np.array([])
>>
>> S_int=np.array([])
>>
>> S_medio=np.array([])
>>
>> desviacion_standard=np.array([])
>>
>> desviacion_standard_nuevo=np.array([])
>>
>> anisotropia_macroscopica_porcentual=np.array([])
>>
>> componente_y=np.array([])
>>
>> componente_x=np.array([])
>> import time
>> for N in cant_de_cadenas:
>>
>>     empieza=time.clock()
>>
>>     PERCOLACION=np.array([])
>>
>>     size_medio_intuitivo = np.array([])
>>     size_medio_nuevo = np.array([])
>>     std_dev_size_medio_intuitivo = np.array([])
>>     std_dev_size_medio_nuevo = np.array([])
>>     comp_y = np.array([])
>>     comp_x = np.array([])
>>
>>
>>     for u in xrange(numero_experimentos):
>>
>>
>>         perco = False
>>
>>         array_x1=uniform.rvs(loc=-b/2, scale=b, size=N)
>>         array_y1=uniform.rvs(loc=-h/2, scale=h, size=N)
>>         array_angle=uniform.rvs(loc=-0.5*(np.pi), scale=np.pi, size=N)
>>
>>
>>         array_pendiente_x=1./np.tan(array_angle)
>>
>>         random=uniform.rvs(loc=-1, scale=2, size=N)
>>         lambda_sign=np.zeros([N])
>>         for t in xrange(N):
>>             if random[t]<0:
>>                 lambda_sign[t]=-1
>>             else:
>>                 lambda_sign[t]=1
>>         array_lambdas=(lambda_sign*Longitud)/np.sqrt(1+array_pendiente_x**2)
>>
>>
>>         array_x2= array_x1 + array_lambdas*array_pendiente_x
>>         array_y2= array_y1 + array_lambdas*1
>>
>>         array_x1 = np.append(array_x1, [-b/2, b/2, -b/2, -b/2])
>>         array_y1 = np.append(array_y1, [-h/2, -h/2, -h/2, h/2])
>>         array_x2 = np.append(array_x2, [-b/2, b/2, b/2, b/2])
>>         array_y2 = np.append(array_y2, [h/2, h/2, -h/2, h/2])
>>
>>         M = np.zeros([N+4,N+4])
>>
>>         for j in xrange(N+4):
>>             if j>0:
>>                 x_A1B1 = array_x2[j]-array_x1[j]
>>                 y_A1B1 = array_y2[j]-array_y1[j]
>>                 x_A1A2 = array_x1[0:j]-array_x1[j]
>>                 y_A1A2 = array_y1[0:j]-array_y1[j]
>>                 x_A2A1 = -1*x_A1A2
>>                 y_A2A1 = -1*y_A1A2
>>                 x_A2B2 = array_x2[0:j]-array_x1[0:j]
>>                 y_A2B2 = array_y2[0:j]-array_y1[0:j]
>>                 x_A1B2 = array_x2[0:j]-array_x1[j]
>>                 y_A1B2 = array_y2[0:j]-array_y1[j]
>>                 x_A2B1 = array_x2[j]-array_x1[0:j]
>>                 y_A2B1 = array_y2[j]-array_y1[0:j]
>>
>>                 p1 = x_A1B1*y_A1A2 - y_A1B1*x_A1A2
>>                 p2 = x_A1B1*y_A1B2 - y_A1B1*x_A1B2
>>                 p3 = x_A2B2*y_A2B1 - y_A2B2*x_A2B1
>>                 p4 = x_A2B2*y_A2A1 - y_A2B2*x_A2A1
>>
>>                 condicion_1=p1*p2
>>                 condicion_2=p3*p4
>>
>>                 for k in xrange (j):
>>                     if condicion_1[k]<=0 and condicion_2[k]<=0:
>>                         M[j,k]=1
>>                 del condicion_1
>>                 del condicion_2
>>
>>
>>             if j+1<N+4:
>>                 x_A1B1 = array_x2[j]-array_x1[j]
>>                 y_A1B1 = array_y2[j]-array_y1[j]
>>                 x_A1A2 = array_x1[j+1:]-array_x1[j]
>>                 y_A1A2 = array_y1[j+1:]-array_y1[j]
>>                 x_A2A1 = -1*x_A1A2
>>                 y_A2A1 = -1*y_A1A2
>>                 x_A2B2 = array_x2[j+1:]-array_x1[j+1:]
>>                 y_A2B2 = array_y2[j+1:]-array_y1[j+1:]
>>                 x_A1B2 = array_x2[j+1:]-array_x1[j]
>>                 y_A1B2 = array_y2[j+1:]-array_y1[j]
>>                 x_A2B1 = array_x2[j]-array_x1[j+1:]
>>                 y_A2B1 = array_y2[j]-array_y1[j+1:]
>>
>>                 p1 = x_A1B1*y_A1A2 - y_A1B1*x_A1A2
>>                 p2 = x_A1B1*y_A1B2 - y_A1B1*x_A1B2
>>                 p3 = x_A2B2*y_A2B1 - y_A2B2*x_A2B1
>>                 p4 = x_A2B2*y_A2A1 - y_A2B2*x_A2A1
>>
>>                 condicion_1=p1*p2
>>                 condicion_2=p3*p4
>>
>>                 for k in xrange ((N+4)-j-1):
>>                     if condicion_1[k]<=0 and condicion_2[k]<=0:
>>                         M[j,k+j+1]=1
>>                 del condicion_1
>>                 del condicion_2
>>
>>         M[N,N+2]=0
>>         M[N,N+3]=0
>>         M[N+1,N+2]=0
>>         M[N+1,N+3]=0
>>         M[N+2,N]=0
>>         M[N+2,N+1]=0
>>         M[N+3,N]=0
>>         M[N+3,N+1]=0
>>
>>
>>         CD=np.array([])
>>
>>         POPOPO=[]
>>         for g in xrange(N):
>>
>>             lala=0
>>             r=False
>>             while lala<=len(POPOPO)-1:
>>                 esta= g in POPOPO[lala]
>>
>>                 if esta is True:
>>                     lala=len(POPOPO)
>>                     r=True
>>                 else:
>>                     lala=lala+1
>>             if r is False:
>>
>>
>>
>>                 L=np.array([g])
>>                 for s in xrange(N):
>>                     if M[g,s] != 0:
>>                         L=np.append(L,s)
>>
>>                 x=0
>>                 while x<= N:
>>                     for l in xrange(N):
>>                         z= l in L
>>                         d=L[x]
>>                         if z is False and M[d,l] != 0:
>>                             L=np.append(L,l)
>>                     if x+1<len(L):
>>                         x+=1
>>                     else:
>>                         x=N+1.
>>
>>                 q= len (L)
>>                 CD=np.append(CD, q)
>>                 POPOPO.append(L)
>>
>>
>>         M_horizontal=M.copy()
>>         M_horizontal[:,N+2] = np.zeros(N+4)
>>         M_horizontal[:,N+3] = np.zeros(N+4)
>>         M_horizontal[N+2] = np.zeros(N+4)
>>         M_horizontal[N+3] = np.zeros(N+4)
>>
>>
>>
>>         L=np.array([N])
>>         for s in xrange(N+4):
>>             if M_horizontal[N,s] != 0:
>>                 L=np.append(L,s)
>>
>>         x=0
>>         while x<= N+4:
>>             for l in xrange(N+4):
>>                 z= l in L
>>                 d=L[x]
>>                 if z is False and M_horizontal[d,l] != 0:
>>                     L=np.append(L,l)
>>             if x+1<len(L):
>>                 x+=1
>>             else:
>>                 x=(N+4)+1.
>>
>>         LV1_in_L = N in L
>>         LV2_in_L= (N+1) in L
>>
>>         if LV1_in_L is True and LV2_in_L is True:
>>             perc_horiz=True
>>         else:
>>             perc_horiz=False
>>
>>
>>         M_vertical=M.copy()
>>
>>         M_vertical[:,N] = np.zeros(N+4)
>>         M_vertical[:,N+1] = np.zeros(N+4)
>>         M_vertical[N] = np.zeros(N+4)
>>         M_vertical[N+1] = np.zeros(N+4)
>>
>>
>>
>>         L=np.array([N+2])
>>         for s in xrange(N+4):
>>             if M_vertical[N+2,s] != 0:
>>                 L=np.append(L,s)
>>
>>         x=0
>>         while x<= N+4:
>>             for l in xrange(N+4):
>>                 z= l in L
>>                 d=L[x]
>>                 if z is False and M_vertical[d,l] != 0:
>>                     L=np.append(L,l)
>>             if x+1<len(L):
>>                 x+=1
>>             else:
>>                 x=(N+4)+1.
>>
>>         LH1_in_L = (N+2) in L
>>         LH2_in_L= (N+3) in L
>>
>>         if LH1_in_L is True and LH2_in_L is True:
>>             perc_ver = True
>>         else:
>>             perc_ver = False
>>
>>
>>
>>         if perc_ver is True or perc_horiz is True:
>>             PERCOLACION=np.append(PERCOLACION,1)
>>             perco=True
>>
>>
>>         D = np.array([])
>>         W = np.array([])
>>
>>         for c in xrange (int(min(CD)), int(max(CD)+1),1):
>>             D=np.append(D,c)
>>             frec = sum (CD == c)
>>             W = np.append(W,frec)
>>
>>         if perco is True:
>>             posicion=np.argmax(D)
>>             D=np.delete(D,posicion)
>>             W=np.delete(W,posicion)
>>
>>         if len(D) == 0 and len(W)==0:
>>             S_medio_intuitivo_exp_u=0
>>             S_medio_nuevo_exp_u = 0
>>             std_dev_exp_u = 0
>>             std_dev_nuevo_exp_u = 0
>>         else:
>>
>>             S_medio_intuitivo_exp_u = np.average (D,weights=W)
>>
>>             peso_nuevo=D*W
>>
>>             S_medio_nuevo_exp_u = np.average (D,weights=peso_nuevo)
>>
>>
>>             tipos=sum(W)
>>             X=W*((D-S_medio_intuitivo_exp_u)**2)
>>             S=sum(X)
>>
>>             std_dev_exp_u = np.sqrt(S/(tipos-1.))
>>
>>             tipos_nuevo=sum(peso_nuevo)
>>             X_nuevo=peso_nuevo*((D-S_medio_nuevo_exp_u)**2)
>>             S_nuevo=sum(X_nuevo)
>>
>>             std_dev_nuevo_exp_u = np.sqrt(S_nuevo/(tipos_nuevo-1.))
>>
>>
>>         componente_longitudinal=Longitud*np.abs(np.cos(array_angle))
>>         comp_y=np.append(comp_y, sum(componente_longitudinal)/N)
>>
>>         componente_transversal=Longitud*np.abs(np.sin(array_angle))
>>         comp_x=np.append(comp_x, sum(componente_transversal)/N)
>>
>>         std_dev_size_medio_intuitivo=np.append(std_dev_size_medio_intuitivo, std_dev_exp_u)
>>
>>         std_dev_size_medio_nuevo=np.append(std_dev_size_medio_nuevo, std_dev_nuevo_exp_u)
>>
>>         size_medio_intuitivo=np.append(size_medio_intuitivo, S_medio_intuitivo_exp_u)
>>
>>         size_medio_nuevo=np.append(size_medio_nuevo, S_medio_nuevo_exp_u)
>>
>>
>>     percolation_probability=sum(PERCOLACION)/numero_experimentos
>>
>>     prob_perc=np.append(prob_perc, percolation_probability)
>>
>>     S_int = np.append (S_int, sum(size_medio_intuitivo)/numero_experimentos)
>>
>>     S_medio=np.append (S_medio, sum(size_medio_nuevo)/numero_experimentos)
>>
>>     desviacion_standard = np.append (desviacion_standard, sum(std_dev_size_medio_intuitivo)/numero_experimentos)
>>
>>     desviacion_standard_nuevo=np.append (desviacion_standard_nuevo, sum(std_dev_size_medio_nuevo)/numero_experimentos)
>>
>>     tiempos=np.append(tiempos, time.clock()-empieza)
>>
>>     componente_y=np.append(componente_y, sum(comp_y)/numero_experimentos)
>>     componente_x=np.append(componente_x, sum(comp_x)/numero_experimentos)
>>
>>     anisotropia_macroscopica_porcentual=100*(1-(componente_y/componente_x))
>>
>> I tryed with gc and gc.collect() and 'del'command for deleting arrays
>> after his use and nothing work!
>>
>> What am I doing wrong? Why the memory becomes full while running (starts
>> with 10% of RAM used and in 1-2hour is totally full used)?
>>
>> Please help me, I'm totally stuck!
>> Thanks a lot!
>>
>>
With the following code in the very beginning (not the cause):

cant_de_cadenas =[700,800,900]

cantidad_de_cadenas=np.array([])
for kkkkk in cant_de_cadenas:
    cantidad_de_cadenas=np.append(cantidad_de_cadenas,kkkkk)

cantidad_de_cadenas=np.transpose(cantidad_de_cadenas)


That can'e be a very good sign.  The thing to remember about np.append() is
that it only exists as a convenience and should be used as infrequently as
possible. Calling append makes a new copy each time.  Instead, if you know
the length of the array ahead of time, initialize those arrays to those
sizes before looping and just assign the value into them.  For example,
with the above example from your code, I would write that as
"cantidad_de_cadenas = np.array([[700, 800, 900]])". A lot of the code you
have here can be greatly simplified. I would start with just trying to get
rid of appends as much as possible and use preallocated arrays with
np.empty() or np.ones() or the likes.

Cheers!
Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130823/232ced56/attachment.html>

From chris.barker at noaa.gov  Fri Aug 23 10:59:47 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Fri, 23 Aug 2013 07:59:47 -0700
Subject: [Numpy-discussion] What is a numpy.long type?
In-Reply-To: <CAGY4rcWmjrKQVEWeZPge8tzmG=5CXLoQAgKk0jNZLWSkoXh_bA@mail.gmail.com>
References: <CALGmxELkBR3+J8o20MF6_Jj9Sep7hCFDONVTETa4PytVMsvkUA@mail.gmail.com>
	<CAGY4rcWmjrKQVEWeZPge8tzmG=5CXLoQAgKk0jNZLWSkoXh_bA@mail.gmail.com>
Message-ID: <7276594298807670183@unknownmsgid>

On Aug 22, 2013, at 11:57 PM, David Cournapeau <cournape at gmail.com> wrote:

npy_long is indeed just an alias to C long,


Which means it's likely broken on 32 bit platforms and 64 bit MSVC.

np.long is an alias to python's long:

But python's long is an unlimited type--it can't be mapped to a c type at
all.

arch -32 python -c "import numpy as np; print np.dtype(np.int); print
np.dtype(np.long)"
int32
int64


So this is giving us a 64 bit int--not a bad compromise, but not a python
long--I've got to wonder why the alias is there at all.

arch -64 python -c "import numpy as np; print np.dtype(np.int); print
np.dtype(np.long)"
int64
int64

Same thing on 64 bit.

So while np.long is an alias to python long--it apparently is translated
internally as 64 bit -- everywhere?

So apparently there is no way to get a "platform long". ( or, for that
matter, a platform anything else, it's just that there is more consistancy
among common platforms for the others)

-Chris


All this is on python 2.7, I am not sure how/if that changes on python 3
(that consolidated python int/long).

David

>
> -Chris
>
>
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chris.Barker at noaa.gov
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130823/577e9be3/attachment.html>

From sebastian at sipsolutions.net  Fri Aug 23 11:11:56 2013
From: sebastian at sipsolutions.net (Sebastian Berg)
Date: Fri, 23 Aug 2013 17:11:56 +0200
Subject: [Numpy-discussion] What is a numpy.long type?
In-Reply-To: <7276594298807670183@unknownmsgid>
References: <CALGmxELkBR3+J8o20MF6_Jj9Sep7hCFDONVTETa4PytVMsvkUA@mail.gmail.com>
	<CAGY4rcWmjrKQVEWeZPge8tzmG=5CXLoQAgKk0jNZLWSkoXh_bA@mail.gmail.com>
	<7276594298807670183@unknownmsgid>
Message-ID: <1377270716.12817.4.camel@sebastian-laptop>

On Fri, 2013-08-23 at 07:59 -0700, Chris Barker - NOAA Federal wrote:
> On Aug 22, 2013, at 11:57 PM, David Cournapeau <cournape at gmail.com>
> wrote:
> 
> 
<snip>
> 
> > arch -32 python -c "import numpy as np; print np.dtype(np.int);
> > print np.dtype(np.long)"
> > int32
> > int64
> 
> 
> So this is giving us a 64 bit int--not a bad compromise, but not a
> python long--I've got to wonder why the alias is there at all.
> 
It is there because you can't remove it :).
> 
> > arch -64 python -c "import numpy as np; print np.dtype(np.int);
> > print np.dtype(np.long)"
> > int64
> > int64
> > 
> > 
> Same thing on 64 bit.
> 
> 
> So while np.long is an alias to python long--it apparently is
> translated internally as 64 bit -- everywhere?
> 
Not sure how a python long is translated...
> 
> So apparently there is no way to get a "platform long". ( or, for that
> matter, a platform anything else, it's just that there is more
> consistancy among common platforms for the others)
> 
An np.int_ is a platform long, since the python ints are C longs. It is
a bit weird naming, but it is transparent. Check
http://docs.scipy.org/doc/numpy-dev/reference/arrays.scalars.html#built-in-scalar-types

you got everything platform dependend there really. `intc` is an int,
`int_` is a long, and you got `longlong`, as well as `intp` which is an
ssize_t, etc.

- Sebastian

> 
> -Chris
> 
> 
> 
> 
> > All this is on python 2.7, I am not sure how/if that changes on
> > python 3 (that consolidated python int/long).
> > 
> > 
> > David
> >         
> >         -Chris
> >         
> >         
> >         
> >         
> >         --
> >         
> >         Christopher Barker, Ph.D.
> >         Oceanographer
> >         
> >         Emergency Response Division
> >         NOAA/NOS/OR&R            (206) 526-6959   voice
> >         7600 Sand Point Way NE   (206) 526-6329   fax
> >         Seattle, WA  98115       (206) 526-6317   main reception
> >         
> >         Chris.Barker at noaa.gov
> >         _______________________________________________
> >         NumPy-Discussion mailing list
> >         NumPy-Discussion at scipy.org
> >         http://mail.scipy.org/mailman/listinfo/numpy-discussion
> > 
> > 
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> > 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion




From charlesr.harris at gmail.com  Fri Aug 23 11:15:08 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Fri, 23 Aug 2013 09:15:08 -0600
Subject: [Numpy-discussion] What is a numpy.long type?
In-Reply-To: <7276594298807670183@unknownmsgid>
References: <CALGmxELkBR3+J8o20MF6_Jj9Sep7hCFDONVTETa4PytVMsvkUA@mail.gmail.com>
	<CAGY4rcWmjrKQVEWeZPge8tzmG=5CXLoQAgKk0jNZLWSkoXh_bA@mail.gmail.com>
	<7276594298807670183@unknownmsgid>
Message-ID: <CAB6mnxJ633z7NX4sfGy3_SQF372oLsXaeMczbQSYsDmpruvPuQ@mail.gmail.com>

On Fri, Aug 23, 2013 at 8:59 AM, Chris Barker - NOAA Federal <
chris.barker at noaa.gov> wrote:

> On Aug 22, 2013, at 11:57 PM, David Cournapeau <cournape at gmail.com> wrote:
>
> npy_long is indeed just an alias to C long,
>
>
> Which means it's likely broken on 32 bit platforms and 64 bit MSVC.
>
> np.long is an alias to python's long:
>
> But python's long is an unlimited type--it can't be mapped to a c type at
> all.
>
> arch -32 python -c "import numpy as np; print np.dtype(np.int); print
> np.dtype(np.long)"
> int32
> int64
>
>
> So this is giving us a 64 bit int--not a bad compromise, but not a python
> long--I've got to wonder why the alias is there at all.
>
> arch -64 python -c "import numpy as np; print np.dtype(np.int); print
> np.dtype(np.long)"
> int64
> int64
>
> Same thing on 64 bit.
>
> So while np.long is an alias to python long--it apparently is translated
> internally as 64 bit -- everywhere?
>
> So apparently there is no way to get a "platform long". ( or, for that
> matter, a platform anything else, it's just that there is more consistancy
> among common platforms for the others)
>
>
I use 'bBhHiIlLqQ' for the C types. Long varies between 32 & 64 bit,
depending on the platform and 64 bit convention chosen. The C int is always
32 bits as far as I know.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130823/86d059ac/attachment.html>

From chris.barker at noaa.gov  Fri Aug 23 11:58:39 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Fri, 23 Aug 2013 08:58:39 -0700
Subject: [Numpy-discussion] What is a numpy.long type?
In-Reply-To: <1377270716.12817.4.camel@sebastian-laptop>
References: <CALGmxELkBR3+J8o20MF6_Jj9Sep7hCFDONVTETa4PytVMsvkUA@mail.gmail.com>
	<CAGY4rcWmjrKQVEWeZPge8tzmG=5CXLoQAgKk0jNZLWSkoXh_bA@mail.gmail.com>
	<7276594298807670183@unknownmsgid>
	<1377270716.12817.4.camel@sebastian-laptop>
Message-ID: <CALGmxELKMnhqs9tiD10UaLCNp1uuaNqX-dsiJLyT_bzsNyXMPQ@mail.gmail.com>

On Fri, Aug 23, 2013 at 8:11 AM, Sebastian Berg
<sebastian at sipsolutions.net> wrote:
>> So this is giving us a 64 bit int--not a bad compromise, but not a
>> python long--I've got to wonder why the alias is there at all.
>>
> It is there because you can't remove it :).

sure we could -- not that we'd want to....

>> So while np.long is an alias to python long--it apparently is
>> translated internally as 64 bit -- everywhere?
>>
> Not sure how a python long is translated...

The big mystery!

> An np.int_ is a platform long, since the python ints are C longs. It is
> a bit weird naming, but it is transparent. Check
> http://docs.scipy.org/doc/numpy-dev/reference/arrays.scalars.html#built-in-scalar-types

cool, thanks.

> you got everything platform dependend there really. `intc` is an int,
> `int_` is a long, and you got `longlong`, as well as `intp` which is an
> ssize_t, etc.

great, thanks. That's helpful.

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From chris.barker at noaa.gov  Fri Aug 23 12:00:47 2013
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Fri, 23 Aug 2013 09:00:47 -0700
Subject: [Numpy-discussion] What is a numpy.long type?
In-Reply-To: <CAB6mnxJ633z7NX4sfGy3_SQF372oLsXaeMczbQSYsDmpruvPuQ@mail.gmail.com>
References: <CALGmxELkBR3+J8o20MF6_Jj9Sep7hCFDONVTETa4PytVMsvkUA@mail.gmail.com>
	<CAGY4rcWmjrKQVEWeZPge8tzmG=5CXLoQAgKk0jNZLWSkoXh_bA@mail.gmail.com>
	<7276594298807670183@unknownmsgid>
	<CAB6mnxJ633z7NX4sfGy3_SQF372oLsXaeMczbQSYsDmpruvPuQ@mail.gmail.com>
Message-ID: <CALGmxEK7Rk19A5X0wxieP-JhT5z7NzNT7hBaJoF-MeB0Hh+-Qg@mail.gmail.com>

On Fri, Aug 23, 2013 at 8:15 AM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
> I use 'bBhHiIlLqQ' for the C types. Long varies between 32 & 64 bit,
> depending on the platform and 64 bit convention chosen. The C int is always
> 32 bits as far as I know.

Well, not in the spec, but in practice, probably. Maybe not on some
embedded platfroms?

not my issue...

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From davidmenhur at gmail.com  Fri Aug 23 12:15:33 2013
From: davidmenhur at gmail.com (=?UTF-8?B?RGHPgGlk?=)
Date: Fri, 23 Aug 2013 18:15:33 +0200
Subject: [Numpy-discussion] RAM problem during code execution - Numpya
	arrays
In-Reply-To: <CANNq6F=q5i1naS+=MrGZ-+98qeJqk6RS3_D+=wx4KtPGXAq+oA@mail.gmail.com>
References: <1377266281.65599.YahooMailNeo@web142304.mail.bf1.yahoo.com>
	<CANF6gwp2DSf-NHnUvprVgSahKBMg+d8O9qTavid7YHaNqaw+OA@mail.gmail.com>
	<CANNq6F=q5i1naS+=MrGZ-+98qeJqk6RS3_D+=wx4KtPGXAq+oA@mail.gmail.com>
Message-ID: <CAJhcF=0pM55pjyMeOxAExx5OMAqdSs_vG8e3bR9bMvk3Ex+U1w@mail.gmail.com>

On 23 August 2013 16:59, Benjamin Root <ben.root at ou.edu> wrote:
> A lot of the code you have here can be greatly simplified. I would start
> with just trying to get rid of appends as much as possible and use
> preallocated arrays with np.empty() or np.ones() or the likes.

Also, if you don't know beforehand the final size of the array (I find
difficult to follow the flow of the program, it is quite lengthy), you
can use lists as a temporary thing:

results = []
while SomeCondition:
   do_something()
   results.append(res)
results = np.array(res)

Also, it may also help to trace things down encapsulate pieces of code
inside functions. There are two reasons for this: it will make the
code more readable, easier to test, and you could run independently
pieces of code to find where is your memory growing.


I hope it is of any help.

David.


From rowen at uw.edu  Fri Aug 23 16:32:29 2013
From: rowen at uw.edu (Russell E. Owen)
Date: Fri, 23 Aug 2013 13:32:29 -0700
Subject: [Numpy-discussion] OS X binaries for releases
References: <CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ@mail.gmail.com>
	<rowen-5A7B80.12141122082013@news.gmane.org>
	<CAH6Pt5o32Otdhk2Ms5Cy5Zo=Mn48H8X2WbsWK92etUb4MmrooQ@mail.gmail.com>
Message-ID: <rowen-F698F0.13322823082013@news.gmane.org>

In article 
<CAH6Pt5o32Otdhk2Ms5Cy5Zo=Mn48H8X2WbsWK92etUb4MmrooQ at mail.gmail.com>,
 Matthew Brett <matthew.brett at gmail.com> wrote:

> Hi,
> 
> On Thu, Aug 22, 2013 at 12:14 PM, Russell E. Owen <rowen at uw.edu> wrote:
> > In article
> > <CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ at mail.gmail.com>,
> >  Ralf Gommers <ralf.gommers at gmail.com> wrote:
> >
> >> Hi all,
> >>
> >> Building binaries for releases is currently quite complex and
> >> time-consuming. For OS X we need two different machines, because we still
> >> provide binaries for OS X 10.5 and PPC machines. I propose to not do this
> >> anymore. It doesn't mean we completely drop support for 10.5 and PPC, just
> >> that we don't produce binaries. PPC was phased out in 2006 and OS X 10.6
> >> came out in 2009, so there can't be a lot of demand for it (and the
> >> download stats at
> >> http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/confirm this).
> >>
> >> Furthermore I propose to not provide 2.6 binaries anymore. Downloads of 2.6
> >> OS X binaries were <5% of the 2.7 ones. We did the same with 2.4 for a long
> >> time - support it but no binaries.
> >>
> >> So what we'd have left at the moment is only the 64-bit/32-bit universal
> >> binary for 10.6 and up. What we finally need to add is 3.x OS X binaries.
> >> We can make an attempt to build these on 10.8 - since we have access to a
> >> hosted 10.8 Mac Mini it would allow all devs to easily do a release
> >> (leaving aside the Windows issue). If anyone has tried the 10.6 SDK on 10.8
> >> and knows if it actually works, that would be helpful.
> >>
> >> Any concerns, objections?
> >
> > I am in strong agreement.
> >
> > I'll be interested to learn how you make binary installers for python
> > 3.x because the standard version of bdist_mpkg will not do it. I have
> > heard of two other projects (forks or variants of bdist_mpkg) that will,
> > but I have no idea of either is supported.
> 
> I think I'm the owner of one of the forks; I supporting it, but I
> should certainly make a release soon too.

That sounds promising. Can you suggest a non-released commit that is 
stable enough to try, or should we wait for a release?

Also, is there a way to combine multiple packages into one binary 
installer? (matplotib used to include python-dateutil, pytz and six, but 
1.3 does not).

> > I have been able to building packages on 10.8 using
> > MACOSX_DEPLOYMENT_TARGET=10.6 that will run on 10.6, so it will probably
> > work. However I have run into several odd problems over the years
> > building a binary installer on a newer system only to find it won't work
> > on older systems for various reasons. Thus my personal recommendation is
> > that you build on 10.6 if you want an installer that reliably works for
> > 10.6 and later. I keep an older computer around for this reason. In fact
> > that is one good reason to drop support for ancient operating systems
> > and PPC.
> 
> I'm sitting next to a 10.6 machine you are welcome to use; just let me
> know, I'll give you login access.

Thank you. Personally I keep an older laptop I keep around that can run 
10.6 (and even 10.4 and 10.5, which was handy when I made binaries that 
supported 10.3.9 and later -- no need for that these days), so I don't 
need it, but somebody else working on matplotlib binaries might.

-- Russell



From matthew.brett at gmail.com  Fri Aug 23 16:38:57 2013
From: matthew.brett at gmail.com (Matthew Brett)
Date: Fri, 23 Aug 2013 13:38:57 -0700
Subject: [Numpy-discussion] OS X binaries for releases
In-Reply-To: <rowen-F698F0.13322823082013@news.gmane.org>
References: <CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ@mail.gmail.com>
	<rowen-5A7B80.12141122082013@news.gmane.org>
	<CAH6Pt5o32Otdhk2Ms5Cy5Zo=Mn48H8X2WbsWK92etUb4MmrooQ@mail.gmail.com>
	<rowen-F698F0.13322823082013@news.gmane.org>
Message-ID: <CAH6Pt5oExOUH8m2gvwFCPcEqAjr9qBRYJYr+yK5YMTxnLLA29g@mail.gmail.com>

Hi,

On Fri, Aug 23, 2013 at 1:32 PM, Russell E. Owen <rowen at uw.edu> wrote:
> In article
> <CAH6Pt5o32Otdhk2Ms5Cy5Zo=Mn48H8X2WbsWK92etUb4MmrooQ at mail.gmail.com>,
>  Matthew Brett <matthew.brett at gmail.com> wrote:
>
>> Hi,
>>
>> On Thu, Aug 22, 2013 at 12:14 PM, Russell E. Owen <rowen at uw.edu> wrote:
>> > In article
>> > <CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ at mail.gmail.com>,
>> >  Ralf Gommers <ralf.gommers at gmail.com> wrote:
>> >
>> >> Hi all,
>> >>
>> >> Building binaries for releases is currently quite complex and
>> >> time-consuming. For OS X we need two different machines, because we still
>> >> provide binaries for OS X 10.5 and PPC machines. I propose to not do this
>> >> anymore. It doesn't mean we completely drop support for 10.5 and PPC, just
>> >> that we don't produce binaries. PPC was phased out in 2006 and OS X 10.6
>> >> came out in 2009, so there can't be a lot of demand for it (and the
>> >> download stats at
>> >> http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/confirm this).
>> >>
>> >> Furthermore I propose to not provide 2.6 binaries anymore. Downloads of 2.6
>> >> OS X binaries were <5% of the 2.7 ones. We did the same with 2.4 for a long
>> >> time - support it but no binaries.
>> >>
>> >> So what we'd have left at the moment is only the 64-bit/32-bit universal
>> >> binary for 10.6 and up. What we finally need to add is 3.x OS X binaries.
>> >> We can make an attempt to build these on 10.8 - since we have access to a
>> >> hosted 10.8 Mac Mini it would allow all devs to easily do a release
>> >> (leaving aside the Windows issue). If anyone has tried the 10.6 SDK on 10.8
>> >> and knows if it actually works, that would be helpful.
>> >>
>> >> Any concerns, objections?
>> >
>> > I am in strong agreement.
>> >
>> > I'll be interested to learn how you make binary installers for python
>> > 3.x because the standard version of bdist_mpkg will not do it. I have
>> > heard of two other projects (forks or variants of bdist_mpkg) that will,
>> > but I have no idea of either is supported.
>>
>> I think I'm the owner of one of the forks; I supporting it, but I
>> should certainly make a release soon too.
>
> That sounds promising. Can you suggest a non-released commit that is
> stable enough to try, or should we wait for a release?

It has hardly changed since the Python 3 port - the current head
should be fine, I'm using it for our installers.  But I will get to a
release soon.

> Also, is there a way to combine multiple packages into one binary
> installer? (matplotib used to include python-dateutil, pytz and six, but
> 1.3 does not).

Well - yes - by hacking.  I did something like this to make huge
scientific python installer for a course I'm teaching:

https://github.com/matthew-brett/reginald

Basically, you build the mpkg files for each thing you want to
install, then copy the sub-packages from the mpkg into a mpkg
megapackage (see the README for what I mean).

I should really automate this better - it was pretty easy to build a
large and useful distribution this way.

Cheers,

Matthew


From joseluismietta at yahoo.com.ar  Fri Aug 23 18:09:31 2013
From: joseluismietta at yahoo.com.ar (=?iso-8859-1?Q?Jos=E8_Luis_Mietta?=)
Date: Fri, 23 Aug 2013 15:09:31 -0700 (PDT)
Subject: [Numpy-discussion] Stick (line segments) percolation algorithm -
	graph theory?
Message-ID: <1377295771.604.YahooMailNeo@web142305.mail.bf1.yahoo.com>

Hi experts!
I wrote an algorithm for study stick percolation (i.e.: networks between line segments that intersect between them). In my algorithm N sticks (line segments) are created inside a rectanglar box of sides 'b' and 'h' and then, one by one, the algorithm explores the intersection between all line segments. This is a Monte Carlo simulation, so the 'experiment' is executed many times (no less than 100 times). Writen like that, very much RAM is consumed:
array_x1=uniform.rvs(loc=-b/2,scale=b,size=N)?
array_y1=uniform.rvs(loc=-h/2,scale=h,size=N)
array_x2=uniform.rvs(loc=-b/2,scale=b,size=N)?
array_y2=uniform.rvs(loc=-h/2,scale=h,size=N)?

M =np.zeros([N,N])?

foru inxrange(100):?---->This'100'isthe number of experiments.
? ? forj inxrange(N):
? ? ? ? ifj>0:
? ? ? ? ? ? x_A1B1 =array_x2[j]-array_x1[j]
? ? ? ? ? ? y_A1B1 =array_y2[j]-array_y1[j]
? ? ? ? ? ? x_A1A2 =array_x1[0:j]-array_x1[j]
? ? ? ? ? ? y_A1A2 =array_y1[0:j]-array_y1[j]? ? ?
? ? ? ? ? ? x_A2A1 =-1*x_A1A2
? ? ? ? ? ? y_A2A1 =-1*y_A1A2
? ? ? ? ? ? x_A2B2 =array_x2[0:j]-array_x1[0:j]
? ? ? ? ? ? y_A2B2 =array_y2[0:j]-array_y1[0:j]
? ? ? ? ? ? x_A1B2 =array_x2[0:j]-array_x1[j]
? ? ? ? ? ? y_A1B2 =array_y2[0:j]-array_y1[j]
? ? ? ? ? ? x_A2B1 =array_x2[j]-array_x1[0:j]
? ? ? ? ? ? y_A2B1 =array_y2[j]-array_y1[0:j]

? ? ? ? ? ? p1 =x_A1B1*y_A1A2 -y_A1B1*x_A1A2
? ? ? ? ? ? p2 =x_A1B1*y_A1B2 -y_A1B1*x_A1B2
? ? ? ? ? ? p3 =x_A2B2*y_A2B1 -y_A2B2*x_A2B1
? ? ? ? ? ? p4 =x_A2B2*y_A2A1 -y_A2B2*x_A2A1

? ? ? ? ? ? condition_1=p1*p2
? ? ? ? ? ? condition_2=p3*p4 ? ? ? ? ? ? ? ? ? ? ? ? 

? ? ? ? ? ? fork inxrange (j):
? ? ? ? ? ? ? ? ifcondicion_1[k]<=0andcondicion_2[k]<=0:
? ? ? ? ? ? ? ? ? ? M[j,k]=1

? ? ? ? ifj+1<N+4:
? ? ? ? ? ? x_A1B1 =array_x2[j]-array_x1[j]
? ? ? ? ? ? y_A1B1 =array_y2[j]-array_y1[j]
? ? ? ? ? ? x_A1A2 =array_x1[j+1:]-array_x1[j]
? ? ? ? ? ? y_A1A2 =array_y1[j+1:]-array_y1[j]? ? 
? ? ? ? ? ? x_A2A1 =-1*x_A1A2
? ? ? ? ? ? y_A2A1 =-1*y_A1A2
? ? ? ? ? ? x_A2B2 =array_x2[j+1:]-array_x1[j+1:]
? ? ? ? ? ? y_A2B2 =array_y2[j+1:]-array_y1[j+1:]
? ? ? ? ? ? x_A1B2 =array_x2[j+1:]-array_x1[j]
? ? ? ? ? ? y_A1B2 =array_y2[j+1:]-array_y1[j]
? ? ? ? ? ? x_A2B1 =array_x2[j]-array_x1[j+1:]
? ? ? ? ? ? y_A2B1 =array_y2[j]-array_y1[j+1:]

? ? ? ? ? ? p1 =x_A1B1*y_A1A2 -y_A1B1*x_A1A2
? ? ? ? ? ? p2 =x_A1B1*y_A1B2 -y_A1B1*x_A1B2
? ? ? ? ? ? p3 =x_A2B2*y_A2B1 -y_A2B2*x_A2B1
? ? ? ? ? ? p4 =x_A2B2*y_A2A1 -y_A2B2*x_A2A1

? ? ? ? ? ? condicion_1=p1*p2
? ? ? ? ? ? condicion_2=p3*p4 ? ? ? ? ? ? ? ? ? ? ? ? 

? ? ? ? ? ? fork inxrange (N-j-1):
? ? ? ? ? ? ? ? ifcondicion_1[k]<=0andcondicion_2[k]<=0:
? ? ? ? ? ? ? ? ? ? M[j,k+j+1]=1

Here, the element Mij=1 if stick i intersect stick j and Mij=0 if not.
How can i optimize my algorithm? Graph theory is usefull in this case?
Waiting for your answers.
Thanks a lot!
Best regards
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130823/1e8597c1/attachment.html>

From matthew.brett at gmail.com  Sat Aug 24 00:34:44 2013
From: matthew.brett at gmail.com (Matthew Brett)
Date: Fri, 23 Aug 2013 21:34:44 -0700
Subject: [Numpy-discussion] OS X binaries for releases
In-Reply-To: <CAH6Pt5oExOUH8m2gvwFCPcEqAjr9qBRYJYr+yK5YMTxnLLA29g@mail.gmail.com>
References: <CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ@mail.gmail.com>
	<rowen-5A7B80.12141122082013@news.gmane.org>
	<CAH6Pt5o32Otdhk2Ms5Cy5Zo=Mn48H8X2WbsWK92etUb4MmrooQ@mail.gmail.com>
	<rowen-F698F0.13322823082013@news.gmane.org>
	<CAH6Pt5oExOUH8m2gvwFCPcEqAjr9qBRYJYr+yK5YMTxnLLA29g@mail.gmail.com>
Message-ID: <CAH6Pt5r99Qj3gAe7sc=wtGkosfUGp0zjMjLRfj8k3Rq=RYVLfw@mail.gmail.com>

Hi,

On Fri, Aug 23, 2013 at 1:38 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
> Hi,
>
> On Fri, Aug 23, 2013 at 1:32 PM, Russell E. Owen <rowen at uw.edu> wrote:
>> In article
>> <CAH6Pt5o32Otdhk2Ms5Cy5Zo=Mn48H8X2WbsWK92etUb4MmrooQ at mail.gmail.com>,
>>  Matthew Brett <matthew.brett at gmail.com> wrote:
>>
>>> Hi,
>>>
>>> On Thu, Aug 22, 2013 at 12:14 PM, Russell E. Owen <rowen at uw.edu> wrote:
>>> > In article
>>> > <CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ at mail.gmail.com>,
>>> >  Ralf Gommers <ralf.gommers at gmail.com> wrote:
>>> >
>>> >> Hi all,
>>> >>
>>> >> Building binaries for releases is currently quite complex and
>>> >> time-consuming. For OS X we need two different machines, because we still
>>> >> provide binaries for OS X 10.5 and PPC machines. I propose to not do this
>>> >> anymore. It doesn't mean we completely drop support for 10.5 and PPC, just
>>> >> that we don't produce binaries. PPC was phased out in 2006 and OS X 10.6
>>> >> came out in 2009, so there can't be a lot of demand for it (and the
>>> >> download stats at
>>> >> http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/confirm this).
>>> >>
>>> >> Furthermore I propose to not provide 2.6 binaries anymore. Downloads of 2.6
>>> >> OS X binaries were <5% of the 2.7 ones. We did the same with 2.4 for a long
>>> >> time - support it but no binaries.
>>> >>
>>> >> So what we'd have left at the moment is only the 64-bit/32-bit universal
>>> >> binary for 10.6 and up. What we finally need to add is 3.x OS X binaries.
>>> >> We can make an attempt to build these on 10.8 - since we have access to a
>>> >> hosted 10.8 Mac Mini it would allow all devs to easily do a release
>>> >> (leaving aside the Windows issue). If anyone has tried the 10.6 SDK on 10.8
>>> >> and knows if it actually works, that would be helpful.
>>> >>
>>> >> Any concerns, objections?
>>> >
>>> > I am in strong agreement.
>>> >
>>> > I'll be interested to learn how you make binary installers for python
>>> > 3.x because the standard version of bdist_mpkg will not do it. I have
>>> > heard of two other projects (forks or variants of bdist_mpkg) that will,
>>> > but I have no idea of either is supported.
>>>
>>> I think I'm the owner of one of the forks; I supporting it, but I
>>> should certainly make a release soon too.
>>
>> That sounds promising. Can you suggest a non-released commit that is
>> stable enough to try, or should we wait for a release?
>
> It has hardly changed since the Python 3 port - the current head
> should be fine, I'm using it for our installers.  But I will get to a
> release soon.

I did a release : https://pypi.python.org/pypi/bdist_mpkg/

Please let me know if you hit any problems...

Cheers,

Matthew


From tom.bennett at mail.zyzhu.net  Sat Aug 24 13:13:52 2013
From: tom.bennett at mail.zyzhu.net (Tom Bennett)
Date: Sat, 24 Aug 2013 12:13:52 -0500
Subject: [Numpy-discussion] numpy dot returns [nan nan nan]
Message-ID: <CAEYEPaQrRTcKCYw0FPDNhUfyirVbiGCuRyuP32F5vLnrx4HxdQ@mail.gmail.com>

Hi All,

I have two arrays, A and B.A is 3 x 100,000 and B is 100,000. If I do
np.dot(A,B), I get [nan, nan, nan].

However, np.any(np.isnan(A))==False and np.any(no.isnan(B))==False. And
also np.seterr(all='print') does not print anything.

I am not wondering what is going on and how to avoid.

In case it is important, A and B are from the normal equation of doing
regression. I am regressing 100,000 observations on 3 100,000 long factors.

Thanks,
Tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130824/77b7b7f6/attachment.html>

From warren.weckesser at gmail.com  Sat Aug 24 13:27:46 2013
From: warren.weckesser at gmail.com (Warren Weckesser)
Date: Sat, 24 Aug 2013 13:27:46 -0400
Subject: [Numpy-discussion] numpy dot returns [nan nan nan]
In-Reply-To: <CAEYEPaQrRTcKCYw0FPDNhUfyirVbiGCuRyuP32F5vLnrx4HxdQ@mail.gmail.com>
References: <CAEYEPaQrRTcKCYw0FPDNhUfyirVbiGCuRyuP32F5vLnrx4HxdQ@mail.gmail.com>
Message-ID: <CAGzF1ueJFvGr+hwgF=RfqWK_oZydohwaPOVQQAkzH5Dz0=OYnw@mail.gmail.com>

On 8/24/13, Tom Bennett <tom.bennett at mail.zyzhu.net> wrote:
> Hi All,
>
> I have two arrays, A and B.A is 3 x 100,000 and B is 100,000. If I do
> np.dot(A,B), I get [nan, nan, nan].
>
> However, np.any(np.isnan(A))==False and np.any(no.isnan(B))==False. And
> also np.seterr(all='print') does not print anything.
>
> I am not wondering what is going on and how to avoid.
>
> In case it is important, A and B are from the normal equation of doing
> regression. I am regressing 100,000 observations on 3 100,000 long factors.
>
> Thanks,
> Tom
>

What are the data types of the arrays, and what are the typical sizes
of the values in these arrays?  I can get all nans from np.dot if the
values are huge floating point values:

```
In [79]: x = 1e160*np.random.randn(3, 10)

In [80]: y = 1e160*np.random.randn(10)

In [81]: np.dot(x, y)
Out[81]: array([ nan,  nan,  nan])
```

Warren


From warren.weckesser at gmail.com  Sat Aug 24 13:39:53 2013
From: warren.weckesser at gmail.com (Warren Weckesser)
Date: Sat, 24 Aug 2013 13:39:53 -0400
Subject: [Numpy-discussion] numpy dot returns [nan nan nan]
In-Reply-To: <CAGzF1ueJFvGr+hwgF=RfqWK_oZydohwaPOVQQAkzH5Dz0=OYnw@mail.gmail.com>
References: <CAEYEPaQrRTcKCYw0FPDNhUfyirVbiGCuRyuP32F5vLnrx4HxdQ@mail.gmail.com>
	<CAGzF1ueJFvGr+hwgF=RfqWK_oZydohwaPOVQQAkzH5Dz0=OYnw@mail.gmail.com>
Message-ID: <CAGzF1uf+zfCDry0X749uAZEnDa4LG+SsVHzx-g8su4w6yQdptg@mail.gmail.com>

On 8/24/13, Warren Weckesser <warren.weckesser at gmail.com> wrote:
> On 8/24/13, Tom Bennett <tom.bennett at mail.zyzhu.net> wrote:
>> Hi All,
>>
>> I have two arrays, A and B.A is 3 x 100,000 and B is 100,000. If I do
>> np.dot(A,B), I get [nan, nan, nan].
>>
>> However, np.any(np.isnan(A))==False and np.any(no.isnan(B))==False. And
>> also np.seterr(all='print') does not print anything.
>>
>> I am not wondering what is going on and how to avoid.
>>
>> In case it is important, A and B are from the normal equation of doing
>> regression. I am regressing 100,000 observations on 3 100,000 long
>> factors.
>>
>> Thanks,
>> Tom
>>
>
> What are the data types of the arrays, and what are the typical sizes
> of the values in these arrays?  I can get all nans from np.dot if the
> values are huge floating point values:
>
> ```
> In [79]: x = 1e160*np.random.randn(3, 10)
>
> In [80]: y = 1e160*np.random.randn(10)
>
> In [81]: np.dot(x, y)
> Out[81]: array([ nan,  nan,  nan])
> ```

...and that happens because some intermediate terms overflow to inf or
-inf, and adding these gives nan:

```
In [89]: x = np.array([1e300])

In [90]: y = np.array([1e10])

In [91]: np.dot(x,y)
Out[91]: inf

In [92]: x2 = np.array([1e300, 1e300])

In [93]: y2 = np.array([1e10,-1e10])

In [94]: np.dot(x2, y2)
Out[94]: nan
```

Warren


>
> Warren
>


From tom.bennett at mail.zyzhu.net  Sat Aug 24 14:00:50 2013
From: tom.bennett at mail.zyzhu.net (Tom Bennett)
Date: Sat, 24 Aug 2013 13:00:50 -0500
Subject: [Numpy-discussion] numpy dot returns [nan nan nan]
In-Reply-To: <CAGzF1uf+zfCDry0X749uAZEnDa4LG+SsVHzx-g8su4w6yQdptg@mail.gmail.com>
References: <CAEYEPaQrRTcKCYw0FPDNhUfyirVbiGCuRyuP32F5vLnrx4HxdQ@mail.gmail.com>
	<CAGzF1ueJFvGr+hwgF=RfqWK_oZydohwaPOVQQAkzH5Dz0=OYnw@mail.gmail.com>
	<CAGzF1uf+zfCDry0X749uAZEnDa4LG+SsVHzx-g8su4w6yQdptg@mail.gmail.com>
Message-ID: <CAEYEPaT9J1KP9paUrG=np1B50EsE=TL1HTWQ=7K4HHuDr1PL+g@mail.gmail.com>

Hi Warren,

Yes you are absolutely right. I had some values close to log(x), where x is
almost 0. That caused the problem.

Thanks,
Tom


On Sat, Aug 24, 2013 at 12:39 PM, Warren Weckesser <
warren.weckesser at gmail.com> wrote:

> On 8/24/13, Warren Weckesser <warren.weckesser at gmail.com> wrote:
> > On 8/24/13, Tom Bennett <tom.bennett at mail.zyzhu.net> wrote:
> >> Hi All,
> >>
> >> I have two arrays, A and B.A is 3 x 100,000 and B is 100,000. If I do
> >> np.dot(A,B), I get [nan, nan, nan].
> >>
> >> However, np.any(np.isnan(A))==False and np.any(no.isnan(B))==False. And
> >> also np.seterr(all='print') does not print anything.
> >>
> >> I am not wondering what is going on and how to avoid.
> >>
> >> In case it is important, A and B are from the normal equation of doing
> >> regression. I am regressing 100,000 observations on 3 100,000 long
> >> factors.
> >>
> >> Thanks,
> >> Tom
> >>
> >
> > What are the data types of the arrays, and what are the typical sizes
> > of the values in these arrays?  I can get all nans from np.dot if the
> > values are huge floating point values:
> >
> > ```
> > In [79]: x = 1e160*np.random.randn(3, 10)
> >
> > In [80]: y = 1e160*np.random.randn(10)
> >
> > In [81]: np.dot(x, y)
> > Out[81]: array([ nan,  nan,  nan])
> > ```
>
> ...and that happens because some intermediate terms overflow to inf or
> -inf, and adding these gives nan:
>
> ```
> In [89]: x = np.array([1e300])
>
> In [90]: y = np.array([1e10])
>
> In [91]: np.dot(x,y)
> Out[91]: inf
>
> In [92]: x2 = np.array([1e300, 1e300])
>
> In [93]: y2 = np.array([1e10,-1e10])
>
> In [94]: np.dot(x2, y2)
> Out[94]: nan
> ```
>
> Warren
>
>
> >
> > Warren
> >
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130824/1f4c8a58/attachment.html>

From warren.weckesser at gmail.com  Sat Aug 24 14:25:48 2013
From: warren.weckesser at gmail.com (Warren Weckesser)
Date: Sat, 24 Aug 2013 14:25:48 -0400
Subject: [Numpy-discussion] numpy dot returns [nan nan nan]
In-Reply-To: <CAEYEPaT9J1KP9paUrG=np1B50EsE=TL1HTWQ=7K4HHuDr1PL+g@mail.gmail.com>
References: <CAEYEPaQrRTcKCYw0FPDNhUfyirVbiGCuRyuP32F5vLnrx4HxdQ@mail.gmail.com>
	<CAGzF1ueJFvGr+hwgF=RfqWK_oZydohwaPOVQQAkzH5Dz0=OYnw@mail.gmail.com>
	<CAGzF1uf+zfCDry0X749uAZEnDa4LG+SsVHzx-g8su4w6yQdptg@mail.gmail.com>
	<CAEYEPaT9J1KP9paUrG=np1B50EsE=TL1HTWQ=7K4HHuDr1PL+g@mail.gmail.com>
Message-ID: <CAGzF1ucWOu+VfiEfh93O6P9jEQThVEW6p_GEc9nwEJ9Vesws_A@mail.gmail.com>

On 8/24/13, Tom Bennett <tom.bennett at mail.zyzhu.net> wrote:
> Hi Warren,
>
> Yes you are absolutely right. I had some values close to log(x), where x is
> almost 0. That caused the problem.
>
> Thanks,
> Tom


Now the question is: why does `np.dot` mask the overflow warning?

In numpy 1.7.1, the default is that overflow should generate a warning:

In [1]: np.seterr()
Out[1]: {'divide': 'warn', 'invalid': 'warn', 'over': 'warn', 'under': 'ignore'}


But `np.dot` does not generate a warning:

In [2]: x = np.array([1e300])

In [3]: y = np.array([1e10])

In [4]: np.dot(x, y)
Out[4]: inf


Multiplying `x` and `y` generates the warning, as expected:

In [5]: x*y
/home/warren/anaconda/bin/ipython:1: RuntimeWarning: overflow
encountered in multiply
  #!/home/warren/anaconda/bin/python
Out[5]: array([ inf])


Warren


>
>
> On Sat, Aug 24, 2013 at 12:39 PM, Warren Weckesser <
> warren.weckesser at gmail.com> wrote:
>
>> On 8/24/13, Warren Weckesser <warren.weckesser at gmail.com> wrote:
>> > On 8/24/13, Tom Bennett <tom.bennett at mail.zyzhu.net> wrote:
>> >> Hi All,
>> >>
>> >> I have two arrays, A and B.A is 3 x 100,000 and B is 100,000. If I do
>> >> np.dot(A,B), I get [nan, nan, nan].
>> >>
>> >> However, np.any(np.isnan(A))==False and np.any(no.isnan(B))==False.
>> >> And
>> >> also np.seterr(all='print') does not print anything.
>> >>
>> >> I am not wondering what is going on and how to avoid.
>> >>
>> >> In case it is important, A and B are from the normal equation of doing
>> >> regression. I am regressing 100,000 observations on 3 100,000 long
>> >> factors.
>> >>
>> >> Thanks,
>> >> Tom
>> >>
>> >
>> > What are the data types of the arrays, and what are the typical sizes
>> > of the values in these arrays?  I can get all nans from np.dot if the
>> > values are huge floating point values:
>> >
>> > ```
>> > In [79]: x = 1e160*np.random.randn(3, 10)
>> >
>> > In [80]: y = 1e160*np.random.randn(10)
>> >
>> > In [81]: np.dot(x, y)
>> > Out[81]: array([ nan,  nan,  nan])
>> > ```
>>
>> ...and that happens because some intermediate terms overflow to inf or
>> -inf, and adding these gives nan:
>>
>> ```
>> In [89]: x = np.array([1e300])
>>
>> In [90]: y = np.array([1e10])
>>
>> In [91]: np.dot(x,y)
>> Out[91]: inf
>>
>> In [92]: x2 = np.array([1e300, 1e300])
>>
>> In [93]: y2 = np.array([1e10,-1e10])
>>
>> In [94]: np.dot(x2, y2)
>> Out[94]: nan
>> ```
>>
>> Warren
>>
>>
>> >
>> > Warren
>> >
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>


From tim at cerazone.net  Sun Aug 25 20:30:47 2013
From: tim at cerazone.net (Cera, Tim)
Date: Sun, 25 Aug 2013 20:30:47 -0400
Subject: [Numpy-discussion] Strange behavior with boolean slices...
Message-ID: <CAO5s+D9DeRKsm=a=knnw8HeD4gERXtZSET34T8V+FbaZ9aaT8A@mail.gmail.com>

I have done this before, but am now really confused.

Created an array 'day' specifying the 'f' type

In [29]: day
Out[29]: array([ 5.,  5.], dtype=float32)

# Have a mask...
In [30]: mask
Out[30]: array([ True, False], dtype=bool)

# So far, so good...
In [31]: day[mask]
Out[31]: array([ 5.], dtype=float32)

In [32]: day[mask] = 10

# What?
In [33]: day
Out[33]: array([ 10.,  10.], dtype=float32)


So I created an integer array 'a'

In [38]: a
Out[38]: array([11,  1])

In [39]: a[mask]
Out[39]: array([11])

In [40]: a[mask] = 12

# This is what I expect.
In [41]: a
Out[41]: array([12,  1])

Am I missing something?  Is this supposed to happen?

Version 1.7.1.

Kindest regards,
Tim


From efiring at hawaii.edu  Sun Aug 25 20:44:30 2013
From: efiring at hawaii.edu (Eric Firing)
Date: Sun, 25 Aug 2013 14:44:30 -1000
Subject: [Numpy-discussion] Strange behavior with boolean slices...
In-Reply-To: <CAO5s+D9DeRKsm=a=knnw8HeD4gERXtZSET34T8V+FbaZ9aaT8A@mail.gmail.com>
References: <CAO5s+D9DeRKsm=a=knnw8HeD4gERXtZSET34T8V+FbaZ9aaT8A@mail.gmail.com>
Message-ID: <521AA4EE.7000601@hawaii.edu>

On 2013/08/25 2:30 PM, Cera, Tim wrote:
> I have done this before, but am now really confused.
>
> Created an array 'day' specifying the 'f' type
>
> In [29]: day
> Out[29]: array([ 5.,  5.], dtype=float32)
>
> # Have a mask...
> In [30]: mask
> Out[30]: array([ True, False], dtype=bool)
>
> # So far, so good...
> In [31]: day[mask]
> Out[31]: array([ 5.], dtype=float32)
>
> In [32]: day[mask] = 10
>
> # What?
> In [33]: day
> Out[33]: array([ 10.,  10.], dtype=float32)

I'm not getting that with 1.7.0:
In [2]: np.__version__
Out[2]: '1.7.0'

In [3]: mask = np.array([True, False], dtype=bool)

In [4]: day = np.array([5, 5], dtype=np.float32)

In [5]: day
Out[5]: array([ 5.,  5.], dtype=float32)

In [6]: mask
Out[6]: array([ True, False], dtype=bool)

In [7]: day[mask]
Out[7]: array([ 5.], dtype=float32)

In [8]: day[mask] = 10

In [9]: day
Out[9]: array([ 10.,   5.], dtype=float32)

Eric


>
>
> So I created an integer array 'a'
>
> In [38]: a
> Out[38]: array([11,  1])
>
> In [39]: a[mask]
> Out[39]: array([11])
>
> In [40]: a[mask] = 12
>
> # This is what I expect.
> In [41]: a
> Out[41]: array([12,  1])
>
> Am I missing something?  Is this supposed to happen?
>
> Version 1.7.1.
>
> Kindest regards,
> Tim
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>



From efiring at hawaii.edu  Sun Aug 25 20:48:19 2013
From: efiring at hawaii.edu (Eric Firing)
Date: Sun, 25 Aug 2013 14:48:19 -1000
Subject: [Numpy-discussion] Strange behavior with boolean slices...
In-Reply-To: <CAO5s+D9DeRKsm=a=knnw8HeD4gERXtZSET34T8V+FbaZ9aaT8A@mail.gmail.com>
References: <CAO5s+D9DeRKsm=a=knnw8HeD4gERXtZSET34T8V+FbaZ9aaT8A@mail.gmail.com>
Message-ID: <521AA5D3.7000003@hawaii.edu>

On 2013/08/25 2:30 PM, Cera, Tim wrote:
> I have done this before, but am now really confused.
>
> Created an array 'day' specifying the 'f' type
>
> In [29]: day
> Out[29]: array([ 5.,  5.], dtype=float32)
>
> # Have a mask...
> In [30]: mask
> Out[30]: array([ True, False], dtype=bool)
>
> # So far, so good...
> In [31]: day[mask]
> Out[31]: array([ 5.], dtype=float32)
>
> In [32]: day[mask] = 10
>
> # What?
> In [33]: day
> Out[33]: array([ 10.,  10.], dtype=float32)
>

I don't get it with 1.7.1, either:

In [2]: np.__version__
Out[2]: '1.7.1'

In [3]: %paste
mask = np.array([True, False], dtype=bool)
day = np.array([5, 5], dtype=np.float32)
day
mask
day[mask]
day[mask] = 10
day
## -- End pasted text --
Out[3]: array([ 10.,   5.], dtype=float32)

My 1.7.0 example is on a Mac, the 1.7.1 is on a Linux virtual machine, 
both 64-bit.

Eric

>
> So I created an integer array 'a'
>
> In [38]: a
> Out[38]: array([11,  1])
>
> In [39]: a[mask]
> Out[39]: array([11])
>
> In [40]: a[mask] = 12
>
> # This is what I expect.
> In [41]: a
> Out[41]: array([12,  1])
>
> Am I missing something?  Is this supposed to happen?
>
> Version 1.7.1.
>
> Kindest regards,
> Tim
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>



From tim at cerazone.net  Sun Aug 25 20:53:05 2013
From: tim at cerazone.net (Cera, Tim)
Date: Sun, 25 Aug 2013 20:53:05 -0400
Subject: [Numpy-discussion] Strange behavior with boolean slices...
In-Reply-To: <521AA4EE.7000601@hawaii.edu>
References: <CAO5s+D9DeRKsm=a=knnw8HeD4gERXtZSET34T8V+FbaZ9aaT8A@mail.gmail.com>
	<521AA4EE.7000601@hawaii.edu>
Message-ID: <CAO5s+D8Gr3XJsUVm7Lwm_2d9VHKA78F4CP+v5iGucCr-+wnhKA@mail.gmail.com>

Figured it out.  I created 'day' as a broadcast array.  Does this
catch other people?  Basically changing day[0] would change the entire
'day' array.  I guess all other elements of the day array are views of
day[0].  Made a copy and everything works as expected.

Kindest regards,
Tim

On Sun, Aug 25, 2013 at 8:44 PM, Eric Firing <efiring at hawaii.edu> wrote:
> On 2013/08/25 2:30 PM, Cera, Tim wrote:
>> I have done this before, but am now really confused.
>>
>> Created an array 'day' specifying the 'f' type
>>
>> In [29]: day
>> Out[29]: array([ 5.,  5.], dtype=float32)
>>
>> # Have a mask...
>> In [30]: mask
>> Out[30]: array([ True, False], dtype=bool)
>>
>> # So far, so good...
>> In [31]: day[mask]
>> Out[31]: array([ 5.], dtype=float32)
>>
>> In [32]: day[mask] = 10
>>
>> # What?
>> In [33]: day
>> Out[33]: array([ 10.,  10.], dtype=float32)
>
> I'm not getting that with 1.7.0:
> In [2]: np.__version__
> Out[2]: '1.7.0'
>
> In [3]: mask = np.array([True, False], dtype=bool)
>
> In [4]: day = np.array([5, 5], dtype=np.float32)
>
> In [5]: day
> Out[5]: array([ 5.,  5.], dtype=float32)
>
> In [6]: mask
> Out[6]: array([ True, False], dtype=bool)
>
> In [7]: day[mask]
> Out[7]: array([ 5.], dtype=float32)
>
> In [8]: day[mask] = 10
>
> In [9]: day
> Out[9]: array([ 10.,   5.], dtype=float32)
>
> Eric
>
>
>>
>>
>> So I created an integer array 'a'
>>
>> In [38]: a
>> Out[38]: array([11,  1])
>>
>> In [39]: a[mask]
>> Out[39]: array([11])
>>
>> In [40]: a[mask] = 12
>>
>> # This is what I expect.
>> In [41]: a
>> Out[41]: array([12,  1])
>>
>> Am I missing something?  Is this supposed to happen?
>>
>> Version 1.7.1.
>>
>> Kindest regards,
>> Tim
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion


From tim at cerazone.net  Sun Aug 25 21:06:35 2013
From: tim at cerazone.net (Cera, Tim)
Date: Sun, 25 Aug 2013 21:06:35 -0400
Subject: [Numpy-discussion] Strange behavior with boolean slices...
In-Reply-To: <CAO5s+D8Gr3XJsUVm7Lwm_2d9VHKA78F4CP+v5iGucCr-+wnhKA@mail.gmail.com>
References: <CAO5s+D9DeRKsm=a=knnw8HeD4gERXtZSET34T8V+FbaZ9aaT8A@mail.gmail.com>
	<521AA4EE.7000601@hawaii.edu>
	<CAO5s+D8Gr3XJsUVm7Lwm_2d9VHKA78F4CP+v5iGucCr-+wnhKA@mail.gmail.com>
Message-ID: <CAO5s+D9SJJO9p3yhzgwpEH9HMjkTtaKXcs+_i+k-jjTsqSdwCQ@mail.gmail.com>

Pardon the noise.  The behavior is described right there in the
documentation of broadcast_arrays.

Kindest regards,
Tim

On Sun, Aug 25, 2013 at 8:53 PM, Cera, Tim <tim at cerazone.net> wrote:
> Figured it out.  I created 'day' as a broadcast array.  Does this
> catch other people?  Basically changing day[0] would change the entire
> 'day' array.  I guess all other elements of the day array are views of
> day[0].  Made a copy and everything works as expected.
>
> Kindest regards,
> Tim
>
> On Sun, Aug 25, 2013 at 8:44 PM, Eric Firing <efiring at hawaii.edu> wrote:
>> On 2013/08/25 2:30 PM, Cera, Tim wrote:
>>> I have done this before, but am now really confused.
>>>
>>> Created an array 'day' specifying the 'f' type
>>>
>>> In [29]: day
>>> Out[29]: array([ 5.,  5.], dtype=float32)
>>>
>>> # Have a mask...
>>> In [30]: mask
>>> Out[30]: array([ True, False], dtype=bool)
>>>
>>> # So far, so good...
>>> In [31]: day[mask]
>>> Out[31]: array([ 5.], dtype=float32)
>>>
>>> In [32]: day[mask] = 10
>>>
>>> # What?
>>> In [33]: day
>>> Out[33]: array([ 10.,  10.], dtype=float32)
>>
>> I'm not getting that with 1.7.0:
>> In [2]: np.__version__
>> Out[2]: '1.7.0'
>>
>> In [3]: mask = np.array([True, False], dtype=bool)
>>
>> In [4]: day = np.array([5, 5], dtype=np.float32)
>>
>> In [5]: day
>> Out[5]: array([ 5.,  5.], dtype=float32)
>>
>> In [6]: mask
>> Out[6]: array([ True, False], dtype=bool)
>>
>> In [7]: day[mask]
>> Out[7]: array([ 5.], dtype=float32)
>>
>> In [8]: day[mask] = 10
>>
>> In [9]: day
>> Out[9]: array([ 10.,   5.], dtype=float32)
>>
>> Eric
>>
>>
>>>
>>>
>>> So I created an integer array 'a'
>>>
>>> In [38]: a
>>> Out[38]: array([11,  1])
>>>
>>> In [39]: a[mask]
>>> Out[39]: array([11])
>>>
>>> In [40]: a[mask] = 12
>>>
>>> # This is what I expect.
>>> In [41]: a
>>> Out[41]: array([12,  1])
>>>
>>> Am I missing something?  Is this supposed to happen?
>>>
>>> Version 1.7.1.
>>>
>>> Kindest regards,
>>> Tim
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion


From ralf.gommers at gmail.com  Mon Aug 26 09:28:59 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Mon, 26 Aug 2013 13:28:59 +0000
Subject: [Numpy-discussion] OS X binaries for releases
In-Reply-To: <CAH6Pt5o32Otdhk2Ms5Cy5Zo=Mn48H8X2WbsWK92etUb4MmrooQ@mail.gmail.com>
References: <CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ@mail.gmail.com>
	<rowen-5A7B80.12141122082013@news.gmane.org>
	<CAH6Pt5o32Otdhk2Ms5Cy5Zo=Mn48H8X2WbsWK92etUb4MmrooQ@mail.gmail.com>
Message-ID: <CABL7CQg1KYiz9CoK=Pp3oYPdgoCoB5gNPCNrV9h3fX_neF87UQ@mail.gmail.com>

On Thu, Aug 22, 2013 at 10:19 PM, Matthew Brett <matthew.brett at gmail.com>wrote:

> Hi,
>
> On Thu, Aug 22, 2013 at 12:14 PM, Russell E. Owen <rowen at uw.edu> wrote:
> > In article
> > <CABL7CQjaCXp2GrtT8HVmaYAjRm0xmtn1Qt71WKdnbGq7dLU0cQ at mail.gmail.com>,
> >  Ralf Gommers <ralf.gommers at gmail.com> wrote:
> >
> >> Hi all,
> >>
> >> Building binaries for releases is currently quite complex and
> >> time-consuming. For OS X we need two different machines, because we
> still
> >> provide binaries for OS X 10.5 and PPC machines. I propose to not do
> this
> >> anymore. It doesn't mean we completely drop support for 10.5 and PPC,
> just
> >> that we don't produce binaries. PPC was phased out in 2006 and OS X 10.6
> >> came out in 2009, so there can't be a lot of demand for it (and the
> >> download stats at
> >> http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/confirm this).
> >>
> >> Furthermore I propose to not provide 2.6 binaries anymore. Downloads of
> 2.6
> >> OS X binaries were <5% of the 2.7 ones. We did the same with 2.4 for a
> long
> >> time - support it but no binaries.
> >>
> >> So what we'd have left at the moment is only the 64-bit/32-bit universal
> >> binary for 10.6 and up. What we finally need to add is 3.x OS X
> binaries.
> >> We can make an attempt to build these on 10.8 - since we have access to
> a
> >> hosted 10.8 Mac Mini it would allow all devs to easily do a release
> >> (leaving aside the Windows issue). If anyone has tried the 10.6 SDK on
> 10.8
> >> and knows if it actually works, that would be helpful.
> >>
> >> Any concerns, objections?
> >
> > I am in strong agreement.
> >
> > I'll be interested to learn how you make binary installers for python
> > 3.x because the standard version of bdist_mpkg will not do it. I have
> > heard of two other projects (forks or variants of bdist_mpkg) that will,
> > but I have no idea of either is supported.
>
> I think I'm the owner of one of the forks; I supporting it, but I
> should certainly make a release soon too.
>
> > I have been able to building packages on 10.8 using
> > MACOSX_DEPLOYMENT_TARGET=10.6 that will run on 10.6, so it will probably
> > work. However I have run into several odd problems over the years
> > building a binary installer on a newer system only to find it won't work
> > on older systems for various reasons. Thus my personal recommendation is
> > that you build on 10.6 if you want an installer that reliably works for
> > 10.6 and later.
>

That has been my experience as well, unfortunately. I was just willing to
try again with 10.8 --> 10.6.


> I keep an older computer around for this reason. In fact
> > that is one good reason to drop support for ancient operating systems
> > and PPC.
>
> I'm sitting next to a 10.6 machine you are welcome to use; just let me
> know, I'll give you login access.
>

I have a 10.6 machine as well, so for me it's not an issue. The aim is to
make it easy for others to build binaries, and there's a hosted 10.8
machine available to use for all numpy/scipy devs.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130826/678c30b8/attachment.html>

From ralf.gommers at gmail.com  Mon Aug 26 11:01:07 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Mon, 26 Aug 2013 15:01:07 +0000
Subject: [Numpy-discussion] 1.8.0 branch reminder
In-Reply-To: <CAB6mnxLrzFM=2Rt4u137URt6gbRo1wnBXxLvNO0mZK7GP106vA@mail.gmail.com>
References: <CAB6mnx+G5JY-szhVH__-sFv=YF6bJ1JvCFQqHm=Nih_-9ACMjg@mail.gmail.com>
	<CAB6mnxLrzFM=2Rt4u137URt6gbRo1wnBXxLvNO0mZK7GP106vA@mail.gmail.com>
Message-ID: <CABL7CQj7a=q9YwKSn781nOb+xUGa8MbLBBKDkro+u-CYQ3ZCmw@mail.gmail.com>

On Sun, Aug 18, 2013 at 6:36 PM, Charles R Harris <charlesr.harris at gmail.com
> wrote:

>
>
>
> On Sun, Aug 18, 2013 at 12:17 PM, Charles R Harris <
> charlesr.harris at gmail.com> wrote:
>
>> Just a reminder that 1.8.0 will be branched tonight. I've put up a big STY:
>> PR <https://github.com/numpy/numpy/pull/3635> that removes trailing
>> whitespace and fixes spacing after commas. I would like to apply before the
>> branch, but it may cause merge difficulties down the line. I'd like
>> feedback on that option.
>>
>>
> I've also run autopep8 on the code and it does a nice job of cleaning up
> things. It gets a little lost in deeply nested lists, but there aren't too
> many of those. By default it doesn't fix spaces about operator (it seems).
> I can apply that also if there is interest in doing so.
>

Depends on how many lines of code it touches. For scipy we decided not to
do this, because it would make "git blame" pretty much useless.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130826/b8dae121/attachment.html>

From ben.root at ou.edu  Mon Aug 26 11:49:29 2013
From: ben.root at ou.edu (Benjamin Root)
Date: Mon, 26 Aug 2013 11:49:29 -0400
Subject: [Numpy-discussion] 1.8.0 branch reminder
In-Reply-To: <CABL7CQj7a=q9YwKSn781nOb+xUGa8MbLBBKDkro+u-CYQ3ZCmw@mail.gmail.com>
References: <CAB6mnx+G5JY-szhVH__-sFv=YF6bJ1JvCFQqHm=Nih_-9ACMjg@mail.gmail.com>
	<CAB6mnxLrzFM=2Rt4u137URt6gbRo1wnBXxLvNO0mZK7GP106vA@mail.gmail.com>
	<CABL7CQj7a=q9YwKSn781nOb+xUGa8MbLBBKDkro+u-CYQ3ZCmw@mail.gmail.com>
Message-ID: <CANNq6Fm=6tB=2FFOuM3yagz0==+ULfXjp2MHb2ZuQEHdtF2UVQ@mail.gmail.com>

On Mon, Aug 26, 2013 at 11:01 AM, Ralf Gommers <ralf.gommers at gmail.com>wrote:

>
>
>
> On Sun, Aug 18, 2013 at 6:36 PM, Charles R Harris <
> charlesr.harris at gmail.com> wrote:
>
>>
>>
>>
>> On Sun, Aug 18, 2013 at 12:17 PM, Charles R Harris <
>> charlesr.harris at gmail.com> wrote:
>>
>>> Just a reminder that 1.8.0 will be branched tonight. I've put up a big STY:
>>> PR <https://github.com/numpy/numpy/pull/3635> that removes trailing
>>> whitespace and fixes spacing after commas. I would like to apply before the
>>> branch, but it may cause merge difficulties down the line. I'd like
>>> feedback on that option.
>>>
>>>
>> I've also run autopep8 on the code and it does a nice job of cleaning up
>> things. It gets a little lost in deeply nested lists, but there aren't too
>> many of those. By default it doesn't fix spaces about operator (it seems).
>> I can apply that also if there is interest in doing so.
>>
>
> Depends on how many lines of code it touches. For scipy we decided not to
> do this, because it would make "git blame" pretty much useless.
>
> Ralf
>

At some point, you just have to bite the bullet. Matplotlib has been doing
pep8 work for about a year now. We adopted very specific rules on how that
work was to be done (make pep8 only PRs, each pep8 PR would be for at most
one module at a time, etc). Yes, it does look like NelleV has taken over
the project, but the trade-off is readability. We even discovered a
non-trivial number of bugs this way. For a core library like NumPy that has
lots of very obscure-looking code that almost never gets changed, avoiding
PEP8 is problematic because it always becomes "Somebody else's problem".

Of course, it is entirely up to you, the devs, on what to do for NumPy and
SciPy, but that is what matplotlib is doing.

Cheers!
Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130826/82aff418/attachment.html>

From ralf.gommers at gmail.com  Mon Aug 26 12:19:24 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Mon, 26 Aug 2013 16:19:24 +0000
Subject: [Numpy-discussion] 1.8.0 branch reminder
In-Reply-To: <CANNq6Fm=6tB=2FFOuM3yagz0==+ULfXjp2MHb2ZuQEHdtF2UVQ@mail.gmail.com>
References: <CAB6mnx+G5JY-szhVH__-sFv=YF6bJ1JvCFQqHm=Nih_-9ACMjg@mail.gmail.com>
	<CAB6mnxLrzFM=2Rt4u137URt6gbRo1wnBXxLvNO0mZK7GP106vA@mail.gmail.com>
	<CABL7CQj7a=q9YwKSn781nOb+xUGa8MbLBBKDkro+u-CYQ3ZCmw@mail.gmail.com>
	<CANNq6Fm=6tB=2FFOuM3yagz0==+ULfXjp2MHb2ZuQEHdtF2UVQ@mail.gmail.com>
Message-ID: <CABL7CQihme6sWOy3rBop2mqMHaFDzBGZ+hz9oUMXTAzJKgowHQ@mail.gmail.com>

On Mon, Aug 26, 2013 at 3:49 PM, Benjamin Root <ben.root at ou.edu> wrote:

>
> On Mon, Aug 26, 2013 at 11:01 AM, Ralf Gommers <ralf.gommers at gmail.com>wrote:
>
>>
>>
>>
>> On Sun, Aug 18, 2013 at 6:36 PM, Charles R Harris <
>> charlesr.harris at gmail.com> wrote:
>>
>>>
>>>
>>>
>>> On Sun, Aug 18, 2013 at 12:17 PM, Charles R Harris <
>>> charlesr.harris at gmail.com> wrote:
>>>
>>>> Just a reminder that 1.8.0 will be branched tonight. I've put up a big STY:
>>>> PR <https://github.com/numpy/numpy/pull/3635> that removes trailing
>>>> whitespace and fixes spacing after commas. I would like to apply before the
>>>> branch, but it may cause merge difficulties down the line. I'd like
>>>> feedback on that option.
>>>>
>>>>
>>> I've also run autopep8 on the code and it does a nice job of cleaning up
>>> things. It gets a little lost in deeply nested lists, but there aren't too
>>> many of those. By default it doesn't fix spaces about operator (it seems).
>>> I can apply that also if there is interest in doing so.
>>>
>>
>> Depends on how many lines of code it touches. For scipy we decided not to
>> do this, because it would make "git blame" pretty much useless.
>>
>> Ralf
>>
>
> At some point, you just have to bite the bullet. Matplotlib has been doing
> pep8 work for about a year now. We adopted very specific rules on how that
> work was to be done (make pep8 only PRs, each pep8 PR would be for at most
> one module at a time, etc). Yes, it does look like NelleV has taken over
> the project, but the trade-off is readability. We even discovered a
> non-trivial number of bugs this way. For a core library like NumPy that has
> lots of very obscure-looking code that almost never gets changed, avoiding
> PEP8 is problematic because it always becomes "Somebody else's problem".
>

We've been doing a lot of PEP8 cleanups in scipy - there's even ``tox -e
pep8`` with a limited number of exceptions. And Chuck has just done a lot
of cleanup for numpy, which is quite useful. So it's definitely not just
someone else's problem.

The question was specifically about spaces around operators, which doesn't
necessarily even make things look better (even apart from destroying
history). This:

    a*x[0] + b*x[1] + c*x[2]**2

look imho better than if you'd "fix" it like:

    a * x[0] + b * x[1] + c * x[2] ** 2

It's about finding the right balance, not about blindly running pep8 fixer
tools.

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130826/40577ac9/attachment.html>

From josef.pktd at gmail.com  Mon Aug 26 12:26:11 2013
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Mon, 26 Aug 2013 12:26:11 -0400
Subject: [Numpy-discussion] 1.8.0 branch reminder
In-Reply-To: <CABL7CQihme6sWOy3rBop2mqMHaFDzBGZ+hz9oUMXTAzJKgowHQ@mail.gmail.com>
References: <CAB6mnx+G5JY-szhVH__-sFv=YF6bJ1JvCFQqHm=Nih_-9ACMjg@mail.gmail.com>
	<CAB6mnxLrzFM=2Rt4u137URt6gbRo1wnBXxLvNO0mZK7GP106vA@mail.gmail.com>
	<CABL7CQj7a=q9YwKSn781nOb+xUGa8MbLBBKDkro+u-CYQ3ZCmw@mail.gmail.com>
	<CANNq6Fm=6tB=2FFOuM3yagz0==+ULfXjp2MHb2ZuQEHdtF2UVQ@mail.gmail.com>
	<CABL7CQihme6sWOy3rBop2mqMHaFDzBGZ+hz9oUMXTAzJKgowHQ@mail.gmail.com>
Message-ID: <CAMMTP+DM5Pkg4VFmT3B2EC_OWZxgCP3uCbW8qtw2y6e2xpLK5w@mail.gmail.com>

On Mon, Aug 26, 2013 at 12:19 PM, Ralf Gommers <ralf.gommers at gmail.com>wrote:

>
>
>
> On Mon, Aug 26, 2013 at 3:49 PM, Benjamin Root <ben.root at ou.edu> wrote:
>
>>
>> On Mon, Aug 26, 2013 at 11:01 AM, Ralf Gommers <ralf.gommers at gmail.com>wrote:
>>
>>>
>>>
>>>
>>> On Sun, Aug 18, 2013 at 6:36 PM, Charles R Harris <
>>> charlesr.harris at gmail.com> wrote:
>>>
>>>>
>>>>
>>>>
>>>> On Sun, Aug 18, 2013 at 12:17 PM, Charles R Harris <
>>>> charlesr.harris at gmail.com> wrote:
>>>>
>>>>> Just a reminder that 1.8.0 will be branched tonight. I've put up a big STY:
>>>>> PR <https://github.com/numpy/numpy/pull/3635> that removes trailing
>>>>> whitespace and fixes spacing after commas. I would like to apply before the
>>>>> branch, but it may cause merge difficulties down the line. I'd like
>>>>> feedback on that option.
>>>>>
>>>>>
>>>> I've also run autopep8 on the code and it does a nice job of cleaning
>>>> up things. It gets a little lost in deeply nested lists, but there aren't
>>>> too many of those. By default it doesn't fix spaces about operator (it
>>>> seems). I can apply that also if there is interest in doing so.
>>>>
>>>
>>> Depends on how many lines of code it touches. For scipy we decided not
>>> to do this, because it would make "git blame" pretty much useless.
>>>
>>> Ralf
>>>
>>
>> At some point, you just have to bite the bullet. Matplotlib has been
>> doing pep8 work for about a year now. We adopted very specific rules on how
>> that work was to be done (make pep8 only PRs, each pep8 PR would be for at
>> most one module at a time, etc). Yes, it does look like NelleV has taken
>> over the project, but the trade-off is readability. We even discovered a
>> non-trivial number of bugs this way. For a core library like NumPy that has
>> lots of very obscure-looking code that almost never gets changed, avoiding
>> PEP8 is problematic because it always becomes "Somebody else's problem".
>>
>
> We've been doing a lot of PEP8 cleanups in scipy - there's even ``tox -e
> pep8`` with a limited number of exceptions. And Chuck has just done a lot
> of cleanup for numpy, which is quite useful. So it's definitely not just
> someone else's problem.
>
> The question was specifically about spaces around operators, which doesn't
> necessarily even make things look better (even apart from destroying
> history). This:
>
>     a*x[0] + b*x[1] + c*x[2]**2
>
> look imho better than if you'd "fix" it like:
>
>     a * x[0] + b * x[1] + c * x[2] ** 2
>

that's not pep-8 (anymore)
http://www.python.org/dev/peps/pep-0008/#other-recommendations

Josef



>
> It's about finding the right balance, not about blindly running pep8 fixer
> tools.
>
> Cheers,
> Ralf
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130826/369e86bf/attachment.html>

From ralf.gommers at gmail.com  Mon Aug 26 12:51:46 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Mon, 26 Aug 2013 16:51:46 +0000
Subject: [Numpy-discussion] 1.8.0 branch reminder
In-Reply-To: <CAMMTP+DM5Pkg4VFmT3B2EC_OWZxgCP3uCbW8qtw2y6e2xpLK5w@mail.gmail.com>
References: <CAB6mnx+G5JY-szhVH__-sFv=YF6bJ1JvCFQqHm=Nih_-9ACMjg@mail.gmail.com>
	<CAB6mnxLrzFM=2Rt4u137URt6gbRo1wnBXxLvNO0mZK7GP106vA@mail.gmail.com>
	<CABL7CQj7a=q9YwKSn781nOb+xUGa8MbLBBKDkro+u-CYQ3ZCmw@mail.gmail.com>
	<CANNq6Fm=6tB=2FFOuM3yagz0==+ULfXjp2MHb2ZuQEHdtF2UVQ@mail.gmail.com>
	<CABL7CQihme6sWOy3rBop2mqMHaFDzBGZ+hz9oUMXTAzJKgowHQ@mail.gmail.com>
	<CAMMTP+DM5Pkg4VFmT3B2EC_OWZxgCP3uCbW8qtw2y6e2xpLK5w@mail.gmail.com>
Message-ID: <CABL7CQj1h5WgPZW5+YXMmnryH3Zu7VFrEO2d2LrG8=FGTr1CiA@mail.gmail.com>

On Mon, Aug 26, 2013 at 4:26 PM, <josef.pktd at gmail.com> wrote:

>
>
>
> On Mon, Aug 26, 2013 at 12:19 PM, Ralf Gommers <ralf.gommers at gmail.com>wrote:
>
>>
>>
>>
>> On Mon, Aug 26, 2013 at 3:49 PM, Benjamin Root <ben.root at ou.edu> wrote:
>>
>>>
>>> On Mon, Aug 26, 2013 at 11:01 AM, Ralf Gommers <ralf.gommers at gmail.com>wrote:
>>>
>>>>
>>>>
>>>>
>>>> On Sun, Aug 18, 2013 at 6:36 PM, Charles R Harris <
>>>> charlesr.harris at gmail.com> wrote:
>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Sun, Aug 18, 2013 at 12:17 PM, Charles R Harris <
>>>>> charlesr.harris at gmail.com> wrote:
>>>>>
>>>>>> Just a reminder that 1.8.0 will be branched tonight. I've put up a
>>>>>> big STY: PR <https://github.com/numpy/numpy/pull/3635> that removes
>>>>>> trailing whitespace and fixes spacing after commas. I would like to apply
>>>>>> before the branch, but it may cause merge difficulties down the line. I'd
>>>>>> like feedback on that option.
>>>>>>
>>>>>>
>>>>> I've also run autopep8 on the code and it does a nice job of cleaning
>>>>> up things. It gets a little lost in deeply nested lists, but there aren't
>>>>> too many of those. By default it doesn't fix spaces about operator (it
>>>>> seems). I can apply that also if there is interest in doing so.
>>>>>
>>>>
>>>> Depends on how many lines of code it touches. For scipy we decided not
>>>> to do this, because it would make "git blame" pretty much useless.
>>>>
>>>> Ralf
>>>>
>>>
>>> At some point, you just have to bite the bullet. Matplotlib has been
>>> doing pep8 work for about a year now. We adopted very specific rules on how
>>> that work was to be done (make pep8 only PRs, each pep8 PR would be for at
>>> most one module at a time, etc). Yes, it does look like NelleV has taken
>>> over the project, but the trade-off is readability. We even discovered a
>>> non-trivial number of bugs this way. For a core library like NumPy that has
>>> lots of very obscure-looking code that almost never gets changed, avoiding
>>> PEP8 is problematic because it always becomes "Somebody else's problem".
>>>
>>
>> We've been doing a lot of PEP8 cleanups in scipy - there's even ``tox -e
>> pep8`` with a limited number of exceptions. And Chuck has just done a lot
>> of cleanup for numpy, which is quite useful. So it's definitely not just
>> someone else's problem.
>>
>> The question was specifically about spaces around operators, which
>> doesn't necessarily even make things look better (even apart from
>> destroying history). This:
>>
>>     a*x[0] + b*x[1] + c*x[2]**2
>>
>> look imho better than if you'd "fix" it like:
>>
>>     a * x[0] + b * x[1] + c * x[2] ** 2
>>
>
> that's not pep-8 (anymore)
> http://www.python.org/dev/peps/pep-0008/#other-recommendations
>

Thanks for pointing that out. Unfortunately it still is exactly what
autopep8 will do:

$ cat tmp.py
a*x[0] + b*x[1] + c*x[2]**2
a * x[0] + b * x[1] + c * x[2] ** 2

$ autopep8 -d tmp.py
--- original/tmp.py
+++ fixed/tmp.py
@@ -1,2 +1,2 @@
-a*x[0] + b*x[1] + c*x[2]**2
 a * x[0] + b * x[1] + c * x[2] ** 2
+a * x[0] + b * x[1] + c * x[2] ** 2

Maybe you can get it to ignore certain operators, but I doubt it will be
smart enough to follow the PEP8 recommendations on spacing dependent on
operator precedence.

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130826/56a8b0d7/attachment.html>

From brett.olsen at gmail.com  Mon Aug 26 13:08:02 2013
From: brett.olsen at gmail.com (Brett Olsen)
Date: Mon, 26 Aug 2013 12:08:02 -0500
Subject: [Numpy-discussion] Stick (line segments) percolation algorithm
 - graph theory?
In-Reply-To: <1377295771.604.YahooMailNeo@web142305.mail.bf1.yahoo.com>
References: <1377295771.604.YahooMailNeo@web142305.mail.bf1.yahoo.com>
Message-ID: <CAFq1z2VtjKie6W6LGobMgcQpSTDAQuA6kgKDduZA1SAbKwdWpA@mail.gmail.com>

I can see a couple opportunities for improvements in your algorithm.
 Running your code on a single experiment, I get about 2.9 seconds to run.
I get this down to about 1.0 seconds by (1) exploiting the symmetry of the
M matrix and (2) avoiding the costly inner loop over k in favor of array
operations:

def check_segments(j, others, data):
    x1, y1, x2, y2 = data

    x_A1B1 = x2[j]-x1[j]
    y_A1B1 = y2[j]-y1[j]

    x_A1A2 = x1[others]-x1[j]
    y_A1A2 = y1[others]-y1[j]

    x_A2A1 = -1*x_A1A2
    y_A2A1 = -1*y_A1A2

    x_A2B2 = x2[others]-x1[others]
    y_A2B2 = y2[others]-y1[others]

    x_A1B2 = x2[others]-x1[j]
    y_A1B2 = y2[others]-y1[j]

    x_A2B1 = x2[j]-x1[others]
    y_A2B1 = y2[j]-y1[others]

    p1 = x_A1B1*y_A1A2 - y_A1B1*x_A1A2
    p2 = x_A1B1*y_A1B2 - y_A1B1*x_A1B2
    p3 = x_A2B2*y_A2B1 - y_A2B2*x_A2B1
    p4 = x_A2B2*y_A2A1 - y_A2B2*x_A2A1

    condition_1=p1*p2
    condition_2=p3*p4

    return (p1 * p2 <= 0) & (p3 * p4 <= 0)

for j in xrange(1, N):
    valid = check_segments(j, range(j), (x1, y1, x2, y2))
    M[j,0:j] = valid
    M[0:j,j] = valid

I don't see any other particularly simple ways to improve this.  You could
probably add an interval check to ensure that the x and y intervals for the
segments of interest overlap before doing the full check, but how much that
would help would depend on the implementations.

~Brett

On Fri, Aug 23, 2013 at 5:09 PM, Jos? Luis Mietta <
joseluismietta at yahoo.com.ar> wrote:

> I wrote an algorithm for study stick percolation (i.e.: networks between
> line segments that intersect between them). In my algorithm N sticks (line
> segments) are created inside a rectangular box of sides 'b' and 'h' and
> then, one by one, the algorithm explores the intersection between all line
> segments. This is a Monte Carlo simulation, so the 'experiment' is executed
> many times (no less than 100 times). Written like that, very much RAM is
> consumed:  Here, the element Mij=1 if stick i intersects stick j and Mij=0
> if not.
> How can I optimize my algorithm? Graph theory is useful in this case?
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130826/4c5d6ba5/attachment.html>

From pav at iki.fi  Mon Aug 26 13:44:00 2013
From: pav at iki.fi (Pauli Virtanen)
Date: Mon, 26 Aug 2013 20:44:00 +0300
Subject: [Numpy-discussion] numpy dot returns [nan nan nan]
In-Reply-To: <CAGzF1ucWOu+VfiEfh93O6P9jEQThVEW6p_GEc9nwEJ9Vesws_A@mail.gmail.com>
References: <CAEYEPaQrRTcKCYw0FPDNhUfyirVbiGCuRyuP32F5vLnrx4HxdQ@mail.gmail.com>
	<CAGzF1ueJFvGr+hwgF=RfqWK_oZydohwaPOVQQAkzH5Dz0=OYnw@mail.gmail.com>
	<CAGzF1uf+zfCDry0X749uAZEnDa4LG+SsVHzx-g8su4w6yQdptg@mail.gmail.com>
	<CAEYEPaT9J1KP9paUrG=np1B50EsE=TL1HTWQ=7K4HHuDr1PL+g@mail.gmail.com>
	<CAGzF1ucWOu+VfiEfh93O6P9jEQThVEW6p_GEc9nwEJ9Vesws_A@mail.gmail.com>
Message-ID: <kvg44o$vv2$1@ger.gmane.org>

24.08.2013 21:25, Warren Weckesser kirjoitti:
[clip]
> Now the question is: why does `np.dot` mask the overflow warning?

That probably depends on whatever BLAS is used. I wouldn't be
surprised if it's BLAS that swallows the floating point invalid flags.

-- 
Pauli Virtanen



From matthew.brett at gmail.com  Mon Aug 26 17:32:24 2013
From: matthew.brett at gmail.com (Matthew Brett)
Date: Mon, 26 Aug 2013 14:32:24 -0700
Subject: [Numpy-discussion] Fwd: New release of bdist_mpkg
In-Reply-To: <CAH6Pt5rZX_djs8JuqLoAqStAANz2HPJEbTGvRurTZt5ksvT51g@mail.gmail.com>
References: <CAH6Pt5rZX_djs8JuqLoAqStAANz2HPJEbTGvRurTZt5ksvT51g@mail.gmail.com>
Message-ID: <CAH6Pt5riGjftq8f7OdeXH639ue=qe=vcrgzf60qFp+JRBX_CxA@mail.gmail.com>

Hi,

Just in case y'all didn't catch this from other lists:

I just did a new release of bdist_mpkg:

https://pypi.python.org/pypi/bdist_mpkg/

The main new feature is Python 3 compatibility:

https://github.com/matthew-brett/bdist_mpkg/blob/master/Changelog

Thanks to Bob Ippolito for the original code and Ronald Oussoren for
taking care of it.

Please do let me know if you hit any bugs or misfeatures,

Cheers,

Matthew


From juanlu001 at gmail.com  Tue Aug 27 07:46:36 2013
From: juanlu001 at gmail.com (Juan Luis Cano)
Date: Tue, 27 Aug 2013 11:46:36 +0000
Subject: [Numpy-discussion] Right way to build Fortran 90 module using
	numpy.distutils
Message-ID: <CAFmivUVARcP-b8yfyYABJYFYTgS0E0qAWW0YV09SuGUywKqFvA@mail.gmail.com>

I'm having some problems to properly build a module written in Fortran
90 using numpy.distutils. It only contains one subroutine, but:

* If I write a .f90 file with the single subroutine (no module ..
contains), then f2py tries to compile it using a f77 compiler, and
that's not what I want.
* If I put it inside a module definition, then at the final
compilation stage modulef2pywrapper.c is missing the corresponding
.mod file and I have to make the build fail, then copy the .mod from
the temp directory and rebuild. Weird.

On the other hand, with the second approach f2py creates a unnecessary
level of nesting in the corresponding python module and I don't feel
quite ok with that.

What is the right way to do this? I find the documentation on
numpy.distutils a bit lacking, I almost always find the solution by
trial and error. I can push the code, it's a short file (part of the
rewrite of odeint I'm doing in SciPy).

Regards

Juan Luis Cano


From kyle.mandli at gmail.com  Tue Aug 27 11:27:37 2013
From: kyle.mandli at gmail.com (Kyle Mandli)
Date: Tue, 27 Aug 2013 10:27:37 -0500
Subject: [Numpy-discussion] Right way to build Fortran 90 module using
	numpy.distutils
In-Reply-To: <CAFmivUVARcP-b8yfyYABJYFYTgS0E0qAWW0YV09SuGUywKqFvA@mail.gmail.com>
References: <CAFmivUVARcP-b8yfyYABJYFYTgS0E0qAWW0YV09SuGUywKqFvA@mail.gmail.com>
Message-ID: <CAO1YWcY5kNDLgps+xkjA--cmAtGDqd5S=UbpVA2uwAp6a097uQ@mail.gmail.com>

On Tue, Aug 27, 2013 at 6:46 AM, Juan Luis Cano <juanlu001 at gmail.com> wrote:

> I'm having some problems to properly build a module written in Fortran
> 90 using numpy.distutils. It only contains one subroutine, but:
>
> * If I write a .f90 file with the single subroutine (no module ..
> contains), then f2py tries to compile it using a f77 compiler, and
> that's not what I want.
> * If I put it inside a module definition, then at the final
> compilation stage modulef2pywrapper.c is missing the corresponding
> .mod file and I have to make the build fail, then copy the .mod from
> the temp directory and rebuild. Weird.
>
> On the other hand, with the second approach f2py creates a unnecessary
> level of nesting in the corresponding python module and I don't feel
> quite ok with that.
>
> What is the right way to do this? I find the documentation on
> numpy.distutils a bit lacking, I almost always find the solution by
> trial and error. I can push the code, it's a short file (part of the
> rewrite of odeint I'm doing in SciPy).
>
> Regards
>
> Juan Luis Cano
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>

You should be able to specify the compiler when invoking f2py, something
like:

f2py --fcompiler=$COMPILER -m $MODULE_NAME -c $SOURCE_FILE

Kyle
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130827/7ac9ed39/attachment.html>

From juanlu001 at gmail.com  Tue Aug 27 14:26:29 2013
From: juanlu001 at gmail.com (Juan Luis Cano)
Date: Tue, 27 Aug 2013 18:26:29 +0000
Subject: [Numpy-discussion] Right way to build Fortran 90 module using
	numpy.distutils
In-Reply-To: <CAO1YWcY5kNDLgps+xkjA--cmAtGDqd5S=UbpVA2uwAp6a097uQ@mail.gmail.com>
References: <CAFmivUVARcP-b8yfyYABJYFYTgS0E0qAWW0YV09SuGUywKqFvA@mail.gmail.com>
	<CAO1YWcY5kNDLgps+xkjA--cmAtGDqd5S=UbpVA2uwAp6a097uQ@mail.gmail.com>
Message-ID: <CAFmivUW4-5uN1vmDy6rK9TJNzDakTGBNd-Nd+v1838U-EpPCmg@mail.gmail.com>

El 27/08/2013 17:27, "Kyle Mandli" <kyle.mandli at gmail.com> escribi?:
>
> On Tue, Aug 27, 2013 at 6:46 AM, Juan Luis Cano <juanlu001 at gmail.com>
wrote:
>>
>> I'm having some problems to properly build a module written in Fortran
>> 90 using numpy.distutils. It only contains one subroutine, but:
>>
>> * If I write a .f90 file with the single subroutine (no module ..
>> contains), then f2py tries to compile it using a f77 compiler, and
>> that's not what I want.
>> * If I put it inside a module definition, then at the final
>> compilation stage modulef2pywrapper.c is missing the corresponding
>> .mod file and I have to make the build fail, then copy the .mod from
>> the temp directory and rebuild. Weird.
>>
>> On the other hand, with the second approach f2py creates a unnecessary
>> level of nesting in the corresponding python module and I don't feel
>> quite ok with that.
>>
>> What is the right way to do this? I find the documentation on
>> numpy.distutils a bit lacking, I almost always find the solution by
>> trial and error. I can push the code, it's a short file (part of the
>> rewrite of odeint I'm doing in SciPy).
>>
>> Regards
>>
>> Juan Luis Cano
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
> You should be able to specify the compiler when invoking f2py, something
like:
>
> f2py --fcompiler=$COMPILER -m $MODULE_NAME -c $SOURCE_FILE

That won't work since gfortran is used for both F77 and F90. If you look on
rules.py in f2py F77 wrappers are generated if the subroutine is not inside
a module, and then the compilation fails because it contains invalid
statements for F77. Bug?

Any other suggestions?

> Kyle
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130827/97aa969a/attachment.html>

From kyle.mandli at gmail.com  Tue Aug 27 15:21:45 2013
From: kyle.mandli at gmail.com (Kyle Mandli)
Date: Tue, 27 Aug 2013 14:21:45 -0500
Subject: [Numpy-discussion] Right way to build Fortran 90 module using
	numpy.distutils
In-Reply-To: <CAFmivUW4-5uN1vmDy6rK9TJNzDakTGBNd-Nd+v1838U-EpPCmg@mail.gmail.com>
References: <CAFmivUVARcP-b8yfyYABJYFYTgS0E0qAWW0YV09SuGUywKqFvA@mail.gmail.com>
	<CAO1YWcY5kNDLgps+xkjA--cmAtGDqd5S=UbpVA2uwAp6a097uQ@mail.gmail.com>
	<CAFmivUW4-5uN1vmDy6rK9TJNzDakTGBNd-Nd+v1838U-EpPCmg@mail.gmail.com>
Message-ID: <CAO1YWcbJ9Ry64hbXEbMSn2qX_1wWgKMsSjhR22aVnEJpfcvAFw@mail.gmail.com>

On Tue, Aug 27, 2013 at 1:26 PM, Juan Luis Cano <juanlu001 at gmail.com> wrote:

>
> El 27/08/2013 17:27, "Kyle Mandli" <kyle.mandli at gmail.com> escribi?:
>
> >
> > On Tue, Aug 27, 2013 at 6:46 AM, Juan Luis Cano <juanlu001 at gmail.com>
> wrote:
> >>
> >> I'm having some problems to properly build a module written in Fortran
> >> 90 using numpy.distutils. It only contains one subroutine, but:
> >>
> >> * If I write a .f90 file with the single subroutine (no module ..
> >> contains), then f2py tries to compile it using a f77 compiler, and
> >> that's not what I want.
> >> * If I put it inside a module definition, then at the final
> >> compilation stage modulef2pywrapper.c is missing the corresponding
> >> .mod file and I have to make the build fail, then copy the .mod from
> >> the temp directory and rebuild. Weird.
> >>
> >> On the other hand, with the second approach f2py creates a unnecessary
> >> level of nesting in the corresponding python module and I don't feel
> >> quite ok with that.
> >>
> >> What is the right way to do this? I find the documentation on
> >> numpy.distutils a bit lacking, I almost always find the solution by
> >> trial and error. I can push the code, it's a short file (part of the
> >> rewrite of odeint I'm doing in SciPy).
> >>
> >> Regards
> >>
> >> Juan Luis Cano
> >> _______________________________________________
> >> NumPy-Discussion mailing list
> >> NumPy-Discussion at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
> >
> > You should be able to specify the compiler when invoking f2py, something
> like:
> >
> > f2py --fcompiler=$COMPILER -m $MODULE_NAME -c $SOURCE_FILE
>
> That won't work since gfortran is used for both F77 and F90. If you look
> on rules.py in f2py F77 wrappers are generated if the subroutine is not
> inside a module, and then the compilation fails because it contains invalid
> statements for F77. Bug?
>
> Any other suggestions?
>
> > Kyle
>
> >
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at scipy.org
> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
> >
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
Can you post your fortran source code?  The other key would be see what
f2py thinks it's doing, for instance in the output of f2py there two lines
that read:

```
Reading fortran codes...
Reading file 'test_file.f90' (format:free)
```

For the following source code:

```
subroutine my_func(a, b)

    implicit none

    integer, intent(in) :: a
    real(kind=8), intent(out) :: b

    b = real(a, kind=8) * 5.d0

end subroutine my_func
```

compiled with `f2py -c test_file.f90 -m my_module`
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130827/c1b7c5de/attachment.html>

From juanlu001 at gmail.com  Tue Aug 27 16:18:01 2013
From: juanlu001 at gmail.com (Juan Luis Cano)
Date: Tue, 27 Aug 2013 20:18:01 +0000
Subject: [Numpy-discussion] Right way to build Fortran 90 module using
	numpy.distutils
In-Reply-To: <CAO1YWcbJ9Ry64hbXEbMSn2qX_1wWgKMsSjhR22aVnEJpfcvAFw@mail.gmail.com>
References: <CAFmivUVARcP-b8yfyYABJYFYTgS0E0qAWW0YV09SuGUywKqFvA@mail.gmail.com>
	<CAO1YWcY5kNDLgps+xkjA--cmAtGDqd5S=UbpVA2uwAp6a097uQ@mail.gmail.com>
	<CAFmivUW4-5uN1vmDy6rK9TJNzDakTGBNd-Nd+v1838U-EpPCmg@mail.gmail.com>
	<CAO1YWcbJ9Ry64hbXEbMSn2qX_1wWgKMsSjhR22aVnEJpfcvAFw@mail.gmail.com>
Message-ID: <CAFmivUVZ1x=FBJqn8DKXR5TGTfqNCqLcnH9kBUkJfhpMKNswmg@mail.gmail.com>

El 27/08/2013 21:21, "Kyle Mandli" <kyle.mandli at gmail.com> escribi?:
>
> On Tue, Aug 27, 2013 at 1:26 PM, Juan Luis Cano <juanlu001 at gmail.com>
wrote:
>>
>>
>> El 27/08/2013 17:27, "Kyle Mandli" <kyle.mandli at gmail.com> escribi?:
>>
>>
>> >
>> > On Tue, Aug 27, 2013 at 6:46 AM, Juan Luis Cano <juanlu001 at gmail.com>
wrote:
>> >>
>> >> I'm having some problems to properly build a module written in Fortran
>> >> 90 using numpy.distutils. It only contains one subroutine, but:
>> >>
>> >> * If I write a .f90 file with the single subroutine (no module ..
>> >> contains), then f2py tries to compile it using a f77 compiler, and
>> >> that's not what I want.
>> >> * If I put it inside a module definition, then at the final
>> >> compilation stage modulef2pywrapper.c is missing the corresponding
>> >> .mod file and I have to make the build fail, then copy the .mod from
>> >> the temp directory and rebuild. Weird.
>> >>
>> >> On the other hand, with the second approach f2py creates a unnecessary
>> >> level of nesting in the corresponding python module and I don't feel
>> >> quite ok with that.
>> >>
>> >> What is the right way to do this? I find the documentation on
>> >> numpy.distutils a bit lacking, I almost always find the solution by
>> >> trial and error. I can push the code, it's a short file (part of the
>> >> rewrite of odeint I'm doing in SciPy).
>> >>
>> >> Regards
>> >>
>> >> Juan Luis Cano
>> >> _______________________________________________
>> >> NumPy-Discussion mailing list
>> >> NumPy-Discussion at scipy.org
>> >> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>> >
>> >
>> > You should be able to specify the compiler when invoking f2py,
something like:
>> >
>> > f2py --fcompiler=$COMPILER -m $MODULE_NAME -c $SOURCE_FILE
>>
>> That won't work since gfortran is used for both F77 and F90. If you look
on rules.py in f2py F77 wrappers are generated if the subroutine is not
inside a module, and then the compilation fails because it contains invalid
statements for F77. Bug?
>>
>> Any other suggestions?
>>
>> > Kyle
>>
>>
>> >
>> > _______________________________________________
>> > NumPy-Discussion mailing list
>> > NumPy-Discussion at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
>> >
>>
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>
> Can you post your fortran source code?  The other key would be see what
f2py thinks it's doing, for instance in the output of f2py there two lines
that read:
>
> ```
> Reading fortran codes...
> Reading file 'test_file.f90' (format:free)
> ```
>
> For the following source code:
>
> ```
> subroutine my_func(a, b)
>
>     implicit none
>
>     integer, intent(in) :: a
>     real(kind=8), intent(out) :: b
>
>     b = real(a, kind=8) * 5.d0
>
> end subroutine my_func
> ```
>
> compiled with `f2py -c test_file.f90 -m my_module`
>

Solved! It was as easy as adding both the .pyf and the .f90 to sources, I
was having a silly name conflict. I will push the source next week. Thanks!

> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130827/3d396475/attachment.html>

From hoffmanm at cs.ubc.ca  Wed Aug 28 06:08:02 2013
From: hoffmanm at cs.ubc.ca (Matt Hoffman)
Date: Wed, 28 Aug 2013 11:08:02 +0100
Subject: [Numpy-discussion] Use numpy.distutils to build fortran with BLAS
Message-ID: <CAL4fS8FDOyrBAhc3vRpo0JsB9+Ov_xL28-P7m=oFEYZSz-MpFw@mail.gmail.com>

I have some fortran code that I would like to build and make accessible
from python which needs to call a few BLAS routines. Building it in a
one-off manner is not a problem, but I can't seem to find a good way to
distribute it in a way that just works.

So really I'm wondering if there is a "correct" way use numpy.distutils in
order to link against the same BLAS libraries used by numpy. I'll paste my
current approach below, but I'm not sure that this is the
best/most-portable way to do it. (And it certainly breaks on MacOS due to
the weird way that macs use rpath.)

Anyway my setup contains something like:

    from numpy.distutils.core import setup
    from numpy.distutils.system_info import get_info, NotFoundError
    from numpy.distutils.misc_util import Configuration

    sources = ['foo.f']
    extra_info = {}
    extra_link_args = []

    try:
        extra_info = get_info('mkl', 2)
        extra_link_args = ['-Wl,-rpath,'+path for path in
extra_info['library_dirs']],

    except NotFoundError:
        try:
            extra_info = get_info('blas', 2)
            extra_link_args = ['-Wl,-rpath,'+path for path in
extra_info['library_dirs']],

        except NotFoundError:
            # couldn't find anything. just fall back to building the
functions we need ourselves.
            sources += ['ddot.f']

    config = Configuration()
    config.add_extension(
        sources=sources,
        extra_info=extra_info,
        extra_link_args=extra_link_args
        )
    setup(**config.todict())

Thanks!
-matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130828/70030aa3/attachment.html>

From ndbecker2 at gmail.com  Wed Aug 28 11:32:15 2013
From: ndbecker2 at gmail.com (Neal Becker)
Date: Wed, 28 Aug 2013 11:32:15 -0400
Subject: [Numpy-discussion] lots of warnings with python3
Message-ID: <kvl55n$mab$2@ger.gmane.org>

I tried running python2 -3 on some code, and found numpy
produces a lot of warnings.

Many like:
python -3 -c 'import numpy'
...
/usr/lib64/python2.7/site-packages/numpy/lib/polynomial.py:928: 
DeprecationWarning: Overriding __eq__ blocks inheritance of __hash__ in 3.x

But also:
/usr/lib64/python2.7/site-packages/numpy/lib/shape_base.py:838: 
DeprecationWarning: classic int division
  n /= max(dim_in,1)



From njs at pobox.com  Wed Aug 28 11:50:11 2013
From: njs at pobox.com (Nathaniel Smith)
Date: Wed, 28 Aug 2013 16:50:11 +0100
Subject: [Numpy-discussion] lots of warnings with python3
In-Reply-To: <kvl55n$mab$2@ger.gmane.org>
References: <kvl55n$mab$2@ger.gmane.org>
Message-ID: <CAPJVwBmv0cCHM4cgv2uzhqqBRyWH+wxO0+103mkOyH4rYJCong@mail.gmail.com>

On Wed, Aug 28, 2013 at 4:32 PM, Neal Becker <ndbecker2 at gmail.com> wrote:
> I tried running python2 -3 on some code, and found numpy
> produces a lot of warnings.
>
> Many like:
> python -3 -c 'import numpy'
> ...
> /usr/lib64/python2.7/site-packages/numpy/lib/polynomial.py:928:
> DeprecationWarning: Overriding __eq__ blocks inheritance of __hash__ in 3.x

Looks like a bug... in 2.x you *must* define __hash__ if you define
__eq__, even if it's just as:

def __hash__(self):
    return NotImplemented

> But also:
> /usr/lib64/python2.7/site-packages/numpy/lib/shape_base.py:838:
> DeprecationWarning: classic int division
>   n /= max(dim_in,1)

Also looks like a bug, should be //=.

Want to file bugs/PRs?

-n


From charlesr.harris at gmail.com  Wed Aug 28 12:03:06 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 28 Aug 2013 10:03:06 -0600
Subject: [Numpy-discussion] lots of warnings with python3
In-Reply-To: <kvl55n$mab$2@ger.gmane.org>
References: <kvl55n$mab$2@ger.gmane.org>
Message-ID: <CAB6mnxKgN6wpZ439ppCjS+9DwFxE+Cb4NjQvAFpwoKSk5OXyBA@mail.gmail.com>

On Wed, Aug 28, 2013 at 9:32 AM, Neal Becker <ndbecker2 at gmail.com> wrote:

> I tried running python2 -3 on some code, and found numpy
> produces a lot of warnings.
>
> Many like:
> python -3 -c 'import numpy'
> ...
> /usr/lib64/python2.7/site-packages/numpy/lib/polynomial.py:928:
> DeprecationWarning: Overriding __eq__ blocks inheritance of __hash__ in 3.x
>
> But also:
> /usr/lib64/python2.7/site-packages/numpy/lib/shape_base.py:838:
> DeprecationWarning: classic int division
>   n /= max(dim_in,1)
>
>
Some of these are bogus.
```
DeprecationWarning: CObject type is not supported in 3.x. Please use
capsule objects instead.
```
Capsule objects aren't available in 2.6 and we don't use them for any of
the 2.x Python series, only for 3.x.

The `__hash__` being blocked hasn't seemed to be a problem, but we should
probably fix it anyway.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130828/2511a9e2/attachment.html>

From charlesr.harris at gmail.com  Wed Aug 28 13:36:06 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 28 Aug 2013 11:36:06 -0600
Subject: [Numpy-discussion] lots of warnings with python3
In-Reply-To: <CAB6mnxKgN6wpZ439ppCjS+9DwFxE+Cb4NjQvAFpwoKSk5OXyBA@mail.gmail.com>
References: <kvl55n$mab$2@ger.gmane.org>
	<CAB6mnxKgN6wpZ439ppCjS+9DwFxE+Cb4NjQvAFpwoKSk5OXyBA@mail.gmail.com>
Message-ID: <CAB6mnxK3GpLRph4dp=CNGz+hNy_1Y6JPF8Zg3Mpgj9_7f31DhQ@mail.gmail.com>

On Wed, Aug 28, 2013 at 10:03 AM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
>
>
>
> On Wed, Aug 28, 2013 at 9:32 AM, Neal Becker <ndbecker2 at gmail.com> wrote:
>>
>> I tried running python2 -3 on some code, and found numpy
>> produces a lot of warnings.
>>
>> Many like:
>> python -3 -c 'import numpy'
>> ...
>> /usr/lib64/python2.7/site-packages/numpy/lib/polynomial.py:928:
>> DeprecationWarning: Overriding __eq__ blocks inheritance of __hash__ in
>> 3.x
>>
>> But also:
>> /usr/lib64/python2.7/site-packages/numpy/lib/shape_base.py:838:
>> DeprecationWarning: classic int division
>>   n /= max(dim_in,1)
>>
For the rest, defining `__hash__ = None` should work, at least for the
numpy.polynomial classes, as they aren't immutable.

Chuck


From charlesr.harris at gmail.com  Wed Aug 28 15:20:04 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Wed, 28 Aug 2013 13:20:04 -0600
Subject: [Numpy-discussion] lots of warnings with python3
In-Reply-To: <CAB6mnxK3GpLRph4dp=CNGz+hNy_1Y6JPF8Zg3Mpgj9_7f31DhQ@mail.gmail.com>
References: <kvl55n$mab$2@ger.gmane.org>
	<CAB6mnxKgN6wpZ439ppCjS+9DwFxE+Cb4NjQvAFpwoKSk5OXyBA@mail.gmail.com>
	<CAB6mnxK3GpLRph4dp=CNGz+hNy_1Y6JPF8Zg3Mpgj9_7f31DhQ@mail.gmail.com>
Message-ID: <CAB6mnxKUXGsBJ-qJB1kuyMkET2qn6ca7faRsPZM6mgsMQ_f2GA@mail.gmail.com>

On Wed, Aug 28, 2013 at 11:36 AM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
> On Wed, Aug 28, 2013 at 10:03 AM, Charles R Harris
> <charlesr.harris at gmail.com> wrote:
>>
>>
>>
>> On Wed, Aug 28, 2013 at 9:32 AM, Neal Becker <ndbecker2 at gmail.com> wrote:
>>>
>>> I tried running python2 -3 on some code, and found numpy
>>> produces a lot of warnings.
>>>
>>> Many like:
>>> python -3 -c 'import numpy'
>>> ...
>>> /usr/lib64/python2.7/site-packages/numpy/lib/polynomial.py:928:
>>> DeprecationWarning: Overriding __eq__ blocks inheritance of __hash__ in
>>> 3.x
>>>
>>> But also:
>>> /usr/lib64/python2.7/site-packages/numpy/lib/shape_base.py:838:
>>> DeprecationWarning: classic int division
>>>   n /= max(dim_in,1)
>>>
> For the rest, defining `__hash__ = None` should work, at least for the
> numpy.polynomial classes, as they aren't immutable.
>

See https://github.com/numpy/numpy/pull/3657

Chuck


From luethi at vaw.baug.ethz.ch  Thu Aug 29 07:00:20 2013
From: luethi at vaw.baug.ethz.ch (Martin Luethi)
Date: Thu, 29 Aug 2013 13:00:20 +0200
Subject: [Numpy-discussion] Array addition inconsistency
Message-ID: <87eh9czrqz.fsf@vaw.baug.ethz.ch>

Dear all,

After some surprise, I noticed an inconsistency while adding array
slices:

> a = np.arange(5)
> a[1:] = a[1:] + a[:-1]
> a
array([0, 1, 3, 5, 7])

versus inplace

> a = np.arange(5)
> a[1:] += a[:-1]
> a
array([ 0,  1,  3,  6, 10])

My suspicition is that the second variant does not create intermediate
storage, and thus works on the intermediate result, effectively
performing a.cumsum().

This behaviour is certainly surprising, and leads to unexpected errors
if used without testing.

Best, Martin


-- 
Dr. Martin L?thi   luethi at vaw.baug.ethz.ch
VAW Glaciology     http://www.vaw.ethz.ch/gz
ETH Z?rich         http://people.ee.ethz.ch/~luethim
8093 Z?rich        Tel: +41 44 632 40 93




From robert.kern at gmail.com  Thu Aug 29 08:04:58 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 29 Aug 2013 13:04:58 +0100
Subject: [Numpy-discussion] Array addition inconsistency
In-Reply-To: <87eh9czrqz.fsf@vaw.baug.ethz.ch>
References: <87eh9czrqz.fsf@vaw.baug.ethz.ch>
Message-ID: <CAF6FJitTh5b4niDUOx49-jRYKsnwif5F6_Efk9AmRwTH07wWcg@mail.gmail.com>

On Thu, Aug 29, 2013 at 12:00 PM, Martin Luethi <luethi at vaw.baug.ethz.ch>
wrote:
>
> Dear all,
>
> After some surprise, I noticed an inconsistency while adding array
> slices:
>
> > a = np.arange(5)
> > a[1:] = a[1:] + a[:-1]
> > a
> array([0, 1, 3, 5, 7])
>
> versus inplace
>
> > a = np.arange(5)
> > a[1:] += a[:-1]
> > a
> array([ 0,  1,  3,  6, 10])
>
> My suspicition is that the second variant does not create intermediate
> storage, and thus works on the intermediate result, effectively
> performing a.cumsum().

Correct. Not creating intermediate storage is the point of using augmented
assignment.

--
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130829/15ad10da/attachment.html>

From ben.root at ou.edu  Thu Aug 29 09:39:06 2013
From: ben.root at ou.edu (Benjamin Root)
Date: Thu, 29 Aug 2013 09:39:06 -0400
Subject: [Numpy-discussion] Array addition inconsistency
In-Reply-To: <CAF6FJitTh5b4niDUOx49-jRYKsnwif5F6_Efk9AmRwTH07wWcg@mail.gmail.com>
References: <87eh9czrqz.fsf@vaw.baug.ethz.ch>
	<CAF6FJitTh5b4niDUOx49-jRYKsnwif5F6_Efk9AmRwTH07wWcg@mail.gmail.com>
Message-ID: <CANNq6Fm1VOurkB4_mgvSzxv4uBAcoLsu6k-UCamUOgrpF90wJw@mail.gmail.com>

On Thu, Aug 29, 2013 at 8:04 AM, Robert Kern <robert.kern at gmail.com> wrote:

> On Thu, Aug 29, 2013 at 12:00 PM, Martin Luethi <luethi at vaw.baug.ethz.ch>
> wrote:
> >
> > Dear all,
> >
> > After some surprise, I noticed an inconsistency while adding array
> > slices:
> >
> > > a = np.arange(5)
> > > a[1:] = a[1:] + a[:-1]
> > > a
> > array([0, 1, 3, 5, 7])
> >
> > versus inplace
> >
> > > a = np.arange(5)
> > > a[1:] += a[:-1]
> > > a
> > array([ 0,  1,  3,  6, 10])
> >
> > My suspicition is that the second variant does not create intermediate
> > storage, and thus works on the intermediate result, effectively
> > performing a.cumsum().
>
> Correct. Not creating intermediate storage is the point of using augmented
> assignment.
>
>
This can be very sneaky.

> a = np.arange(5)
> a[:-1] = a[:-1] + a[1:]
> a
array([1, 3, 5, 7, 4])

> a = np.arange(5)
> a[:-1] += a[1:]
> a
array([1, 3, 5, 7, 4])

So, if someone is semi-careful and tries out that example, they might
(incorrectly) assume that such operations are safe without realizing that
it was safe because the values of a[1:] were ahead of the values of a[:-1]
in memory. I could easily imagine a situation where views of an array are
passed around only to finally end up in an in-place operation like this and
sometimes be right and sometimes be wrong. Maybe there can be some simple
check that could be performed to detect this sort of situation?

Cheers!
Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130829/2642021f/attachment.html>

From anubhab91 at gmail.com  Thu Aug 29 11:33:40 2013
From: anubhab91 at gmail.com (Anubhab Baksi)
Date: Thu, 29 Aug 2013 21:03:40 +0530
Subject: [Numpy-discussion] Relative speed
Message-ID: <CANHk5kyhQMB_9AHVonvPmAOt5iiTbOGP7-pZ88wAFM6xCuTZhQ@mail.gmail.com>

Hi,
I need to know about the relative speed (i.e., which one is faster) of the
followings:
1. list and numpy array, tuples and numpy array
2. list of tuples and numpy matrix (first one is rectangular)
3. random.randint() and numpy.random.random_integers()

Thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130829/84a548a2/attachment.html>

From josef.pktd at gmail.com  Thu Aug 29 11:39:33 2013
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Thu, 29 Aug 2013 11:39:33 -0400
Subject: [Numpy-discussion] Array addition inconsistency
In-Reply-To: <CANNq6Fm1VOurkB4_mgvSzxv4uBAcoLsu6k-UCamUOgrpF90wJw@mail.gmail.com>
References: <87eh9czrqz.fsf@vaw.baug.ethz.ch>
	<CAF6FJitTh5b4niDUOx49-jRYKsnwif5F6_Efk9AmRwTH07wWcg@mail.gmail.com>
	<CANNq6Fm1VOurkB4_mgvSzxv4uBAcoLsu6k-UCamUOgrpF90wJw@mail.gmail.com>
Message-ID: <CAMMTP+AccqvE0iqPP4z-m+U6wx=qyzypokU_RJW87A26MrgSPA@mail.gmail.com>

On Thu, Aug 29, 2013 at 9:39 AM, Benjamin Root <ben.root at ou.edu> wrote:

>
>
> On Thu, Aug 29, 2013 at 8:04 AM, Robert Kern <robert.kern at gmail.com>wrote:
>
>> On Thu, Aug 29, 2013 at 12:00 PM, Martin Luethi <luethi at vaw.baug.ethz.ch>
>> wrote:
>> >
>> > Dear all,
>> >
>> > After some surprise, I noticed an inconsistency while adding array
>> > slices:
>> >
>> > > a = np.arange(5)
>> > > a[1:] = a[1:] + a[:-1]
>> > > a
>> > array([0, 1, 3, 5, 7])
>> >
>> > versus inplace
>> >
>> > > a = np.arange(5)
>> > > a[1:] += a[:-1]
>> > > a
>> > array([ 0,  1,  3,  6, 10])
>> >
>> > My suspicition is that the second variant does not create intermediate
>> > storage, and thus works on the intermediate result, effectively
>> > performing a.cumsum().
>>
>> Correct. Not creating intermediate storage is the point of using
>> augmented assignment.
>>
>>
> This can be very sneaky.
>
> > a = np.arange(5)
> > a[:-1] = a[:-1] + a[1:]
> > a
> array([1, 3, 5, 7, 4])
>
> > a = np.arange(5)
> > a[:-1] += a[1:]
> > a
> array([1, 3, 5, 7, 4])
>
> So, if someone is semi-careful and tries out that example, they might
> (incorrectly) assume that such operations are safe without realizing that
> it was safe because the values of a[1:] were ahead of the values of a[:-1]
> in memory. I could easily imagine a situation where views of an array are
> passed around only to finally end up in an in-place operation like this and
> sometimes be right and sometimes be wrong. Maybe there can be some simple
> check that could be performed to detect this sort of situation?
>


I think the main message is that you don't use inplace operation with
mutables unless you know what you are doing, and you really need them.

inplace cumsum in python

>>> a = range(5)
>>> for i in xrange(1, 5): a[i] += a[i-1]
...
>>> a
[0, 1, 3, 6, 10]

Defensive programming.

Josef


>
> Cheers!
> Ben Root
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130829/41edae91/attachment.html>

From jniehof at lanl.gov  Thu Aug 29 11:41:56 2013
From: jniehof at lanl.gov (Jonathan T. Niehof)
Date: Thu, 29 Aug 2013 09:41:56 -0600
Subject: [Numpy-discussion] Relative speed
In-Reply-To: <CANHk5kyhQMB_9AHVonvPmAOt5iiTbOGP7-pZ88wAFM6xCuTZhQ@mail.gmail.com>
References: <CANHk5kyhQMB_9AHVonvPmAOt5iiTbOGP7-pZ88wAFM6xCuTZhQ@mail.gmail.com>
Message-ID: <521F6BC4.5070807@lanl.gov>

On 08/29/2013 09:33 AM, Anubhab Baksi wrote:
> Hi,
> I need to know about the relative speed (i.e., which one is faster) of
> the followings:
> 1. list and numpy array, tuples and numpy array
> 2. list of tuples and numpy matrix (first one is rectangular)
> 3. random.randint() and numpy.random.random_integers()

African or European?

It really depends on what you're doing with it. The ipython %timeit 
magic is pretty useful for answering that question. Note that the answer 
may change dramatically based on the size of the data set.

-- 
Jonathan Niehof
ISR-3 Space Data Systems
Los Alamos National Laboratory
MS-D466
Los Alamos, NM 87545

Phone: 505-667-9595
email: jniehof at lanl.gov

Correspondence /
Technical data or Software Publicly Available


From kyle.mandli at gmail.com  Thu Aug 29 12:38:36 2013
From: kyle.mandli at gmail.com (Kyle Mandli)
Date: Thu, 29 Aug 2013 11:38:36 -0500
Subject: [Numpy-discussion] Use numpy.distutils to build fortran with
	BLAS
In-Reply-To: <CAL4fS8FDOyrBAhc3vRpo0JsB9+Ov_xL28-P7m=oFEYZSz-MpFw@mail.gmail.com>
References: <CAL4fS8FDOyrBAhc3vRpo0JsB9+Ov_xL28-P7m=oFEYZSz-MpFw@mail.gmail.com>
Message-ID: <CAO1YWcYtJjF_Dc-85nW+sDOFOn5S7HA0daumrjhXVp9ezAdFTg@mail.gmail.com>

In the Clawpack projects (specifically the Riemann solvers) we compile
against LAPACK and the BLAS using f2py via the `--link-lapack_opt` flag.
 This does cause some problems in terms of portability though, Aron Ahmadia
might be able to shed some light on this as he has looked into it most
recently.


On Wed, Aug 28, 2013 at 5:08 AM, Matt Hoffman <hoffmanm at cs.ubc.ca> wrote:

> I have some fortran code that I would like to build and make accessible
> from python which needs to call a few BLAS routines. Building it in a
> one-off manner is not a problem, but I can't seem to find a good way to
> distribute it in a way that just works.
>
> So really I'm wondering if there is a "correct" way use numpy.distutils in
> order to link against the same BLAS libraries used by numpy. I'll paste my
> current approach below, but I'm not sure that this is the
> best/most-portable way to do it. (And it certainly breaks on MacOS due to
> the weird way that macs use rpath.)
>
> Anyway my setup contains something like:
>
>     from numpy.distutils.core import setup
>     from numpy.distutils.system_info import get_info, NotFoundError
>     from numpy.distutils.misc_util import Configuration
>
>     sources = ['foo.f']
>     extra_info = {}
>     extra_link_args = []
>
>     try:
>         extra_info = get_info('mkl', 2)
>         extra_link_args = ['-Wl,-rpath,'+path for path in
> extra_info['library_dirs']],
>
>     except NotFoundError:
>         try:
>             extra_info = get_info('blas', 2)
>             extra_link_args = ['-Wl,-rpath,'+path for path in
> extra_info['library_dirs']],
>
>         except NotFoundError:
>             # couldn't find anything. just fall back to building the
> functions we need ourselves.
>             sources += ['ddot.f']
>
>     config = Configuration()
>     config.add_extension(
>         sources=sources,
>         extra_info=extra_info,
>         extra_link_args=extra_link_args
>         )
>     setup(**config.todict())
>
> Thanks!
> -matt
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130829/c8981024/attachment.html>

From ralf.gommers at gmail.com  Thu Aug 29 14:00:49 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Thu, 29 Aug 2013 18:00:49 +0000
Subject: [Numpy-discussion] Relative speed
In-Reply-To: <521F6BC4.5070807@lanl.gov>
References: <CANHk5kyhQMB_9AHVonvPmAOt5iiTbOGP7-pZ88wAFM6xCuTZhQ@mail.gmail.com>
	<521F6BC4.5070807@lanl.gov>
Message-ID: <CABL7CQjq6wZdfFmBgMhF5kFGpgQxqCY-Nv20=ZbMtLwPXDDa-w@mail.gmail.com>

On Thu, Aug 29, 2013 at 3:41 PM, Jonathan T. Niehof <jniehof at lanl.gov>wrote:

> On 08/29/2013 09:33 AM, Anubhab Baksi wrote:
> > Hi,
> > I need to know about the relative speed (i.e., which one is faster) of
> > the followings:
> > 1. list and numpy array, tuples and numpy array
> > 2. list of tuples and numpy matrix (first one is rectangular)
> > 3. random.randint() and numpy.random.random_integers()
>

Hi Anubhab, if you have a reasonably large amount of data (say O(100)),
always try to use numpy arrays and not lists or tuples - it'll be faster.
I'd recommend not to use numpy.matrix, it's speed will be similar to numpy
arrays but it has some peculiarities that you'd rather not deal with. For
the random numbers I'm not sure without checking, just timing it in ipython
with %timeit is indeed the way to go.

Cheers,
Ralf


> African or European?
>

Why on earth would you ask that?



> It really depends on what you're doing with it. The ipython %timeit
> magic is pretty useful for answering that question. Note that the answer
> may change dramatically based on the size of the data set.
>
> --
> Jonathan Niehof
> ISR-3 Space Data Systems
> Los Alamos National Laboratory
> MS-D466
> Los Alamos, NM 87545
>
> Phone: 505-667-9595
> email: jniehof at lanl.gov
>
> Correspondence /
> Technical data or Software Publicly Available
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130829/eca7e68f/attachment.html>

From ewm at redtetrahedron.org  Thu Aug 29 14:10:51 2013
From: ewm at redtetrahedron.org (Eric Moore)
Date: Thu, 29 Aug 2013 14:10:51 -0400
Subject: [Numpy-discussion] Relative speed
In-Reply-To: <CABL7CQjq6wZdfFmBgMhF5kFGpgQxqCY-Nv20=ZbMtLwPXDDa-w@mail.gmail.com>
References: <CANHk5kyhQMB_9AHVonvPmAOt5iiTbOGP7-pZ88wAFM6xCuTZhQ@mail.gmail.com>
	<521F6BC4.5070807@lanl.gov>
	<CABL7CQjq6wZdfFmBgMhF5kFGpgQxqCY-Nv20=ZbMtLwPXDDa-w@mail.gmail.com>
Message-ID: <521F8EAB.4090506@redtetrahedron.org>


>     African or European?
>
>
> Why on earth would you ask that?
>
>

Its a Monty Python and the Holy Grail reference.

Eric


From ralf.gommers at gmail.com  Thu Aug 29 15:48:20 2013
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Thu, 29 Aug 2013 19:48:20 +0000
Subject: [Numpy-discussion] Relative speed
In-Reply-To: <521F8EAB.4090506@redtetrahedron.org>
References: <CANHk5kyhQMB_9AHVonvPmAOt5iiTbOGP7-pZ88wAFM6xCuTZhQ@mail.gmail.com>
	<521F6BC4.5070807@lanl.gov>
	<CABL7CQjq6wZdfFmBgMhF5kFGpgQxqCY-Nv20=ZbMtLwPXDDa-w@mail.gmail.com>
	<521F8EAB.4090506@redtetrahedron.org>
Message-ID: <CABL7CQg=Y_USboUy_3Uv0tqWmX9yVsQeKMqpjqGKUvfzp+UZ=A@mail.gmail.com>

On Thu, Aug 29, 2013 at 6:10 PM, Eric Moore <ewm at redtetrahedron.org> wrote:

>
> >     African or European?
> >
> >
> > Why on earth would you ask that?
> >
> >
>
> Its a Monty Python and the Holy Grail reference.


Thanks. I had read that quite differently, and I'm sure I'm not the only
one. Some context would have helped....

Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130829/ff62321d/attachment.html>

From alan.isaac at gmail.com  Thu Aug 29 16:07:08 2013
From: alan.isaac at gmail.com (Alan G Isaac)
Date: Thu, 29 Aug 2013 16:07:08 -0400
Subject: [Numpy-discussion] Relative speed
In-Reply-To: <CABL7CQg=Y_USboUy_3Uv0tqWmX9yVsQeKMqpjqGKUvfzp+UZ=A@mail.gmail.com>
References: <CANHk5kyhQMB_9AHVonvPmAOt5iiTbOGP7-pZ88wAFM6xCuTZhQ@mail.gmail.com>
	<521F6BC4.5070807@lanl.gov>
	<CABL7CQjq6wZdfFmBgMhF5kFGpgQxqCY-Nv20=ZbMtLwPXDDa-w@mail.gmail.com>
	<521F8EAB.4090506@redtetrahedron.org>
	<CABL7CQg=Y_USboUy_3Uv0tqWmX9yVsQeKMqpjqGKUvfzp+UZ=A@mail.gmail.com>
Message-ID: <521FA9EC.1090608@gmail.com>

On 8/29/2013 3:48 PM, Ralf Gommers wrote:
> Some context would have helped.


http://www.youtube.com/watch?v=y2R3FvS4xr4

fwiw,
Alan



From jniehof at lanl.gov  Thu Aug 29 16:11:47 2013
From: jniehof at lanl.gov (Jonathan T. Niehof)
Date: Thu, 29 Aug 2013 14:11:47 -0600
Subject: [Numpy-discussion] Relative speed
In-Reply-To: <CABL7CQg=Y_USboUy_3Uv0tqWmX9yVsQeKMqpjqGKUvfzp+UZ=A@mail.gmail.com>
References: <CANHk5kyhQMB_9AHVonvPmAOt5iiTbOGP7-pZ88wAFM6xCuTZhQ@mail.gmail.com>
	<521F6BC4.5070807@lanl.gov>
	<CABL7CQjq6wZdfFmBgMhF5kFGpgQxqCY-Nv20=ZbMtLwPXDDa-w@mail.gmail.com>
	<521F8EAB.4090506@redtetrahedron.org>
	<CABL7CQg=Y_USboUy_3Uv0tqWmX9yVsQeKMqpjqGKUvfzp+UZ=A@mail.gmail.com>
Message-ID: <521FAB03.8010204@lanl.gov>

On 08/29/2013 01:48 PM, Ralf Gommers wrote:

> Thanks. I had read that quite differently, and I'm sure I'm not the only
> one. Some context would have helped....

My apologies--that was a rather obtuse reference.

In my oddly-wired brain it struck me as a fairly similar, 
suboptimally-posed question: all data structures sit in memory at the 
same speed, it's a question of the operations. And as you pointed out, 
most of the time for non-trivial datasets the numpy operations will be 
faster. (I'm daunted by the notion of trying to do linear algebra on 
lists of tuples, assuming that's the relevant set of operations given 
the comparison to the matrix class.)

-- 
Jonathan Niehof
ISR-3 Space Data Systems
Los Alamos National Laboratory
MS-D466
Los Alamos, NM 87545

Phone: 505-667-9595
email: jniehof at lanl.gov

Correspondence /
Technical data or Software Publicly Available


From zachary.pincus at yale.edu  Thu Aug 29 16:32:41 2013
From: zachary.pincus at yale.edu (Zachary Pincus)
Date: Thu, 29 Aug 2013 16:32:41 -0400
Subject: [Numpy-discussion] Relative speed
In-Reply-To: <521FAB03.8010204@lanl.gov>
References: <CANHk5kyhQMB_9AHVonvPmAOt5iiTbOGP7-pZ88wAFM6xCuTZhQ@mail.gmail.com>
	<521F6BC4.5070807@lanl.gov>
	<CABL7CQjq6wZdfFmBgMhF5kFGpgQxqCY-Nv20=ZbMtLwPXDDa-w@mail.gmail.com>
	<521F8EAB.4090506@redtetrahedron.org>
	<CABL7CQg=Y_USboUy_3Uv0tqWmX9yVsQeKMqpjqGKUvfzp+UZ=A@mail.gmail.com>
	<521FAB03.8010204@lanl.gov>
Message-ID: <F8F5D4BE-EA25-442B-B940-67FFC2BCED45@yale.edu>

> And as you pointed out, 
> most of the time for non-trivial datasets the numpy operations will be 
> faster. (I'm daunted by the notion of trying to do linear algebra on 
> lists of tuples, assuming that's the relevant set of operations given 
> the comparison to the matrix class.)

Note the important and pretty common exception of building up a list one element (or row of elements) at a time. Here, python lists usually rule, unless the final size is known in advance.



From joseluismietta at yahoo.com.ar  Thu Aug 29 18:55:17 2013
From: joseluismietta at yahoo.com.ar (=?iso-8859-1?Q?Jos=E8_Luis_Mietta?=)
Date: Thu, 29 Aug 2013 15:55:17 -0700 (PDT)
Subject: [Numpy-discussion] Stick (line segments) percolation algorithm
	- graph theory?
In-Reply-To: <CAFq1z2VtjKie6W6LGobMgcQpSTDAQuA6kgKDduZA1SAbKwdWpA@mail.gmail.com>
References: <1377295771.604.YahooMailNeo@web142305.mail.bf1.yahoo.com>
	<CAFq1z2VtjKie6W6LGobMgcQpSTDAQuA6kgKDduZA1SAbKwdWpA@mail.gmail.com>
Message-ID: <1377816917.36149.YahooMailNeo@web142303.mail.bf1.yahoo.com>

Thanks a lot!!
?
Jos? Luis


________________________________
 De: Brett Olsen <brett.olsen at gmail.com>
Para: Discussion of Numerical Python <numpy-discussion at scipy.org> 
Enviado: lunes, 26 de agosto de 2013 14:08
Asunto: Re: [Numpy-discussion] Stick (line segments) percolation algorithm - graph theory?
 


I can see a couple opportunities for improvements in your algorithm. ?Running your code on a single experiment, I get about 2.9 seconds to run. I get this down to about 1.0 seconds by (1) exploiting the symmetry of the M matrix and (2) avoiding the costly inner loop over k in favor of array operations:

def check_segments(j, others, data):
? ? x1, y1, x2, y2 = data
? ??
? ? x_A1B1 = x2[j]-x1[j]
? ? y_A1B1 = y2[j]-y1[j]
? ??
? ? x_A1A2 = x1[others]-x1[j]
? ? y_A1A2 = y1[others]-y1[j] ? ? ?
? ??
? ? x_A2A1 = -1*x_A1A2
? ? y_A2A1 = -1*y_A1A2
? ??
? ? x_A2B2 = x2[others]-x1[others]
? ? y_A2B2 = y2[others]-y1[others]
? ??
? ? x_A1B2 = x2[others]-x1[j]
? ? y_A1B2 = y2[others]-y1[j]
? ??
? ? x_A2B1 = x2[j]-x1[others]
? ? y_A2B1 = y2[j]-y1[others]

? ? p1 = x_A1B1*y_A1A2 - y_A1B1*x_A1A2
? ? p2 = x_A1B1*y_A1B2 - y_A1B1*x_A1B2
? ? p3 = x_A2B2*y_A2B1 - y_A2B2*x_A2B1
? ? p4 = x_A2B2*y_A2A1 - y_A2B2*x_A2A1

? ? condition_1=p1*p2
? ? condition_2=p3*p4 ? ? ? ? ? ? ? ? ? ? ? ??

? ? return (p1 * p2 <= 0) & (p3 * p4 <= 0)

for j in xrange(1, N):
? ? valid = check_segments(j, range(j), (x1, y1, x2, y2))
? ? M[j,0:j] = valid
? ? M[0:j,j] = valid

I don't see any other particularly simple ways to improve this. ?You could probably add an interval check to ensure that the x and y intervals for the segments of interest overlap before doing the full check, but how much that would help would depend on the implementations.

~Brett


On Fri, Aug 23, 2013 at 5:09 PM, Jos? Luis Mietta <joseluismietta at yahoo.com.ar> wrote:

I wrote an algorithm for study stick percolation (i.e.: networks between line segments that intersect between them). In my algorithm N sticks (line segments) are created inside a rectangular box of sides 'b' and 'h' and then, one by one, the algorithm explores the intersection between all line segments. This is a Monte Carlo simulation, so the 'experiment' is executed many times (no less than 100 times). Written like that, very much RAM is consumed: ?Here, the element Mij=1 if stick i intersects stick j and Mij=0 if not.
>
>How can I optimize my algorithm? Graph theory is useful in this case?

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130829/ae9a3ba9/attachment.html>

From charlesr.harris at gmail.com  Thu Aug 29 19:27:43 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Thu, 29 Aug 2013 17:27:43 -0600
Subject: [Numpy-discussion] _PyADt
Message-ID: <CAB6mnxLiCTM4rnHHp6csaTP9D3-=UL=OV8RoMnT1Uv7xhroxyQ@mail.gmail.com>

Anyone know what _PyADt is? It turns up in ndarraytypes.h

#define PyDataType_ISBOOL(obj) PyTypeNum_ISBOOL(_PyADt(obj))

and only there. It's not in the build directory, google yields nothing. I
suspect it is an historical artifact turned bug and should be replaced by
((PyArray_Descr*)(obj))->type_num.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130829/8702a9c9/attachment.html>

From cjwilliams43 at gmail.com  Thu Aug 29 20:40:44 2013
From: cjwilliams43 at gmail.com (Colin J. Williams)
Date: Thu, 29 Aug 2013 20:40:44 -0400
Subject: [Numpy-discussion] Matrix peculiarities
Message-ID: <521FEA0C.50503@gmail.com>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130829/da0178a0/attachment.html>

From ben.root at ou.edu  Thu Aug 29 23:26:55 2013
From: ben.root at ou.edu (Benjamin Root)
Date: Thu, 29 Aug 2013 23:26:55 -0400
Subject: [Numpy-discussion] Relative speed
In-Reply-To: <521FAB03.8010204@lanl.gov>
References: <CANHk5kyhQMB_9AHVonvPmAOt5iiTbOGP7-pZ88wAFM6xCuTZhQ@mail.gmail.com>
	<521F6BC4.5070807@lanl.gov>
	<CABL7CQjq6wZdfFmBgMhF5kFGpgQxqCY-Nv20=ZbMtLwPXDDa-w@mail.gmail.com>
	<521F8EAB.4090506@redtetrahedron.org>
	<CABL7CQg=Y_USboUy_3Uv0tqWmX9yVsQeKMqpjqGKUvfzp+UZ=A@mail.gmail.com>
	<521FAB03.8010204@lanl.gov>
Message-ID: <CANNq6Fnom6QShkeNS9S7qgmGD699vWfoV+_DDVTW8rMH80-gag@mail.gmail.com>

On Aug 29, 2013 4:11 PM, "Jonathan T. Niehof" <jniehof at lanl.gov> wrote:
>
> On 08/29/2013 01:48 PM, Ralf Gommers wrote:
>
> > Thanks. I had read that quite differently, and I'm sure I'm not the only
> > one. Some context would have helped....
>
> My apologies--that was a rather obtuse reference.
>

Just for future reference, the language and the community is full of
references like these. IDLE, is named for Eric Idle, one of the members of
Monty Python, while Guido's title of BDFL is a reference to a sketch.

But I am sure you'd never expected that... :-p

Cheers!
Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130829/be050bcf/attachment.html>

From anubhab91 at gmail.com  Fri Aug 30 00:20:10 2013
From: anubhab91 at gmail.com (Anubhab Baksi)
Date: Fri, 30 Aug 2013 09:50:10 +0530
Subject: [Numpy-discussion] Relative speed
In-Reply-To: <CABL7CQjq6wZdfFmBgMhF5kFGpgQxqCY-Nv20=ZbMtLwPXDDa-w@mail.gmail.com>
References: <CANHk5kyhQMB_9AHVonvPmAOt5iiTbOGP7-pZ88wAFM6xCuTZhQ@mail.gmail.com>
	<521F6BC4.5070807@lanl.gov>
	<CABL7CQjq6wZdfFmBgMhF5kFGpgQxqCY-Nv20=ZbMtLwPXDDa-w@mail.gmail.com>
Message-ID: <CANHk5kyzA+hr2rPS4uBPUDdK4dtE=bTzZDEsordzWMM59mHpkA@mail.gmail.com>

Thanks all, my client actually wants the output at a minimum time.

On Thu, Aug 29, 2013 at 11:30 PM, Ralf Gommers <ralf.gommers at gmail.com>wrote:

>
> if you have a reasonably large amount of data (say O(100)),
>
> I need to deal with nearly 2**19 or 2**20 arrays of length about 250 each.


On Thu, Aug 29, 2013 at 11:30 PM, Ralf Gommers <ralf.gommers at gmail.com>wrote:

>
>
>
> On Thu, Aug 29, 2013 at 3:41 PM, Jonathan T. Niehof <jniehof at lanl.gov>wrote:
>
>> On 08/29/2013 09:33 AM, Anubhab Baksi wrote:
>> > Hi,
>> > I need to know about the relative speed (i.e., which one is faster) of
>> > the followings:
>> > 1. list and numpy array, tuples and numpy array
>> > 2. list of tuples and numpy matrix (first one is rectangular)
>> > 3. random.randint() and numpy.random.random_integers()
>>
>
> Hi Anubhab, if you have a reasonably large amount of data (say O(100)),
> always try to use numpy arrays and not lists or tuples - it'll be faster.
> I'd recommend not to use numpy.matrix, it's speed will be similar to numpy
> arrays but it has some peculiarities that you'd rather not deal with. For
> the random numbers I'm not sure without checking, just timing it in ipython
> with %timeit is indeed the way to go.
>
> Cheers,
> Ralf
>
>
>> African or European?
>>
>
> Why on earth would you ask that?
>
>
>
>> It really depends on what you're doing with it. The ipython %timeit
>> magic is pretty useful for answering that question. Note that the answer
>> may change dramatically based on the size of the data set.
>>
>> --
>> Jonathan Niehof
>> ISR-3 Space Data Systems
>> Los Alamos National Laboratory
>> MS-D466
>> Los Alamos, NM 87545
>>
>> Phone: 505-667-9595
>> email: jniehof at lanl.gov
>>
>> Correspondence /
>> Technical data or Software Publicly Available
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130830/c127bbfd/attachment.html>

From stefan at sun.ac.za  Fri Aug 30 04:10:30 2013
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Fri, 30 Aug 2013 10:10:30 +0200
Subject: [Numpy-discussion] Relative speed
In-Reply-To: <CANHk5kyzA+hr2rPS4uBPUDdK4dtE=bTzZDEsordzWMM59mHpkA@mail.gmail.com>
References: <CANHk5kyhQMB_9AHVonvPmAOt5iiTbOGP7-pZ88wAFM6xCuTZhQ@mail.gmail.com>
	<521F6BC4.5070807@lanl.gov>
	<CABL7CQjq6wZdfFmBgMhF5kFGpgQxqCY-Nv20=ZbMtLwPXDDa-w@mail.gmail.com>
	<CANHk5kyzA+hr2rPS4uBPUDdK4dtE=bTzZDEsordzWMM59mHpkA@mail.gmail.com>
Message-ID: <CABDkGQ=3uRyxNGs-0KATKmoDNVayW2RWK9JG6Cb=HNs6LKmtQg@mail.gmail.com>

On Fri, Aug 30, 2013 at 6:20 AM, Anubhab Baksi <anubhab91 at gmail.com> wrote:
> I need to deal with nearly 2**19 or 2**20 arrays of length about 250 each.

As mentioned elsewhere in this thread: what does "deal" mean.  You may
be better off with something like:

http://kwant-project.org/tinyarray/

St?fan


From stefan at sun.ac.za  Fri Aug 30 04:15:53 2013
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Fri, 30 Aug 2013 10:15:53 +0200
Subject: [Numpy-discussion] _PyADt
In-Reply-To: <CAB6mnxLiCTM4rnHHp6csaTP9D3-=UL=OV8RoMnT1Uv7xhroxyQ@mail.gmail.com>
References: <CAB6mnxLiCTM4rnHHp6csaTP9D3-=UL=OV8RoMnT1Uv7xhroxyQ@mail.gmail.com>
Message-ID: <CABDkGQme3=XckHoJ0B_mWt08vf=wSYgp23Sqpmi7gS4BoyVaCg@mail.gmail.com>

On Fri, Aug 30, 2013 at 1:27 AM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
> Anyone know what _PyADt is? It turns up in ndarraytypes.h
>
> #define PyDataType_ISBOOL(obj) PyTypeNum_ISBOOL(_PyADt(obj))

That code looks broken--can't we just remove it?

St?fan


From Nicolas.Rougier at inria.fr  Fri Aug 30 11:26:51 2013
From: Nicolas.Rougier at inria.fr (Nicolas Rougier)
Date: Fri, 30 Aug 2013 17:26:51 +0200
Subject: [Numpy-discussion] Structured array dtype
Message-ID: <382D8659-0481-487A-8B22-E8C63CF337BE@inria.fr>


Hi,

I'm a bit lost with the following example (numpy 1.7.1, osx 10.8):


>>> Z = np.zeros(10, [('a', np.float32, 3), ('b', np.float32, 4)])

>>> Z['a'].dtype
dtype('float32')

>>> Z.dtype['a']
dtype(('<f4', (3,)))


Does that mean that dtype['a'] is the dtype of field 'a' when in Z, while Z['a'].dtype is the dtype of field 'a' when "extracted" or my way of thinking is totally wrong ?


What bothers me the most is that I cannot do:

>>> Z['a'].view(Z.dtype['a'])
ValueError: new type not compatible with array.


Nicolas

From cournape at gmail.com  Fri Aug 30 14:16:05 2013
From: cournape at gmail.com (David Cournapeau)
Date: Fri, 30 Aug 2013 19:16:05 +0100
Subject: [Numpy-discussion] ANN: Scipy 0.13.0 beta 1 release
In-Reply-To: <CABL7CQga-Qkq8_hG1E+_KfpmVjSE-duyRePVcZw_-s-PXay75w@mail.gmail.com>
References: <CABL7CQga-Qkq8_hG1E+_KfpmVjSE-duyRePVcZw_-s-PXay75w@mail.gmail.com>
Message-ID: <CAGY4rcW0svTy_WH9GR2C_JTcAea5gBj0G3uD7sEnVOLqozKMRA@mail.gmail.com>

It looks like it broke the build with MKL as well (in, surprised, ARPACK).
I will investigate this further this WE


On Thu, Aug 22, 2013 at 2:12 PM, Ralf Gommers <ralf.gommers at gmail.com>wrote:

> Hi all,
>
> I'm happy to announce the availability of the first beta release of Scipy
> 0.13.0. Please try this beta and report any issues on the scipy-dev mailing
> list.
>
> Source tarballs and release notes can be found at
> https://sourceforge.net/projects/scipy/files/scipy/0.13.0b1/. Windows and
> OS X installers will follow later (we have a minor infrastructure issue to
> solve, and I'm at EuroScipy now).
>
> Cheers,
> Ralf
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130830/6b53c0f1/attachment.html>

From stefan at sun.ac.za  Sat Aug 31 01:51:36 2013
From: stefan at sun.ac.za (=?iso-8859-1?Q?St=E9fan?= van der Walt)
Date: Sat, 31 Aug 2013 07:51:36 +0200
Subject: [Numpy-discussion] Structured array dtype
In-Reply-To: <382D8659-0481-487A-8B22-E8C63CF337BE@inria.fr>
References: <382D8659-0481-487A-8B22-E8C63CF337BE@inria.fr>
Message-ID: <20130831055136.GD14495@shinobi>

Hi Nicolas

On Fri, 30 Aug 2013 17:26:51 +0200, Nicolas Rougier wrote:
> >>> Z = np.zeros(10, [('a', np.float32, 3), ('b', np.float32, 4)])
> 
> >>> Z['a'].dtype
> dtype('float32')
> 
> >>> Z.dtype['a']
> dtype(('<f4', (3,)))
> 
> 
> Does that mean that dtype['a'] is the dtype of field 'a' when in Z, while Z['a'].dtype is the dtype of field 'a' when "extracted" or my way of thinking is totally wrong ?

Apologies if this is a duplicate response; I'm sending it offline.

In case 1, you are indexing into the array, and querying its dtype.  In case
two, you are indexing into a dtype.

I.e., in case two, you are doing this:

In [18]: dtype = np.dtype([('a', float, 3), ('b', int)])

In [19]: dtype['a']
Out[19]: dtype(('<f8', (3,)))

> What bothers me the most is that I cannot do:
> 
> >>> Z['a'].view(Z.dtype['a'])
> ValueError: new type not compatible with array.

That's quite a tricky operation to perform, since it has to take into account
the underlying strides of the old array as well as calculate a shape for the
new array.  It should be possible to make it work using something similar to
`np.lib.stride_tricks.as_strided`, but my quick attempt failed because of the
following:

In [13]: class Foo:
    __array_interface__ = Z.__array_interface__
   ....:

In [14]: f = Foo()

In [15]: np.asarray(f)
Out[15]:
array([, ,
       , ,
       , ,
       , ,
       , ],
      dtype='|V28')

This does not seem right.

St?fan



From Nicolas.Rougier at inria.fr  Sat Aug 31 03:05:11 2013
From: Nicolas.Rougier at inria.fr (Nicolas Rougier)
Date: Sat, 31 Aug 2013 09:05:11 +0200
Subject: [Numpy-discussion] Structured array dtype
In-Reply-To: <20130831055136.GD14495@shinobi>
References: <382D8659-0481-487A-8B22-E8C63CF337BE@inria.fr>
	<20130831055136.GD14495@shinobi>
Message-ID: <149FF6D8-12E6-4191-91D6-393581CC87B5@inria.fr>



Thanks for the explanation.
In fact, since the 4 components of 'b' are contiguous in memory, I wanted to find a way to express that fact in the dtype.

Z = np.zeros(10, [('a', np.float32, 3), ('b', np.float32, 4)])
Z['b'].strides
(28,4)

Z = np.zeros((10,4), np.float32)
Z.strides
(16,4)

Z = np.zeros(10, (np.float32,4))
Z.strides
(16,4)




Nicolas



On Aug 31, 2013, at 7:51 AM, St?fan van der Walt <stefan at sun.ac.za> wrote:

> Hi Nicolas
> 
> On Fri, 30 Aug 2013 17:26:51 +0200, Nicolas Rougier wrote:
>>>>> Z = np.zeros(10, [('a', np.float32, 3), ('b', np.float32, 4)])
>> 
>>>>> Z['a'].dtype
>> dtype('float32')
>> 
>>>>> Z.dtype['a']
>> dtype(('<f4', (3,)))
>> 
>> 
>> Does that mean that dtype['a'] is the dtype of field 'a' when in Z, while Z['a'].dtype is the dtype of field 'a' when "extracted" or my way of thinking is totally wrong ?
> 
> Apologies if this is a duplicate response; I'm sending it offline.
> 
> In case 1, you are indexing into the array, and querying its dtype.  In case
> two, you are indexing into a dtype.
> 
> I.e., in case two, you are doing this:
> 
> In [18]: dtype = np.dtype([('a', float, 3), ('b', int)])
> 
> In [19]: dtype['a']
> Out[19]: dtype(('<f8', (3,)))
> 
>> What bothers me the most is that I cannot do:
>> 
>>>>> Z['a'].view(Z.dtype['a'])
>> ValueError: new type not compatible with array.
> 
> That's quite a tricky operation to perform, since it has to take into account
> the underlying strides of the old array as well as calculate a shape for the
> new array.  It should be possible to make it work using something similar to
> `np.lib.stride_tricks.as_strided`, but my quick attempt failed because of the
> following:
> 
> In [13]: class Foo:
>    __array_interface__ = Z.__array_interface__
>   ....:
> 
> In [14]: f = Foo()
> 
> In [15]: np.asarray(f)
> Out[15]:
> array([, ,
>       , ,
>       , ,
>       , ,
>       , ],
>      dtype='|V28')
> 
> This does not seem right.
> 
> St?fan
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion



From charlesr.harris at gmail.com  Sat Aug 31 10:09:47 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 31 Aug 2013 08:09:47 -0600
Subject: [Numpy-discussion] Permissions on Sourceforge
Message-ID: <CAB6mnxJGf7sBgsMw0tLKOr6iZt65ijRnnNmEFyWY+Xcy-ZX+OQ@mail.gmail.com>

Hi Travis, Robert, et al

As part of fixing up the release infrastructure it would be helpful if you
could make Ralf and myself admins of the numpy and scipy sites on
Sourceforge. Thoughts?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130831/19e7c7b6/attachment.html>

From robert.kern at gmail.com  Sat Aug 31 10:41:00 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 31 Aug 2013 15:41:00 +0100
Subject: [Numpy-discussion] Permissions on Sourceforge
In-Reply-To: <CAB6mnxJGf7sBgsMw0tLKOr6iZt65ijRnnNmEFyWY+Xcy-ZX+OQ@mail.gmail.com>
References: <CAB6mnxJGf7sBgsMw0tLKOr6iZt65ijRnnNmEFyWY+Xcy-ZX+OQ@mail.gmail.com>
Message-ID: <CAF6FJiuvvi1ekJuvkQ=Zq95EXutv22QfAKbrauZqz_KpO01EYw@mail.gmail.com>

On Sat, Aug 31, 2013 at 3:09 PM, Charles R Harris <charlesr.harris at gmail.com
> wrote:

> Hi Travis, Robert, et al
>
> As part of fixing up the release infrastructure it would be helpful if you
> could make Ralf and myself admins of the numpy and scipy sites on
> Sourceforge. Thoughts?
>

What is your SF user name? I have taken care of Ralf on numpy. I am not an
admin on scipy.

-- 
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130831/962733d5/attachment.html>

From charlesr.harris at gmail.com  Sat Aug 31 11:18:19 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 31 Aug 2013 09:18:19 -0600
Subject: [Numpy-discussion] Permissions on Sourceforge
In-Reply-To: <CAF6FJiuvvi1ekJuvkQ=Zq95EXutv22QfAKbrauZqz_KpO01EYw@mail.gmail.com>
References: <CAB6mnxJGf7sBgsMw0tLKOr6iZt65ijRnnNmEFyWY+Xcy-ZX+OQ@mail.gmail.com>
	<CAF6FJiuvvi1ekJuvkQ=Zq95EXutv22QfAKbrauZqz_KpO01EYw@mail.gmail.com>
Message-ID: <CAB6mnx+B87165GjGG8cGiVHTCF3UDSmeb94eEB2rfWoLCpLqxg@mail.gmail.com>

On Sat, Aug 31, 2013 at 8:41 AM, Robert Kern <robert.kern at gmail.com> wrote:

> On Sat, Aug 31, 2013 at 3:09 PM, Charles R Harris <
> charlesr.harris at gmail.com> wrote:
>
>> Hi Travis, Robert, et al
>>
>> As part of fixing up the release infrastructure it would be helpful if
>> you could make Ralf and myself admins of the numpy and scipy sites on
>> Sourceforge. Thoughts?
>>
>
> What is your SF user name? I have taken care of Ralf on numpy. I am not an
> admin on scipy.
>
>
charris208

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130831/2ce67bca/attachment.html>

From robert.kern at gmail.com  Sat Aug 31 11:19:48 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 31 Aug 2013 16:19:48 +0100
Subject: [Numpy-discussion] Permissions on Sourceforge
In-Reply-To: <CAB6mnx+B87165GjGG8cGiVHTCF3UDSmeb94eEB2rfWoLCpLqxg@mail.gmail.com>
References: <CAB6mnxJGf7sBgsMw0tLKOr6iZt65ijRnnNmEFyWY+Xcy-ZX+OQ@mail.gmail.com>
	<CAF6FJiuvvi1ekJuvkQ=Zq95EXutv22QfAKbrauZqz_KpO01EYw@mail.gmail.com>
	<CAB6mnx+B87165GjGG8cGiVHTCF3UDSmeb94eEB2rfWoLCpLqxg@mail.gmail.com>
Message-ID: <CAF6FJitn9s2FWUJjJNhkacqzsKtE2-VD2fGNGjFy6rtGBZThkQ@mail.gmail.com>

On Sat, Aug 31, 2013 at 4:18 PM, Charles R Harris <charlesr.harris at gmail.com
> wrote:

>
> On Sat, Aug 31, 2013 at 8:41 AM, Robert Kern <robert.kern at gmail.com>wrote:
>
>> On Sat, Aug 31, 2013 at 3:09 PM, Charles R Harris <
>> charlesr.harris at gmail.com> wrote:
>>
>>> Hi Travis, Robert, et al
>>>
>>> As part of fixing up the release infrastructure it would be helpful if
>>> you could make Ralf and myself admins of the numpy and scipy sites on
>>> Sourceforge. Thoughts?
>>>
>>
>> What is your SF user name? I have taken care of Ralf on numpy. I am not
>> an admin on scipy.
>>
>>
> charris208
>

Done, for numpy. Someone else will have to help with scipy.

-- 
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130831/2d93240c/attachment.html>

From charlesr.harris at gmail.com  Sat Aug 31 11:50:56 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 31 Aug 2013 09:50:56 -0600
Subject: [Numpy-discussion] Permissions on Sourceforge
In-Reply-To: <CAF6FJitn9s2FWUJjJNhkacqzsKtE2-VD2fGNGjFy6rtGBZThkQ@mail.gmail.com>
References: <CAB6mnxJGf7sBgsMw0tLKOr6iZt65ijRnnNmEFyWY+Xcy-ZX+OQ@mail.gmail.com>
	<CAF6FJiuvvi1ekJuvkQ=Zq95EXutv22QfAKbrauZqz_KpO01EYw@mail.gmail.com>
	<CAB6mnx+B87165GjGG8cGiVHTCF3UDSmeb94eEB2rfWoLCpLqxg@mail.gmail.com>
	<CAF6FJitn9s2FWUJjJNhkacqzsKtE2-VD2fGNGjFy6rtGBZThkQ@mail.gmail.com>
Message-ID: <CAB6mnx+DcNsAGr2TSNs44pvDKWy6ARAZSK0JoAEm9TdSQhXdMA@mail.gmail.com>

On Sat, Aug 31, 2013 at 9:19 AM, Robert Kern <robert.kern at gmail.com> wrote:

> On Sat, Aug 31, 2013 at 4:18 PM, Charles R Harris <
> charlesr.harris at gmail.com> wrote:
>
>>
>> On Sat, Aug 31, 2013 at 8:41 AM, Robert Kern <robert.kern at gmail.com>wrote:
>>
>>> On Sat, Aug 31, 2013 at 3:09 PM, Charles R Harris <
>>> charlesr.harris at gmail.com> wrote:
>>>
>>>> Hi Travis, Robert, et al
>>>>
>>>> As part of fixing up the release infrastructure it would be helpful if
>>>> you could make Ralf and myself admins of the numpy and scipy sites on
>>>> Sourceforge. Thoughts?
>>>>
>>>
>>> What is your SF user name? I have taken care of Ralf on numpy. I am not
>>> an admin on scipy.
>>>
>>>
>> charris208
>>
>
> Done, for numpy. Someone else will have to help with scipy.
>
>
Thanks Robert.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130831/7af4afa5/attachment.html>

From faltet at gmail.com  Sat Aug 31 11:58:05 2013
From: faltet at gmail.com (Francesc Alted)
Date: Sat, 31 Aug 2013 17:58:05 +0200
Subject: [Numpy-discussion] [ANN] numexpr 2.2 released
Message-ID: <CAFrp1vrPafSv=yooiLbps+b_xHT_gCMwq8AuCmQSWE5tWukgMA@mail.gmail.com>

==========================
 Announcing Numexpr 2.2
==========================

Numexpr is a fast numerical expression evaluator for NumPy.  With it,
expressions that operate on arrays (like "3*a+4*b") are accelerated
and use less memory than doing the same calculation in Python.

It wears multi-threaded capabilities, as well as support for Intel's
VML library (included in Intel MKL), which allows an extremely fast
evaluation of transcendental functions (sin, cos, tan, exp, log...)
while squeezing the last drop of performance out of your multi-core
processors.

Its only dependency is NumPy (MKL is optional), so it works well as an
easy-to-deploy, easy-to-use, computational kernel for projects that
don't want to adopt other solutions that require more heavy
dependencies.

What's new
==========

This release is mainly meant to fix a problem with the license the
numexpr/win32/pthread.{c,h} files emulating pthreads on Windows. After
persmission from the original authors is granted, these files adopt
the MIT license and can be redistributed without problems.  See issue
#109 for details
(https://code.google.com/p/numexpr/issues/detail?id=110).

Another important improvement is the new algorithm to decide the initial
number of threads to be used.  This was necessary because by default,
numexpr was using a number of threads equal to the detected number of
cores, and this can be just too much for moder systems where this
number can be too high (and counterporductive for performance in many
cases).  Now, the 'NUMEXPR_NUM_THREADS' environment variable is
honored, and in case this is not present, a maximum number of *8*
threads are setup initially.  The new algorithm is fully described in
the Users Guide, in the note of 'General routines' section:
https://code.google.com/p/numexpr/wiki/UsersGuide#General_routines.
Closes #110.

In case you want to know more in detail what has changed in this
version, see:

http://code.google.com/p/numexpr/wiki/ReleaseNotes

or have a look at RELEASE_NOTES.txt in the tarball.

Where I can find Numexpr?
=========================

The project is hosted at Google code in:

http://code.google.com/p/numexpr/

You can get the packages from PyPI as well:

http://pypi.python.org/pypi/numexpr

Share your experience
=====================

Let us know of any bugs, suggestions, gripes, kudos, etc. you may
have.


Enjoy data!

-- 
Francesc Alted
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130831/8758f031/attachment.html>

From travis at continuum.io  Sat Aug 31 12:27:36 2013
From: travis at continuum.io (Travis Oliphant)
Date: Sat, 31 Aug 2013 11:27:36 -0500
Subject: [Numpy-discussion] Permissions on Sourceforge
In-Reply-To: <CAF6FJitn9s2FWUJjJNhkacqzsKtE2-VD2fGNGjFy6rtGBZThkQ@mail.gmail.com>
References: <CAB6mnxJGf7sBgsMw0tLKOr6iZt65ijRnnNmEFyWY+Xcy-ZX+OQ@mail.gmail.com>
	<CAF6FJiuvvi1ekJuvkQ=Zq95EXutv22QfAKbrauZqz_KpO01EYw@mail.gmail.com>
	<CAB6mnx+B87165GjGG8cGiVHTCF3UDSmeb94eEB2rfWoLCpLqxg@mail.gmail.com>
	<CAF6FJitn9s2FWUJjJNhkacqzsKtE2-VD2fGNGjFy6rtGBZThkQ@mail.gmail.com>
Message-ID: <CAMcnTE79NeJQzJdsvFx1jKJLcVYRYP6cvxcM3xHc4Q1euofk6g@mail.gmail.com>

Done for SciPy.

-Travis



On Sat, Aug 31, 2013 at 10:19 AM, Robert Kern <robert.kern at gmail.com> wrote:

> On Sat, Aug 31, 2013 at 4:18 PM, Charles R Harris <
> charlesr.harris at gmail.com> wrote:
>
>>
>> On Sat, Aug 31, 2013 at 8:41 AM, Robert Kern <robert.kern at gmail.com>wrote:
>>
>>> On Sat, Aug 31, 2013 at 3:09 PM, Charles R Harris <
>>> charlesr.harris at gmail.com> wrote:
>>>
>>>> Hi Travis, Robert, et al
>>>>
>>>> As part of fixing up the release infrastructure it would be helpful if
>>>> you could make Ralf and myself admins of the numpy and scipy sites on
>>>> Sourceforge. Thoughts?
>>>>
>>>
>>> What is your SF user name? I have taken care of Ralf on numpy. I am not
>>> an admin on scipy.
>>>
>>>
>> charris208
>>
>
> Done, for numpy. Someone else will have to help with scipy.
>
> --
> Robert Kern
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>


-- 

Travis Oliphant
Continuum Analytics, Inc.
http://www.continuum.io
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130831/cc6efece/attachment.html>

From charlesr.harris at gmail.com  Sat Aug 31 12:56:46 2013
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 31 Aug 2013 10:56:46 -0600
Subject: [Numpy-discussion] Permissions on Sourceforge
In-Reply-To: <CAMcnTE79NeJQzJdsvFx1jKJLcVYRYP6cvxcM3xHc4Q1euofk6g@mail.gmail.com>
References: <CAB6mnxJGf7sBgsMw0tLKOr6iZt65ijRnnNmEFyWY+Xcy-ZX+OQ@mail.gmail.com>
	<CAF6FJiuvvi1ekJuvkQ=Zq95EXutv22QfAKbrauZqz_KpO01EYw@mail.gmail.com>
	<CAB6mnx+B87165GjGG8cGiVHTCF3UDSmeb94eEB2rfWoLCpLqxg@mail.gmail.com>
	<CAF6FJitn9s2FWUJjJNhkacqzsKtE2-VD2fGNGjFy6rtGBZThkQ@mail.gmail.com>
	<CAMcnTE79NeJQzJdsvFx1jKJLcVYRYP6cvxcM3xHc4Q1euofk6g@mail.gmail.com>
Message-ID: <CAB6mnxJ-HYfhncRKuMykNGU0B24yKTTctuL6F_KGObY+rXVisg@mail.gmail.com>

On Sat, Aug 31, 2013 at 10:27 AM, Travis Oliphant <travis at continuum.io>wrote:

> Done for SciPy.
>
>
Thanks Travis.

Chuck

>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130831/2c284683/attachment.html>