Hi all,
On Fri, Feb 20, 2015 at 10:05 AM, Ralf Gommers <ralf.gommers(a)gmail.com>
wrote:
> Hi all,
>
> It's time to start preparing for this year's Google Summer of Code. There
> is actually one urgent thing to be done (before 19.00 UTC today), which is
> to get our ideas page in decent shape. It doesn't have to be final, but
> there has to be enough on there for the organizers to judge it. This page
> is here: https://github.com/scipy/scipy/wiki/GSoC-project-ideas. I'll be
> reworking it and linking it from the PSF page today, but if you already
> have new ideas please add them there. See
> https://wiki.python.org/moin/SummerOfCode/OrgIdeasPageTemplate for this
> year's template for adding a new idea.
>
The ideas page is now in pretty good shape. More ideas are very welcome
though, especially easy or easy/intermediate ideas. Numpy right now has
zero easy ones and Scipy only one and a half.
What we also need is mentors. All ideas already have a potential mentor
listed, however some ideas are from last year and I'm not sure that all
those mentors really are available this year. And more than one potential
mentor per idea is always good. So can everyone please add/remove his or
her name on that page?
I'm happy to take care of most of the organizational aspects this year,
however I'll be offline for two weeks in July and from the end of August
onwards, so I'll some help in those periods. Any volunteers?
Thanks,
Ralf
An issue was raised yesterday in github, regarding np.may_share_memory when
run on a class exposing an array using the __array__ method. You can check
the details here:
https://github.com/numpy/numpy/issues/5604
Looking into it, I found out that NumPy doesn't really treat objects
exposing __array__,, __array_interface__, or __array_struct__ as if they
were proper arrays:
1. When converting these objects to arrays using PyArray_Converter, if
the arrays returned by any of the array interfaces is not C contiguous,
aligned, and writeable, a copy that is will be made. Proper arrays and
subclasses are passed unchanged. This is the source of the error reported
above.
2. When converting these objects using PyArray_OutputConverter, as well
as in similar code in the ufucn machinery, anything other than a proper
array or subclass raises an error. This means that, contrary to what the
docs on subclassing say, see below, you cannot use an object exposing the
array interface as an output parameter to a ufunc
The following classes can be used to test this behavior:
class Foo:
def __init__(self, arr):
self.arr = arr
def __array__(self):
return self.arr
class Bar:
def __init__(self, arr):
self.arr = arr
self.__array_interface__ = arr.__array_interface__
class Baz:
def __init__(self, arr):
self.arr = arr
self.__array_struct__ = arr.__array_struct__
They all behave the same with these examples:
>>> a = Foo(np.ones(5))
>>> np.add(a, a)
array([ 2., 2., 2., 2., 2.])
>>> np.add.accumulate(a)
array([ 1., 2., 3., 4., 5.])
>>> np.add(a, a, out=a)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: return arrays must be of ArrayType
>>> np.add.accumulate(a, out=a)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: output must be an array
I think this should be changed, and whatever gets handed by this
methods/interfaces be treated as if it were an array or subclass of it.
This is actually what the docs on subclassing say about __array__ here:
http://docs.scipy.org/doc/numpy/reference/arrays.classes.html#numpy.class._…
This also seems to contradict a rather cryptic comment in the code of
PyArray_GetArrayParamsFromObject, which is part of the call sequence of
this whole mess, see here:
https://github.com/numpy/numpy/blob/maintenance/1.9.x/numpy/core/src/multia…
/*
* If op supplies the __array__ function.
* The documentation says this should produce a copy, so
* we skip this method if writeable is true, because the intent
* of writeable is to modify the operand.
* XXX: If the implementation is wrong, and/or if actual
* usage requires this behave differently,
* this should be changed!
*/
There has already been some discussion in the issue linked above, but I
would appreciate any other thoughts on the idea of treating objects with
some form of array interface as if they were arrays. Does it need a
deprecation cycle? Is there some case I am not considering where this could
go horribly wrong?
Jaime
--
(\__/)
( O.o)
( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes
de dominación mundial.
Dear all,
I have initiated a PR-5597 <https://github.com/numpy/numpy/pull/5597>,
which enables the reading of new flags from the site.cfg file.
@rgommers requested that I posted some information on this site, possibly
somebody could test it on their setup.
So the PR basically enables reading these extra options in each section:
runtime_library_dirs : Add runtime library directories to the shared
libraries (overrides the dreaded LD_LIBRARY_PATH)
extra_compile_args: Adds extra compile flags to the compilation
extra_link_args: Adds extra flags when linking to libraries
Note that this PR will "fix" a lot of issues down the line. Specifically
all software which utilises numpy's distutils will benefit from this.
As an example, I have successfully set runtime_library_dirs for site.cfg in
numpy, where scipy, petsc4py, pygsl, slepc4py utilises these flags and this
enables me to create environments without the need for LD_LIBRARY_PATH.
The other options simply adds to the flexibility of the compilation to test
different optimisations etc.
For instance my OpenBLAS section looks like this:
[openblas]
library_dirs = /opt/openblas/0.2.13/gnu-4.9.2/lib
include_dirs = /opt/openblas/0.2.13/gnu-4.9.2/include
runtime_library_dirs = /opt/openblas/0.2.13/gnu-4.9.2/lib
I hope this can be of use to somebody else than me :)
Feel free to test it and provide feedback!
--
Kind regards Nick
This was raised in SO today:
http://stackoverflow.com/questions/28663142/why-is-np-wheres-result-read-on…
np.nonzero (and np.where for boolean arrays) behave differently for 1-D and
higher dimensional arrays:
In the first case, a tuple with a single behaved base ndarray is returned:
>>> a = np.ma.array(range(6))
>>> np.where(a > 3)
(array([4, 5]),)
>>> np.where(a > 3)[0].flags
C_CONTIGUOUS : True
F_CONTIGUOUS : True
OWNDATA : True
WRITEABLE : True
ALIGNED : True
UPDATEIFCOPY : False
In the second, a tuple with as many arrays as dimensions in the passed
array is returned, but the arrays are not base ndarrays, but of the same
subtype as was passed to the function. These arrays are also set as
non-writeable:
>>> np.where(a.reshape(2, 3) > 3)
(masked_array(data = [1 1],
mask = False,
fill_value = 999999)
, masked_array(data = [1 2],
mask = False,
fill_value = 999999)
)
>>> np.where(a.reshape(2, 3) > 3)[0].flags
C_CONTIGUOUS : False
F_CONTIGUOUS : False
OWNDATA : False
WRITEABLE : False
ALIGNED : True
UPDATEIFCOPY : False
I can't think of any reason that justifies this difference, and believe
they should be made to return similar results. My feeling is that the
proper behavior is the 1-D one, and that the behavior for multidimensional
arrays should match it. Anyone can think of any reason that justifies the
current behavior?
Jaime
--
(\__/)
( O.o)
( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes
de dominación mundial.
The idea of a one-byte string dtype has been extensively discussed twice
before, with a lot of good input and ideas, but no action [1, 2].
tl;dr: Perfect is the enemy of good. Can numpy just add a one-byte string
dtype named 's' that uses latin-1 encoding as a bridge to enable Python 3
usage in the near term?
A key consequence of not having a one-byte string dtype is that handling
ASCII data stored in binary formats such as HDF or FITS is basically broken
in Python 3. Packages like h5py, pytables, and astropy.io.fits all return
text data arrays with the numpy 'S' type, and in fact have no direct
support for the numpy wide unicode 'U' type. In Python 3, the 'S' type
array cannot be compared with the Python str type, so that something like
below fails:
>>> mask = (names_array == "john") # FAIL
Problems like this are now showing up in the wild [3]. Workarounds are
also showing up, like a way to easily convert from 'S' to 'U' within
astropy Tables [4], but this is really not a desirable way to go.
Gigabyte-sized string data arrays are not uncommon, so converting to UCS-4
is a real memory and performance hit.
For a good top-level summary of much of the previous thread discussion, see
[5] from Chris Barker. Condensing this down to just a few points:
- *Changing* the behavior of the existing 'S' type is going to break code
and seems a bad idea.
- *Adding* a new dtype 's' will work and allow highly performant
conversion from 'S' to 's' via view().
- Using the latin-1 encoding will minimize code breakage vis-a-vis what
works in Python 2 [6].
Using latin-1 is a pragmatic compromise that provides continuity to allow
scientists to run their existing code in Python 3 and have things just
work. It isn't perfect and it should not be the end of the story, but it
would be good. This single issue is the *only* thing blocking me and my
team from using Python 3 in operations.
As a final point, I don't know the numpy internals at all, but it *seems*
like this proposal is one of the easiest to implement amongst those that
were discussed.
Cheers,
Tom
[1]:
http://mail.scipy.org/pipermail/numpy-discussion/2014-January/068622.html
[2]: http://mail.scipy.org/pipermail/numpy-discussion/2014-July/070574.html
[3]: https://github.com/astropy/astropy/issues/3311
[4]:
http://astropy.readthedocs.org/en/latest/api/astropy.table.Table.html#astro…
[5]: http://mail.scipy.org/pipermail/numpy-discussion/2014-July/070631.html
[6]: It is not uncommon to store uint8 data in a bytestring
Hi,
NumFOCUS has promotes and supports the ongoing research and development of
open-source computing tools including NumPy.
This year NumFOCUS want to try be a Google Summer of Code
"umbrella" mentoring organization,
Umbrella organizations are mentoring organizations accepted into the Google
Summer of Code program that have other open source organizations working
"under" them. Sometime organizations that work very closely or have very
similar goals or communities may get put together under an "umbrella."
Google stills expects all organizations under the umbrella, whether accepted
into the program under their title or not, to adhere to all the rules and
regulations of the program.
From https://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2…
To help promote and support NumPy.
We encourage NumPy to apply to Google Summer of Code under your own title
and will be very happy if you can also do with us.
If you are interested, please check https://github.com/swcarpentry/gsoc2015
and https://github.com/swcarpentry/gsoc2015/blob/master/CONTRIBUTING.md.
If you have any question, please email me directly.
Thanks in advance,
Raniere
Hi all,
It's time to start preparing for this year's Google Summer of Code. There
is actually one urgent thing to be done (before 19.00 UTC today), which is
to get our ideas page in decent shape. It doesn't have to be final, but
there has to be enough on there for the organizers to judge it. This page
is here: https://github.com/scipy/scipy/wiki/GSoC-project-ideas. I'll be
reworking it and linking it from the PSF page today, but if you already
have new ideas please add them there. See
https://wiki.python.org/moin/SummerOfCode/OrgIdeasPageTemplate for this
year's template for adding a new idea.
Cheers,
Ralf
Hi all,
It looks like I'll be at PyCon this year. Anyone else? Any interest in
organizing a numpy sprint?
-n
--
Nathaniel J. Smith -- http://vorpus.org
ffnet-0.8.0 has been released.
ffnet is a fast and easy-to-use feed-forward neural network training
solution for python
This version supports python 3. Look at ffnet website:
http://ffnet.sourceforge.net for installation instructions and documentation.
Regards,
--
Marek Wojciechowski
---
Politechnika Łódzka
Lodz University of Technology
Treść tej wiadomości zawiera informacje przeznaczone tylko dla adresata.
Jeżeli nie jesteście Państwo jej adresatem, bądź otrzymaliście ją przez pomyłkę
prosimy o powiadomienie o tym nadawcy oraz trwałe jej usunięcie.
This email contains information intended solely for the use of the individual to whom it is addressed.
If you are not the intended recipient or if you have received this message in error,
please notify the sender and delete it from your system.
It seems to be agreed that there are weaknesses in the existing Numpy Matrix
Class.
Some problems are illustrated below.
I'll try to put some suggestions over the coming weeks and would appreciate
comments.
Colin W.
Test Script:
if __name__ == '__main__':
a= mat([4, 5, 6]) # Good
print('a: ', a)
b= mat([4, '5', 6]) # Not the expected result
print('b: ', b)
c= mat([[4, 5, 6], [7, 8]]) # Wrongly accepted as rectangular
print('c: ', c)
d= mat([[1, 2, 3]])
try:
d[0, 1]= 'b' # Correctly flagged, not numeric
except ValueError:
print("d[0, 1]= 'b' # Correctly flagged, not numeric", '
ValueError')
print('d: ', d)
Result:
*** Python 2.7.9 (default, Dec 10 2014, 12:28:03) [MSC v.1500 64 bit
(AMD64)] on win32. ***
>>>
a: [[4 5 6]]
b: [['4' '5' '6']]
c: [[[4, 5, 6] [7, 8]]]
d[0, 1]= 'b' # Correctly flagged, not numeric ValueError
d: [[1 2 3]]
>>>
--
View this message in context: http://numpy-discussion.10968.n7.nabble.com/Matrix-Class-tp39719.html
Sent from the Numpy-discussion mailing list archive at Nabble.com.
