Join us for a datarray sprint on July 28. Several of us will meet at
UC Berkeley from 2pm (Pacific Time) until our fingers bleed from
typing or until 6 or 7pm, whichever comes first.
If you can't be there in person, grab an item from the issue tracker
or create your own. I'm told some of us will hang out on IRC #scipy,
so we can coordinate there.
A datarray is a subclass of a Numpy array that lets you label the axes
and label the elements along each axis. Our target is to get this into
Numpy. We are currently in the prototype stage, so this is a good time
to jump in.
issue tracker: http://github.com/fperez/datarray/issues
The execfile builtin has disappeared in python 3.x, so I'm trying to find
another solution for the use of it in setupegg.py. So far I've tried
if sys.version_info >= 3:
# 3.x doesn't have execfile anymore, so we define our own
# The code below is syntactically valid 2.x, but 2.x thinks that a tuple
# gets passed to the exec statement. (from Twisted)
def execfile(filename, globals=None, locals=None):
exec(compile(open(filename).read(), filename, 'exec'), globals,
if sys.version_info >= 3:
setupfile = imp.load_source('setupfile', 'setup.py')
Both of the above attempts hang at:
$ paver dmg -p 3.1
LDFLAGS='-undefined dynamic_lookup -bundle -arch i386 -arch ppc
Converting to Python3 via 2to3...
Looks like the combination of setuptools, numpy.distutils and 2to3 is too
much for me to wrap my head around. Anyone have an idea?
There is a bug report asking for the example files under doc/cython,
doc/swig etc. to be included in the released binaries. This thread,
seems to indicate the docs should still be integrated in the main
documentation. What do we want to do with the example files, are they still
up to date?
If I create a structured array with vector columns:
>>> array = np.array(zip([[1,2],[1,2],[1,3]]),dtype=[('a',float,2)])
then examine the type of the column, I get:
Then, if I try and view the numerical type, I see:
I have to basically do
to get what I need. I seem to remember that this used not to be the case, and that even for vector columns, one could access array.dtype.type to get the numerical type. Is this a bug, or deliberate?
On May 26th, I sent an email titled "curious about how people would
feel about moving to github." While there were a few concerns raised,
everyone was generally positive and were mainly concerned that this
transition would need to be done carefully with clear workflow
instructions and an clearly marked master repository. Since then
David Cournapeau has done a lot of work looking into how to make the
transition go as smoothly as possible. He has written a script to
convert the svn repository to a git repository. And I've registered a
numpy organization account on github.
Over the next week, David, Stefan, and I will set things up so that
everyone can see how things will work after the transition to git.
David will convert the current trunk to a git repository and put it on
the github site. We will write up instructions on how to use git and
the github site. Everyone who wants to can get a github account and
test out the workflow. At that point everyone can provide feedback
and we can decide if we are ready to move forward. If we are ready to
move forward, we will set up a date for the transition. On that date,
we would turn off the old subversion account and create a new git
repository which would from the point forward be the new master
branch. If the transition to git and github for numpy goes smoothly,
we will turn our attention to scipy.
During the SciPy conference in Austin, we had a Birds-of-a-Feather to
discuss the transition to git and github.
[[ Here is a picture of the git/github BOF (several people joined the
discussion later including Travis Oliphant and Fernando Perez):
At the end of the discussion there was a general consensus that it was
time to make the move. Several questions and concerns were raised:
1. Since there are many possible workflows, it is important to
clearly document are proposed workflow. This document should provide
simple cut-and-paste commands necessary to get developers up and
running with git.
2. The question was raised about how to handle bug reports. It was
pointed out that while our current trac bug report system isn't
perfect, it does work and people are used to it. We decided to keep
our existing trac instance and integrate it with the github site.
Potentially moving from trac to another system like redmine was
brought up, but most people felt it was better to only change one
thing at a time (and besides that no one volunteered to do the work
necessary to move to a new bug tracking system).
3. Since Ralf Gommers is in the middle of making a release, did he
want us to delay any transition preparation until he was finished.
This is Ralf's response when Stefan van der Walt contacted him:
"Thanks for asking! For me the sooner the better, I do everything with
git and haven't touched svn since I discovered Mercurial while writing
my thesis (and that feels like a long long time ago)."
When Stefan contacted Ralf, Ralf raised the following additional concern:
4. "The two things I haven't seen a good solution for are the
svn/externals in scipy which pulls in doc/sphinxext from numpy, and
the vendor branch."
In response, Stefan asked whether submodules would provide a solution.
David Cournapeau responded to Stefan stating "submodules are very
awkward to use." Then David added in response to Ralf's original
"For vendor, it will be a separate repo, and there is no need for
synchronization, so that's easy to deal with. For the sphinx
extension, I would just merge with the subtree strategy from time to
time from a separate repository.
"That's what I do for bento + yaku: yaku has its own repo, and when I
update the copy in bento, I just use git merge -s subtree --no-squash,
so everything is updated in one commit.
"The big advantage is that there is no need to even be aware of the
second repo for users (git clone will bring everything), and there is
no chance of screwing things up:
I ran into a difficulty with floating point arithmetic in python. Namely
>>> 0.001 + 1 - 1
And, as a consequence, in python:
>>> 0.001 + 1 - 1 == 0.001
In more details, my problem is that I have a fonction which needs to
compute (a + b - c) % a. And for b == c, you would expect the result to
be 0 whatever the value of a. But it isn't...
>>> (0.001 + 1 - 1) % 0.001
Is there any way to solve this?
Please forgive me if this is obvious, but this surprised me:
In : x = np.array(['a', 'b'])
In : x == 'a' # this was what I expected
Out: array([ True, False], dtype=bool)
In : x == 1 # this was strange to me
Is it easy to explain why this is?
Thanks a lot,
What are the rules for when 'np.asarray' and 'np.asfortranarray' make a copy?
This makes sense to me:
In : carr = np.arange(3)
In : carr2 = np.asarray(carr)
In : carr2 = 1
In : carr
Out: array([1, 1, 2])
No copy is made.
But doing the same with a fortran array makes a copy:
In : farr = np.arange(3).copy('F')
In : farr2 = np.asfortranarray(farr)
In : farr2 = 1
In : farr
Out: array([0, 1, 2])
Could it be a 1D thing, since it's both C contiguous & F contiguous?
Here's a 2D example:
In : f2D = np.arange(10).reshape((2,5), order='F')
In : f2D2 = np.asfortranarray(f2D)
In : f2D2[0,0] = 10
In : f2D
array([[10, 2, 4, 6, 8],
[ 1, 3, 5, 7, 9]])
So it looks like np.asfortranarray makes an unnecessary copy if the
array is simultaneously 1D, C contiguous and F contiguous.
Coercing the array with np.atleast_2d() makes asfortranarry behave.
Looking further, np.isfortran always returns false if the array is 1D,
even if it's Fortran contiguous (and np.isfortran is documented as
What is the rationale here? Is it a 'column' vs. 'row' thing?