Mailman 3 February 2011 - NumPy-Discussion

Numpy 1.6 schedule (was: Numpy 2.0 schedule)
by Ralf Gommers 07 Mar '11

07 Mar '11

On Mon, Feb 28, 2011 at 10:36 PM, Bruce Southey <bsouthey(a)gmail.com> wrote: > On 02/28/2011 02:00 AM, Ralf Gommers wrote: >> Hi, >> >> On Fri, Jan 28, 2011 at 7:15 AM, Travis Oliphant<oliphant(a)enthought.com> wrote: >>> The reason for a NumPy 1.6 suggestion, is that Mark (and others it would >>> seem) have additional work and features that do not need to wait for the >>> NumPy 2.0 ABI design to finalize in order to get out there. >>> If someone is willing to manage the release of NumPy 1.6, then it sounds >>> like a great idea to me. >> This thread ended without a conclusion a month ago. Now I think master >> is in a better state than a month ago for a release (py 2.4/2.5/3.x >> issues and segfault on OS X fixed, more testing of changes), and I >> have a better idea of my free time for March/April. Basically, I have >> a good amount of time for the next couple of weeks, and not so much at >> the end of March / first half of April due to an inter-continental >> move. But I think we can get out a beta by mid-March, and I can manage >> the release. >> >> I've had a look at the bug tracker, here's a list of tickets for 1.6: >> #1748 (blocker: regression for astype('str')) >> #1619 (issue with dtypes, with patch) >> #1749 (distutils, py 3.2) >> #1601 (distutils, py 3.2) >> #1622 (Solaris segfault, with patch) >> #1713 (Solaris segfault) >> #1631 (Solaris segfault) >> >> I can look at the distutils tickets. >> >> The other thing that needs to be done is some (more) documentation of >> new features. Einsum and the new iterator seem to be well documented, >> but not described in the release notes. Datetime has no docs as far as >> I can see except for two similar NEPs. >> >> Proposed schedule: >> March 15: beta 1 >> March 28: rc 1 >> April 17: rc 2 (if needed) >> April 24: final release >> >> Let me know what you think. Bonus points for volunteering to fix some >> of those tickets:) >> >> Cheers, >> Ralf >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion(a)scipy.org >> http://mail.scipy.org/mailman/listinfo/numpy-discussion > Is this 1.6 or 2.0? > The title is 2.0 but you talk about 1.6 so some tickets listed as 2.0 > may apply to 1.6. 1.6, and renamed the thread to make it clearer. > It would be great to do some 'housekeeping' and try to address some of > the old tickets dealt with before numpy 2.0. For example, I think ticket > 225 (bincount does not accept input of type > N.uint16) has been > addressed but it needs to be checked from windows and 32-bit systems. That would be useful of course, it just requires someone with spare time and the knowledge and motivation to close those tickets. I listed only the ones that I think are essential to fix before a release can be made. Ralf > Created 2006: > #38 strides accepted as an argument to records.array > #57 ufunc methods need improved BUFFER loop > #213 SharedLibrary builder for numpy.distutils > #225 bincount does not accept input of type > N.uint16 > #236 reduceat cornercase > #237 reduceat should handle outlier indices gracefully > #244 Build fails with Intel Visual Fortran compiler > #260 Add mechanism for registering objects to be deallocated and > memory-to-be freed at Python exit > #274 Speed up N-D Boolean indexing > #301 power with negative argument returns 0 > #333 Creating an array from a n-dim dtype type fails > #338 Valgrind warning when calling scipy.interpolate.interp1d > #349 Improve unit tests in linalg > #354 Possible inconsistency in 0-dim and scalar empty array types > #398 Compatibility loader for old Numeric pickles > #400 C API access to fft for C scipy extension ? > #402 newaxis incompatible with array indexing > > > Numpy 1.0 > #450 Make a.min() not copy data > #417 Numpy 1.0.1 compilation fails on IRIX 6.5 > #527 fortran linking flag option... > #1176 deepcopy turns ndarry into string_ > #1143 Improve performance of PyUFunc_Reduce > #931 Records containing zero-length items pickle just fine, but > cannot be unpickled > #803 Assignment problem on matrix advanced selection > > Numpy 1.1 > #1266 Extremely long runtimes in numpy.fft.fft > #963 Object array comparisons eat exceptions > #929 empty_like and zeros_like behave differently from ones_like > #934 Documentation error in site.cfg.example > > Numpy 1.2 > #1374 Ticket 628 not fixed for Solaris (polyfit uses 100% CPU and > does not stop) > #1209 Docstring for numpy.numarray.random_array.multinomial is out of > date. > #1192 integer dot product > #1172 abs does not work with -maxint > #1163 Incorrect conversion to Int64 by loadtxt (traced to _getconv in > numpy.lib.io) > #1161 Errors and/or wrong result with reverse slicing in numpy.delete > #1094 masked array autotest fails with bus error > #1085 Surprising results from in-place operations involving views > #1071 loadtxt fails if the last column contains empty value > > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion(a)scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion >

12 47

{zeros, empty, ...}_like functions behavior with subclasses
by Wes McKinney 06 Mar '11

06 Mar '11

I'm having some trouble with the zeros_like function via np.fix: def zeros_like(a): if isinstance(a, ndarray): res = ndarray.__new__(type(a), a.shape, a.dtype, order=a.flags.fnc) res.fill(0) return res try: wrap = a.__array_wrap__ except AttributeError: wrap = None a = asarray(a) res = zeros(a.shape, a.dtype) if wrap: res = wrap(res) return res As you can see this is going to discard any metadata stored in a subtype. I'm not sure whether this is a bug or a feature but wanted to bring it up. Thanks, Wes

4 5

Numpy 2.0 schedule
by Charles R Harris 02 Mar '11

02 Mar '11

Hi All, Just thought it was time to start discussing a release schedule for numpy 2.0 so we have something to aim at. I'm thinking sometime in the period April-June might be appropriate. There is a lot coming with the next release: the Enthought's numpy refactoring, Mark's float16 and iterator work, and support for IronPython. How do things look to the folks involved in those projects? Chuck

11 27

Adding `offset` argument to np.lib.format.open_memmap and np.load
by Jon Olav Vik 02 Mar '11

02 Mar '11

https://github.com/jonovik/numpy/compare/master...offset_memmap The `offset` argument to np.memmap enables memory-mapping a portion of a file on disk to a memory-mapped Numpy array. Memory-mapping can also be done with np.load, but it uses np.lib.format.open_memmap, which has no offset argument. I have added an offset argument to np.lib.format.open_memmap and np.load as detailed in the link above, and humbly submit the changes for review. This is my first time using git, apologies for any mistakes. Note that the offset is in terms of array elements, not bytes (which is what np.memmap uses), because that was what I had use for. Also, I added a `shape` to np.load to memory-map only a portion of a file. My use case was to preallocate a big record array on disk, then start many processes writing to their separate, memory-mapped segments of the file. The end result was one big array on disk, with the correct shape and data type information. Using a record array makes the data structure more self- documenting. Using open_memmap with mode="w+" is the fastest way I've found to preallocate an array on disk; it does not create the huge array in memory. Letting multiple processes memory-map and read/write to non-overlapping portions without interfering with each other allows for fast, simple parallel I/ O. I've used this extensively on Numpy 1.4.0, but based my Git checkout on the current Numpy trunk. There have been some rearrangements in np.load since then (it used to be in np.lib.io and is now in np.lib.npyio), but as far as I can see, my modifications carry over fine. I haven't had a chance to test with Numpy trunk, though. (What is the best way to set up a test version without affecting my working 1.4.0 setup?) Hope this can be useful, Jon Olav Vik

4 16

Grid indexing
by Wolfgang Kerzendorf 01 Mar '11

01 Mar '11

Hello, I have a n dimensional grid. The grids axes are linear but not intergers. Let's say I want the value in gridcell [3.2,-5.6,0.01]. Is there an easy way to transform the index? Do I have to write my own class to accomplish this. Is there an easy way in numpy to accomplish this. Can I give functions to transform the index somehow to the numpy array? Cheers Wolfgang

3 3

Small typo in fromregex
by Matthieu Brucher 01 Mar '11

01 Mar '11

Hi, I'm sorry I didn't file a bug, I have some troubles getting my old trac account back :| In lib/npyio.py, there is a mistake line 1029. Instead on fh.close(), it should have been file.close(). If fromregex opens the file, it will crash because the name of the file is not correct. Matthieu -- Information System Engineer, Ph.D. Blog: http://matt.eifelle.com LinkedIn: http://www.linkedin.com/in/matthieubrucher

2 1

permute and sum?
by Angus McMorland 01 Mar '11

01 Mar '11

Hi all, I want to create a transpose of a vector, such that if the same index is given in the 'axes' list (as per np.transpose), then the sum of the original values sharing the same index is placed in the corresponding output index. For example: In: data = np.array([5, 7, 9, 6, 2]) In: order = np.array([0, 2, 0, 3, 1]) In: permute_and_sum(data, order) Out: array([14, 2, 7, 6]) I can obviously do this using the following, def permute_and_sum(data, order): result = np.zeros(np.max(order) + 1) for val, dest in zip(data, order): result[dest] += val return result but it seems like there's bound to be a much more elegant method I'm not currently seeing. Can anyone point me in the right direction? Thanks, Angus. -- AJC McMorland Post-doctoral research fellow Neurobiology, University of Pittsburgh

2 1

Example of PyArray_RemoveSmallest use?
by John Salvatier 28 Feb '11

28 Feb '11

Does anyone have or know of an example of how to use PyArray_RemoveSmallest(multiiterator) to make the inner loop fast (how to get the right strides etc.)? I am finding it a bit confusing. Best Regards, John

1 0

ANN: Spyder v2.0.8
by Pierre Raybaut 27 Feb '11

27 Feb '11

Hi all, I am pleased to announced that Spyder v2.0.8 has just been released. As this is mostly a maintenance release, a lot of bugs were fixed (see below) and some minor features were added. Integration Spyder's GUI-based Python shell in your application: Note that this version includes an example of application using the Spyder's internal shell as a debugging console which also demonstrates the py2exe deployment procedure. Spyder (previously known as Pydee) is a free open-source Python development environment providing MATLAB-like features in a simple and light-weighted software, available for Windows XP/Vista/7, GNU/Linux and MacOS X: http://spyderlib.googlecode.com/ Spyder is part of spyderlib, a Python module based on PyQt4, pyflakes and rope (QScintilla's dependency has been removed in version 2.0 and rope features have been integrated since this version as well). Some of Spyder basic features: * Python, C/C++, Fortran source editor with class/function browser, code completion and calltips * consoles: o open as many Python interpreters, IPython consoles or command windows as you need o code completion and calltips o variable explorer with GUI-based editors for a lot of data types (numbers, strings, lists, arrays, dictionaries, ...) * object inspector: provide documentation or source code on any Python object (class, function, module, ...) * online documentation: automatically generated html documentation on installed Python modules * find in files * file explorer * project manager * MATLAB-like PYTHONPATH management dialog box (works with all consoles) * Windows only: current user environment variables editor * direct links to documentation (Python, Qt, Matplotlib, NumPy, Scipy, etc.) * direct link to Python(x,y) launcher * direct links to QtDesigner, QtLinguist and QtAssistant (Qt documentation) Bug fixes (since v2.0.6): Consoles/bugfix: saving history log (see context menu) was not working following a recent code cleaning/refactoring On non-Windows platforms, the file selection dialog "All files (*.*)" filter was not matching files without extension dochelpers.isdefined/bugfix: ignoring syntax errors while evaluating object Preferences Dialog (dialog box + keyboard shortcut page): improved size/resize behavior Editor: when cursor was on the very last line, Duplicate/Delete line features were getting stuck in an infinite loop Editor/duplicate line feature - fixed unexpected behavior: when duplicating selected text, text selection was extended to duplicated part Editor/bugfix with multiple editor windows: when opening file on one editor window, the top-left corner menu (file list) was not updated correctly in other editor windows Editor/fixed unexpected behavior: when clicking on the main window's outline explorer while a separate editor window had focus, the latter was used to show the associated line of code Project explorer: added new debugging options (profiling 'rope' calls) Console/Advanced settings/UMD module list: removing all entries (empty module list) was not working Editor/File list management dialog (Ctrl+E): double-clicking/pressing Return on a listwidget item will switch to the associated file Editor/Tab bar: fixed missing tooltips issue (and missing file switch menu entries) Code completion/bugfix: list widget was not hiding as expected when pressing ':' Editor/fixed unexpected behavior: when some text was selected, "Ctrl+Left mouse click" was trying to "go to definition" instead of doing the standard drag n'drop feature Editor/bugfix: disabling code completion/calltips for non-Python source code (was not working -as expected- but was taking time to simply not work...) Editor/go to line: fixed unicode error Code editor/bugfix: cursor position was not restored when undoing an indent operation with "tab always indent" feature turned on *and* the cursor at the end of the line Tab behavior when "tab always indents" is turned off: inserting 4-(len(leading_text) % 4) spaces (instead of 4) Object inspector/bugfix: ignoring unknown objects when called automatically from editor/console, i.e. do not log, do not show 'no doc available' Other changes (since v2.0.6): Code editor syntax highlighting: added more keywords to Cython syntax highlighter (cpdef, inline, cimport and DEF) Added example of application using the Spyder's internal shell as a debugging console (demonstrates also the py2exe deployment procedure) Object inspector: added "Source" combo box (Console/Editor) -> object inspected from editor are now analyzed only with rope (if available) and then shown in object inspector Added keyboard shortcut to open Preferences Dialog (default: Ctrl+Alt+Shift+P) Editor: added "Copy line" feature (Ctrl+Alt+Down), similar to "Duplicate line" (Ctrl+Alt+Up) but paste text before the current line/selected text (instead of after) Array editor: added option to set row/col labels (resp. ylabels and xlabels) Editor/rope: improved performance for calltips/doc feature Cheers, Pierre

1 0

Re: [Numpy-discussion] Where can I download numpy.i?
by Brandt Belson 27 Feb '11

27 Feb '11

Thanks, but I just downloaded the source for versions 1.5.1 and 1.4.1 and neither appear to have a swig directory in doc or numpy/doc. Are you sure that it is included with the standard download from http://sourceforge.net/projects/numpy/files/NumPy/ ? Is there another way to get the file? Thanks On Sun, Feb 27, 2011 at 1:00 PM, <numpy-discussion-request(a)scipy.org> wrote: > Send NumPy-Discussion mailing list submissions to > numpy-discussion(a)scipy.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.scipy.org/mailman/listinfo/numpy-discussion > or, via email, send a message with subject or body 'help' to > numpy-discussion-request(a)scipy.org > > You can reach the person managing the list at > numpy-discussion-owner(a)scipy.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of NumPy-Discussion digest..." > > > Today's Topics: > > 1. Re: Largest possible numpy array (Sturla Molden) > 2. Where can I download numpy.i? (Brandt Belson) > 3. Re: Where can I download numpy.i? (Bill Spotz) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sat, 26 Feb 2011 19:44:03 +0100 > From: Sturla Molden <sturla(a)molden.no> > Subject: Re: [Numpy-discussion] Largest possible numpy array > To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> > Message-ID: <4D6949F3.5000205(a)molden.no> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > Den 26.02.2011 16:09, skrev Jaidev Deshpande: > > Hi > > > > How can I know the size of the largest possible 2-D array in numpy, > > given a specific 'dtype' and my system memory? > > > > The largest array size is not dicatated by system memory but virtual > memory. > > On Windws the available virtual address space is 2 GB with x86 and 16 TB > with AMD64. > > If you have less physical memory than the array size, the OS will swap > some of it to disk. > > Sturla > > > ------------------------------ > > Message: 2 > Date: Sat, 26 Feb 2011 20:15:11 -0500 > From: Brandt Belson <bbelson(a)princeton.edu> > Subject: [Numpy-discussion] Where can I download numpy.i? > To: numpy-discussion(a)scipy.org > Message-ID: > <AANLkTinJ=UL-OkKDEy1s6gqYXG-fCEKY=zUQNowt1LJg(a)mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > I just want to know exactly how to get the file numpy.i (for SWIG), I've > come across links that seem to take me nowhere. > Thank you. >

2 1