Mailman 3 March 2009 - NumPy-Discussion

Is anyone knowledgeable about dll deployment on windows ?
by David Cournapeau Dec. 2, 2009

Dec. 2, 2009

Hi, My head hurt trying to understand dll management with windows. I wanted to find a sane way to use dll for numscons, but I can't see how to do this, so I was wondering if anyone on this ML had any deep knowledge on how to install dll, and reuse them with python extensions ? The problem is the following: - you have some dll (say MKL) in a path (say C:\program files\MKL). - how do you tell another dll (more exactly a python extension, a .pyd) to look for mkl dll into the MKL path ? It seems that there is no rpath-like capability on windows, and the only way to load dll is to put them into some windows directory, or to put the dll path into PATH. Is it really the only way ? cheers, David

8 25

suggestion for generalizing numpy functions
by Darren Dale Sept. 14, 2009

Sept. 14, 2009

I spent some time over the weekend fixing a few bugs in numpy that were exposed when attempting to use ufuncs with ndarray subclasses. It got me thinking that, with relatively little work, numpy's functions could be made to be more general. For example, the numpy.ma module redefines many of the standard ufuncs in order to do some preprocessing before the builtin ufunc is called. Likewise, in the units/quantities package I have been working on, I would like to perform a dimensional analysis to make sure an operation is allowed before I call a ufunc that might change data in place. Imagine an ndarray subclass with methods like __gfunc_pre__ and __gfunc_post__. __gfunc_pre__ could accept the context that is currently provided to __array_wrap__ (the inputs and the function called), perform whatever preprocessing is desired, and maybe return a dictionary containing metadata. Numpy functions could then be wrapped with a decorator that 1) calls __gfunc_pre__ and obtain any metadata that is returned 2) calls the wrapped functions, and then 3) calls __gfunc_post__, which might be very similar to __array_wrap__ except that it would also accept the metadata created by __gfunc_pre__. In cases where the routines to be called by __gfunc_pre__ and _post__ depend on what function is called, the the subclass could implement routines and store them in a dictionary-like object that is keyed using the function called. I have been exploring this approach with Quantities and it seems to work well. For example: def __gfunc_pre__(self, gfunc, *args): try: return gfunc_pre_registry[gfunc](*args) except KeyError: return {} I think such an approach for generalizing numpy's functions could be implemented without being disruptive to the existing __array_wrap__ framework. The decorator would attempt to identify an input or output array to use to call __gfunc_pre__ and _post__. If it finds them, it uses them. If it doesnt find them, no harm done, the existing __array_wrap__ mechanisms are still in place if the wrapped function is a ufunc. One other nice feature: the metadata that is returned by __gfunc_pre__ could contain an optional flag that the decorator attempts to pass to the wrapped function so that __gfunc_pre__ and _post are not called for any decorated internal functions. That way the subclass could specify that __gfunc_pre__ and _post should be called only for the outer-most function. Comments? Darren

5 24

ANN: line_profiler 1.0b2 released
by Robert Kern July 21, 2009

July 21, 2009

http://pypi.python.org/pypi/line_profiler/ http://packages.python.org/line_profiler/ This release fixes the "negative timings" issue on Windows. Future announcements will occur on python-announce. I just wanted to make sure my earliest users here who ran into this bug are aware of the fix. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

4 6

ANNOUNCE: ETS 3.2.0 Released
by Dave Peterson June 5, 2009

June 5, 2009

Hello, I'm pleased to announce that Enthought Tool Suite (ETS) version 3.2.0 has been tagged and released! Source distributions (.tar.gz) have been uploaded to PyPi, and Windows binaries will be follow shortly. A full install of ETS can be done using Setuptools via a command like: easy_install -U "ets[nonets] >= 3.2.0" NOTE 1: Users of an old ETS release will need to first uninstall prior to installing the new ETS. NOTE 2: If you get a 'SandboxViolation' error, simply re-run the command again -- it may take multiple invocations to get everything installed. (This error appears to be a long-standing incompatibility between numpy.distutils and setuptools.) Please see below for a list of what's new in this release. What Is ETS? =========== The Enthought Tool Suite (ETS) is a collection of components developed by Enthought and the open-source community, which we use every day to construct custom scientific applications. It includes a wide variety of components, including: * an extensible application framework * application building blocks * 2-D and 3-D graphics libraries * scientific and math libraries * developer tools The cornerstone on which these tools rest is the Traits package, which provides explicit type declarations in Python; its features include initialization, validation, delegation, notification, and visualization of typed attributes. More information on ETS is available from the development home page: http://code.enthought.com/projects/index.php Changelog ========= ETS 3.2.0 is a feature-added update to ETS 3.1.0, including numerous bug-fixes. Some of the notable changes include: Chaco ----- * Domain limits - Mappers now can declare the "limits" of their valid domain. PanTool and ZoomTool respect these limits. (pwang) * Adding "hide_grids" parameter to Plot.img_plot() and Plot.contour_plot() so users can override the default behavior of hiding grids. (pwang) * Refactored examples to declare a Demo object so they can be be run with the demo.py example launcher. (vibha) * Adding chaco.overlays package with some canned SVG overlays. (bhendrix) * DragZoom now can scale both X and Y axes independently corresponding to the mouse cursor motion along the X and Y axes (similar to the zoom behavior in Matplotlib). (pwang) * New Examples: * world map (bhendrix) * more financial plots (pwang) * scatter_toggle (pwang) * stacked_axis (pwang) * Fixing the chaco.scales TimeFormatter to use the built-in localtime() instead of the one in the safetime.py module due to Daylight Savings Time issues with timedelta. (r23231, pwang) * Improved behavior of ScatterPlot when it doesn't get the type of metadata it expects in its "selections" and "selection_masks" metadata keys (r23121, pwang) * Setting the .range2d attribute on GridMapper now properly sets the two DataRange1D instances of its sub-mappers. (r23119, pwang) * ScatterPlot.map_index() now respects the index_only flag (r23060, pwang) * Fixed occasional traceback/bug in LinePlot that occurred when data was completely outside the visible range (r23059, pwang) * Implementing is_in() on legends to account for padding and alignment (caused by tools that move the legend) (r23052, bhendrix) * Legend behaves properly when there are no plots to display (r23012, judah) * Fixed LogScale in the chaco.scales package to correctly handle the case when the length of the interval is less than a decade (r22907, warren.weckesser) * Fixed traceback when calling copy_traits() on a DataView (r22894, vibha) * Scatter plots generated by Plot.plot() now properly use the "auto" coloring feature of Plot. (r22727, pwang) * Reduced the size of screenshots in the user manual. (r22720, rkern) Mayavi ------ * 17, 18 March, 2009 (PR): * NEW: A simple example to show how one can use TVTK’s visual module with mlab. [23250] * BUG: The size trait was being overridden and was different from the parent causing a bug with resizing the viewer. [23243] * 15 March, 2009 (GV): * ENH: Add a volume factory to mlab that knows how to set color, vmin and vmax for the volume module [23221]. * 14 March, 2009 (PR): * API/TEST: Added a new testing entry point: ‘mayavi -t’ now runs tests in separate process, for isolation. Added enthought.mayavi.api.test to allow for simple testing from the interpreter [23195]...[23200], [23213], [23214], [23223]. * BUG: The volume module was directly importing the wx_gradient_editor leading to an import error when no wxPython is available. This has been tested and fixed. Thanks to Christoph Bohme for reporting this issue. [23191] * 14 March, 2009 (GV): * BUG: [mlab]: fix positioning for titles [23194], and opacity for titles and text [23193]. * ENH: Add the mlab_source attribute on all objects created by mlab, when possible [23201], [23209]. * ENH: Add a message to help the first-time user, using the new banner feature of the IPython shell view [23208]. * 13 March, 2009 (PR): * NEW/API: Adding a powerful TCP/UDP server for scripting mayavi via the network. This is available in enthought.mayavi.tools.server and is fully documented. It uses twisted and currently only works with wxPython. It is completely insecure though since it allows a remote user to do practically anything from mayavi. * 13 March, 2009 (GV) * API: rename mlab.orientationaxes to mlab.orientation_axes [23184] * 11 March, 2009 (GV) * API: Expose ‘traverse’ in mlab.pipeline [23181] * 10 March, 2009 (PR) * BUG: Fixed a subtle bug that affected the ImagePlaneWidget. This happened because the scalar_type of the output data from the VTKDataSource was not being set correctly. Getting the range of any input scalars also seems to silence warnings from VTK. This should hopefully fix issues with the use of the IPW with multiple scalars. I’ve added two tests for this, one is an integration test since those errors really show up only when the display is used. The other is a traditional unittest. [23166] * 08 March, 2009 (GV) * ENH: Raises an error when the user passes to mlab an array with infinite values [23150] * 07 March, 2009 (PR) * BUG: A subtle bug with a really gross error in the GridPlane component, I was using the extents when I should really have been looking at the dimensions. The extract grid filter was also not flushing the data changes downstream leading to errors that are also fixed now. These errors would manifest when you use an ExtractGrid to select a VOI or a sample rate and then used a grid plane down stream causing very wierd and incorrect rendering of the grid plane (thanks to conflation of extents and dimensions). This bug was seen at NAL for a while and also reported by Fred with a nice CME. The CME was then converted to a nice unittest by Suyog and then improved. Thanks to them all. [23146] * 28 February, 2009 (PR) * BUG: Fixed some issues reported by Ondrej Certik regarding the use Of mlab.options.offscreen, mlab.options.backend = ‘test’, removed cruft from earlier ‘null’ backend, fixed bug with incorrect imports, add_dataset set no longer adds one new null engine each time figure=False is passed, added test case for the options.backend test. [23088] * 23 February, 2009 (PR) * ENH: Updating show so that it supports a stop keyword argument that pops up a little UI that lets the user stop the mainloop temporarily and continue using Python [23049] * 21 February, 2009 (GV) * ENH: Add a richer view for the pipeline to the MayaviScene [23035] * ENH: Add safegards to capture wrong triangle array sizes in mlab.triangular_mesh_source. [23037] * 21 February, 2009 (PR) * ENH: Making the transform data filter recordable. [23033] * NEW: A simple animator class to make it relatively to create animations. [23036] [23039] * 20 February, 2009 (PR) * ENH: Added readers for various image file formats, poly data readers and unstructured grid readers. These include DICOM, GESigna, DEM, MetaImage (mha,mhd) MINC, AVSucd, GAMBIT, Exodus, STL, Points, Particle, PLY, PDB, SLC, OBJ, Facet and BYU files. Also added several tests for most of this functionality along with small data files. These are additions from PR’s project staff, Suyog Jain and Sreekanth Ravindran. [23013] * ENH: We now change the default so the ImagePlaneWidget does not control the LUT. Also made the IPW recordable. [23011] * 18 February, 2009 (GV) * ENH: Add a preference manager view for editing preferences outside envisage [22998] * 08 February, 2009 (GV) * ENH: Center the glyphs created by barchart on the data points, as mentioned by Rauli Ruohonen [22906] * 29 January, 2009 (GV) * ENH: Make it possible to avoid redraws with mlab by using mlab.gcf().scene.disable_render = True [22869] * 28 January, 2009 (PR and GV) * ENH: Make the mlab.pipeline.user_defined factory function usable to add arbitrary filters on the pipeline. [22867], [22865] * 11 January, 2009 (GV) * ENH: Make mlab.imshow use the ImageActor. Enhance the ImageActor to map scalars to colors when needed. [22816] Traits ------ * Fixed a bug whereby faulty error handling in the PyProtocols Pyrex speedup code keeps references to tracebacks that have been handled. In so doing, clean up the same code such that it can be used with a modern Pyrex release (a bare raise can no longer be used outside of an except: clause). * RangeEditor factory now supports a 'logslider' mode: Thanks to Matthew Turk for the patch * TabularEditor factory now supports editing of all columns: Thanks to Didrik Pinte for the patch * DateEditor factory in 'custom' style now supports multi-select feature. * DateEditor and TimeEditor now support the 'readonly' style. * Fixed a bug in the ArrayEditor factory that was causing multiple trait change events to get fired when the underlying array is changed externally to the editor: Thanks to Matthew Turk for he patch. * Fixed a circular import error in Color, Font and RGBColor traits * Fixed a bug in the factory for ArrayViewEditor so it now calls the toolkit backend-specific editor TraitsBackendWX --------------- * RangeEditor now supports a 'logslider' mode: Thanks to Matthew Turk for the patch * TabularEditor now supports editing of all columns: Thanks to Didrik Pinte for the patch * DateEditor in 'custom' style now supports multi-select feature. * DateEditor and TimeEditor now support the 'readonly' style. * Added a trait to the wx pyface workbench View to indicate if the view dock window should be closeable. * Fixed the DirectoryEditor to popup the correct file dialog (thanks to Luca Fasano and Phil Thompson) * Fixed a circular import error in Color, Font and RGBColor traits * Fixed a bug in the ColorEditor that was causing the revert action to not work correctly. * Fixed a bug that caused a traceback when trying to undock a pyface dock window * Fixed a bug in the 'livemodal' view that caused the UI to become unresponsive if the 'updated' event was fired on the contained view. * Fixed bugs in ListEditor (notebook style) that caused a loss of sync between the 'selected' trait and the activated dock window. TraitsBackendQt --------------- * RangeEditor now supports a 'logslider' mode: Thanks to Matthew Turk for the patch * Fixed the DirectoryEditor to popup the correct file dialog (thanks to Luca Fasano and Phil Thompson)

3 2

setmember1d_nu
by Robert Cimrman June 2, 2009

June 2, 2009

Hi all, I have added to the ticket [1] a script that compares the proposed setmember1d_nu() implementations of Neil and Kim. Comments are welcome! [1] http://projects.scipy.org/numpy/ticket/1036 r.

2 2

inplace matrix multiplication
by Frédéric Bastien April 29, 2009

April 29, 2009

Hi, I would like to know how I can make a call to the blas function gemm in numpy. I need a multiply and accumulate for matrix and I don't want to allocate a new matrix each time I do it. thanks for your time Frederic Bastien

4 6

JOB: write numpy docs
by Joe Harrington April 15, 2009

April 15, 2009

Last year's Doc Marathon got us off to a great start on documenting NumPy! But, there's still much work to be done, and SciPy after that. It's time to gear up for doing it again. Critical to last year's success was Stefan van der Walt's committed time, but he will be unable to play that role this year. So, I am looking to hire someone to write NumPy docs and help coordinate the doc project and its volunteers. The job includes working with me, the doc team, doc volunteers, and developers to: write and review a lot of docs, mainly those that others don't want to write help define milestones organize campaigns and volunteer teams to meet them research the NumPy and SciPy source codes to help plan: the eventual SciPy documentation the writing of a good User Manual work with the packaging team to meet their release deadlines perform other duties as assigned I am seeking someone to work full time if possible, and at least half time, from mid-April (or soon thereafter) through at least the (northern) summer. Candidates must be experienced NumPy and SciPy programmers; familiarity under the hood is a strong plus. They must also demonstrate their ability to produce excellent docs on the docs.SciPy.org wiki. Having contributed at a high level to an open-source community, especially to SciPy, is a big plus. Ability to take direction, work with and lead a team, and to work for extended periods without direct supervision on a list of assigned tasks are all critical. The applicant must be able to function well in a Linux environment; familiarity with multiple platforms is a plus. Please reply directly to me by email only. Include the following (PDF or ASCII formats strongly preferred): CV Statement of interest, qualifications per requirements above, availability, and wage expectations. Contact info for at least 3 professional references. Links to doc wiki pages for which you wrote the initial draft Links to doc wiki pages started by others to which you contributed significantly (edited, reviewed, proofed) The position is open until filled; candidates with complete applications by April 15 will receive full consideration. This is an open posting. Candidates who have not written any pages on the doc wiki yet have several weeks in which to do so. Pay will be commensurate with experience (up to a point). Relocation is not necessary. Candidates will need to provide their own computer and internet access. The University of Central Florida is an equal opportunity, equal access, affirmative action employer. --jh-- Prof. Joseph Harrington Department of Physics MAP 414 4000 Central Florida Blvd. University of Central Florida Orlando, FL 32816-2385 (407) 823-3416 voice (407) 823-5112 fax (407) 823-2325 physics office jh(a)physics.ucf.edu

1 1

DVCS at PyCon
by Travis E. Oliphant April 14, 2009

April 14, 2009

FYI from PyCon Here at PyCon, it has been said that Python will be moving towards DVCS and will be using bzr or mecurial, but explicitly *not* git. It would seem that *git* got the "lowest" score in the Developer survey that Brett Cannon did. The reasons seem to be: * git doesn't have good Windows clients * git is not written with Python I think the sample size was pretty small to be making decisions on (especially when most opinions where "un-informed"). I don't know if it matters that NumPy / SciPy use the same DVCS as Python, but it's a data-point. -Travis

23 112

array of matrices
by Bryan Cole April 8, 2009

April 8, 2009

I have a number of arrays of shape (N,4,4). I need to perform a vectorised matrix-multiplication between pairs of them I.e. matrix-multiplication rules for the last two dimensions, usual element-wise rule for the 1st dimension (of length N). (How) is this possible with numpy? thanks, BC

8 10

numpy.ctypeslib.ndpointer and the restype attribute
by Jens Rantil April 5, 2009

April 5, 2009

Hi all, So I have a C-function in a DLL loaded through ctypes. This particular function returns a pointer to a double. In fact I know that this pointer points to the first element in an array of, say for simplicity, 200 elements. How do I convert this pointer to a NumPy array that uses this data (ie. no copy of data in memory)? I am able to create a numpy array using a copy of the data. I have tried using the 'numpy.ctypeslib.ndpointer' but so far failed. In its documentation it claims it should be possible to use for not only argtypes attribute, but also restype. I have not found a single example of this on the web, and I wonder how this is done. As I see it, it would have to use the errcheck attribute to return an ndarray and not just restype. My latest trial was: >>> import ctypes >>> pointer = DLL.my_func() >>> ctypes_arr_type = C.POINTER(200 * ctypes.c_double) >>> ctypes_arr = ctypes_arr_type(pointer) >>> narray = N.ctypeslib.as_array(ctypes_arr) however this didn't work. Any hints would be appreciated. Thanks, Jens

5 11