Mailman 3 August 2013 - NumPy-Discussion

Using as_strided to avoid copy on repeat (with 2-dimensional array)
by Nicolas Rougier 13 Aug '13

13 Aug '13

Hi, I have a (n,2) shaped array representing points and I would like to double each point as in: A = np.arange(10*2).reshape(10,2) B = np.repeat( A, 2, axis=0 ) Is it possible to do the same using 'as_strided' to avoid copy (and still get the same output shape for B) ? I found this reference: http://stackoverflow.com/questions/5564098/repeat-numpy-array-without-repli… but did not manage to adapt it to my case. Bonus question (if previous 'as_strided' trick is possible), would this work or do I need a different construct ? A = np.arange(10*2).reshape(10,2) B = np.repeat( A[::2], 2, axis=0 ) Nicolas

3 2

quickselect
by Charles R Harris 13 Aug '13

13 Aug '13

Hi All, There is a PR <https://github.com/numpy/numpy/pull/3360> adding quickselect to numpy as a function `partition`. Comments on name and exposure in the numpy API are welcome. Chuck

5 9

chebyshev, laguerre, ... exec() template code at each import
by Andrew Dalke 12 Aug '13

12 Aug '13

Hi all, I mostly develop software related cheminformatics. There isn't much direct overlap between the tools of that field and NumPy and SciPy provide, but it's increasing with the use of scikit-learn and pandas. I tend to write command-line tools which indirectly import numpy. I've noticed that 25% of the "import numpy" cost of about 0.081 seconds is due to the chebyshev, laguerre, legendre, hermite_e, and hermite_e modules. Each module takes about 0.004 seconds to import. This is because each of them does a: exec(polytemplate.substitute(name='Chebyshev', nick='cheb', domain='[-1,1]')) during import. It appears that *everyone* takes a 0.02 second overhead during "import numpy" in order to simplify maintenance. This balance doesn't seem correct, given the number of people who use numpy vs. how rarely the polytemplate changes. Last year I submitted a patch which pre-computed all of those templates, so they would only be byte-compiled once. I knew (and still know) almost nothing about git/github, so Scott Sinclair kindly took it up and made it a pull request at: https://github.com/numpy/numpy/pull/334 I see that there's been no activity for at least 10 months. Is there anything more I can do to encourage that this patch be accepted? Cheers, Andrew dalke(a)dalkescientific.com

4 11

Nanmean review
by David Reed 12 Aug '13

12 Aug '13

Hello, This is my first time contributing and I was hoping to get a review of a change I made. Here is the comparison link. https://github.com/dvreed77/numpy/compare/nanmean

3 2

Releaseplan for 1.8?
by Till Stensitzki 12 Aug '13

12 Aug '13

Hi, there already a plan to release 1.8? I would like to play around with gufuncs. greetings Till

3 2

solving matrix
by Sudheer Joseph 11 Aug '13

11 Aug '13

Dear Experts, I am trying to write the below piece of matlab code to a python script. The objective of the code is to find a fit of annual harmonic to a time series and then remove it from the time series. It uses matlabs E\ts to find the coefficients of harmonic fit, I used linlag.lstsq equivalent in python but feel the result is not correct. Can any one advice on this please. ##########Matlab CODE################ function[an_fit,sam_fit,fit_both]=calcharm(t,ts,dt) subplot 211 plot(t,ts) legend('Orig Tser') xlabel('TIME') title('ORIGINAL TIMESERIES') figure(gcf) disp('Paused...');pause %% Prepare Annual signal fann = 2*pi/(365.25/dt); E = [ones(size(t)) cos(fann*t) sin(fann*t)]; a = E\ts; %dfit = a(1) + a(2)*cos(fann*t) + a(3)*sin(fann*t); % % or more easily: an_fit = E*a; #################PYTHON CODE ################## import numpy as np from numpy import pi,c_,ones,size,cos,sin t=np.arange(1,365.25*5) dt=np.average(np.diff(t)) acyc = 2*pi/(365.25/dt) 365.25/4)/dt) E1=c_[ones(size(t)),cos(acyc*t),sin(acyc*t)] linalg.lstsq(E1,ts) an_fit = E*a with best regards, Sudheer *************************************************************** Sudheer Joseph Indian National Centre for Ocean Information Services Ministry of Earth Sciences, Govt. of India POST BOX NO: 21, IDA Jeedeemetla P.O. Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55 Tel:+91-40-23886047(O),Fax:+91-40-23895011(O), Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile) E-mail:sjo.India@gmail.com;sudheer.joseph@yahoo.com Web- http://oppamthadathil.tripod.com ***************************************************************

1 2

a list of all available integer dtypes
by Valentin Haenel 10 Aug '13

10 Aug '13

Hi, I have a quick question: Is there a way to get a list of all available Numpy integer dtypes programatically? thanks, V-

3 3

np.load crashing in Anaconda Python 3.3.2
by Gaël Varoquaux 09 Aug '13

09 Aug '13

Hi list, Just a quick heads up: on Python Anaconda 3.3.2 np.load leads to a segfault. I am not on a computer for a week, but I saw that just before leaving. If I remember right, it is a simple matter of doing: a = np.zeros(10) np.save('a.npy', a) np.load('a.npy') I haven't submitted a bug report, because I didn't have time to check things properly. I suspect that it might be a problem of the compilation, rather then numpy itself, so the Anaconda team might want to check. Sorry for the incomplete report. I thought that it was better to submit an incomplete report now than wait a week. Gaël

1 0

ANN: XDress v0.3 Release
by Anthony Scopatz 09 Aug '13

09 Aug '13

Hello All, I am pleased to announce the latest the latest and greatest release of xdress, which incorporates a lot of feedback received at SciPy 2013. Please see the release notes below for more information. And a HUGE congratulations to the IPython team on their v1.0 release today! Please continue taking over the world ;). Be Well Anthony ======================== XDress 0.3 Release Notes ======================== XDress is an automatic wrapper generator for C/C++ written in pure Python. Currently, xdress may generate Python bindings (via Cython) for C++ classes, functions, and certain variable types. It also contains idiomatic wrappers for C++ standard library containers (sets, vectors, maps). In the future, other tools and bindings will be supported. The main enabling feature of xdress is a dynamic type system that was designed with the purpose of API generation in mind. Release highlights: - C++ template class & function support - Automatic docstring generation via dOxygen - A complete type system implementation refactor - Wrapped names may be different than source names, allowing automatic PEP-8-ification - A plethora of useful bug fixes Please visit the website for more information: http://xdress.org/ Ask questions on the mailing list: xdress(a)googlegroups.com Download the code from GitHub: http://github.com/xdress/xdress XDress is free & open source (BSD 2-clause license) and requires Python 2.7, NumPy 1.5+, Cython 0.19+, and optionally GCC-XML, pycparser, dOxygen, and lxml. New Features ============ C++ Template Support --------------------- The most requested feature following SciPy 2013 was full and complete support for C++ templates. It is here! Template instantiations are now automatically discovered, described, and wrapped. The only exception to this is that template member functions are *not* automatically discovered due to a defect in GCC-XML. Template methods must, therefore, be added by hand in a sidecar file. See the FAQ for more details. dOxygen Docstrings ------------------ A new plugin called ``xdress.doxygen`` has been added which automatically scrapes the source code for documentation using dOxygen. These are then automatically inserted as docstrings into the wrappers that are generated. These docstrings follow the numpydoc convention. Type System Refactor -------------------- The type system has undergone a complete reactor to make it more user-friendly and expose a lot of its internals. Previously, the type system was a collection of functions and module-level private containers. All of this has been moved into a new ``TypeSystem`` class whose attributes and methods are almost all public. This enables a number of potentially useful features. * Type systems may be customized in the xdressrc.py file. * Type systems may be customized in side car files. * Type systems can now persist and load themselves. This enables saving a type system when wrapping one project and loading it in as a dependency in future projects. * Easier testing As a consequence, most other parts of xdress must now hold a reference to a type system object. Before, it was sufficient to import the type system module. API Named Tuple --------------- A new named tuple class ``apiname`` has been added. This allows for a broader selection of how users can choose to write names in the classes, functions, and variables list in the run control file. Possible options now include:: from xdress.utils import apiname classes = [ (srcname, srcfile,), (srcname, srcfile, tarfile,), (srcname, srcfile, tarfile, tarname), [srcname, srcfile, tarfile, tarname], {'srcname': a, 'srcfile': b, 'tarfile': c, 'tarname': d}, apiname(srcname, srcfile, tarfile, tarname), ] This allows the target name (ie the name in the wrapper) to be distinct from the name that is in the C/C++ source code (ie the source name). This is very useful for customizing APIs because xdress takes care of the mapping for you! PEP-8 Compliance Plugin ----------------------- A new plugin called ``xdress.pep8names`` has been added which ensures that all target names for API elements are PEP-8 compliant for that type. If the target name is not PEP-8 complaint then it converts it to a compliant version for you. Subversive? That's the point! Functions and variables are converted to lowercase_with_underscores while classes are converted to CapCase. Description Filtering Plugin ---------------------------- Sometimes autoall will expose too much. To avoid certain API elements while still automatically finding other names, the ``xdress.descfilter`` plugin allows you to skip methods, attributes, and types that the you doesn't feel are appropriate to expose. This is also a useful stop-gap for data types that have not yet been implemented into the xdress type system. Tab Completion in BASH ---------------------- Through the power and magic of the argcomplete package, all arguments and options for all plugins now have tab-completion enabled in BASH. Install argcomplete and then add the following lines to your ``~/.bashrc`` file:: # Enable completion for xdress eval "$(register-python-argcomplete xdress)" Major Bug Fixes =============== * Enums and functions pointers were noticeably absent from the GCC-XML describer. This has been rectified. * Classes and structs are now registered into the type system earlier. This allows them to be used in other plugins. This includes stlwrap. It is now possible and easy to create custom numpy dtypes for any struct or C++ class. * Fixed segfaults with custom dtypes * Fixed build issue when only vectors were present in stlcontainers. * The global namespace is now correctly represented by None, rather than '::', when using GCC-XML. Join in the Fun! ================ If you are interested in using xdress on your project (and need help), contributing back to xdress, starting up a development team, or writing your own code generation plugin tool, please let us know. Participation is very welcome! Authors ======= - [Anthony Scopatz](http://scopatz.com/) - Spencer Lyon - John Wiggins * - Matt McCormick * - Brad Buran An * indicates a first time contributor. Links ===== 1. Homepage - http://xdress.org/ 2. Mailing List - xdress(a)googlegroups.com 3. GitHub Organization - https://github.com/xdress 4. numpydoc - https://pypi.python.org/pypi/numpydoc 5. argcomplete - https://argcomplete.readthedocs.org/en/latest/

1 0

[ANN] IPython 1.0 is finally released, nearly 12 years in the making!
by Fernando Perez 09 Aug '13

09 Aug '13

Hi all, I am incredibly thrilled, on behalf of the amazing IPython Dev Team, to announce the official release of IPython 1.0 today, an effort nearly 12 years in the making. The previous version (0.13) was released on June 30, 2012, and in this development cycle we had: ~12 months of work. ~700 pull requests merged. ~600 issues closed (non-pull requests). contributions from ~150 authors. ~4000 commits. # A little context What does "1.0" mean for IPython? Obviously IPython has been a staple of the scientific Python community for years, and we've made every effort to make it a robust and production ready tool for a long time, so what exactly do we mean by tagging this particular release as 1.0? Basically, we feel that the core design of IPython, and the scope of the project, is where we want it to be. What we have today is what we consider a reasonably complete, design- and scope-wise, IPython 1.0: an architecture for interactive computing, that can drive kernels in a number of ways using a well-defined protocol, and rich and powerful clients that let users control those kernels effectively. Our different clients serve different needs, with the old workhorse of the terminal still being very useful, but much of our current development energy going into the Notebook, obviously. The Notebook enables interactive exploration to become Literate Computing, bridging the gaps from individual work to collaboration and publication, all with an open file format that is a direct record of the underlying communication protocol. There are obviously plenty of open issues (many of them very important) that need fixing, and large and ambitious new lines of development for the years to come. But the work of the last four years, since the summer of 2009 when Brian Granger was able to devote a summer (thanks to funding from the NiPy project - nipy.org) to refactoring the old IPython core code, finally opened up or infrastructure for real innovation. By disentangling what was a useful but impenetrable codebase, it became possible for us to start building a flexible, modern system for interactive computing that abstracted the old REPL model into a generic protocol that kernels could use to talk to clients. This led at first to the creation of the Qt console, and then to the Notebook and out-of-process terminal client. It also allowed us to (finally!) unify our parallel computing machinery with the rest of the interactive system, which Min Ragan-Kelley pulled off in a development tour de force that involved rewriting in a few weeks a huge and complex Twisted-based system. We are very happy with how the Notebook work has turned out, and it seems the entire community agrees with us, as the uptake has been phenomenal. Back from the very first "IPython 0.0.1" that I started in 2001: https://gist.github.com/fperez/1579699 there were already hints of tools like Mathematica: it was my everyday workhorse as a theoretical physicist and I found its Notebook environment invaluable. But as a grad student trying out "just an afternoon hack" (IPython was my very first Python program as I was learning the language), I didn't have the resources, skills or vision to attempt building an entire notebook system, and to be honest the tools of the day would have made that enterprise a miserable one. But those ideas were always driving our efforts, and as IPython started becoming a project with a team, we made multiple attempts to get a good Notebook built around IPython. Those interested can read an old blog post of mine with the history (http://blog.fperez.org/2012/01/ipython-notebook-historical.html). The short story is that in 2011, on our sixth attempt, Brian was again able to devote a focused summer into using our client-server architecture and, with the stack of the modern web (Javascript, CSS, websockets, Tornado, ...), finally build a robust system for Literate Computing across programming languages. Today, thanks to the generous support and vision of Josh Greenberg at the Alfred P. Sloan Foundation, we are working very hard on building the notebook infrastructure, and this release contains major advances on that front. We have high hopes for what we'll do next; as a glimpse of the future that this enables, now there is a native Julia kernel that speaks to our clients, notebook included: https://github.com/JuliaLang/IJulia.jl. # Team I can't stress enough how impressed I am with the work people are doing in IPython, and what a privilege it is to work with colleagues like these. Brian Granger and Min Ragan-Kelley joined IPython around 2005, initially working on the parallel machinery, but since ~ 2009 they have become the heart of the project. Today Min is our top committer and knows our codebase better than anyone else, and I can't imagine better partners for an effort like this. And from regulars in our core team like Thomas Kluyver, Matthias Bussonnier, Brad Froehle and Paul Ivanov to newcomers like Jonathan Frederic and Zach Sailer, in addition to the many more whose names are in our logs, we have a crazy amount of energy being poured into IPython. I hope we'll continue to harness it productively! The full list of contributors to this release can be seen here: http://ipython.org/ipython-doc/rel-1.0.0/whatsnew/github-stats-1.0.html # Release highlights * nbconvert: this is the major piece of new functionality in this cycle, and was an explicit part of our roadmap (https://github.com/ipython/ipython/wiki/Roadmap:-IPython). nbconvert is now an IPython subcommand to convert notebooks into other formats such as HTML or LaTeX, but more importantly, it's a very flexible system that lets you write custom templates to generate new output with arbitrary control over the formatting and transformations that are applied to the input. We want to stress that despite the fact that a huge amount of work went into nbconvert, this should be considered a *tech preview* release. We've come to realize how complex this problem is, and while we'll make every effort to keep the high-level command-line syntax and APIs as stable as possible, it is quite likely that the internals will continue to evolve, possibly in backwards-incompatible ways. So if you start building services and libraries that make heavy use of the nbconvert internals, please be prepared for some turmoil in the months to come, and ping us on the dev list with questions or concerns. * Notebook improvements: there has been a ton of polish work in the notebook at many levels, though the file format remains unchanged from 0.13, so you shouldn't have any problems sharing notebooks with colleagues still using 0.13. - Autosave: probably the most oft-requested feature, the notebook server now autosaves your files! You can still hit Ctrl-S to force a manual save (which also creates a special 'checkpoint' you can come back to). - The notebook supports raw_input(), and thus also %debug. This was probably the main deficiency of the notebook as a client compared to the terminal/qtconsole, and it has been finally fixed. - Add %%html, %%svg, %%javascript, and %%latex cell magics for writing raw output in notebook cells. - Fix an issue parsing LaTeX in markdown cells, which required users to type \\\, instead of \\. -Images support width and height metadata, and thereby 2x scaling (retina support). - %%file has been renamed %%writefile (%%file) is deprecated. * The input transofrmation code has been updated and rationalized. This is a somewhat specialized part of IPython, but of importance to projects that build upon it for custom environments, like Sympy and Sage. Our full release notes are here: http://ipython.org/ipython-doc/rel-1.0.0/whatsnew/version1.0.html and the gory details are here: http://ipython.org/ipython-doc/rel-1.0.0/whatsnew/github-stats-1.0.html # Installation Installation links and instructions are at: http://ipython.org/install.html And IPython is also on PyPI: http://pypi.python.org/pypi/ipython # Requirements IPython 1.0 requires Python ≥ 2.6.5 or ≥ 3.2.1. It does not support Python 3.0, 3.1, or 2.5. # Acknowledgments Last but not least, we'd like to acknowledge the generous support of those who make it possible for us to spend our time working on IPython. In particular, the Alfred P. Sloan Foundation today lets us have a solid team working full-time on the project, and without the support of Enthought Inc at multiple points in our history, we wouldn't be where we are today. The full list of our support is here: http://ipython.org/index.html#support Thanks to everyone! Please enjoy IPython 1.0, and report all bugs as usual! Fernando, on behalf of the IPython Dev Team. -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail

1 0