Mailman 3 August 2018 - NumPy-Discussion

Add pybind11 to docs about writing binding code
by Hans Dembinski 20 Aug '18

20 Aug '18

Dear all, I just joined the numpy mailing list to suggest an enhancement of the docs about writing binding code https://docs.scipy.org/doc/numpy/user/c-info.python-as-glue.html <https://docs.scipy.org/doc/numpy/user/c-info.python-as-glue.html> I hope this is the right place to discuss this. So, this page is one of the first hits in Google when you search for "python bindings numpy". It is an important page for orientation. What I miss is a longer article about pybind11. http://pybind11.readthedocs.io/en/stable/ <http://pybind11.readthedocs.io/en/stable/> https://github.com/pybind/pybind11 <https://github.com/pybind/pybind11> pybind11 is currently the best tool on the market to wrap C++ code to Python. This is my professional opinion. When you look at the facts, it is hard to disagree. Pybind11 is based on the approach of Boost.Python, but is a compact project that doesn't require Boost and is developed independently. If you use it in a Python package, you can add it as a requirement and pip will happily install it. It doesn't require you to learn a new language, the bindings are generated using C++ meta-programming techniques under the hood. pybind11 has outstanding documentation and is extremely popular on github (3800+ stars). Refcounting is done automatically, which is why I would even use it to wrap C code. Naturally, it has excellent support for numpy arrays. Pybind11 is FOSS (BSD-style license). I have used Cython, Boost.Python, SWIG, and pybind11 in small to large projects, and pybind11 is by far the most pleasant and the most powerful. You can do really sophisticated things in pybind11, which I cannot imagine doing with other binding tools, and most importantly, it never chokes over your C++ code. Cython and SWIG both have trouble with certain C++ idioms, which is not surprising because C++ is notoriously difficult to parse and these tools were primarily developed to wrap C (which is much easier to parse). For C++, it is much better to not add a custom parser to the toolchain and just let the C++ compiler generate the low-level binding code. This is what pybind11 does. So far all these reasons and more, it should be mentioned and even highlighted here: https://docs.scipy.org/doc/numpy/user/c-info.python-as-glue.html <https://docs.scipy.org/doc/numpy/user/c-info.python-as-glue.html> I am happy to write a section about it. Disclaimer: I am not at all affiliated with the pybind11 developers, just a thankful user. Best regards, Hans

6 12

NumPy 1.15.1 release
by Charles R Harris 19 Aug '18

19 Aug '18

Hi All, The NumPy 1.15.1 release is ready to go, the release notes may be reviewed at gh-11787 <https://github.com/numpy/numpy/pull/11787>. Pending comments, I will make the release next Tuesday. Chuck

1 0

ANN: NumExpr 2.6.8
by Robert McLeod 19 Aug '18

19 Aug '18

========================== Announcing Numexpr 2.6.8 ========================== Hi everyone, Our attempt to fix the memory leak in 2.6.7 had an unforseen consequence that the `f_locals` from the top-most frame is actually `f_globals`, and clearing it to fix the extra reference count deletes all global variables. Needless to say this is undesired behavior. A check has been added to prevent clearing the globals dict, tested against both `python` and `ipython`. As such, we recommend skipping 2.6.7 and upgrading straight to 2.6.8 from 2.6.6. Project documentation is available at: http://numexpr.readthedocs.io/ Changes from 2.6.7 to 2.6.8 --------------------------- - Add check to make sure that `f_locals` is not actually `f_globals` when we do the `f_locals` clear to avoid the #310 memory leak issue. - Compare NumPy versions using `distutils.version.LooseVersion` to avoid issue #312 when working with NumPy development versions. - As part of `multibuild`, wheels for Python 3.7 for Linux and MacOSX are now available on PyPI. What's Numexpr? --------------- Numexpr is a fast numerical expression evaluator for NumPy. With it, expressions that operate on arrays (like "3*a+4*b") are accelerated and use less memory than doing the same calculation in Python. It has multi-threaded capabilities, as well as support for Intel's MKL (Math Kernel Library), which allows an extremely fast evaluation of transcendental functions (sin, cos, tan, exp, log...) while squeezing the last drop of performance out of your multi-core processors. Look here for a some benchmarks of numexpr using MKL: https://github.com/pydata/numexpr/wiki/NumexprMKL Its only dependency is NumPy (MKL is optional), so it works well as an easy-to-deploy, easy-to-use, computational engine for projects that don't want to adopt other solutions requiring more heavy dependencies. Where I can find Numexpr? ------------------------- The project is hosted at GitHub in: https://github.com/pydata/numexpr You can get the packages from PyPI as well (but not for RC releases): http://pypi.python.org/pypi/numexpr Documentation is hosted at: http://numexpr.readthedocs.io/en/latest/ Share your experience --------------------- Let us know of any bugs, suggestions, gripes, kudos, etc. you may have. Enjoy data! -- Robert McLeod, Ph.D. robbmcleod(a)gmail.com robbmcleod(a)protonmail.com robert.mcleod(a)hitachi-hhtc.ca www.entropyreduction.al

1 0

mplcursors 0.2 release
by Antony Lee 15 Aug '18

15 Aug '18

Dear all, I am please to announce the release of mplcursors 0.2. # Description mplcursors provides interactive data selection cursors for Matplotlib. Features include a callback system allowing full customization of the actions when data selection is triggered, and the ability to add a data selection cursor to any preexisting program using Matplotlib by just setting the MPLCURSORS environment variable, without changes to the source code. # Changelog from mplcursors 0.1 to mplcursors 0.2 - Updated dependency to matplotlib 2.1 (2.0 gives more information about orientation of bar plots; 2.1 improves the handling of step plots). - Setting `MPLCURSORS` hooks `Figure.draw` (once per figure only) instead of `plt.show`, thus supporting figures created after the first call to `plt.show`. - Automatic positioning and alignment of annotation text. - Selections on images now have an index as well. - Selections created on `scatter` plots, `errorbar` plots, and `polar` plots can now be moved. - `PathCollection`\s not created by `plt.scatter` are now picked as paths, not as collections of points. - `Patch`\es now pick on their borders, not their interior. - Improved picking of `Container`\s. - In hover mode, annotations can still be removed by right-clicking. Enjoy, Antony Lee

1 0

Dropping 32-bit from macOS wheels
by Matthew Brett 15 Aug '18

15 Aug '18

Hi, Short version: We are planning to drop 32-bit compatibility from the numpy macOS wheels. This email is to ask for feedback. Long version: macOS has tooling to allow distribution of binaries that have objects for more than one architecture. These are called "fat" binaries, and they were useful when Macs could be PPC, 32-bit or 64-bit intel. Now of course, they are effectively all 64-bit Intel. Amusingly enough, you can use the "lipo" command to operate on fat binaries. For example, system Python (at /usr/bin/python) is a fat binary, containing objects for 32 and 64-bit. $ lipo -info /usr/bin/python Architectures in the fat file: /usr/bin/python are: i386 x86_64 It is possible to run Python in 32-bit mode with: $ arch -i386 /usr/bin/python Up until the last release of numpy, we have build fat (32 and 64 bit) binaries for numpy, so, if you do run Python in 32-bit mode, you can still import and use numpy. In the last release, I accidentally broke the 32-bit part of the build - see https://github.com/numpy/numpy/issues/11625 . We could fix this, but we suspect that there are very few people still using 32-bit on macOS. If you do use it - please let us know, and we'll consider the options. Cheers, Matthew

2 1

ANN: Numexpr 2.6.7
by Robert McLeod 12 Aug '18

12 Aug '18

========================== Announcing Numexpr 2.6.7 ========================== Hi everyone, This is a bug-fix release. Thanks to Lehman Garrison for a fix that could result in memory leak-like behavior. Project documentation is available at: http://numexpr.readthedocs.io/ Changes from 2.6.6 to 2.6.7 --------------------------- - Thanks to Lehman Garrison for finding and fixing a bug that exhibited memory leak-like behavior. The use in `numexpr.evaluate` of `sys._getframe` combined with `.f_locals` from that frame object results an extra refcount on objects in the frame that calls `numexpr.evaluate`, and not `evaluate`'s frame. So if the calling frame remains in scope for a long time (such as a procedural script where `numexpr` is called from the base frame) garbage collection would never occur. - Imports for the `numexpr.test` submodule were made lazy in the `numexpr` module. What's Numexpr? --------------- Numexpr is a fast numerical expression evaluator for NumPy. With it, expressions that operate on arrays (like "3*a+4*b") are accelerated and use less memory than doing the same calculation in Python. It has multi-threaded capabilities, as well as support for Intel's MKL (Math Kernel Library), which allows an extremely fast evaluation of transcendental functions (sin, cos, tan, exp, log...) while squeezing the last drop of performance out of your multi-core processors. Look here for a some benchmarks of numexpr using MKL: https://github.com/pydata/numexpr/wiki/NumexprMKL Its only dependency is NumPy (MKL is optional), so it works well as an easy-to-deploy, easy-to-use, computational engine for projects that don't want to adopt other solutions requiring more heavy dependencies. Where I can find Numexpr? ------------------------- The project is hosted at GitHub in: https://github.com/pydata/numexpr You can get the packages from PyPI as well (but not for RC releases): http://pypi.python.org/pypi/numexpr Documentation is hosted at: http://numexpr.readthedocs.io/en/latest/ Share your experience --------------------- Let us know of any bugs, suggestions, gripes, kudos, etc. you may have. Enjoy data! -- Robert McLeod, Ph.D. robbmcleod(a)gmail.com robbmcleod(a)protonmail.com robert.mcleod(a)hitachi-hhtc.ca www.entropyreduction.al

1 0

Does numpy need a *.dist-info directory? What is the status of setuptools?
by Charles R Harris 11 Aug '18

11 Aug '18

Hi All, A question for any packaging gurus out there, should NumPy have a *.dist-info <https://www.python.org/dev/peps/pep-0376/> directory? I note that currently we have a *.egg-info directory. I also have a question as to what role setuptools should have going forward. Should we still rely on them, or has pip matured to the degree that we no longer need it. Note that we will only be supporting Python >= 3.5 in the near future. Chuck

2 3

Stacklevel for warnings.
by Charles R Harris 11 Aug '18

11 Aug '18

Hi All, Do we have a policy for the stacklevel that should be used in NumPy? How far back should the stack be displayed? I note that the optimum stacklevel may vary between users and developers. Chuck

3 3

Convolution of NumPy arrays of arbitrary dimension
by Foad Sojoodi Farimani 11 Aug '18

11 Aug '18

Hello everyone, My first email here, so sorry in advance if I'm violating any rules. I'm trying to implement the convolution of NumPy arrays of arbitrary dimension in order to get Cauchy product of multivariate power series. However I have some issues. I was wondering maybe you can help and this might also be of your interest. I have explained everything here in this StackOverflow post: http://bit.ly/2MhJCex I would appreciate if you could help me figure this out. Thanks in advance. Best, Foad

2 2

Taking back control of the #numpy irc channel
by Matti Picus 11 Aug '18

11 Aug '18

Over the past few days spambots have been hitting freenode's IRC channels[0, 1]. It turns out the #numpy channel has no operator, so we cannot make the channel mode "|+q $~a"[2] - i.e. only registered freenode users can talk but anyone can listen. I was in touch with the freenode staff, they requested that someone from the steering council reach out to them at ||projects(a)freenode.net, here is the quote from the discussion: " it's pretty much a matter of them sending an email telling us who they'd like to represent them on freenode, which channels and cloak namespaces they want, and any info we might need on the project " In the mean time they set the channel mode appropriately, so this is also a notice that if you want to chat on the #numpy IRC channel you need to register. Hope someone from the council picks this up and reaches out to them, and will decide who is to able to become channel operators (the recommended practice is to use it like sudo, only assume the role when needed then turn it back off). Matti [0] https://freenode.net/news/spambot-attack [1] https://freenode.net/news/spam-shake [2] https://nedbatchelder.com/blog/201808/fighting_spam_on_freenode.html |

5 9