Mailman 3 March 2015 - NumPy-Discussion

Introductory mail and GSoc Project "Vector math library integration"
by Dp Docs 24 Mar '15

24 Mar '15

Hi all, I am a CS 3rd Undergrad. Student from an Indian Institute (III T). I believe I am good in Programming languages like C/C++, Python as I have already done Some Projects using these language as a part of my academics. I really like Coding (Competitive as well as development). I really want to get involved in Numpy Development Project and want to take "Vector math library integration" as a part of my project. I want to here any idea from your side for this project. Thanks For your time for reading this email and responding back. My IRCnickname: dp Real Name: Durgesh Pandey.

8 14

Re: [Numpy-discussion] Asking proposal review/feedback for GSOC 15
by Oğuzhan Ünlü 24 Mar '15

24 Mar '15

Hi again, Thanks Ralf, I understand that. Then, I would like to share a public link to my proposal and I appreciate anybody who take time and leave comment/give feedback. It is on Github Gist. URL: https://gist.github.com/oguzhanunlu/1f8bf3ffc6ac5c420dd1 Thanks in advance, Oguzhan > Hi, > > > > My name is O?uzhan(You may use 'Oguzhan'). I submitted a proposal on the > > system with the title 'NumPy - Vector math library integration'. Ralf > > commented on my proposal and advised to ask for a feedback on mailing > list > > and here I am. > > > > I would appreciate any feedback from community. I think community members > > are able to view my proposal, its visibility is set to 'Organization > > members'. > > > > I preferred my name in its original form, if any mentor would like to > > search, I provide my name on system below. > > Name: O?uzhan ?nl? > > > > Hi O?uzhan, > > There are only a handful of potential mentors signed up in Melange, and > this list is read by hundreds of people. So it would be good to post your > proposal in a publicly accessible place and post the link here. Good > options are on Github or on StackEdit. > > Cheers, > Ralf > > P.S. for those who do have access to Melange: > > http://www.google-melange.com/gsoc/proposal/review/org/google/gsoc2015/blac… >

2 1

numpy.i: passing in in-place ND array as "flat" 1D array
by Tom Krauss 23 Mar '15

23 Mar '15

I have a method on a C++ object that treats all elements the same and modifies the array in-place (quantizes each value). Usually I just have a vector, i.e. a 1D array. But today I wanted to quantize a 2D array, and the (DATA_TYPE* INPLACE_ARRAY1, DIM_TYPE DIM1) failed to do the trick, because of the require_dimensions(array, 1) call. So I created a new typemap in numpy.i (DATA_TYPE* INPLACE_ARRAY_FLAT, DIM_TYPE DIM_FLAT) that omits the call to "require_dimensions", and behold! It works for both 1D and 2D (and really any D). Does this seem like a reasonable addition to numpy.i? Or is there another way to do this that I am missing? Regards, Tom

1 0

Rewrite np.histogram in c?
by Robert McGibbon 23 Mar '15

23 Mar '15

Hi, Numpy.histogram is implemented in python, and is a little sluggish. This has been discussed previously on the mailing list, [1, 2]. It came up in a project that I maintain, where a new feature is bottlenecked by numpy.histogram, and one developer suggested a faster implementation in cython [3]. Would it make sense to reimplement this function in c? or cython? Is moving functions like this from python to c to improve performance within the scope of the development roadmap for numpy? I started implementing this a little bit in c, [4] but I figured I should check in here first. -Robert [1] http://scipy-user.10969.n7.nabble.com/numpy-histogram-is-slow-td17208.html [2] http://numpy-discussion.10968.n7.nabble.com/Fast-histogram-td9359.html [3] https://github.com/mdtraj/mdtraj/pull/734 [4] https://github.com/rmcgibbo/numpy/tree/histogram

7 11

Asking proposal review/feedback for GSOC 15
by Oğuzhan Ünlü 23 Mar '15

23 Mar '15

Hi, My name is Oğuzhan(You may use 'Oguzhan'). I submitted a proposal on the system with the title 'NumPy - Vector math library integration'. Ralf commented on my proposal and advised to ask for a feedback on mailing list and here I am. I would appreciate any feedback from community. I think community members are able to view my proposal, its visibility is set to 'Organization members'. I preferred my name in its original form, if any mentor would like to search, I provide my name on system below. Name: Oğuzhan Ünlü Thanks in advance,

2 1

Reminder - Summer School "Advanced Scientific Programming in Python" in Munich, Germany
by Tiziano Zito 23 Mar '15

23 Mar '15

Reminder: Deadline for application is 23:59 UTC, March 31, 2015. Advanced Scientific Programming in Python ========================================= a Summer School by the G-Node, the Bernstein Center for Computational Neuroscience Munich and the Graduate School of Systemic Neurosciences Scientists spend more and more time writing, maintaining, and debugging software. While techniques for doing this efficiently have evolved, only few scientists have been trained to use them. As a result, instead of doing their research, they spend far too much time writing deficient code and reinventing the wheel. In this course we will present a selection of advanced programming techniques, incorporating theoretical lectures and practical exercises tailored to the needs of a programming scientist. New skills will be tested in a real programming project: we will team up to develop an entertaining scientific computer game. We use the Python programming language for the entire course. Python works as a simple programming language for beginners, but more importantly, it also works great in scientific simulations and data analysis. We show how clean language design, ease of extensibility, and the great wealth of open source libraries for scientific computing and data visualization are driving Python to become a standard tool for the programming scientist. This school is targeted at Master or PhD students and Post-docs from all areas of science. Competence in Python or in another language such as Java, C/C++, MATLAB, or Mathematica is absolutely required. Basic knowledge of Python is assumed. Participants without any prior experience with Python should work through the proposed introductory materials before the course. Date and Location ================= August 31—September 5, 2015. Munich, Germany. Preliminary Program =================== Day 0 (Mon Aug 31) — Best Programming Practices • Best Practices for Scientific Computing • Version control with git and how to contribute to Open Source with github • Object-oriented programming & design patterns Day 1 (Tue Sept 1) — Software Carpentry • Test-driven development, unit testing & quality assurance • Debugging, profiling and benchmarking techniques • Advanced Python: generators, decorators, and context managers Day 2 (Wed Sept 2) — Scientific Tools for Python • Advanced NumPy • The Quest for Speed (intro): Interfacing to C with Cython • Contributing to Open Source Software/Programming in teams Day 3 (Thu Sept 3) — The Quest for Speed • Writing parallel applications in Python • Python 3: why should I care • Programming project Day 4 (Fri Sept 4) — Efficient Memory Management • When parallelization does not help: the starving CPUs problem • Programming project Day 5 (Sat Sept 5) — Practical Software Development • Programming project • The Pelita Tournament Every evening we will have the tutors' consultation hour: Tutors will answer your questions and give suggestions for your own projects. Applications ============ You can apply on-line at https://python.g-node.org Applications must be submitted before 23:59 UTC, March 31, 2015. Notifications of acceptance will be sent by May 1, 2015. No fee is charged but participants should take care of travel, living, and accommodation expenses. Candidates will be selected on the basis of their profile. Places are limited: acceptance rate is usually around 20%. Prerequisites: You are supposed to know the basics of Python to participate in the lectures Preliminary Faculty =================== • Pietro Berkes, Enthought Inc., UK • Marianne Corvellec, Plotly Technologies Inc., Montréal, Canada • Kathryn D. Huff, Department of Nuclear Engineering, University of California - Berkeley, USA • Zbigniew Jędrzejewski-Szmek, Krasnow Institute, George Mason University, USA • Eilif Muller, Blue Brain Project, École Polytechnique Fédérale de Lausanne, Switzerland • Juan Nunez-Iglesias, Victorian Life Sciences Computation Initiative, University of Melbourne, Australia • Rike-Benjamin Schuppner, Institute for Theoretical Biology, Humboldt-Universität zu Berlin, Germany • Bartosz Teleńczuk, European Institute for Theoretical Neuroscience, CNRS, Paris, France • Nelle Varoquaux, Centre for Computational Biology Mines ParisTech, Institut Curie, U900 INSERM, Paris, France • Tiziano Zito, Forschungszentrum Jülich GmbH, Germany Organized by Tiziano Zito (head) and Zbigniew Jędrzejewski-Szmek for the German Neuroinformatics Node of the INCF Germany, Christopher Roppelt for the German Center for Vertigo and Balance Disorders (DSGZ) and the Graduate School of Systemic Neurosciences (GSN) of the Ludwig-Maximilians-Universität Munich Germany, Christoph Hartmann for the Frankfurt Institute for Advanced Studies (FIAS) and International Max Planck Research School (IMPRS) for Neural Circuits, Frankfurt Germany, and Jakob Jordan for the Institute of Neuroscience and Medicine (INM-6) and Institute for Advanced Simulation (IAS-6), Jülich Research Centre and JARA. Additional funding provided by the Bernstein Center for Computational Neuroscience (BCCN) Munich. Website: https://python.g-node.org Contact: python-info(a)g-node.org

1 0

ANN: pandas 0.16.0 released
by Jeff Reback 23 Mar '15

23 Mar '15

Hello, We are proud to announce v0.16.0 of pandas, a major release from 0.15.2. This release includes a small number of API changes, several new features, enhancements, and performance improvements along with a large number of bug fixes. This was 4 months of work by 60 authors encompassing 204 issues. We recommend that all users upgrade to this version. *Highlights:* - - *DataFrame.assign* method, see here <http://pandas.pydata.org/pandas-docs/stable/whatsnew.html#whatsnew-0160-enh…> - *Series.to_coo/from_coo* methods to interact with *scipy.sparse*, see here <http://pandas.pydata.org/pandas-docs/stable/whatsnew.html#whatsnew-0160-enh…> - Backwards incompatible change to *Timedelta* to conform the *.seconds* attribute with *datetime.timedelta*, see here <http://pandas.pydata.org/pandas-docs/stable/whatsnew.html#whatsnew-0160-api…> - Changes to the *.loc* slicing API to conform with the behavior of *.ix* see here <http://pandas.pydata.org/pandas-docs/stable/whatsnew.html#whatsnew-0160-api…> - Changes to the default for ordering in the *Categorical* constructor, see here <http://pandas.pydata.org/pandas-docs/stable/whatsnew.html#whatsnew-0160-api…> - Enhancement to the *.str* accessor to make string operations easier, see here <http://pandas.pydata.org/pandas-docs/stable/whatsnew.html#whatsnew-0160-enh…> - The *pandas.tools.rplot*, *pandas.sandbox.qtpandas* and *pandas.rpy* modules are deprecated. - We refer users to external packages like seaborn <http://stanford.edu/~mwaskom/software/seaborn/>, pandas-qt <https://github.com/datalyze-solutions/pandas-qt> and rpy2 <http://rpy.sourceforge.net/> for similar or equivalent functionality, see here for more detail <http://pandas.pydata.org/pandas-docs/stable/whatsnew.html#deprecations> See a full description of the Whatsnew for v0.16.0 <http://pandas.pydata.org/pandas-docs/stable/whatsnew.html> *What is it:* *pandas* is a Python package providing fast, flexible, and expressive data structures designed to make working with “relational” or “labeled” data both easy and intuitive. It aims to be the fundamental high-level building block for doing practical, real world data analysis in Python. Additionally, it has the broader goal of becoming the most powerful and flexible open source data analysis / manipulation tool available in any language. Documentation: http://pandas.pydata.org/pandas-docs/stable/ Source tarballs, windows wheels, macosx wheels are available on PyPI: https://pypi.python.org/pypi/pandas windows binaries are courtesy of Christoph Gohlke and are built on Numpy 1.9 macosx wheels are courtesy of Matthew Brett and are built on Numpy 1.7.1 Please report any issues here: https://github.com/pydata/pandas/issues Thanks The Pandas Development Team

1 0

element-wise array segmental function operation?
by oyster 23 Mar '15

23 Mar '15

Hi, all I want to know wether there is a terse way to apply a function to every array element, where the function behaves according to the element value. for example [code] def fun(v): if 0<=v<60: return f1(v) #where f1 is a function elif 60<=v<70: return f2(v) elif 70<=v<80: return f3(v) ...and so on... [/code] for 'a=numpy.array([20,50,75])', I hope to get numpy.array([f1(20), f1(50), f3(75)]) thanks in advance Lee

3 2

Installation on Windows
by Per Tunedal 23 Mar '15

23 Mar '15

Hi, how do I install Numpy on Windows? I've tried the setup.py file, but get an error message: setup.py install gives: No module named msvccompiler in numpy.distutils; trying from distutils error: Unable to find vcvarsall.bat Yours, Per Tunedal

4 4

GSoC projects
by Lulu Li 23 Mar '15

23 Mar '15

My apology if I am posting to the wrong mailing list. I am interested in NumPy project ideas for Google Summer of Code 2015 as posted here https://github.com/scipy/scipy/wiki/GSoC-project-ideas. In particular, knowing C and Python, I am interested in porting parts of bumpy from C to Cython or pythonic types. I wonder if these projects are still looking for participants? If not I will be excited to put together a proposal and work on them these summer. Lulu

2 1

Use of NameValidator in np.genfromtxt is inconsistent with the rules for naming structured array fields
by Alistair Muldal 21 Mar '15

21 Mar '15

Hi all, I originally posted this to the issue tracker (https://github.com/numpy/numpy/issues/5686), and am posting here as well at the request of charris. Currently, np.genfromtxt uses a numpy.lib._iotools.NameValidator which mangles field names by replacing spaces and stripping out certain non-alphanumeric characters etc.: import numpy as np from io import BytesIO s = 'name,name with spaces,2*(x-1)!\n1,2,3\n4,5,6' x = np.genfromtxt(BytesIO(s), delimiter=',', names=True) print(repr(x)) # array([(1.0, 2.0, 3.0), (4.0, 5.0, 6.0)], # dtype=[('name', '<f8'), ('name_with_spaces', '<f8'), ('2x1', '<f8')]) This behaviour has been the cause of some confusion in the past, e.g. http://stackoverflow.com/q/29097917/1461210, http://stackoverflow.com/q/16020137/1461210. Part of the issue is that it's currently not very well covered by the documentation for np.genfromtext - at best, it's alluded to in the descriptions for some of the keyword arguments ('deletechars', 'autostrip', 'replace_space' etc.). However, I think the more fundamental problem is that this behaviour seems to be inconsistent with the rules for naming the fields in structured arrays. In the example above, all of the original field names are perfectly legal: names = ['name', 'name with spaces', '2*(x-1)!'] types = ('f',) * 3 dtype = zip(names, types) x2 = np.empty(2, dtype=dtype) x2[:] = [(1.0, 2.0, 3.0), (4.0, 5.0, 6.0)] print(repr(x2)) # array([(1.0, 2.0, 3.0), (4.0, 5.0, 6.0)], # dtype=[('name', '<f4'), ('name with spaces', '<f4'), ('2*(x-1)!', '<f4')]) print(x2['2*(x-1)!']) # [3. 6.] What is the rationale behind the use of NameValidator here? One possible reason would be to ensure that the field names would also be legal for an np.recarray. However, this doesn't make sense for several reasons: Firstly, the names above also seem to be legal field names for a recarray: xr = x2.view(np.recarray) print(repr(x2)) # rec.array([(1.0, 2.0, 3.0), (4.0, 5.0, 6.0)], # dtype=[('name', '<f4'), ('name with spaces', '<f4'), ('2*(x-1)!', '<f4')]) Obviously if the field names aren't valid Python identifiers then it won't be possible to access them via 'xr.fieldname' syntax, but dict-style indexing is still fine, e.g. xr['2*(x-1)!']. Also, if the goal of NameValidator were to ensure that the field names were always valid Python identifiers then it currently fails at this anyway, since in my first example, '2x1' is not a valid Python identifier. What is perhaps most confusing is the fact that np.genfromtxt will even mangle field names that you pass in directly via the 'names' keyword argument. Suppose you wanted to specify field names that NameValidator doesn't like. You might try something like this: print(repr(np.genfromtxt(BytesIO(s), delimiter=',', names=names, skip_header=1))) # array([(1.0, 2.0, 3.0), (4.0, 5.0, 6.0)], # dtype=[('name', '<f8'), ('name_with_spaces', '<f8'), ('2x1', '<f8')]) Or even this: print(repr(np.genfromtxt(BytesIO(s), delimiter=',', names=names, skip_header=1, deletechars=[], replace_space=False, excludelist=[], autostrip=False))) # array([(1.0, 2.0, 3.0), (4.0, 5.0, 6.0)], # dtype=[('name', '<f8'), ('name_with_spaces', '<f8'), ('2x1', '<f8')]) Still no luck! As far as I can tell, there is no option in np.genfromtxt that allows you to preserve field names that don't conform to NameValidator's seemingly arbitrary rules. What should be done about this? Personally, I think that either things like spaces and non-alphanumeric characters should be disallowed in structured array field names altogether (and my second example should raise an exception), or np.genfromtxt should leave field names alone by default. It would also be a good idea to raise a SyntaxWarning in a case where the user creates a recarray containing field names that are not valid Python identifiers (and are therefore incompatible with the dot indexing syntax). This is essentially what PyTables does for non-conforming HDF5 node names: https://github.com/PyTables/PyTables/blob/13047c897d28b7278cbeab732f12feadb…. Any thoughts on this? Alistair

1 0

how to optimize numpy code for Markovian path
by Jianhong Wang 20 Mar '15

20 Mar '15

Below is a python function to generate Markov path (the travelling salesman problem). def generate_travel_path(markov_matrix, n): assert markov_matrix.shape[0] == markov_matrix.shape[1] assert n <= markov_matrix.shape[0] p = markov_matrix.copy() path = [0] * n for k in range(1, n): k1 = path[k-1] row_sums = 1 / (1 - p[:, k1]) p *= row_sums[:, np.newaxis] p[:, k1] = 0 path[k] = np.random.multinomial(1, p[k1, :]).argmax() assert len(set(path)) == n return path markov_matrix is a predefined Markov transition matrix. The code generates a path starting from node zero and visit every node once based on this matrix. However I feel the function is quite slow. Below is the line-by-line profile with a 53x53 markov_matrix: Timer unit: 3.49943e-07 s Total time: 0.00551195 sFile: <ipython-input-29-37e4c9b5469e>Function: generate_travel_path at line 1Line # Hits Time Per Hit % Time Line Contents============================================================== 1 def generate_travel_path(markov_matrix, n): 2 1 31 31.0 0.2 assert markov_matrix.shape[0] == markov_matrix.shape[1] 3 1 12 12.0 0.1 assert n <= markov_matrix.shape[0] 4 5 1 99 99.0 0.6 p = markov_matrix.copy() 6 1 12 12.0 0.1 path = [0] * n 7 53 416 7.8 2.6 for k in range(1, n): 8 52 299 5.8 1.9 k1 = path[k-1] 9 52 3677 70.7 23.3 row_sums = 1 / (1 - p[:, k1]) 10 52 4811 92.5 30.5 p = p * row_sums[:, np.newaxis] 11 52 1449 27.9 9.2 p[:, k1] = 0 12 52 4890 94.0 31.0 path[k] = np.random.multinomial(1, p[k1, :]).argmax() 13 14 1 51 51.0 0.3 assert len(set(path)) == n 15 1 4 4.0 0.0 return path If I ran this function 25000 times, it will take me more than 125 seconds. Any headroom to improve the speed? Below is a simple function to generate a Markov matrix. def initial_trans_matrix(n): x = np.ones((n, n)) / (n - 1) np.fill_diagonal(x, 0.0) return x

1 0

Problem with _dotblas.pyd when using Matplotlib for 3d plot
by Geoffrey Mégardon 19 Mar '15

19 Mar '15

Hi, I am on Windows 8.1, 64-bit, using the usual version of Anaconda-64-bit for windows. I just come from the mailing list of matplotlib: my PC is not able to do 3D plot with matplotlib. Python crashes completely if I try to plot something in 3D and to show it: "Python.exe has stopped working..." The main problem in resolving my issue is that the python traceback and the faulthandler package do not succeed in catching the errors. Only the Windows crash report allows us to see what happened before the crash. The windows report tells that the file _dotblas.pyd from numpy is likely responsible for the crash. So I tried a simple: import numpy; numpy.test() numpy.test() does not work! I dont get any traceback from python. It just crashes as before: "Python.exe has stopped working..." And again, the Windows report blames _dotblas.pyd :) Here the new Windows report for the numpy.test(), I highlighted the lines which blame _dotblas.pyd: Version=1 EventType=APPCRASH EventTime=130608138227608275 ReportType=2 Consent=1 UploadTime=130608138229728384 ReportIdentifier=216b16d2-6f5c-11e4-bec3-48d22435da2b IntegratorReportIdentifier=216b16d1-6f5c-11e4-bec3-48d22435da2b NsAppName=python.exe Response.BucketId=398b7eee350a0fd8a7a96f705d3488f6 Response.BucketTable=4 Response.LegacyBucketId=85979911964 Response.type=4 Sig[0].Name=Application Name Sig[0].Value=python.exe Sig[1].Name=Application Version Sig[1].Value=0.0.0.0 Sig[2].Name=Application Timestamp Sig[2].Value=53b4679e *Sig[3].Name=Fault Module NameSig[3].Value=_dotblas.pyd* Sig[4].Name=Fault Module Version Sig[4].Value=0.0.0.0 Sig[5].Name=Fault Module Timestamp Sig[5].Value=545678cb Sig[6].Name=Exception Code Sig[6].Value=c000001d Sig[7].Name=Exception Offset Sig[7].Value=0000000000324022 DynamicSig[1].Name=OS Version DynamicSig[1].Value=6.3.9600.2.0.0.768.101 DynamicSig[2].Name=Locale ID DynamicSig[2].Value=2057 DynamicSig[22].Name=Additional Information 1 DynamicSig[22].Value=f32f DynamicSig[23].Name=Additional Information 2 DynamicSig[23].Value=f32feb95f950f918532aa47b4372840e DynamicSig[24].Name=Additional Information 3 DynamicSig[24].Value=9882 DynamicSig[25].Name=Additional Information 4 DynamicSig[25].Value=98823b6c7f579b24e92112ab827fe4a1 UI[2]=C:\Anaconda\python.exe UI[3]=python.exe has stopped working UI[4]=Windows can check online for a solution to the problem. UI[5]=Check online for a solution and close the program UI[6]=Check online for a solution later and close the program UI[7]=Close the program LoadedModule[0]=C:\Anaconda\python.exe LoadedModule[1]=C:\WINDOWS\SYSTEM32\ntdll.dll LoadedModule[2]=C:\WINDOWS\system32\KERNEL32.DLL LoadedModule[3]=C:\WINDOWS\system32\KERNELBASE.dll LoadedModule[4]=C:\Anaconda\python27.dll LoadedModule[5]=C:\WINDOWS\WinSxS\amd64_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.8387_none_08e793bfa83a89b5\MSVCR90.dll LoadedModule[6]=C:\WINDOWS\system32\USER32.dll LoadedModule[7]=C:\WINDOWS\system32\ADVAPI32.dll LoadedModule[8]=C:\WINDOWS\system32\SHELL32.dll LoadedModule[9]=C:\WINDOWS\system32\GDI32.dll LoadedModule[10]=C:\WINDOWS\system32\msvcrt.dll LoadedModule[11]=C:\WINDOWS\SYSTEM32\sechost.dll LoadedModule[12]=C:\WINDOWS\system32\RPCRT4.dll LoadedModule[13]=C:\WINDOWS\SYSTEM32\combase.dll LoadedModule[14]=C:\WINDOWS\system32\SHLWAPI.dll LoadedModule[15]=C:\WINDOWS\system32\IMM32.DLL LoadedModule[16]=C:\WINDOWS\system32\MSCTF.dll LoadedModule[17]=C:\Anaconda\DLLs\_socket.pyd LoadedModule[18]=C:\WINDOWS\system32\WS2_32.dll LoadedModule[19]=C:\WINDOWS\system32\NSI.dll LoadedModule[20]=C:\Anaconda\DLLs\_ssl.pyd LoadedModule[21]=C:\Anaconda\DLLs\_ctypes.pyd LoadedModule[22]=C:\WINDOWS\system32\ole32.dll LoadedModule[23]=C:\WINDOWS\system32\OLEAUT32.dll LoadedModule[24]=C:\Anaconda\DLLs\_hashlib.pyd LoadedModule[25]=C:\WINDOWS\SYSTEM32\CRYPTSP.dll LoadedModule[26]=C:\WINDOWS\system32\rsaenh.dll LoadedModule[27]=C:\WINDOWS\SYSTEM32\bcrypt.dll LoadedModule[28]=C:\WINDOWS\SYSTEM32\CRYPTBASE.dll LoadedModule[29]=C:\WINDOWS\SYSTEM32\bcryptPrimitives.dll LoadedModule[30]=C:\Users\User\AppData\Roaming\Python-Eggs\faulthandler-2.4-py2.7-win-amd64.egg-tmp\faulthandler.pyd LoadedModule[31]=C:\Anaconda\lib\site-packages\numpy\core\multiarray.pyd LoadedModule[32]=C:\Anaconda\lib\site-packages\numpy\core\umath.pyd LoadedModule[33]=C:\Anaconda\lib\site-packages\numpy\core\_dotblas.pyd LoadedModule[34]=C:\Anaconda\lib\site-packages\numpy\core\libiomp5md.dll LoadedModule[35]=C:\Anaconda\lib\site-packages\numpy\core\scalarmath.pyd LoadedModule[36]=C:\Anaconda\lib\site-packages\numpy\lib\_compiled_base.pyd LoadedModule[37]=C:\Anaconda\lib\site-packages\numpy\linalg\lapack_lite.pyd LoadedModule[38]=C:\Anaconda\lib\site-packages\numpy\linalg\_umath_linalg.pyd LoadedModule[39]=C:\Anaconda\lib\site-packages\numpy\fft\fftpack_lite.pyd LoadedModule[40]=C:\Anaconda\lib\site-packages\numpy\random\mtrand.pyd LoadedModule[41]=C:\Anaconda\DLLs\_multiprocessing.pyd State[0].Key=Transport.DoneStage1 State[0].Value=1 FriendlyEventName=Stopped working ConsentKey=APPCRASH AppName=python.exe AppPath=C:\Anaconda\python.exe NsPartner=windows NsGroup=windows8 ApplicationIdentity=5B036AF1EC2E20F320DBF28D119DE93D Any idea, of what I can do? I try to reinstall anaconda, I also try conda remove numpy and conda install numpy. That does not seem to change anything. -- -- MEGARDON Geoffrey

5 4

Porting C to Python
by Marcos . 19 Mar '15

19 Mar '15

Dear colleagues, My name is Marcos Chaves, I'm an undergraduate student from Brazil. I study at the State University of Campinas, currently at the end of my graduation. I'm deeply interested in applying to GSOC this year, specifically to work with NumPy. About some months ago me and my friends at work started porting some legacy C/C++ code to Python, a code that we used to evaluate some images using OpenCV. We made use of NumPy and I liked it. I believe I can be of some help in making it better, particularly in solving the issue with porting parts of it from C (while maintaining optimal performance). I've never contributed to NumPy in any way and I see that its a requirement for me to apply for such task. I would like to know if there's a way I can participate, because the end of student registration is coming soon and I'm not sure if there is enough time for me to do something like that, but I'm interested. Some input from the potential mentor, Nathaniel Smith, along with some extra info on this task would be very nice for me. Thank you!

1 0

SciPy 2015 Call for Propsals Open - tutorial & talk submissions due April 1st
by Courtenay Godshall (Enthought) 18 Mar '15

18 Mar '15

**SciPy 2015 Conference (Scientific Computing with Python) Call for Proposals: Submit Your Tutorial and Talk Ideas by April 1, 2015 at http://scipy2015.scipy.org.** SciPy 2015, the fourteenth annual Scientific Computing with Python conference, will be held July 6-12, 2015 in Austin, Texas. SciPy is a community dedicated to the advancement of scientific computing through open source Python software for mathematics, science, and engineering. The annual SciPy Conference brings together over 500 participants from industry, academia, and government to showcase their latest projects, learn from skilled users and developers, and collaborate on code development. The full program will consist of two days of tutorials by followed by three days of presentations, and concludes with two days of developer sprints. More info available on the conference website at http://scipy2015.scipy.org; you can also sign up on the website for mailing list updates or follow @scipyconf on Twitter. We hope you'll join us - early bird registration is open until May 15, 2015 at http://scipy2015.scipy.org We encourage you to submit tutorial or talk proposals in the categories below; please also share with others who you'd like to see participate! Submit via the conference website @ http://scipy2015.scipy.org. *SCIPY TUTORIAL SESSION PROPOSALS - DEADLINE EXTENDED TO WED APRIL 1, 2015* The SciPy experience kicks off with two days of tutorials. These sessions provide extremely affordable access to expert training, and consistently receive fantastic feedback from participants. We're looking for submissions on topics from introductory to advanced - we'll have attendees across the gamut looking to learn. Whether you are a major contributor to a scientific Python library or an expert-level user, this is a great opportunity to share your knowledge and stipends are available. Submit Your Tutorial Proposal on the SciPy 2015 website: http://scipy2015.scipy.org *SCIPY TALK AND POSTER SUBMISSIONS - DUE April 1, 2015* SciPy 2015 will include 3 major topic tracks and 7 mini-symposia tracks. Submit Your Talk Proposal on the SciPy 2015 website: http://scipy2015.scipy.org Major topic tracks include: - Scientific Computing in Python (General track) - Python in Data Science - Quantitative and Computational Social Sciences Mini-symposia will include the applications of Python in: - Astronomy and astrophysics - Computational life and medical sciences - Engineering - Geographic information systems (GIS) - Geophysics - Oceanography and meteorology - Visualization, vision and imaging If you have any questions or comments, feel free to contact us at: scipy-organizers(a)scipy.org.

1 0

Re: [Numpy-discussion] 3D array and the right hand rule
by Dieter Van Eessen 17 Mar '15

17 Mar '15

Hello, Sorry to disturb again, but the topic still bugs me somehow... I'll try to rephrase the question: - What's the influence of the type of N-array representation with respect to TENSOR-calculus? - Are multiple representations possible? - I assume that the order of the dimensions plays a major role in for example TENSOR product. Is this assumption correct? As I said before, my math skills are lacking in this area... I hope you consider this a valid question. kind regards, Dieter On Fri, Jan 30, 2015 at 2:32 AM, Alexander Belopolsky <ndarray(a)mac.com> wrote: > > On Mon, Jan 26, 2015 at 6:06 AM, Dieter Van Eessen < > dieter.van.eessen(a)gmail.com> wrote: > >> I've read that numpy.array isn't arranged according to the >> 'right-hand-rule' (right-hand-rule => thumb = +x; index finger = +y, bend >> middle finder = +z). This is also confirmed by an old message I dug up from >> the mailing list archives. (see message below) >> > > Dieter, > > It looks like you are confusing dimensionality of the array with the > dimensionality of a vector that it might store. If you are interested in > using numpy for 3D modeling, you will likely only encounter 1-dimensional > arrays (vectors) of size 3 and 2-dimensional arrays (matrices) of size 9 > or shape (3, 3). > > A 3-dimensional array is a stack of matrices and the 'right-hand-rule' > does not really apply. The notion of C/F-contiguous deals with the order > of axes (e.g. width first or depth first) while the right-hand-rule is > about the direction of the axes (if you "flip" the middle finger right hand > becomes left.) In the case of arrays this would probably correspond to > little-endian vs. big-endian: is a[0] stored at a higher or lower address > than a[1]. However, whatever the answer to this question is for a > particular system, it is the same for all axes in the array, so right-hand > - left-hand distinction does not apply. > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion(a)scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > -- gtz, Dieter VE

2 1

ANN: HoloViews 1.0 released
by James A. Bednar 17 Mar '15

17 Mar '15

We are pleased to announce the first public release of HoloViews, a Python package for scientific and engineering data visualization: http://ioam.github.io/holoviews HoloViews provides composable, sliceable, declarative data structures for building even complex visualizations easily. It's designed to exploit the rich ecosystem of scientific Python tools already available, using Numpy for data storage, matplotlib and mpld3 as plotting backends, and integrating fully with IPython Notebook to make your data instantly visible. If you look at the website for just about any other visualization package, you'll see a long list of pretty pictures, each one of which has a page or two of code putting it together. There are pretty pictures in HoloViews too, but there is *no* hidden code -- *all* of the steps needed to build a given figure are shown right before the HoloViews plot, with just a few lines needed for nearly all of our examples, even complex multi-figure subplots and animations. This concise but flexible specification makes it practical to explore and analyze your data interactively, while leaving a full record for later reproducibility in the notebook. It may sound like magic, but it's not -- HoloViews simply lets you annotate your data with appropriate metadata, and then the data can display itself! HoloViews provides a set of general, compositional, multidimensional data structures suitable for both discrete and continuous real-world data, and pairs them with separate customizable plotting classes to visualize them without extensive coding. An large collection of continuously tested IPython Notebook tutorials accompanies HoloViews, showing you precisely the small number of steps required to generate any of the plots. Some of the most important features: - Freely available under a BSD license - Python 2 and 3 compatible - Minimal external dependencies -- easy to integrate into your workflow - Builds figures by slicing, sampling, and composing your data - Builds web-embeddable animations without any extra coding - Easily customizable without obscuring the underlying data objects - Includes interfaces to pandas and Seaborn - Winner of the 2015 UK Open Source Award For the rest, check out ioam.github.io/holoviews! Jean-Luc Stevens Philipp Rudiger James A. Bednar The University of Edinburgh School of Informatics -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

1 0

should views into structured arrays be reversible?
by Allan Haldane 17 Mar '15

17 Mar '15

Hello all, I've introduced PR 5548 <https://github.com/numpy/numpy/pull/5548> which, through more careful safety checks, allows views of object arrays. However, I had to make 'partial views' into structured arrays irreversible, and I want to check with the list that that's ok. With the PR, if you only view certain fields of an array you cannot take a 'reverse' view of the resulting object to get back the original array: >>> arr = np.array([(1,2),(4,5)], dtype=[('A', 'i'), ('B', 'i')]) >>> varr = arr.view({'names': ['A'], 'formats': ['i'], ... 'itemsize': arr.dtype.itemsize}) >>> varr.view(arr.dtype) TypeError: view would access data parent array doesn't own Ie., with this PR you can only take views into parts of an array that have fields. This was necessary in order to guarantee that we never interpret memory containing a python Object as another type, which could cause a segfault. I have a more extensive discussion & motivation in the PR, including an alternative idea. So does this limitation seem reasonable? Cheers, Allan

1 0

Fastest way to compute summary statistics for a specific axis
by Dave Hirschfeld 17 Mar '15

17 Mar '15

I have a number of large arrays for which I want to compute the mean and standard deviation over a particular axis - e.g. I want to compute the statistics for axis=1 as if the other axes were combined so that in the example below I get two values back In [1]: a = randn(30, 2, 10000) For the mean this can be done easily like: In [2]: a.mean(0).mean(-1) Out[2]: array([ 0.0007, -0.0009]) ...but this won't work for the std. Using some transformations we can come up with something which will work for either: In [3]: a.transpose(2,0,1).reshape(-1, 2).mean(axis=0) Out[3]: array([ 0.0007, -0.0009]) In [4]: a.transpose(1,0,2).reshape(2, -1).mean(axis=-1) Out[4]: array([ 0.0007, -0.0009]) If we look at the performance of these equivalent methods: In [5]: %timeit a.transpose(2,0,1).reshape(-1, 2).mean(axis=0) 100 loops, best of 3: 14.5 ms per loop In [6]: %timeit a.transpose(1,0,2).reshape(2, -1).mean(axis=-1) 100 loops, best of 3: 5.05 ms per loop we can see that the latter version is a clear winner. Investigating further, both methods appear to copy the data so the performance is likely down to better cache utilisation. In [7]: np.may_share_memory(a, a.transpose(2,0,1).reshape(-1, 2)) Out[7]: False In [8]: np.may_share_memory(a, a.transpose(1,0,2).reshape(2, -1)) Out[8]: False Both methods are however significantly slower than the initial attempt: In [9]: %timeit a.mean(0).mean(-1) 1000 loops, best of 3: 1.2 ms per loop Perhaps because it allocates a smaller temporary? For those who like a challenge: is there a faster way to achieve what I'm after? Cheers, Dave

4 4

numpy.stack -- which function, if any, deserves the name?
by Stephan Hoyer 16 Mar '15

16 Mar '15

In the past months there have been two proposals for new numpy functions using the name "stack": 1. np.stack for stacking like np.asarray(np.bmat(...)) http://thread.gmane.org/gmane.comp.python.numeric.general/58748/ https://github.com/numpy/numpy/pull/5057 2. np.stack for stacking along an arbitrary new axis (this was my proposal) http://thread.gmane.org/gmane.comp.python.numeric.general/59850/ https://github.com/numpy/numpy/pull/5605 Both functions generalize the notion of stacking arrays from the existing hstack, vstack and dstack, but in two very different ways. Both could be useful -- but we can only call one "stack". Which one deserves that name? The existing *stack functions use the word "stack" to refer to combining arrays in two similarly different ways: a. For ND -> ND stacking along an existing dimensions (like numpy.concatenate and proposal 1) b. For ND -> (N+1)D stacking along new dimensions (like proposal 2). I think it would be much cleaner API design if we had different words to denote these two different operations. Concatenate for "combine along an existing dimension" already exists, so my thought (when I wrote proposal 2), was that the verb "stack" could be reserved (going forward) for "combine along a new dimension." This also has the advantage of suggesting that "concatenate" and "stack" are the two fundamental operations for combining N-dimensional arrays. The documentation on this is currently quite confusing, mostly because no function like that in proposal 2 currently exists. Of course, the *stack functions have existed for quite some time, and in many cases vstack and hstack are indeed used for concatenate like functionality (e.g., whenever they are used for 2D arrays/matrices). So the case is not entirely clear-cut. (We'll never be able to remove this functionality from NumPy.) In any case, I would appreciate your thoughts. Best, Stephan

3 3

Fix masked arrays to properly edit views
by John Kirkham 15 Mar '15

15 Mar '15

The sample case of the issue ( https://github.com/numpy/numpy/issues/5558 ) is shown below. A proposal to address this behavior can be found here ( https://github.com/numpy/numpy/pull/5580 ). Please give me your feedback. I tried to change the mask of `a` through a subindexed view, but was unable. Using this setup I can reproduce this in the 1.9.1 version of NumPy. import numpy as np a = np.arange(6).reshape(2,3) a = np.ma.masked_array(a, mask=np.ma.getmaskarray(a), shrink=False) b = a[1:2,1:2] c = np.zeros(b.shape, b.dtype) c = np.ma.masked_array(c, mask=np.ma.getmaskarray(c), shrink=False) c[:] = np.ma.masked This yields what one would expect for `a`, `b`, and `c` (seen below). masked_array(data = [[0 1 2] [3 4 5]], mask = [[False False False] [False False False]], fill_value = 999999) masked_array(data = [[4]], mask = [[False]], fill_value = 999999) masked_array(data = [[--]], mask = [[ True]], fill_value = 999999) Now, it would seem reasonable that to copy data into `b` from `c` one can use `__setitem__` (seen below). b[:] = c This results in new data and mask for `b`. masked_array(data = [[--]], mask = [[ True]], fill_value = 999999) This should, in turn, change `a`. However, the mask of `a` remains unchanged (seen below). masked_array(data = [[0 1 2] [3 0 5]], mask = [[False False False] [False False False]], fill_value = 999999) Best, John

2 1

Error message
by Danny Kramer 14 Mar '15

14 Mar '15

Hi, I am getting the following error message: C:\Python27\lib\site-packages\numpy\lib\npyio.py:819: UserWarning: loadtxt: Empty input file: "[]" warnings.warn('loadtxt: Empty input file: "%s"' % fname) main loop list assignment index out of range 1. Why is it happening? 2. How can I fix it? Thanks a lot in advance. Danny

2 1

Numpy where
by Charles R Harris 14 Mar '15

14 Mar '15

Hi All, This is apropos gh-5582 <https://github.com/numpy/numpy/pull/5582> dealing with some corner cases of np.where. The following are the current behavior >>> import numpy >>> numpy.where(True) # case 1 ... (array([0]),) >>> numpy.where(True, None, None) # case 2 ... array(None, dtype=object) >>> numpy.ma.where(True) # case 3 ... (array([0]),) >>> numpy.ma.where(True, None, None) # case 4 ... (array([0]),) The question is, what exactly should be done in these cases? I'd be inclined to raise an error for cases 1 and 3. Case two looks correct to me if we agree that scalar inputs are acceptable. Case 4 looks wrong. Thoughts? Chuck

3 6

Custom __array_interface__ error
by Daniel Smith 13 Mar '15

13 Mar '15

Greetings everyone, I have a new project that deals with core and disk tensors wrapped into a single object so that the expressions are transparent to the user after the tensor is formed. I would like to add __array_interface__ to the core tensor and provide a reasonable error message if someone tries to call the __array_interface__ for a disk tensor. I may be missing something, but I do not see an obvious way to do this in the python layer. Currently I do something like: if ttype == “Core": self.__array_interface__ = self.tensor.ndarray_interface() else: self.__array_interface__ = {'typestr’: 'Only Core tensor types are supported.’} Which provides at least a readable error message if it is not a core tensor: TypeError: data type "Only Core tensor types are supported." not understood A easy solution I see is to change numpy C side __array_interface__ error message to throw custom strings. In numpy/core/src/multiarray/ctors.c:2100 we have the __array_interface__ conversion: if (!PyDict_Check(iface)) { Py_DECREF(iface); PyErr_SetString(PyExc_ValueError, "Invalid __array_interface__ value, must be a dict"); return NULL; } It could simply be changed to: if (!PyDict_Check(iface)) { if (PyString_Check(iface)){ PyErr_SetString(PyExc_ValueError, iface); } else{ PyErr_SetString(PyExc_ValueError, "Invalid __array_interface__ value, must be a dict”); } Py_DECREF(iface); return NULL; } Thoughts? Cheers, -Daniel Smith

3 2

random.RandomState and deepcopy
by Neal Becker 13 Mar '15

13 Mar '15

It is common that to guarantee good statistical independence between various random generators, a singleton instance of an RNG is shared between them. So I typically have various random generator objects, which (sometimes several levels objects deep) embed an instance of RandomState. Now I have a requirement to copy a generator object (without knowing exactly what that generator object is). My solution is to use deepcopy on the top-level object. But I need to overload __deepcopy__ on the singleton RandomState object. Unfortunately, RandomState doesn't allow customization of __deepcopy__ (or anything else). And it has no __dict__. My solution is: class shared_random_state (object): def __init__ (self, rs): self.rs = rs def __getattr__ (self, attr): return getattr (self.rs, attr) def __deepcopy__ (self): return self An example usage: rs = shared_random_state (RandomState(0)) from exponential import exponential e = exponential (rs, 1) where exponential is: class exponential (object): def __init__ (self, rs, mu): self.rs = rs self.mu = mu def __call__ (self, size=None): if size == None: return self.rs.exponential (self.mu, 1)[0] else: return self.rs.exponential (self.mu, size) def __repr__ (self): return 'exp(%s)' % self.mu I wonder if anyone has any other suggestions? Personally, I would prefer if numpy provided a more direct solution to this. Either by providing for overloading RandomState deepcopy, or by making the copy behavior switchable with a flag to the constructor. -- Those who fail to understand recursion are doomed to repeat it

2 3