NumPy-Discussion October 2005

numpy-discussion@python.org

37 participants
50 discussions

Re: [Numpy-discussion] Can you enumerarte an array?
by Chris Barker 12 Oct '05

12 Oct '05

Travis Oliphant wrote: > All flavors of Numeric would return > > 0 a[0] > 1 a[1] > . > . > . > n-1 a[n-1] > > where n=a.shape[0] > > in response to index, item in enumerate(A): Right, as expected, and as Anrd pointed out, this is useful behaviour. > In scipy you could also get > > 0 a.flat[0] > 1 a.flat[1] I do like the new flat! > To do something like he wants, I think we would have to construct a > different iterator then enumerate. Exactly,. I didn't mean to imply that that's what the built-in enumerate should do. > It wouldn't be that hard using the > a.flat iterator (it's just a remapping of the 1-d index). Cool. then again, consider this a feature request, for a nd_enumerate, or whatever you want to call it. I'm sorry that implimenting something like this myself is way out my depth! -Chris

3 5

question on differential results in windows and debian
by meng＠are.berkeley.edu 11 Oct '05

11 Oct '05

Hi there- I have a matrix X, calculate cX=matrixmultiply(transpose(X), X) and finally invert it by icX = linear_algebra.inverse(cX) cX is kind of ill-conditioned, which caused the icX differs a lot between windows and debian, after a tiny difference on cX across platforms. Here are the results: In windows: >>> x.min() -0.51187150837988826 >>> x.max() 0.81517690875232773 >>> cX = matrixmultiply(transpose(x), x) >>> cX.min() -666.96857541904126 >>> cX.max() 1073.3945530725441 >>> import numarray.linear_algebra as la >>> icX = la.inverse(cX) >>> icX.min() -3287030277.3580685 >>> icX.max() 0.0 In debian: >>> x.min() -0.51187150837988826 >>> x.max() 0.81517690875232773 >>> cX = matrixmultiply(transpose(x), x) >>> cX.min() -666.96857541898294 >>> cX.max() 1073.3945530726264 >>> import numarray.linear_algebra as la >>> icX = la.inverse(cX) >>> icX.min() 0.0 >>> icX.max() 84778028554.337051 I don't understand why a multiplication of a matrix and its transpose will create a tiny difference across windows and debian, which could be disastrous when inverting it. Please help me. Thanks a lot for your time! BTW, I am using Python2.4 and numarray 1.3.2 in both machines. Best, Xiangyi Xiangyi Meng Department of Agricultural and Resource Economics Room 303, Giannini Hall #3310 University of California, Berkeley Berkeley, CA 94720-3310 Tel: (510) 643-4124 Fax: (510) 643-8911 Email: meng(a)are.berkeley.edu

1 0

ASP status, docs, and supporting development
by Joe Harrington 11 Oct '05

11 Oct '05

After SciPy '04, Perry, Janet, and a few others and I put together a roadmap and framework, called ASP, for people to contribute to all the parts of SciPy other than the software: docs, packaging, website, etc. There was much interest expressed, and a few people even signed on the electronic dotted line to be involved: http://www.scipy.org/wikis/accessible_scipy/AccessibleSciPy The roadmap is laid out in this thread, which is linked in the first paragraph of the page above: http://www.scipy.org/mailinglists/mailman?fn=scipy-dev/2004-October/002412.… The first thing we did was to gather all the external projects using SciPy that we could find, and make an index page. The community found them and Fernando Perez did the hard work (more than he bargained for) of collating everything into the index page: http://www.scipy.org/wikis/topical_software/TopicalSoftware That is now a live wiki page, so if your project isn't on there, please add it! We were gearing up for effort #2, a web-site overhaul, early this year, when Travis announced his intention to heal the rift between the small- and large-array communities. We held up our push on web development, which was a few days from kickoff, so that it wouldn't take volunteers from his more-crucial effort. We all know the story of last year: Travis worked like crazy, called for volunteers, got only a few, and finished the job anyway. Now he's publishing a book in hopes of supporting some of his work from the revenues. He's made it clear that he will not be offended by or opposed to free docs, and may even contribute to them. He's also still Head Nummie and is leading the hard work of testing and bug swatting. Of course, we all want him to continue, and most of us freely admit our getting more than we are giving, and our gratitude to Travis, Todd, Robert, and the other core developers. Meanwhile, we have the problem of needing some basic docs that are free, and there seems to be quite a bit of interest in the community for doing one or more free docs. This seems to have more community energy behind it now than a web overhaul, so let's do it. The wiki and procedures are all set up to do some docs, and have been for about a year now. If you're interested in doing some docs, either as the lead author of a doc, as a contributor, or as a reviewer, please sign up on http://www.scipy.org/wikis/accessible_scipy/AccessibleSciPy The goal of the signups page is to make it easy for people to find each other: for lead authors to find people who will help them, for the community to identify who is taking part in what efforts, for low-level-of-effort volunteers to become hooked up with bigger projects, etc. There should be dozens of names there, not just three! So: If you're interested in LEADING A DOC, please add your name to the page, make a page for your doc on the wiki, and hang it off the main page, as "Scipy Cookbook" has done (there's a help link at the top of the page with instructions). A project can be anything from writing a collaborative book from scratch, to writing a monograph, to editing and revising existing docs. Announce your project on scipy-dev. If you would like to do a little work but not take the lead on something, you can contribute to the Cookbook or sign up to be a reviewer or contributor, either on an existing doc or at large. Or, contact a doc lead directly and sign up under that project. Please read the roadmap for ideas of docs we thought the community needs. The roadmap document is meant to be amended. For example, is the idea of using the docstrings to make a full-blown reference manual a good idea? I think so, since it's a rather self-updating format, but it will require some substantial work to get them all fleshed out and up to par. Discuss plans, changes, and ongoing efforts for docs on the scipy-dev mailing. It would be nice to have each project have a home on scipy.org, and Plone has excellent workflow-management tools. But, it's ok to home a project on your own site and just put a link on scipy.org. Finally, PLEASE everyone buy Travis's book if you can! Wait for the price to go up. Buy copies for everyone in your lab, if you can afford that. Buy one for your grandmother. It looks like it will be a really nice book, but it's more than that. This is a way to support development, which everyone desperately needs but few have the time (and fewer the skill and experience) to do. If you're at a company that benefits from SciPy and that hasn't already contributed resources to the effort, please consider parting with a larger chunk of change, either by buying more copies or by hiring Travis or others directly to do ongoing maintenance and package development. --jh--

1 0

Problems compiling ufuncs example
by Kosta Gaitanis 10 Oct '05

10 Oct '05

Hi all, I'm trying to run the UFunc example in the Numarray manual (http://stsdas.stsci.edu/numarray/numarray-1.1.html/node32.html) but when I run the command python setup.py buil_ext I get the following output : running build_ext error: Python was built with version 7.1 of Visual Studio, and extensions need t o be built with the same version of the compiler, but it isn't installed so I installed Visual C++ Toolkit 2003 and then run the same command again and got the following output : running build_ext error: the .NET Framework SDK needs to be installed before building extensions f or Python so I installed .NET Framework SDK 1.1 and then tried again. I got back the first error... does anyone have an idea ? What am I doing wrong ? thanks in advance Kosta I'm using Python24 and NumArray 1.3.3 -- setup.py -- """ This setup demonstrates how to use the numarray code generation facilities to define additional ufuncs at the user level. Universal functions apply operators or C-functions elementwise to all of the elements of an array or pair of arrays. This setup generates code defining three universal functions which are installed as the numarray.foo package: Cos, bessel, and bar. The ufuncs are used like this: >>> import numarray as na >>> import numarray.foo as foo >>> a = na.array([1,2,3,4], type='Int32') >>> foo.Cos(a) array([ 0.54030228, -0.41614684, -0.9899925 , -0.65364361 ], type=Float32) Note that since a is an Int32 array, Cos(a) first does blockwise conversion of a to Float32 before calling the cosine library function. """ import distutils, os from distutils.core import setup from numarray.numarrayext import NumarrayExtension from numarray.codegenerator import UfuncModule, make_stub # ====================================================================== # # Generate the C-code for the numarray.foo._foo extension: # m = UfuncModule("_foo") # Inline the code for a couple C functions to be turned into ufuncs. # This method lets you define your function here in the setup.py. An # alternate approach would be to link with a libarary or additional C # module. m.add_code(""" static double c_bessel(double x, double y) { double x2=x*x, y2=y*y, diff=x2-y2; return diff*diff/(x2+y2); } static UInt8 c_bar(UInt32 x ) { return (x >> 24) & 0xFF; } """) # Method add_ufunc() defines the name of a ufunc, it's corresponding C # function which is applied element-wise to all elements of an array, # The arity of the ufunc (unary or binary only), and the I/O type # signatures which are directly supported by the ufunc. Binary Ufunc # "bessel" is implemented by the inline function "c_bessel" from the # add_code() call above. m.add_ufunc("bessel", "c_bessel", 2, [('Float32', 'Float32'), ('Float64', 'Float64')]) # Unary Ufunc "bar" only supports one builtin type signature: # UInt32-->UInt8. Other types are blockwise converted by the ufunc # machinery to UInt32 before "c_bar" is called. m.add_ufunc("bar", "c_bar", 1, [('UInt32','UInt8')]) # Unary Ufunc "cos" is implemented by the C standard library function # "cos". Given a Float32 array, it returns a Float32 array. Given a # Float64 array, it returns a Float64 array. For other arrays, # transparent conversion to one of the supported input types is performed # by the ufunc machinery. m.add_ufunc("cos", "cos", 1, [('Float32', 'Float32'), ('Float64', 'Float64')]) # The generate() method writes out the complete extension module to the # specified C file. m.generate("foo/Src/_foo.c") # ====================================================================== # Create foo's __init__.py for defining UFuncs corresponding to CFuncs # in _foo. make_stub() emits boiler-plate which makes your extension # a package. The __init__ file imports all the public symbols from # C extension _foo making them visible as numarray.foo. make_stub("foo/Lib/__init__", "_foo") # ====================================================================== # # Standard distutils setup for the generated code. # setup(name = "foo", version = "0.1", maintainer = "Todd Miller", maintainer_email = "jmiller(a)stsci.edu", description = "foo universal functions for numarray", url = "http://www.stsci.edu/projects/foo/", packages = ["numarray.foo"], package_dir = { "numarray.foo":"foo/Lib" }, ext_modules = [ NumarrayExtension( 'numarray.foo._foo', ['foo/Src/_foo.c'], # libraries = ['ttf'], # include_dirs = ['include'], # library_dirs = ['lib'] ) ] )

1 1

Re: [Numpy-discussion] Re: Purchasing Documentation
by Tim Churches 10 Oct '05

10 Oct '05

Paul F. Dubois wrote: > The original sources were in Framemaker. I am not positive where they > are. In any case they are copyrighted by the Regents of the University > of California. I am not a lawyer and don't know what the consequences of > that are. LLNL granted free distribution of the printed document with > the Numeric source code but I don't know what their position would be on > using their copyrighted text in a new document or on giving away the > sources. OK, thanks Paul. That may have implications for you, Travis, if you are planning to base your SciPy Core book on the existing NumPy documentation. Given the circumstance which Paul describes, perhaps the only way forward is to create a freely available addendum to the freely available, existing NumPy documentation which describes how and where SciPy Core differs or extends NumPy? Or to write entirely new documentation from scratch. > I believe that under U.S. copyright law, a person may lend their copy of > a book to anyone but not reproduce it and give them a copy. Yes, likewise here in Oz, and in most countries. Of course, the publisher of a book can impose additional restrictions to which the purchaser of the book must agree, such as 'You may not show or lend your copy of this book to your colleagues, or to anyone else.' However, who in their right mind would buy a book which was sold with such restrictions attached to it? > Given the way open source capitalism works, I would not be surprised if > someone produces a 'quick reference guide' that they give away, or > writes a book on 'Using scipy.core in biology' that they sell for $88. > Let a thousand flowers bloom. Yes, I agree entirely. Travis, you are at perfect liberty to create commercial documentation for SciPy Core, but please don't object if others try to organise to create free open source documentation as well. Tim C > Tim Churches wrote: > >> Travis Oliphant wrote: >> >>> Tim Churches wrote: >>> >>> >>>> Travis Oliphant wrote: >>>> >>>> >>>> >>>>> Tim Churches wrote: >>>>> >>>>> >>>> >>>> >>>> Travis, are you saying that this agreement only allows a single person >>>> to read the single printed copy? If so, I think you need a formally >>>> worded legal license to make that stick - certainly Australian >>>> copyright >>>> law (nor copyright law in other countries, I suspect) does not provide >>>> any support for such severe restrictions in the use of a printed >>>> document. Under copyright law, you may not make unauthorised copies >>>> of a >>>> printed document, but you can certainly lend that copy to others, or >>>> sell or give it to them, or share it as a bench manual. >>>> >>>> >>> >>> I'm just stating what I think is fair. I'm not going to try and use the >>> power of the Australian state or any other to try and enforce it. >> >> >> >> My point was that you won't get any help from the State if what you >> think is fair extends to restricting who can read a single printed copy >> of your documentation. You will need to resort to tort law to enforce >> such restrictions, and thus you really need a formal documentation >> license, if that is your aim. >> >> >>>> Many of those downloads are for "t[y|i]re-kicking" - potential users >>>> determining whether it meets their needs. If those potential users have >>>> to pay for documentation up-front, then a large proportion will go >>>> elsewhere. The rest are probably existing NumPy users upgrading (or >>>> testing the upgrade waters). The latter group may well pay for >>>> documentation, but how large is that group? For example, how many >>>> people >>>> are subscribed to the numpy-discussion mailing list? >>>> >>>> >>> >>> Don't know. But, there is enough free documentation to get started, I >>> think. Lack of documentation has hampered more people than cost of >>> existing documentation, I think. >> >> >> >> I have to disagree on this point. Granted the existing documentation for >> NumPy is not fantastic - some aspects of it are positively obscure - but >> overall it is better than good-enough, we have found, and was not a >> barrier to our decision to use NumPy. Having to cough up $50 to even >> peek at the documentation would be a far, far greater barrier to uptake >> than the occasional obscure wording of some of the function >> descriptions, IMHO. >> >> >>>> Travis, although many of us are grateful for your efforts on SciPy >>>> Core, >>>> no-one made you do it. If you wanted to earn (more) money by doing >>>> other >>>> things, you should have done them. >>>> >>> >>> I'm really thinking about the future here. If I can succesfully raise >>> money to support work on this using a simple time-delay documentation >>> encouragement, then perhaps others will do the same, and we can all have >>> more. Waiting for people to "donate" their time to develop scientific >>> python is rather slow.... I've been freely donating time for a long >>> time, and there are few who help. So, we'll try a slightly different >>> route and see where that goes. >> >> >> >> Yes, fair enough. My point was that 7 years is far too long a time >> delay. Given that SciPy Core will probably take another year to become >> rock-solid, I would happily buy several copies of your documentation at >> $50 per copy if I knew that in a year the documentation could be freely >> distributed to all God's children. But in 7 years? Nope, too long to >> wait, not interested in supporting that. >> >> >>>> I think there needs to be some community debate about this. Is there >>>> sufficient interest for people other than Travis to start with the >>>> Numeric documentation and update it as necessary to become a free SciPy >>>> Core documentation? The NumPy documentation is available in HTML format >>>> as the basis of this - perhaps the original source (LaTeX?) for the >>>> Numeric docs is also available? >>> >>> >>> I don't know why you would want to undermine my efforts in this way by >>> duplicating effort. >> >> >> >> Travis, I don't want to undermine your efforts, but you must understand >> that one of the attractions of NumPy and thus Scipy Core is that they >> are free, open source software, which implies that there exists adequate >> free, open source documentation for them, or if such free, open source >> documentation does not exist, then taht users are collectively at >> liberty to create it. That's a basic FOSS tenet. What you are are >> proposing is the creation of proprietary documentation for SciPy Core. >> No-one objects to this, but only if it is supplementary to free, open >> source "core" documentation, not instead of it. >> >> >>> Perhaps, instead you could have people donate $$ >>> instead of time to releasing the documentation. I give away copies of >>> the documentation to people who participate in the development process >>> all the time (and that comes off the total price --- though I haven't >>> advertised this as of yet). So, why don't you encourage people who >>> don't have the money to contribute to the project instead. >> >> >> >> I understand that open source software and its documentation don't just >> appear out of thin air, and I fully support the idea of funded or >> collectively-commissioned open source development. But only if the >> results of that funded or commissioned development is, in fact, free and >> open sourced. By preventing free distribution of the documentation you >> are creating for 7 years, the end product cannot even remotely be >> described as open source, thus personally I am disinterested in funding >> it. If the delay until open sourcing was just one year, then that's a >> different matter, but you do not seem to be proposing that. >> >> Hence I think the NumPy/SciPy Core user community should establish a >> SciPy Core documentation project, with the aim of updating and enhancing >> the existing NumPy documentation so that it covers the new and changed >> features of SciPy Core, and making that documentation available on teh >> same free, open source basis as NumPy/SciPy Core itself. >> >> To that end, are the original LaTeX or whatever sources for the currenet >> NumPy documenattion available somewhere (apart from the HTML and PDF >> versions, that is). >> >> Tim C >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: >> Power Architecture Resource Center: Free content, downloads, discussions, >> and more. http://solutions.newsforge.com/ibmarch.tmpl >> _______________________________________________ >> Numpy-discussion mailing list >> Numpy-discussion(a)lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> >

7 12

ma is in scipy_core
by Eric Firing 10 Oct '05

10 Oct '05

Paul, Tim, Masked arrays *are* in scipy_core. import scipy.base.ma as ma In addition, a quick look indicates that NaN-handling is in good shape. Eric

9 22

Dcoument strings for ndarray
by Colin J. Williams 09 Oct '05

09 Oct '05

With a view to exploring Numeric3, below is a script to check the availability of doc strings. Some are not yet available. There is a method __class__ module but the purpose is not clear. Colin W. # checkDocs.py ''' To check the availabilty of doc strings for the ndarray class. Among the attributes of ndarray is: __class__ module(name[, doc]) Create a module object. The name must be a string; the optional doc argument can have any type. ndarray is referenced as a class and reported as "<type 'type'>" but the common Python usage is <type 'classobj'> see the last output line. ''' import scipy.base.multiarray as M print type(M.ndarray) print '>', type( M) print 'module doc:', M.__doc__ for i in dir(M.ndarray): print i, try: print eval('M.'+i+'.__doc__') except: print 'No doc' print '\n', M.ndarray.__doc__ print type(M.ndarray) # Check normal usage import timeit print timeit.Timer, type(timeit.Timer)

3 6

Can you enumerarte an array?
by Chris Barker 09 Oct '05

09 Oct '05

Hi all, A freind of mine that I jsut introduced to NumPy asked me about enumerating an array. What he'd like to see is something like: A = N.ones((3,4)) for indexes, item in enumeate(A): print indexes result in: (0,0) (0,1) (0,2) (1,0) . . . You get the idea. Can you do this with any of the three NumPys? If not consider this a feature request for scipy_base. -Chris

2 1

Question about 64-bit integers being cast to double precision
by Travis Oliphant 09 Oct '05

09 Oct '05

What is the opinion of people here regarding the casting of 64-bit integers to double precision. In scipy (as in Numeric), there is the concept of "Casting safely" to a type. This concept is used when choosing a ufunc, for example. My understanding is that a 64-bit integer cannot be cast safely to a double-precision floating point number, because precision is lost in the conversion. However, at least a signed 64-bit integer can usually be cast safely to a long double precision floating point number. This is not too big a deal on 32-bit systems where people rarely request 64-bit integers. However, on some 64-bit systems (where the C long is 64-bit), Python's default integer is 64-bit. Therefore, simple expressions like sqrt(2) which require conversion to floating point will look for the first floating point number that it can convert a 64-bit integer to safely. This can only be a long double. The result is that on 64-bit systems, the long double type gets used a lot more. Is this acceptable? expected? What do those of you on 64-bit systems think? -Travis

1 0

Another PEP needed for the Numeric community.
by Travis Oliphant 08 Oct '05

08 Oct '05

Full, 64-bit support in Python is problematic because of the buffer and sequence protocols which require int's in their interfaces. Right now, numarray support memory mapped arrays. SciPy can easily support memory mapped arrays by modifying the same module that numarray has forged (using the frombuffer function). However, the module will be limited on 64-bit systems because memory-map support in Python is limited to 32-bit files even on 64-bit systems. This was an intentional limitation of the memory map module in Python so that the sequence and buffer protocols could be supported. However, the choice to not even allow a 64-bit size memory map to be created seems wrong. There is no need to go through the sequence and buffer interface at all times. If the memory-map module in Python were re-written to inherit from a big-map object that did not export the buffer and sequence protocols (or did so in a limited fashion), then all that is needed is for the object to export it's data pointer and size. Then a function could be written to create an array in scipy that used the mmap's exported buffer as its data region. This would allow very big memory mapped arrays without waiting for Python to fix it's sequence and buffer protocols (which may not be fixed until Python 3.0). Thus, a PEP stating how the mmap module should change to support 64-bit systems is needed. Another possible project idea for any lurking people desiring to help. The other possibility is to just copy the nice muliplatform code in the Python mmap module and use an ndarray as the memory-mapped object. -Travis

1 0

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

NumPy-Discussion October 2005