Mailman 3 July 2007 - NumPy-Discussion

ANN: PyQwt-5.0.1 released
by Gerard Vermeulen 26 Jul '07

26 Jul '07

What is PyQwt ( http://pyqwt.sourceforge.net ) ? - it is a set of Python bindings for the Qwt C++ class library which extends the Qt framework with widgets for scientific and engineering applications. It provides a widget to plot 2-dimensional data and various widgets to display and control bounded or unbounded floating point values. - it requires and extends PyQt, a set of Python bindings for Qt. - it supports the use of PyQt, Qt, Qwt, and optionally NumPy or SciPy in a GUI Python application or in an interactive Python session. - it runs on POSIX, Mac OS X and Windows platforms (practically any platform supported by Qt and Python). - it plots fast: displaying data with 100,000 points takes about 0.1 s (PyQwt with Qt-3 is faster than with Qt-4). - it is licensed under the GPL with an exception to allow dynamic linking with non-free releases of Qt and PyQt. The most important new features of PyQwt-5.0.1 are: - support for Qt-4.3, SIP-4.7, and PyQt-4.3 - support for the N-D array interface specification ( http://numpy.scipy.org/array_interface.shtml ). - the iqt module also supports an interactive Python session without the help of the GNU readline module. - PyQwt is now part of the PyQt-Py2.5-gpl-4.3 binary installer for Windows ( http://www.riverbankcomputing.com/Downloads/PyQt4/GPL/ ) The most important bug fix in PyQwt-5.0.1 is: - removal of a huge memory leak in the conversion from an array to a QImage. PyQwt-5.0.1 supports: 1. Python-2.5, or -2.4. 2. PyQt-3.17. 3. PyQt-4.3, or PyQt-4.2. 3 SIP-4.7, or SIP-4.6. 4. Qt-3.3, or Qt-3.2. 5. Qt-4.3, or Qt-4.2. 6. Recent versions of NumPy, numarray, and/or Numeric. Enjoy -- Gerard Vermeulen

1 0

Re: [Numpy-discussion] Compile extension modules with Visual Studio 2005
by Ray Schumacher 26 Jul '07

26 Jul '07

Geoffrey Zhu wrote: > Hi, > > I am about to write a C extension module. C functions in the module will > take and return numpy arrays. I found a tutorial online, but I am not > sure about the following: I agree with others that ctypes might be your best path. The codeGenerator is magic, if you ask me: http://starship.python.net/crew/theller/ctypes/old/codegen.html But, if the function is simple, why not weave.inline? What I have done is run the function once, hunt down the long-named library, copy it to the local directory, then include it explicitly and call its function. This eliminates some overhead time for the call. I use it to convert packed IEEE data from an ADC data read function, and it's faster than the manufacturer's own function version that returns scaled integers! Ray

5 5

output arguments
by Thomas Breuel 25 Jul '07

25 Jul '07

Hi, core NumPy doesn't seem to support a lot of output arguments, or common composite operations. For example, a common operation is something like a = outer(b,c) or a += outer(b,c) There are some workarounds, but they aren't pretty. Consistently providing output arguments throughout NumPy would help; is there any reason this isn't being done? For example, it would be nice if "outer" supported: outer(b,c,output=a) outer(b,c,increment=a) outer(b,c,increment=a,scale=eps) or maybe one could specify an accumulation ufunc, with addition, multiplication, min, and max being fast, and with an optional scale parameter. Another approach might be to provide, in addition to the convenient high-level NumPy operations, direct bindings for BLAS and/or similar libraries, with Fortran-like procedural interfaces, but I can't find any such libraries in NumPy or SciPy. Am I missing something? It seems like writing native code to speed up these kinds of operations isn't really so great because (1) it unnecessarily complicates development and packaging, and (2) is likely to perform worse than BLAS and similar libraries. How are other people handling these cases? Thanks, Tom

2 2

Should I use numpy array?
by Geoffrey Zhu 25 Jul '07

25 Jul '07

Hi, I am writing a function that would take a list of datetime objects and a list of single letter characters (such as ["A","B","C"]). The number of items tend to be big and both the list and the numpy array have all the functionalities I need. Do you think I should use numpy arrays or the regular lists? Thanks, cg

2 1

Re: [Numpy-discussion] tofile speed
by Lars Friedrich 25 Jul '07

25 Jul '07

Hello, I tried the following: ####### start code a = N.random.rand(1000000) myFile = file('test.bin', 'wb') for i in range(100): a.tofile(myFile) myFile.close() ####### end code And this gives roughly 50 MB/s on my office-machine but only 6.5 MB/s on the machine that I was reporting about. Both computers use Python 2.4.3 with enthought 1.0.0 and numpy 1.0.1 So I think I will go and check the harddisk-drivers. array.tofile does not seem to be the problem and actually seems to be very fast. Any other recommendations? Thanks Lars

2 1

Compile extension modules with Visual Studio 2005
by Geoffrey Zhu 24 Jul '07

24 Jul '07

Hi, I am about to write a C extension module. C functions in the module will take and return numpy arrays. I found a tutorial online, but I am not sure about the following: 1. Can I compile my extension with Visual Studio 2005? My impression is that I will have to link with numpy libraries, and, if numpy was compiled with a different compiler, I might have problems. However, if numpy is a DLL, maybe there is a way that I can build a LIB file based on the DLL and link with the LIB file. Does anyone have experience in doing this? 2. I am new to writing python extensions. The tutorial is doing things by hand. Does anyone know what is the best way to do this? How about SWIG? Thanks, Geoffrey

3 6

numpy-1.0.3 compile-problems on WindowsXP/Visual Studio 2005
by Csaba 24 Jul '07

24 Jul '07

Hi, did somebody suceeded to compile/install numpy-1.0.3 from the sources under WindowsXP with Visual Studio 2005? Shortly after issuing the > python setup.py build command, the build breaks with a strange error and even stranger recommendation: > ............................. > building extension "numpy.core.multiarray" sources > Generating build\src.win32-2.5\numpy\core\config.h > No module named msvccompiler in numpy.distutils, trying from distutils.. > error: Python was built with Visual Studio 2003; > extensions must be built with a compiler than can generate compatible binaries. > Visual Studio 2003 was not found on this system. If you have Cygwin installed, > you can try compiling with MingW32, by passing "-c mingw32" to setup.py. > I successfully built and used Python-2.5, Qt-4.3.0, sip-4.5.2 and PyQwt-5.0.0 from the very same environment ( WindowsXP, nmake from VS-2005, Platform SDK, etc.). Everything is built with the very same binaries and tools behind, why would numpy-1.0.3's setup.py beleave that I had or ever used VS-2003? > extensions must be built with a compiler than can generate compatible binaries. I don't really understand this neither, compattible to whom or what? Please help me. Best Regards, Csaba F. Kiss - csaba.kiss(a)scienomics.com

1 0

Re: [Numpy-discussion] getting numPy happening for sciPy
by Tim Mortimer 24 Jul '07

24 Jul '07

Thanks for the responses, http://code.enthought.com/enstaller/ I'll probably skip this. Although it looks like a useful tool, the computer / Python 2.5 installation i'm "managing" isn't on the internet, so any synchronisation or what not probably won't be possible there. ( i think that's what that was all about...) Happy to wait for a new NumPy package, plenty to keep me busy till then. Will watch this list for announcements. thanks to you all.

1 0

Re: [Numpy-discussion] tofile speed
by Lars Friedrich 24 Jul '07

24 Jul '07

Hello everyone, thank you for the replies. Sebastian, the chunk size is roughly 4*10^6 samples, with two byte per sample, this is about 8MB. I can vary this size, but increasing it only helps for much smaller values. For example, when I use a size of 100 Samples, I am much too slow. It gets better for 1000 Samples, 10000 Samples and so on. But since I already reached a chunksize in the region of megabytes, I have difficulties to increase my buffer size further. I also have the feeling that increasing does not help in this size region. (correct me if I am wrong...) Chuck, I am using a Windows XP system with a new (few months old) Maxtor SATA-drive. Lars

2 2

tofile speed
by Lars Friedrich 23 Jul '07

23 Jul '07

Hello everyone, I am using array.tofile successfully for a data-acqusition-streaming application. I mean that I do the following: for a long time: temp = dataAcquisisionDevice.getData() temp.tofile(myDataFile) temp is a numpy array that is used for storing the data temporarily. The data acquisition device is acquiring continuously and writing the data to a buffer from which I can read with .getData(). This works fine, but of course, when I turn the sample rate higher, there is a point when temp.toFile is too slow. The dataAcquisitionDevice's buffer will run full before I can fetch the data again. (temp has a size of ~Mbyte, and the for loop has a period of ~0.5 seconds so that increasing the chunk size won't help) I have no idea how efficient array.tofile() is. Maybe it is terribly efficient and what I see is just the limitation of my hardware (harddisk). Currently I can stream with roughly 4 Mbyte/s, which is quite fast, I guess. However, if anyone can point me to a way to write my data to harddisk faster, I would be very happy! Thanks Lars -- Dipl.-Ing. Lars Friedrich Photonic Measurement Technology Department of Microsystems Engineering -- IMTEK University of Freiburg Georges-Köhler-Allee 102 D-79110 Freiburg Germany phone: +49-761-203-7531 fax: +49-761-203-7537 room: 01 088 email: lfriedri(a)imtek.de

3 2