Mailman 3 November 2008 - SciPy-User

calculations using the datetime information of timeseries
by Tim Michelsen 02 Dec '08

02 Dec '08

Hello, I need to perform calculations for a time series that use the datetime of each data point as input. An example: def myfunction(datetime_obj, scaling_factor): pass I found out that I can get the datetime for each entry with for i in range(0, series.size): series[i] = myfunction(series.dates.tolist()[i], 10.) Now, I noticed a strange thing. If I have a base series "base_series" and assige it to a new one with new_series = base_series The base_series gets updated/changed according to all calculations I perform on new_series (Please see method 1 below). The only way I could imagine to make my code work is createding lots of template series like in method 3 below. This way lets me calculate my new values in new_series using the datetime information and still retrain base_series with its original values. I kindly ask you to shed some light why the base_series get changed when I change derived series. Is there a more efficient way to acomplish my task that I may haven't thought of so far? Thanks in advance! Kind regards, Timmie #### BELOW A SAMPLE SCRIPT THAT MAY ILLUSTRATE #### #!/usr/bin/env python # -*- coding: utf-8 -*- import datetime import scikits.timeseries as ts import numpy as np #create dummy series data = np.zeros(600)+1 now = datetime.datetime.now() start = datetime.datetime(now.year, now.month, now.day) #print start start_date = ts.Date('H', datetime=start) #print start_date series_dummy = ts.time_series(data, dtype=np.float_, freq='H', start_date=start_date) snew = series_dummy ###method 1 for i in range(0,snew.size): snew[i] = snew[i]* 2 #snew.dates[i].datetime print "method 1:", snew.sum()-series_dummy.sum() ###method 2 for i in range(0,snew.size): snew = snew*2 print "method 2:", snew.sum()-series_dummy.sum() #method 3: data = np.zeros(series_dummy.size)+1 dt_arr = series_dummy.dates cser = ts.time_series(data.astype(np.float_), dt_arr) for i in range(0,cser.size): # note: cser.dates[i].datetime.hour is just used as an example # my function performes calculations based on the value of the datetime of each data point for each data point (current datetime is the input parameter). cser[i] = cser.dates[i].datetime.hour print "method 3:", cser.sum()-series_dummy.sum()

3 4

weave problems, weave_imp.o no such file or directory
by Søren Nielsen 02 Dec '08

02 Dec '08

Hi, I've done a fresh install of MinGw 5.14, Python 2.4.4, Scipy 0.6.0, Numpy 1.1.0 and numarray 1.5.2 When I try to use weave i get this: C:\>test_weave.py <weave: compiling> running build_ext running build_src building extension "sc_552cccf5dbf4f6eadd273cdcbd5860523" sources customize Mingw32CCompiler customize Mingw32CCompiler using build_ext customize Mingw32CCompiler customize Mingw32CCompiler using build_ext building 'sc_552cccf5dbf4f6eadd273cdcbd5860523' extension compiling C++ sources C compiler: g++ -mno-cygwin -O2 -Wall compile options: '-IC:\Python24\lib\site-packages\scipy\weave -IC:\Python24\lib\ site-packages\scipy\weave\scxx -IC:\Python24\lib\site-packages\numpy\core\includ e -IC:\Python24\include -IC:\Python24\PC -c' g++ -mno-cygwin -O2 -Wall -IC:\Python24\lib\site-packages\scipy\weave -IC:\Pytho n24\lib\site-packages\scipy\weave\scxx -IC:\Python24\lib\site-packages\numpy\cor e\include -IC:\Python24\include -IC:\Python24\PC -c C:\Python24\lib\site-package s\scipy\weave\scxx\weave_imp.cpp -o c:\docume~1\lisear~1\lokale~1\temp\Lise Arle th\python24_intermediate\compiler_08edc7e348e1c33f63a33ab500aef08e\Release\pytho n24\lib\site-packages\scipy\weave\scxx\weave_imp.o Found executable C:\MinGw\bin\g++.exe g++.exe: Arleth\python24_intermediate\compiler_08edc7e348e1c33f63a33ab500aef08e\ Release\python24\lib\site-packages\scipy\weave\scxx\weave_imp.o: No such file or directory Traceback (most recent call last): File "C:\test_weave.py", line 341, in ? main() File "C:\test_weave.py", line 224, in main weave.inline('printf("%d\\n",a);',['a'], verbose=2, type_converters=converte rs.blitz) #, compiler = 'msvc', verbpse=2, type_converters=converters.blitz, au to_downcast=0) #'msvc' or 'gcc' or 'mingw32' File "C:\Python24\Lib\site-packages\scipy\weave\inline_tools.py", line 338, in inline auto_downcast = auto_downcast, File "C:\Python24\Lib\site-packages\scipy\weave\inline_tools.py", line 447, in compile_function verbose=verbose, **kw) File "C:\Python24\Lib\site-packages\scipy\weave\ext_tools.py", line 365, in co mpile verbose = verbose, **kw) File "C:\Python24\Lib\site-packages\scipy\weave\build_tools.py", line 269, in build_extension setup(name = module_name, ext_modules = [ext],verbose=verb) File "C:\Python24\Lib\site-packages\numpy\distutils\core.py", line 184, in set up return old_setup(**new_attr) File "C:\Python24\lib\distutils\core.py", line 166, in setup raise SystemExit, "error: " + str(msg) distutils.errors.CompileError: error: Command "g++ -mno-cygwin -O2 -Wall -IC:\Py thon24\lib\site-packages\scipy\weave -IC:\Python24\lib\site-packages\scipy\weave \scxx -IC:\Python24\lib\site-packages\numpy\core\include -IC:\Python24\include - IC:\Python24\PC -c C:\Python24\lib\site-packages\scipy\weave\scxx\weave_imp.cpp -o c:\docume~1\lisear~1\lokale~1\temp\Lise Arleth\python24_intermediate\compiler _08edc7e348e1c33f63a33ab500aef08e\Release\python24\lib\site-packages\scipy\weave \scxx\weave_imp.o" failed with exit status 1 the test_weave file was something i found on the scipy dev wiki... I also tried some of my older files that also uses weave and they all give the same error... Can anyone help me with this?

4 5

Is it possible to pass Fortran derived data types to Python via C and SWIG?
by John Salvatier 01 Dec '08

01 Dec '08

I have a Fortran 90 algorithm which uses a derived data type to return data, and I would like to make a python wrapper for this algorithm. I understand that f2py cannot wrap derived data types; is it possible to do so with a C interface for the Fortran algorithm and SWIG? I would have to pass the derived data type into a C struct and then to Python. Best Regard, John Salvatier

6 5

ANN: Veusz 1.2.1 - a scientific plotting package
by Jeremy Sanders 30 Nov '08

30 Nov '08

Note that this release includes binaries for Linux/Windows/MacOSX. The embedding interface is now more robust and works under all platforms, including Windows. Veusz 1.2.1 ----------- Velvet Ember Under Sky Zenith ----------------------------- http://home.gna.org/veusz/ Veusz is Copyright (C) 2003-2008 Jeremy Sanders <jeremy(a)jeremysanders.net> Licenced under the GPL (version 2 or greater). Veusz is a scientific plotting package. It is written in Python, using PyQt4 for display and user-interfaces, and numpy for handling the numeric data. Veusz is designed to produce publication-ready Postscript/PDF output. The user interface aims to be simple, consistent and powerful. Veusz provides a GUI, command line, embedding and scripting interface (based on Python) to its plotting facilities. It also allows for manipulation and editing of datasets. Change in 1.2.1: * Fix crash when adding a key without any key text defined. Changes in 1.2: * Boxes, ellipses, lines, arrows and image files can now be added to the plot or page and interactively adjusted. * Page sizes, graphs, grids and axes can be interactively adjusted. * Plot keys can have multiple columns. * Error bars can have cross-ends. * Several user interface usability enhancements. * Embedding interface has been rewritten to be more robust. It now uses multiple processes and sockets. * Embedding now works fully on Windows. * Embedding interface has been expanded: - Zoom width, height and page options for zooming graph to window - Dynamically change update interval - Move between pages of documents - Open up more than one view onto a document * PDF export fixed for recent versions of Qt * Quite a lot of minor bug fixes Features of package: * X-Y plots (with errorbars) * Line and function plots * Contour plots * Images (with colour mappings and colorbars) * Stepped plots (for histograms) * Fitting functions to data * Stacked plots and arrays of plots * Plot keys * Plot labels * Shapes and arrows on plots * LaTeX-like formatting for text * EPS/PDF/PNG/SVG export * Scripting interface * Dataset creation/manipulation * Embed Veusz within other programs * Text, CSV and FITS importing Requirements: Python (2.3 or greater required) http://www.python.org/ Qt >= 4.3 (free edition) http://www.trolltech.com/products/qt/ PyQt >= 4.3 (SIP is required to be installed first) http://www.riverbankcomputing.co.uk/pyqt/ http://www.riverbankcomputing.co.uk/sip/ numpy >= 1.0 http://numpy.scipy.org/ Optional: Microsoft Core Fonts (recommended for nice output) http://corefonts.sourceforge.net/ PyFITS >= 1.1 (optional for FITS import) http://www.stsci.edu/resources/software_hardware/pyfits For documentation on using Veusz, see the "Documents" directory. The manual is in pdf, html and text format (generated from docbook). Issues: * Can be very slow to plot large datasets if antialiasing is enabled. Right click on graph and disable antialias to speed up output. If you enjoy using Veusz, I would love to hear from you. Please join the mailing lists at https://gna.org/mail/?group=veusz to discuss new features or if you'd like to contribute code. The latest code can always be found in the SVN repository. Jeremy Sanders

1 0

How to use Open Opt
by Vicent Giner-Bosch 29 Nov '08

29 Nov '08

Hello. I am looking for infomation about the Open Opt (OO) project, and I've been referred to this group. I've been reading the official documentation about OO, but it seems a little confusing to me. My question is: if I want to use OO, what must I do? In fact, if I want to develop a new optimization algorithm in Python, how can I use OO? I mean, in which part of the process can / should I use OO? What are the advantages of using OO? Is it just a "bunch" or library of available optimization algorithms, or does it also provide a general framework (for example, a general predefined Object Oriented structure, or some general functions in order to manage algorithms...) in order to build an test or run our own algorithms? What are the key features of OO? I hope I've been clear enough about my questions. Any answer will be appreciated. Thank you very much in advance. -- Vicent Giner-Bosch, Valencia (Spain)

5 7

trouble saving sparse matrix
by Robin 28 Nov '08

28 Nov '08

Hi, I have a large sparse matrix (about 9GB): In [18]: a.A Out[18]: <21699x1048575 sparse matrix of type '<type 'numpy.int8'>' with 1035272192 stored elements in Compressed Sparse Column format> but I am having trouble saving it. I am on 64 bit linux. The problem is whatever I try I get : SystemError: Negative size passed to PyString_FromStringAndSize This happens with cPickle.dump, np.save, sp.io.savemat etc. I am guessing something is overflowing a 32 bit integer. The matrix itself seems ok... I was wondering if anyone had any ideas for another way to save it - or perhaps if I have made a mistake and it is really too big? Cheers Robin

3 2

Gram-Schmidt orthogonalization
by David Warde-Farley 27 Nov '08

27 Nov '08

Hi, Is there a built-in, somewhere in NumPy or SciPy, that implements Gram- Schmidt orthogonalization? Thanks, David

5 9

getting values from traits objects
by gross 26 Nov '08

26 Nov '08

I'm obviously missing something very basic, can someone please explain to me what i've done wrong in this example: [code] from enthought.traits.api import HasTraits, Float, Int from scipy import linspace class InputData(HasTraits): xmin=Float(default_value=.5) xmax=Float(default_value=2.0) xres=Int(default_value=128) x=linspace(xmin, xmax, xres) if __name__ == "__main__": window = InputData() window.configure_traits() [/code] Executing this file results in: [quote] Traceback (most recent call last): File "<stdin>", line 1, in <module> File "t2.py", line 4, in <module> class InputData(HasTraits): File "t2.py", line 8, in InputData x=linspace(xmin, xmax, xres) File "/usr/lib/python2.5/site-packages/numpy/lib/function_base.py", line 74, in linspace num = int(num) TypeError: int() argument must be a string or a number, not 'Int' [/quote] Printing the value of xres after execution returns the (by me) expected value of 128 as an int, but it seems it doesen't work in the same way when the script runs? Believe me, i've studied examples and searched for hours, but it can be really difficult to find straight answers to such general questions as this one... -- View this message in context: http://www.nabble.com/getting-values-from-traits-objects-tp20623191p2062319… Sent from the Scipy-User mailing list archive at Nabble.com.

2 2

ndimage zero-ignorant filters, or other ways to fill holes
by Vincent Schut 26 Nov '08

26 Nov '08

Guys, I feel a bit hesistant to ask this because I know: as a fairly intensive user of scipy i should try to give something back to the community and code it up myself. However, I don't feel confident enough with ndimage's c code, and feel I'd spend much too much time and energy on it. That being said, I'm dying (well, almost) for a zero-ignorant version of ndimage.uniform_filter. In other words: a nd (2-d would be enough for me) windowing average filter that gives me back the mean of the non-zero pixels in the window. All current ndimage filters just incorporate the zero's, NaN's propagate through, and ndimage doesn't work with masks or masked arrays. I often find myself looking for a simple way to fill gaps (patches of zeros) in an image with values resembling the average surroundings of the gap, or smoothing an image that has zero-filled gaps of data that should not be taken into account while smoothing. I could solve such a thing very easily by iterating a version of uniform_filter that would not incorporate the zero-cells in its calculation. Hmm, I feel it's hard to explain. I hope someone understands what I mean... If someone would be willing to contribute such a feature to ndimage, please know that you'd at least make me very happy :) If someone comes up with another brillant idea to fill the zero-gaps in my images with values that are in reasonable range of the gap's surroundings, I'd also be very grateful. Keep in mind that the images typically are pretty large, though. 7000x7000 pixels is no exception. Regards, Vincent Schut.

3 3

ANNOUNCE: EPD with Py2.5 version 4.0.30002 RC2 available for testing
by Travis E. Oliphant 26 Nov '08

26 Nov '08

Hello, We've recently posted the beta1 build of EPD (the Enthought Python Distribution) with Python 2.5 version 4.1.30001 to the EPD website. You may download the beta from here: http://www.enthought.com/products/epdearlyaccess.php You can check out the release notes here: https://svn.enthought.com/epd/wiki/Python2.5.2/4.1.300/Beta1 Please help us test it out and provide feedback on the EPD Trac instance: https://svn.enthought.com/epd or via e-mail to epd-support(a)enthought.com. If everything goes well, we are planning a final release for December. About EPD --------- The Enthought Python Distribution (EPD) is a "kitchen-sink-included" distribution of the Python™ Programming Language, including over 60 additional tools and libraries. The EPD bundle includes NumPy, SciPy, IPython, 2D and 3D visualization, database adapters, GUI building libraries, and a lot of other tools right out of the box. http://www.enthought.com/products/epd.php It is currently available as a single-click installer for Windows XP (x86), Mac OS X (a universal binary for OS X 10.4 and above), and RedHat 3 and 4 (x86 and amd64). EPD is free for academic use. An annual subscription and installation support are available for individual commercial use. Enterprise subscriptions with support for particular deployment environments are also available for commercial purchase. Enthought Build Team

1 0