Mailman 3 April 2006 - SciPy-User

some statistical models / formulas
by Jonathan Taylor April 20, 2006

April 20, 2006

i have made a numpy/scipy package for some linear statistical models http://www-stat.stanford.edu/~jtaylo/scipy_stats_models-0.01a.tar.gz i was hoping that it might someday get into scipy.stats, maybe as scipy.stats.models? anyways, i am sure the code needs work and more docs with examples, but right now there is basic functionality for the following (the tests give some examples): - model formulae as in R (to some extent) - OLS (ordinary least square regression) - WLS (weighted least square regression) - AR1 regression (non-diagonal covariance -- right now just AR1 but easy to extend to ARp) - generalized linear models (all of R's links and variance functions but extensible as well -- not everything has been rigorously tested but logistic agrees with R, for instance) - robust linear models using M estimators (with a number of standard default robust norms as in R's rlm) - robust scale estimates (MAD, Huber's proposal 2). it would be nice to add a few things over time, too, like: - mixed effects models - generalized additive models (gam), generalized estimating equations (gee).... - nonlinear regression (i have some quasi working code for this, too, but it is not yet included). + anything else people want to add. -- jonathan -- ------------------------------------------------------------------------ I'm part of the Team in Training: please support our efforts for the Leukemia and Lymphoma Society! http://www.active.com/donate/tntsvmb/tntsvmbJTaylor GO TEAM !!! ------------------------------------------------------------------------ Jonathan Taylor Tel: 650.723.9230 Dept. of Statistics Fax: 650.725.8977 Sequoia Hall, 137 www-stat.stanford.edu/~jtaylo 390 Serra Mall Stanford, CA 94305

1 0

?Inclusion of stats.py (GPL)
by Norman Shelley April 19, 2006

April 19, 2006

Should scipy use stats.py? The current version at least is GPL and "infects" other software on installation or importation. I got rid of it after seeing this header. # Copyright (c) 1999-2002 Gary Strangman; All Rights Reserved. # # This software is distributable under the terms of the GNU # General Public License (GPL) v2, the text of which can be found at # http://www.gnu.org/copyleft/gpl.html. Installing, importing or otherwise # using this module constitutes acceptance of the terms of this License. # http://www.nmr.mgh.harvard.edu/Neural_Systems_Group/gary/python/stats.py

2 1

Errors compiling matplotlib on intel Mac (with fix)
by Robert Hetland April 19, 2006

April 19, 2006

I have had errors complaining about uint and ushort being defined in both types.h and numpy/arrayobject.h. I fixed this problem by commenting out the lines in arrayobject where these things were redefined -- matplotlib compiles fine then. This hack is unnecessary on the PPC Mac (as well as other platforms, I imagine). Will this hack give me trouble with other packages? Do any of you have suggestions for a more permanent fix? -Rob ----- Rob Hetland, Assistant Professor Dept of Oceanography, Texas A&M University p: 979-458-0096, f: 979-845-6331 e: hetland(a)tamu.edu, w: http://pong.tamu.edu

2 1

Re: [SciPy-user] accessing a class's code
by LATORNELL, Doug April 19, 2006

April 19, 2006

I think the inspect module (http://docs.python.org/lib/module-inspect.html) in the standard library might enable you to do what you want. I've never used inspect (just read its docs when they caught my eye a couple of weeks ago), nor done what you describe though, so I can't promise that I'm not sending you on a wild goose chase :-) Doug > -----Original Message----- > From: scipy-user-bounces(a)scipy.net > [mailto:scipy-user-bounces@scipy.net] On Behalf Of Ryan Krauss > Sent: April 19, 2006 08:40 > To: SciPy Users List > Subject: [SciPy-user] accessing a class's code > > I don't have the best of luck with comp.lang.python and I get > great results from this list. So, I am mildly sorry about > the general python question and the double post. > > I have a set of Python classes that represent elements in a > structural model for vibration modeling (sort of like FEA). > Some of the parameters of the model are initially unknown and > I do some system identification to determine the parameters. > After I determine these unknown parameters, I would like to > substitute them back into the model and save the model as a > new python class. To do this, I think each element needs to > be able to read in the code for its __init__ method, make the > substitutions and then write the new __init__ method to a > file defining a new class with the now known parameters. > > Is there a way for a Python instance to access its own code > (especially the __init__ method)? And if there is, is there a clean > way to write the modified code back to a file? I assume that if I > can get the code as a list of strings, I can output it to a > file easily enough. > > I am tempted to just read in the code and write a little > Python script to parse it to get me the __init__ methods, but > that seems like reinventing the wheel. > > I don't just want to read and write a vector of coefficients > because after I have identified the unknown structural > parameters, I will use the model for control design and there > will be a new set of unknown control parameters. > > Thanks, > > Ryan > > _______________________________________________ > SciPy-user mailing list > SciPy-user(a)scipy.net > http://www.scipy.net/mailman/listinfo/scipy-user > This email and any files transmitted with it may contain privileged or confidential information and may be read or used only by the intended recipient. If you are not the intended recipient of the email or any of its attachments, please be advised that you have received this email in error and any use, dissemination, distribution, forwarding, printing or copying of this email or any attached files is strictly prohibited. If you have received this email in error, please immediately purge it and all attachments and notify the sender by reply email or contact the sender at the number listed.

2 1

accessing a class's code
by Ryan Krauss April 19, 2006

April 19, 2006

I don't have the best of luck with comp.lang.python and I get great results from this list. So, I am mildly sorry about the general python question and the double post. I have a set of Python classes that represent elements in a structural model for vibration modeling (sort of like FEA). Some of the parameters of the model are initially unknown and I do some system identification to determine the parameters. After I determine these unknown parameters, I would like to substitute them back into the model and save the model as a new python class. To do this, I think each element needs to be able to read in the code for its __init__ method, make the substitutions and then write the new __init__ method to a file defining a new class with the now known parameters. Is there a way for a Python instance to access its own code (especially the __init__ method)? And if there is, is there a clean way to write the modified code back to a file? I assume that if I can get the code as a list of strings, I can output it to a file easily enough. I am tempted to just read in the code and write a little Python script to parse it to get me the __init__ methods, but that seems like reinventing the wheel. I don't just want to read and write a vector of coefficients because after I have identified the unknown structural parameters, I will use the model for control design and there will be a new set of unknown control parameters. Thanks, Ryan

1 0

Trac Wikis closed for anonymous edits until further notice
by Robert Kern April 18, 2006

April 18, 2006

We've been hit badly by spammers, so I can only presume our Trac sites are now on the traded spam lists. I am going to turn off anonymous edits for now. Ticket creation will probably still be left open for now. Many thanks to David Cooke for quickly removing the spam. I am looking into ways to allow people to register themselves with the Trac sites so they can edit the Wikis and submit tickets without needing to be added by a project admin. -- Robert Kern robert.kern(a)gmail.com "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

2 1

No module named _umfpack
by Ron Kneusel April 18, 2006

April 18, 2006

Hi- I'm building numpy and scipy according to the directions on the web site (Steve Baum's doc). Everything works fine and compiles properly. When I load scipy: >>>import scipy I'm told that there is no module named '_umfpack'. Looking through the archives I see a recent post where someone said that setup.py for linsolve was screwy and was building __umfpack instead of _umfpack. What the post didn't mention is how to fix this error. What needs to be modified? Thanks! Ron

5 12

Building SciPy with IBM XL Fortran compiler on Mac OS X
by Neil Martinsen-Burrell April 16, 2006

April 16, 2006

I am having a problem build SciPy (svn r1854) with the IBM XL 8.1 fortran compiler on Mac OS X 10.4.6. Starting from a clean checkout, it builds a number of libraries successfully, but fails one the final one. Here's the log (elided, full results at http://braeburn.amath.unc.edu/~nburrell/scipy.build.log) [...] building 'scipy.fftpack._fftpack' extension compiling C sources gcc options: '-fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -DNDEBUG -g -O3 -Wall -Wstrict-prototypes' creating build/temp.darwin-8.5.0-Power_Macintosh-2.4/build creating build/temp.darwin-8.5.0-Power_Macintosh-2.4/build/src creating build/temp.darwin-8.5.0-Power_Macintosh-2.4/build/src/Lib creating build/temp.darwin-8.5.0-Power_Macintosh-2.4/build/src/Lib/fftpack creating build/temp.darwin-8.5.0-Power_Macintosh-2.4/Lib/fftpack/src compile options: '-Ibuild/src -I/Network/Servers/core.amath.unc.edu/Volumes/data/home/nburrell/usr/lib/python2.4/site-packages/numpy/core/include -I/Network/Servers/core.amath.unc.edu/Volumes/data/home/nburrell/usr/include/python2.4 -c' gcc: Lib/fftpack/src/drfft.c gcc: Lib/fftpack/src/zfftnd.c gcc: build/src/fortranobject.c In file included from /Network/Servers/core.amath.unc.edu/Volumes/data/home/nburrell/usr/include/python2.4/Python.h:55, from build/src/fortranobject.h:7, from build/src/fortranobject.c:2: /Network/Servers/core.amath.unc.edu/Volumes/data/home/nburrell/usr/include/python2.4/pyport.h:396: warning: 'struct winsize' declared inside parameter list /Network/Servers/core.amath.unc.edu/Volumes/data/home/nburrell/usr/include/python2.4/pyport.h:397: warning: 'struct winsize' declared inside parameter list gcc: build/src/Lib/fftpack/_fftpackmodule.c In file included from /Network/Servers/core.amath.unc.edu/Volumes/data/home/nburrell/usr/include/python2.4/Python.h:55, from build/src/Lib/fftpack/_fftpackmodule.c:16: /Network/Servers/core.amath.unc.edu/Volumes/data/home/nburrell/usr/include/python2.4/pyport.h:396: warning: 'struct winsize' declared inside parameter list /Network/Servers/core.amath.unc.edu/Volumes/data/home/nburrell/usr/include/python2.4/pyport.h:397: warning: 'struct winsize' declared inside parameter list gcc: Lib/fftpack/src/zfft.c gcc: Lib/fftpack/src/zrfft.c Traceback (most recent call last): File "setup.py", line 50, in ? setup_package() File "setup.py", line 42, in setup_package configuration=configuration ) File "/Network/Servers/core.amath.unc.edu/Volumes/data/home/nburrell/usr/lib/python2.4/site-packages/numpy/distutils/core.py", line 153, in setup return old_setup(**new_attr) File "/Network/Servers/core.amath.unc.edu/Volumes/data/home/nburrell/usr/lib/python2.4/distutils/core.py", line 149, in setup dist.run_commands() File "/Network/Servers/core.amath.unc.edu/Volumes/data/home/nburrell/usr/lib/python2.4/distutils/dist.py", line 946, in run_commands self.run_command(cmd) File "/Network/Servers/core.amath.unc.edu/Volumes/data/home/nburrell/usr/lib/python2.4/distutils/dist.py", line 966, in run_command cmd_obj.run() File "/Network/Servers/core.amath.unc.edu/Volumes/data/home/nburrell/usr/lib/python2.4/site-packages/numpy/distutils/command/install.py", line 11, in run r = old_install.run(self) File "/Network/Servers/core.amath.unc.edu/Volumes/data/home/nburrell/usr/lib/python2.4/distutils/command/install.py", line 506, in run self.run_command('build') File "/Network/Servers/core.amath.unc.edu/Volumes/data/home/nburrell/usr/lib/python2.4/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/Network/Servers/core.amath.unc.edu/Volumes/data/home/nburrell/usr/lib/python2.4/distutils/dist.py", line 966, in run_command cmd_obj.run() File "/Network/Servers/core.amath.unc.edu/Volumes/data/home/nburrell/usr/lib/python2.4/distutils/command/build.py", line 112, in run self.run_command(cmd_name) File "/Network/Servers/core.amath.unc.edu/Volumes/data/home/nburrell/usr/lib/python2.4/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/Network/Servers/core.amath.unc.edu/Volumes/data/home/nburrell/usr/lib/python2.4/distutils/dist.py", line 966, in run_command cmd_obj.run() File "/Network/Servers/core.amath.unc.edu/Volumes/data/home/nburrell/usr/lib/python2.4/site-packages/numpy/distutils/command/build_ext.py", line 109, in run self.build_extensions() File "/Network/Servers/core.amath.unc.edu/Volumes/data/home/nburrell/usr/lib/python2.4/distutils/command/build_ext.py", line 405, in build_extensions self.build_extension(ext) File "/Network/Servers/core.amath.unc.edu/Volumes/data/home/nburrell/usr/lib/python2.4/site-packages/numpy/distutils/command/build_ext.py", line 301, in build_extension link = self.fcompiler.link_shared_object AttributeError: 'NoneType' object has no attribute 'link_shared_object' It appears that self.fcompiler is None in build_ext.py because IBMFCompiler.get_version() returns None even when there is a valid IBM fortran compiler installed. Digging into numpy/distutils/fcompiler/ibm.py, in get_version(), the version discovery code contains: [...] if not l: from distutils.version import LooseVersion self.version = version = LooseVersion(l[0]) return version I believe that this should be "if l". Making that change shows the IBM compiler as available in python setup.py config_fc --help-fcompiler and scipy builds correctly. Can someone make this change to numpy/distutils/fcompiler/ibm.py? Thanks. Peace, -Neil -- Neil Martinsen-Burrell nmb(a)unc.edu

2 1

64-bit ndimage
by Matthew Brett April 13, 2006

April 13, 2006

Hi Travis, > >Sorry to keep bombarding the list with 64-bit troubles. > > > >In brief - scipy.ndimage.test() segfaults on my x86-64 P4 system, but > >passes on other x86-32 systems. > > > > > ndimage is not 64-bit ready at this point. I would disable it in the > setup.py script until it is (or remove it from the scipy directory). Just to ask - is 64-bit compatibility for ndimage in progress? I may have some time to help, but have little experience of scipy internals; do you have a feel for how much work this would be? Is there a good place to start looking for pointers? Thanks a lot, Matthew

1 0

[bug?] scipy.ndimage.rotate
by Alok Singhal April 13, 2006

April 13, 2006

Hi, I am using scipy 0.4.8 and numpy 0.9.6 on a Linux machine. I am trying to figure out scipy.ndimage.rotate() function. From info(rotate): ---- rotate(input, angle, axes=(-1, -2), reshape=True, output_type=None, output=None, order=3, mode='constant', cval=0.0, prefilter=True) Rotate an array. The array is rotated in the plane defined by the two axes given by the axes parameter using spline interpolation of the requested order. The angle is given in degrees. Points outside the boundaries of the input are filled according to the given mode. If reshape is true, the output shape is adapted so that the input array is contained completely in the output. The parameter prefilter determines if the input is pre- filtered before interpolation, if False it is assumed that the input is already filtered. ---- I am trying to rotate an array by an arbitrary angle, but the rotation does not seem to work well unless the angle is a multiple of 90 degrees[1]: >>> a = eye(4, dtype=Float64) >>> a array([[ 1., 0., 0., 0.], [ 0., 1., 0., 0.], [ 0., 0., 1., 0.], [ 0., 0., 0., 1.]]) >>> # Rotation by 90 degrees >>> b = rotate(a, 90) >>> b # looks good array([[ -2.37169225e-20, 1.23341550e-16, -2.22044605e-16, 1.00000000e+00], [ 1.23338162e-16, -2.12801781e-16, 1.00000000e+00, -2.21990395e-16], [ -1.70206189e-16, 1.00000000e+00, -3.51959130e-17, 7.89366944e-17], [ 1.00000000e+00, -3.99677578e-16, 1.08562519e-16, 3.69983991e-18]]) >>> # Rotation by 45 degrees >>> b = rotate(a, 45) >>> b array([[ 0. , 0. , 0. , 0.02360487, 0. , 0. ], [ 0. , 0. , 0.01925501, 0.01925501, 0.27416806, 0. ], [ 0. , 0.22915545, 0.44416261, 0.44416261, 0.22915545, 0.44507896], [ 0. , 0.22915545, 0.44416261, 0.44416261, 0.22915545, 0.44507896], [ 0. , 0. , 0.01925501, 0.01925501, 0.27416806, 0. ], [ 0. , 0. , 0. , 0.02360487, 0. , 0. ]]) >>> b = rotate(a, 45, reshape=False) >>> b array([[ 0. , 0.01925501, 0.01925501, 0.27416806], [ 0.22915545, 0.44416261, 0.44416261, 0.22915545], [ 0.22915545, 0.44416261, 0.44416261, 0.22915545], [ 0. , 0.01925501, 0.01925501, 0.27416806]]) I cannot understand the output in this case. Rotating the Identity matrix by 45 degrees (or any arbitrary angle) should result in a matrix that still has 1's "along a line" and zeros everywhere else (or something close to that). But as can be seen above, the resulting matrix does not have that property. Am I missing something, or is it a bug in ndimage.rotate()? Thanks, Alok [1] Even the case for multiples of 90 degrees "does not work": >>> a = eye(5, dtype=Float64) >>> b = rotate(a, 270) >>> b array([[ 1.23480463e-17, 3.97766672e-18, 9.09628682e-17, 0.00000000e+00, 0.00000000e+00], [ 2.21922632e-18, 1.87363688e-17, -6.40458552e-17, 1.00000000e+00, 0.00000000e+00], [ 2.81757040e-17, 5.07155895e-16, 1.00000000e+00, -5.28853491e-16, 0.00000000e+00], [ 6.07356504e-17, 1.00000000e+00, -5.87237778e-16, 1.79130528e-16, 0.00000000e+00], [ 1.00000000e+00, -1.00977847e-15, 2.66090318e-16, -6.32022104e-17, 0.00000000e+00]]) >>> # b[0][4] is not 1 >>> b = rotate(a, 90) >>> b array([[ 1.23531285e-17, -2.15688470e-17, 1.23355102e-16, -2.96048179e-16, 1.00000000e+00], [ -2.46749168e-17, 6.01901612e-17, -2.06574395e-16, 1.00000000e+00, 6.07898606e-17], [ -7.70630575e-18, -1.12540186e-16, 1.00000000e+00, -2.06567619e-16, 2.81824802e-17], [ -4.17228101e-16, 1.00000000e+00, -8.78678098e-17, 1.87363688e-17, -9.66972813e-18], [ 1.00000000e+00, -6.22545499e-16, 1.86127020e-16, -1.98154888e-17, 1.13553237e-17]]) >>> # Looks OK -- Alok Singhal * * Graduate Student, dept. of Astronomy * * * University of Virginia http://www.astro.virginia.edu/~as8ca/ * *

1 0