Mailman 3 January 2011 - SciPy-Dev

future of maxentropy module (was: sparse rmatvec and maxentropy)
by Ralf Gommers Jan. 25, 2011

Jan. 25, 2011

(excuse the cross-post, but this may be of interest to scipy-user and the scikits.learn crowd) On Sat, Jan 22, 2011 at 10:44 PM, <josef.pktd(a)gmail.com> wrote: > On Sat, Jan 22, 2011 at 8:50 AM, Ralf Gommers > <ralf.gommers(a)googlemail.com> wrote: > > On Thu, Jan 20, 2011 at 10:13 PM, Skipper Seabold <jsseabold(a)gmail.com> > > wrote: > >> > >> I picked up the montecarlo code when I was playing around with these. > >> > >> > http://bazaar.launchpad.net/~jsseabold/statsmodels/statsmodels-skipper-maxe…<http://bazaar.launchpad.net/%7Ejsseabold/statsmodels/statsmodels-skipper-ma…> > >> > >> I'm curious if the maxentropy stuff as it is in scipy wouldn't find > >> more use and maintenance in scikits.learn. The implementation is > >> somewhat use specific (natural language processing), though this is > >> not by any means set in stone. > >> > > Probably, but wouldn't it need a lot of work before it could be moved? It > > has a grand total of one test, mostly non-working examples, and is > obviously > > hardly used at all (see r6919 and r6920 for more examples of broken > code). > > > > Perhaps it's worth asking the scikits.learn guys, and otherwise consider > > deprecating it if they're not interested? > > I haven't seen or heard anyone using it besides Skipper. There are > also still some features that where designed for pysparse and never > fully updated to scipy.sparse. > http://projects.scipy.org/scipy/ticket/856 > > I also thought deprecating and removing maxentropy will be the best > idea, if nobody volunteers to give it a workout. > So I guess we just have to ask this out loud: is anyone using the scipy.maxentropy module or interested in doing so? If you are, would you be interested in putting some work into it, like making the examples work and adding some tests? The current status is that 3 out of 4 examples are broken, the module has only a single test, and from broken code that went unnoticed for a long time it is clear that there are very few users. If no one steps up, I propose to deprecate the module for the 0.10 release. If there are any users out there that missed this email and step up then, we can always un-deprecate again. To the scikits.learn developers: would this code fit better and see more use in scikits.learn than in scipy? Would you be interested to pick it up? Ralf

4 4

[feature request] Smoothing data library
by David Kremer Jan. 25, 2011

Jan. 25, 2011

Hello, I'm a scipy user that use it for academic research. I would like to ask you if you could design a scipy.smooth module, because currently the smoothing abilitise of scipy are rather poor. Moreover, academic paper on smoothing 1D and 2D data set does exist, so it would be cool if you would like to implement some of them. Thank you very much

2 2

Re: [SciPy-Dev] [feature request] Smoothing data library
by Jonathan Stickel Jan. 24, 2011

Jan. 24, 2011

On 1/24/11 11:00 , scipy-dev-request(a)scipy.org wrote: > Date: Mon, 24 Jan 2011 16:15:43 +0100 > From: David Kremer<david.kremer.dk(a)gmail.com> > Subject: [SciPy-Dev] [feature request] Smoothing data library > > Hello, I'm a scipy user that use it for academic research. > > I would like to ask you if you could design a scipy.smooth module, > because currently the smoothing abilitise of scipy are rather poor. > > Moreover, academic paper on smoothing 1D and 2D data set does exist, > so it would be cool if you would like to implement some of them. > > Thank you very much There are a few things scattered about. Smoothing splines are available in scipy.interpolate: http://docs.scipy.org/doc/scipy/reference/interpolate.html I submitted a scikit for smoothing by regularization of 1D arrays: http://pypi.python.org/pypi/scikits.datasmooth/0.5 I am open to contributions for improved packaging, documentation, and examples. I, too, think it would be nice if things were collected in a scipy module. Regards, Jonathan

2 1

You know she wants it big
by Adam Machnik Jan. 23, 2011

Jan. 23, 2011

Huge discounts on huge growth http://argilaceous.pillclaytabletsdrugstore.net/

1 0

Edit rights for doc.scipy.org
by Loïc Berthe Jan. 23, 2011

Jan. 23, 2011

Hi ! Could you give me edit rights for doc.scipy.org ? My username is loic. I would be happy to review some documentation and correct some mistakes. -- Loic

2 1

sparse rmatvec and maxentropy
by josef.pktd＠gmail.com Jan. 22, 2011

Jan. 22, 2011

I don't know much about these two parts of scipy. Running an example for scipy.maxentropy raises an exception after upgrading to scipy trunk I didn't manage to find out with a Trac search whether rmatvec still exists. scipy.maxentropy maxentutils is still using it. Traceback (most recent call last): File "C:\Josef\eclipsegworkspace\statsmodels-josef-experimental-gsoc\scikits\statsmodels\sandbox\examples\example_maxent.py", line 29, in <module> model.fit(K[i]) File "\Programs\Python25\Lib\site-packages\scipy\maxentropy\maxentropy.py", line 226, in fit File "\Programs\Python25\Lib\site-packages\scipy\optimize\optimize.py", line 636, in fmin_cg File "\Programs\Python25\Lib\site-packages\scipy\optimize\optimize.py", line 176, in function_wrapper File "\Programs\Python25\Lib\site-packages\scipy\maxentropy\maxentropy.py", line 420, in grad File "\Programs\Python25\Lib\site-packages\scipy\maxentropy\maxentropy.py", line 702, in expectations File "\Programs\Python25\Lib\site-packages\scipy\maxentropy\maxentropy.py", line 737, in pmf File "\Programs\Python25\Lib\site-packages\scipy\maxentropy\maxentropy.py", line 719, in logpmf File "\Programs\Python25\Lib\site-packages\scipy\maxentropy\maxentutils.py", line 381, in innerprodtranspose File "C:\Josef\_progs\Subversion\scipy_trunk_g09\dist\scipy-0.10.0.dev7027.win32\Programs\Python25\Lib\site-packages\scipy\sparse\base.py", line 384, in __getattr__ raise AttributeError(attr + " not found") AttributeError: rmatvec not found Josef

3 4

postponing 0.9.0rc1 by one week
by Ralf Gommers Jan. 20, 2011

Jan. 20, 2011

Hi all, There is still one release blocker for 0.9.0, the change in behavior of convolve/correlate. See the old_behavior keyword in correlate/convolve/correlate2d/convolve2d, and also http://projects.scipy.org/scipy/ticket/1301. Therefore the release of RC1 is postponed by one week. If anyone has time to look at this, that would be great. Three other things that could be fixed as well are the following: 1. Medfilt test failure on linux 64-bit, see http://news.gmane.org/gmane.comp.python.scientific.devel. This will be marked as knownfail for this release if no one has time to investigate. 2. CorrelateComplex192 failures on Windows, see the other email I just sent. 3. Test warnings from interpolate and linalg, see below. Will just be filtered out for 0.9.0 unless they are fixed in time. /Users/rgommers/Code/scipy/scipy/interpolate/fitpack2.py:673: UserWarning: The coefficients of the spline returned have been computed as the minimal norm least-squares solution of a (numerically) rank deficient system (deficiency=7). If deficiency is large, the results may be inaccurate. Deficiency may strongly depend on the value of eps. warnings.warn(message) ....../Users/rgommers/Code/scipy/scipy/interpolate/fitpack2.py:604: UserWarning: The required storage space exceeds the available storage space: nxest or nyest too small, or s too small. The weighted least-squares spline corresponds to the current set of knots. warnings.warn(message) /Users/rgommers/Code/scipy/scipy/linalg/decomp_qr.py:91: ComplexWarning: Casting complex values to real discards the imaginary part qr, tau, work, info = geqrf(a1, lwork=lwork, overwrite_a=overwrite_a) /Users/rgommers/Code/scipy/scipy/linalg/decomp_qr.py:126: ComplexWarning: Casting complex values to real discards the imaginary part Q, work, info = gor_un_gqr(qqr, tau, lwork=lwork, overwrite_a=1) ......................../Users/rgommers/Code/scipy/scipy/linalg/decomp_schur.py:72: ComplexWarning: Casting complex values to real discards the imaginary part result = gees(lambda x: None, a, lwork=result[-2][0], overwrite_a=overwrite_a) Cheers, Ralf

1 1

correlate complex192 failures on win32
by Ralf Gommers Jan. 20, 2011

Jan. 20, 2011

Hi Warren, In r6982 you added correlate/convolve tests that are now failing on Windows for longdouble. Can you please have a look at that? If you don't have time soon I think it can be marked knownfail for 0.9.0 on Windows. Offending code: + +# Create three classes, one for each complex data type: TestCorrelateComplex64, +# TestCorrelateComplex128 and TestCorrelateComplex256. +# The second number in the pairs is used in the 'decimal' keyword argument of +# the array comparisons in the tests. +for i, decimal in [(np.csingle, 5), (np.cdouble, 10), (np.clongdouble, 15)]: Resulting in 16 failures like: ====================================================================== FAIL: test_rank1_full (test_signaltools.TestCorrelateComplex192) ---------------------------------------------------------------------- Traceback (most recent call last): File "Z:\Users\rgommers\.wine\drive_c\Python26\lib\site-packages\scipy\signal\tests\test_signaltools.py", line 630, in test_rank1_full assert_array_almost_equal(y, y_r, decimal=self.decimal) File "Z:\Users\rgommers\.wine\drive_c\Python26\lib\site-packages\numpy\testing\utils.py", line 774, in assert_array_almost_equal header='Arrays are not almost equal') File "Z:\Users\rgommers\.wine\drive_c\Python26\lib\site-packages\numpy\testing\utils.py", line 618, in assert_array_compare raise AssertionError(msg) AssertionError: Arrays are not almost equal (mismatch 11.7647058824%) x: array([0.17161931+0.20853512j, 0.22923514+0.78246389j, -1.3237712+0.47060531j, -3.2990924+-0.50721318j, -2.9089895+0.084727311j, 1.5646499+-0.56960514j,... y: array([0.17161931+0.20853512j, 0.22923514+0.78246389j, -1.3237712+0.47060531j, -3.2990924+-0.50721318j, -2.9089895+0.084727311j, 1.5646499+-0.56960514j,... Ralf

2 2

[Ann] EuroScipy 2011 - Call for papers
by Tiziano Zito Jan. 20, 2011

Jan. 20, 2011

========================= Announcing EuroScipy 2011 ========================= --------------------------------------------- The 4th European meeting on Python in Science --------------------------------------------- **Paris, Ecole Normale Supérieure, August 25-28 2011** We are happy to announce the 4th EuroScipy meeting, in Paris, August 2011. The EuroSciPy meeting is a cross-disciplinary gathering focused on the use and development of the Python language in scientific research. This event strives to bring together both users and developers of scientific tools, as well as academic research and state of the art industry. Main topics =========== - Presentations of scientific tools and libraries using the Python language, including but not limited to: - vector and array manipulation - parallel computing - scientific visualization - scientific data flow and persistence - algorithms implemented or exposed in Python - web applications and portals for science and engineering. - Reports on the use of Python in scientific achievements or ongoing projects. - General-purpose Python tools that can be of special interest to the scientific community. Tutorials ========= There will be two tutorial tracks at the conference, an introductory one, to bring up to speed with the Python language as a scientific tool, and an advanced track, during which experts of the field will lecture on specific advanced topics such as advanced use of numpy, scientific visualization, software engineering... Keynote Speaker: Fernando Perez =============================== We are excited to welcome Fernando Perez (UC Berkeley, Helen Wills Neuroscience Institute, USA) as our keynote speaker. Fernando Perez is the original author of the enhanced interactive python shell IPython and a very active contributor to the Python for Science ecosystem. Important dates =============== Talk submission deadline: Sunday May 8 Program announced: Sunday May 29 Tutorials tracks: Thursday August 25 - Friday August 26 Conference track: Saturday August 27 - Sunday August 28 Call for papers =============== We are soliciting talks that discuss topics related to scientific computing using Python. These include applications, teaching, future development directions, and research. We welcome contributions from the industry as well as the academic world. Indeed, industrial research and development as well academic research face the challenge of mastering IT tools for exploration, modeling and analysis. We look forward to hearing your recent breakthroughs using Python! Submission guidelines ===================== - We solicit talk proposals in the form of a one-page long abstract. - Submissions whose main purpose is to promote a commercial product or service will be refused. - All accepted proposals must be presented at the EuroSciPy conference by at least one author. The one-page long abstracts are for conference planing and selection purposes only. We will later select papers for publication of post-proceedings in a peer-reviewed journal. How to submit an abstract ========================= To submit a talk to the EuroScipy conference follow the instructions here: http://www.euroscipy.org/card/euroscipy2011_call_for_papers Organizers ========== Chairs: - Gaël Varoquaux (INSERM, Unicog team, and INRIA, Parietal team) - Nicolas Chauvat (Logilab) Local organization committee: - Emmanuelle Gouillart (Saint-Gobain Recherche) - Jean-Philippe Chauvat (Logilab) Tutorial chair: - Valentin Haenel (MKP, Technische Universität Berlin) Program committee: - Chair: Tiziano Zito (MKP, Technische Universität Berlin) - Romain Brette (ENS Paris, DEC) - Emmanuelle Gouillart (Saint-Gobain Recherche) - Eric Lebigot (Laboratoire Kastler Brossel, Université Pierre et Marie Curie) - Konrad Hinsen (Soleil Synchrotron, CNRS) - Hans Petter Langtangen (Simula laboratories) - Jarrod Millman (UC Berkeley, Helen Wills NeuroScience institute) - Mike Müller (Python Academy) - Didrik Pinte (Enthought Inc) - Marc Poinot (ONERA) - Christophe Pradal (CIRAD/INRIA, Virtual Plantes team) - Andreas Schreiber (DLR) - Stéfan van der Walt (University of Stellenbosch) Website ======= http://www.euroscipy.org/conference/euroscipy_2011

1 0

Re: [SciPy-Dev] postponing 0.9.0rc1 by one week
by Fabian Pedregosa Jan. 18, 2011

Jan. 18, 2011

On Mon, Jan 17, 2011 at 9:52 AM, Fabian Pedregosa <fabian.pedregosa(a)inria.fr> wrote: > On Sun, Jan 16, 2011 at 3:59 PM, Ralf Gommers > <ralf.gommers(a)googlemail.com> wrote: >> Hi all, >> >> There is still one release blocker for 0.9.0, the change in behavior of >> convolve/correlate. See the old_behavior keyword in >> correlate/convolve/correlate2d/convolve2d, and also >> http://projects.scipy.org/scipy/ticket/1301. Therefore the release of RC1 is >> postponed by one week. If anyone has time to look at this, that would be >> great. >> >> Three other things that could be fixed as well are the following: >> 1. Medfilt test failure on linux 64-bit, see >> http://news.gmane.org/gmane.comp.python.scientific.devel. This will be >> marked as knownfail for this release if no one has time to investigate. >> 2. CorrelateComplex192 failures on Windows, see the other email I just sent. >> 3. Test warnings from interpolate and linalg, see below. Will just be >> filtered out for 0.9.0 unless they are fixed in time. >> >> /Users/rgommers/Code/scipy/scipy/interpolate/fitpack2.py:673: UserWarning: >> The coefficients of the spline returned have been computed as the >> minimal norm least-squares solution of a (numerically) rank deficient >> system (deficiency=7). If deficiency is large, the results may be >> inaccurate. Deficiency may strongly depend on the value of eps. >> warnings.warn(message) >> ....../Users/rgommers/Code/scipy/scipy/interpolate/fitpack2.py:604: >> UserWarning: >> The required storage space exceeds the available storage space: nxest >> or nyest too small, or s too small. >> The weighted least-squares spline corresponds to the current set of >> knots. >> warnings.warn(message) >> >> /Users/rgommers/Code/scipy/scipy/linalg/decomp_qr.py:91: ComplexWarning: >> Casting complex values to real discards the imaginary part >> qr, tau, work, info = geqrf(a1, lwork=lwork, overwrite_a=overwrite_a) >> /Users/rgommers/Code/scipy/scipy/linalg/decomp_qr.py:126: ComplexWarning: >> Casting complex values to real discards the imaginary part >> Q, work, info = gor_un_gqr(qqr, tau, lwork=lwork, overwrite_a=1) > > These warnings come from the fact that lwork returned from a workspace > query has the same dtype as the principal matrix, but when we call the > computational routine it expects an integer value. A simple workaround > is here: I've updated my patch to also fix the warning in schur(): https://github.com/scipy/scipy-svn/pull/2 Fabian.

2 1