Mailman 3 July 2013 - yt-dev

Fwd: [IPython-dev] [ANN] IPython 1.0.0a1
by Matthew Turk July 22, 2013

July 22, 2013

---------- Forwarded message ---------- From: MinRK <benjaminrk(a)gmail.com> Date: Sun, Jul 21, 2013 at 9:43 PM Subject: [IPython-dev] [ANN] IPython 1.0.0a1 To: IPython Development list <ipython-dev(a)scipy.org>, ipython user list <IPython-User(a)scipy.org> First alpha of IPython 1.0.0 is now available, please give it a test drive. http://archive.ipython.org/testing/1.0.0 Probably the quickest way to install: pip install http://archive.ipython.org/testing/1.0.0/ipython-1.0.0a1.tar.gz Lots of new stuff, especially in the notebook. Also significant performance improvements in IPython.parallel and qtconsole, and a revamped input transform system that should be much better behaved than ever before. Plus loads of bugfixes all over the place, of course. For a quick summary, see what's new. The most significant addition is nbconvert, accessible now as ipython nbconvert [mynotebook.ipynb]. nbconvert is going to be decidedly in ‘tech preview’ mode for 1.0, so we hope to hear from early adopters, particularly those using it to integrate IPython with blogging engines or writing papers or documentation. We full expect that by 2.0 there will need to be enough changes that some APIs will not survive from 1.0 to 2.0, so you should be aware of that. There will still be a few more adjustments to nbconvert, and more fleshing out of documentation before 1.0 is released. Please report any installation issues you encounter, that's the most important part of this first alpha. We are happy to get this out before our big all hands dev meeting this week, and only six days behind schedule (who said that July 15 was the target for final release of 1.0?) Thanks for everyone's contributions in getting us this far! -MinRK et al. IPython Devs _______________________________________________ IPython-dev mailing list IPython-dev(a)scipy.org http://mail.scipy.org/mailman/listinfo/ipython-dev

1 0

Public testing dashboard
by Nathan Goldbaum July 17, 2013

July 17, 2013

Hi all, Kacper Kowalik has recently put a lot of work into improving yt's testing infrastructure. In the past day, he's moved our continuous integration service from a server behind a firewall at his university in Torun to a publicly accessible server: http://piernik.astri.umk.pl:8080/ The Jenkins instance Kacper has set up will automatically run the full testing suite for all incoming pull requests and direct commits to both the main yt_analysis/yt repo as well as the yt_analysis/yt-3.0 repo. The results are reported in real time both on the jenkins dashboard as well as via an IRC bot that idles in the #yt chatroom on freenode. If you're on IRC, you just need to open the link sent by the yt-fido bot when testing finishes for your PR. Just as an example, this is the test results for my PR #555: http://piernik.astri.umk.pl:8080/job/py2.7-yt-2.x_testing/11/ Since the CI we run for yt_analysis/yt include answer tests, Kacper has set up subprojects the unit tests, answer tests for the slower frontends, and plot window answer tests. This allows the subprojects to run in parallel, speeding up test reporting. Generally, you'll want to click on the link for a failing subproject and then click the link for the console output to get a traceback or test failure report. Another cool thing: here is the flake8 (code static analysis and style checking) report for yt-3.0: http://piernik.astri.umk.pl:8080/job/py2.7-yt_analysis-3.0/violations/? Jenkins does a ton more stuff and there are still some loose ends to tie up and some new features Kacper wants to add, but this should be a very useful service going forward. I want to take the opportunity to thank Kacper for all the time he's put into this. We now have a really excellent CI service which will be a big aid in the work of improving the test coverage of the code base, improving PEP8 compliance, and instantly detecting regressions. -Nathan

1 0

Proposed: Moving yt from GPLv3 to BSD-like license
by Matthew Turk July 17, 2013

July 17, 2013

Hi everyone, I apologize for the long email, but because this is a relatively important subject I need to address a few items in detail. This is meant as a discussion point. I would like to ensure investment in a change like this from the yt community, both developers and users, and I think before a decision can be made to either change or remain the same we should ensure that community members are able to voice concerns or suggestions. = What = I'm writing to propose a change in the licesning scheme for yt. Currently, yt is licensed under the GPLv3 license. This license brings with it an idealogy that I support -- free (as in freedom) and open source software, and attempts to ensure that the spread of software spreads those freedoms as well. This is done through terms of licensing; while there are several subtleties to how this plays out, and the goals of FLOSS and the GPL align very well with my own, I don't believe that it is appropriate for yt to be licensed under the GPL any more. = Why = In the scientific software community, for the most part codes and platforms are released under a permissive, BSD-like license. This is not universally true. Within the scientific python ecosystem (including projects such as AstroPy), BSD-like licenses are especially prevalent. These licenses place no restrictions on redistribution, passing on freedoms to end users, or making a piece of software closed-source. A side effect is that if a piece of software is BSD licensed, it cannot rely on GPL'd software without itself being subject to those terms. Specifically, a BSD licensed package that requires an import of a GPL'd package is likely itself then subject to the GPL -- this is why it has been termed "viral" in the past. As examples, many BSD-licensed packages exist in the scientific software community: VisIt, ParaView, MayaVi, NumPy, Matplotlib, IPython, Python itself, mpi4py, h5py, SymPy, SciPy, most of the scikits, scikits-learn, NetworkX and so on. Collaboration with these projects is currently one-way because of our license. When I initially decided on a license for yt (seven years ago) it seemed appropriate to use the licensing terms as a mechanism to encourage contributions upstream. However, within the current ecosystem, it is clear that because of the virality of the GPL and the prevailing mindsets of developers, it is actually an impediment to receiving contributions and receiving mindshare. John Hunter described it very clearly in this document: http://nipy.sourceforge.net/software/license/johns_bsd_pitch.html#johns-bsd… While he focuses on commercial utilization, I believe that within the scientific python ecosystem the picture can be broadened to include any piece of software that is under a permissive license. yt cannot be used or relied upon as a primary component without that piece of software then becoming subject to the terms of the GPL. Additionally, some private and public institutions are averse to providing code under the GPL, specifically version 3 of the GPL. By transitioning to a permissive license, we may be able to receive more contributions and collaborate more widely. As a few examples, this could include more direct collaborations with packages such as Glue, IPython, VisIt, ParaView, and even utilization and exposing of yt methods and operations in other, permissively-licensed packages. For example, deep integration between permissively-licensed simulation codes will benefit from this. Furthermore, individuals who otherwise could not contribute code under the GPL (due to employer restrictions) will be able to contribute code under a permissive license. The GPL is designed to prevent turning FLOSS code proprietary. Changing to a BSD license does not allow another entity to prevent us from continuing to develop or make available any yt code. It simply means that others can utilize it however they see fit. I believe that we stand to gain considerably more than we stand to lose from such a transition. More to the point, a few years ago on this mailing list we came up with a mission statement for yt. I think we can better serve that mission statement by enabling broader collaborations within the scientific software ecosystem. This is not motivated by any desire to create a proprietary distribution of yt -- in fact, exactly the opposite. Recently I have begun to think about the future of yt if I, personally, end up having to depart the community. I believe that in the current ecosystem of scientific software, yt will be more sustainable if it is under a permissive license. = License= I specifically believe that the IPython license, which is itself the modified BSD license, is the license we should move to. I am willing to entertain other options (such as the PSF license, which matplotlib uses, or an MIT-like license). While it is tempting to me to suggest that we license under the LGPL, I don't think that this would provide any benefits over moving directly to a fully permissive license. http://ipython.org/ipython-doc/dev/about/license_and_copyright.html While we are doing this, I would like to suggest we enumerate the copyright model that IPython does, as that is exactly the copyright model we exist under currently but it is not specifically identified in the code base. yt will continue to have a GPL-mode, where we can link against GPL'd libraries such as Rockstar, ExtJS and so on. However, this mode will be activated by a specific set of imports or items, and the core yt functionality will by default not activate any of this mode. This is similar to how packages like VisIt and ParaView link against R, which is itself GPL'd. = How = If we do decide to make this change, this is the process that it will take: 1) I have created a spreadsheet of every contributor to yt. This spreadsheet will be public. 2) I will individually write to each one, asking for their consent to relicense their contributions under the new license. 3) All responses will be directed to yt-svn (rather than yt-dev) which is a publicly accessible and archived mailing list. A link to the archive post for each message will be placed in the spreadsheet. 4) If we as a project decide that we are committed to moving to a permissive license and an individual does not consent to change the license of their code, we will need to either rewrite or remove it. Alternately, this could serve as a veto for a transition. We will discuss this if it arises. 5) Once complete, I will issue a pull request that changes the license and makes explicit the copyright policy. All previous releases of yt will remain under the GPL, as will any previous changesets of yt prior to the acceptance of item #5. = Votes / Discussion = At this time, I would like to request and solicit feedback or votes. If you have contributed a large amount of code to yt, please write back to this email with a Yea or a Nay to this change and any comments you have about it, or suggestions about a particular license other than the IPython license. In particular, if you are opposed to this change, *please* make your voice heard so that we can have a productive discussion about it. = Thank you = Thank you very much for taking the time to read this. This email and initiative is the result of a lot of soul-searching on my part. I have come to believe very strongly that as a project we can do more to support goals of open science, open source, and build a stronger community by rethinking how we fit into an ecosystem of scientific software. -Matt

15 19

Issue #609: Remove delaunay triangulation (yt_analysis/yt)
by Matthew Turk July 17, 2013

July 17, 2013

New issue 609: Remove delaunay triangulation https://bitbucket.org/yt_analysis/yt/issue/609/remove-delaunay-triangulation Matthew Turk: As noted in #608, we still ship C++ in the form of the Delaunay Triangulation code. This code is now included and maintained in Matplotlib. We should remove ours and rely on the upstream version. Additionally, in service of removing C++ code we can remove the `_ramses_headers` and the entire RAMSES frontend from 2.X. Responsible: xarthisius

1 0

Issue #608: Install script needlessly depends on C++ (yt_analysis/yt)
by Kacper Kowalik July 17, 2013

July 17, 2013

New issue 608: Install script needlessly depends on C++ https://bitbucket.org/yt_analysis/yt/issue/608/install-script-needlessly-de… Kacper Kowalik: The only consumer is hdf5, but we do not use C++ bindings anywhere. IMO it should be built with --disable-cxx.

1 0

Issue #607: ncurses dev required (yt_analysis/yt)
by VISTA synoptic tools July 16, 2013

July 16, 2013

New issue 607: ncurses dev required https://bitbucket.org/yt_analysis/yt/issue/607/ncurses-dev-required VISTA synoptic tools: Linux version 2.6.38.8-32.fc15.x86_64 (mockbuild(a)x86-13.phx2.fedoraproject.org) (gcc version 4.6.0 20110530 (Red Hat 4.6.0-9) (GCC) ) #1 SMP Mon Jun 13 19:49:05 UTC 2011 On running the yt install script: Got a few warnings but it seemed to churn through OK Issue appears on ipython session from: "/home/???????/yt-x86_64/bin/./ipython" as user: Since NO "tcl"-like features, <up-arrow> for example puts out spurious chars. Solution: Appears ncurses-dev is required. So as root I ran: >yum install ncurses-devel This seemed to fix the issue. So the reason readline can't compile is it can't find libncurses -<ngoldbaum>

1 0

Issue #606: Support cooling*.out file in RAMSES (yt_analysis/yt)
by Matthew Turk July 11, 2013

July 11, 2013

New issue 606: Support cooling*.out file in RAMSES https://bitbucket.org/yt_analysis/yt/issue/606/support-cooling-out-file-in-… Matthew Turk: We should be able to calculate species abundances in RAMSES from the cooling.out files. Responsible: MatthewTurk

1 0

round-off error in TotalQuantity
by Douglas Harvey Rudd July 10, 2013

July 10, 2013

Hi all, I'm working on getting the ARTIO frontend working with a script used by the AGORA collaboration to measure density profiles. This script uses the "TotalQuantity" quantity to sum all particle masses within a given sphere, then differences a nested set of spheres to measure the mean density in spherical shells. I noticed a 2% difference at large radius from my own profiling routines, and tracked it down to the precision of the sum, which used the same precision as the input variable, float32 for ARTIO. Since there are a large number of particles, summing 1e5 x 10^-7 floats led to O(1%) error. While this is clearly not an ideal way to measure density profiles, it's probably a good idea to help avoid such issues in future by changing TotalQuantity (and other places, like _TotalMass) to use a 64-bit accumulator, e.g. totals.append(data[field].sum(dtype=np.float64)) The alternative is to promote all float32 input variables to float64, which is wasteful of memory, or otherwise warn the user that they are likely to encounter loss of precision. Douglas Rudd Scientific Computing Consultant Research Computing Center drudd(a)uchicago.edu

4 4

NumberDensity in clumps and on FLASH data
by Matthew Turk July 10, 2013

July 10, 2013

Hi all, Mark Richardson wrote to the yt_analysis project with a problem with clumps on FLASH datasets. Here's his email: "Below includes my script and the output (it wouldn't let me attach two files). I get some output in the clump file up until I'd expect to see the Jeans Mass dumped. The output makes me think that it can't calculate this point because Gamma is unknown, or some other essential variable needed to get the number density. I've attempted to change the data_structure.py frontend for Flash to alias Gamma to gamc and game but I still get the 'cannot find Gamma' when loading my dataset." The issue from the traceback is that JeansMassMsun requires MeanMolecularWeight which needs NumberDensity. This can't be generated for his data. I am not an expert on FLASH, but it seems that this should be straightforward to fix, I just don't know how. Anybody from the FLASH or Clump side have any ideas? -Matt

5 5

Issue #605: PlanckTransferFunction does not support grey_opacity (yt_analysis/yt)
by Nathan Goldbaum July 9, 2013

July 9, 2013

New issue 605: PlanckTransferFunction does not support grey_opacity https://bitbucket.org/yt_analysis/yt/issue/605/plancktransferfunction-does-… Nathan Goldbaum: >From wlad on IRC: File "/Users/wlyra/yt-x86_64/src/yt-hg/yt/visualization/volume_rendering/camera.py", line 611, in finalize_image if self.transfer_function.grey_opacity is False: AttributeError: 'PlanckTransferFunction' object has no attribute 'grey_opacity' @jwise77, @samskillman do you have any thoughts about this? I've never used the PlanckTransferFunction but that's mostly because it's not documented very well. If we could shore it up I think it would be a very useful thing to have around.

1 0