yt-dev October 2018

yt-dev@python.org

6 participants
9 discussions

[ANN] yt_astro_analysis 1.0 released
by Britton Smith 16 Oct '18

16 Oct '18

Hi everyone, In conjunction with the release of yt 3.5, I'm pleased to announce the inaugural release of the yt_astro_analysis package. yt_astro_analysis is an extension of yt for astro-specific analysis modules. This functionality previously existed in yt as the analysis_modules submodule, but has been spun off into its own package to allow development to occur on its own timeline independent of yt. yt_astro_analysis includes: - halo finding - halo analysis - light cones - data export to RADMC-3D radiative transfer code Switching from yt's analysis_modules to yt_astro_analysis should be straightforward. yt_astro_analysis can be installed with pip: pip install yt_astro_analysis Then, analysis scripts need only change their import lines from "yt.analysis_modules" to "yt.extensions.astro_analysis". The yt_astro_analysis package lives at: https://github.com/yt-project/yt_astro_analysis Documentation can be found at: https://yt-astro-analysis.readthedocs.io Just like yt, contributions to yt_astro_analysis are highly encouraged! Regards, The yt development team yt_astro_analysis has received contributions from the following people: Pengfei Chen Bili Dong Hilary Egan Nathan Goldbaum Cameron Hummels Kacper Kowalik Matthew Krafczyk Michael Kuhlen Sam Leitner Joshua Moloney Christopher Moody Andrew Myers Mark Richardson Douglas Rudd Anthony Scopatz Sam Skillman Stephen Skory Britton Smith Geoffrey So Casey W. Stark Stephanie Tonnesen Matthew Turk Miguel de Val-Borro Mike Warren John Wise John ZuHone

1 0

[ANN] yt 3.5.0 released
by Nathan Goldbaum 16 Oct '18

16 Oct '18

The yt community is proud to announce the release of yt 3.5.0. This is a major release and includes a number of new features, minor improvements, and bugfixes. yt (http://yt-project.org) is an open source, community developed toolkit for the analysis and visualization of volumetric data. If you are interested in contributing to yt, please see our repository on GitHub at http://github.com/yt-project/yt. Since yt 3.4.0 we have merged 328 pull requests from 41 contributors, including 23 new contributors. We would like to thank the following people who made code contributions to this release, new contributors are marked with an asterisk. Ashley Kelly * Fabian Koller Enrico Garaldi * Hugo Pfister * Josh Borrow * Kiran Eiden * Matthew Turk Prateek Gupta * Ricarda Beckmann Kacper Kowalik Andrew Myers Brandon Qiao * Britton Smith Christoph Behrens * Cameron Hummels Claire Kopenhafer * Clayton Strawn * Corentin Cadiou David Pérez-Suárez * Donald E Willcox * Hilary Egan Abhishek Singh * Hsi-Yu Scive Suoqing Ji Josh Soref * John ZuHone Max Gronke * Max Katz Miguel de Val-Borro Marianne Corvellec * Madicken Munk * Clément Robert * Nick Gnedin * Nathan Goldbaum Philip Grete * Bili Dong Sean Larkin * Stephanie Ho * Yi-Hao Chen Ying-Chao Lu * Mike Zingale We would also like to thank everyone who has contributed in other ways, including by reporting issues, participating on the mailing list, and evangelizing yt to your colleagues. Additionally, we would like to highlight the contributions of Ashley Kelly and Abhishek Singh, who contributed to this release in the context of Google Summery of Code projects. Both students passed their evaluations and made substantial improvements to yt. You can find out more about Ashley and Abhishek's projects on the Summer of Code website: https://summerofcode.withgoogle.com/projects/#5101882905722880 https://summerofcode.withgoogle.com/projects/#5325639024902144 Finally, we would also like to highlight the contributions of Corentin Cadiou, who has made substantial performance and functionality improvements to the RAMSES frontend as part of this release. Corentin's contributions came in on a volunteer basis and we thank him for his tireless efforts to improve yt's support for RAMSES data. Binaries for yt 3.5.0 are available via pip and conda. If you installed via the install script or use conda to manage your python installation, you can update yt via: $ conda update -c conda-forge yt Conda-forge package updates may not be up quite yet but should be uploaded by the end of the day. And if you manage your python installation with pip: $ pip install -U yt As always, if you have any questions, concerns, or run into any trouble updating please don’t hesitate to send a message to the mailing list or stop by our Slack or IRC channel. yt is the product of a large community of developers and users and we are extraordinarily grateful for and proud of their contributions. Please forward this announcement on to any interested parties. Best, The yt development team What's new? ----------- Please see the release notes for full details (http://yt-project.org/docs/3.5.0/reference/changelog.html#version-3-5-0), but we highlight a few major changes and improvements here. * yt.analysis_modules has been deprecated in favor of the new yt_astro_analysis package. New features and new astronomy-specific analysis modules will go into yt_astro_analysis and importing from yt.analysis_modules will raise a noisy warning. We will remove yt.analysis_modules in a future release * Vector fields and derived fields depending on vector fields have been systematically updated to account for a bulk correction field parameter. For example, for the velocity field, all derived fields that depend on velocity will now account for the "bulk_velocity" field parameter. In addition, we have defined "relative_velocity" and "relative_magnetic_field" fields that include the bulk correction. Both of these are vector fields, to access the components, use e.g. "relative_velocity_x". The "particle_position_relative" and "particle_velocity_relative" fields have been deprecated. * The RAMSES frontend has seen considerable improvements, with support for the new RAMSES self-describing particle output format, considerable speed improvements for I/O and initially loading data, and support for restricting the domain to a subset of the full simulation volume for imrpoved speed and reduced memory usage for large datasets. * Aliases to spatial fields with the "gas" field type will now be returned in the default unit system for the dataset. As an example the "x" field might resolve to the field tuples ("index", "x") or ("gas", "x"). Accessing the former will return data in code units while the latter will return data in whatever unit system the dataset is configured to use (CGS, by default). This means that to ensure the units of a spatial field will always be consistent, one must access the field as a tuple, explicitly specifying the field type. Accessing a spatial field using a string field name may return data in either code units or the dataset’s default unit system depending on the history of field accesses prior to accessing that field. In the future accessing fields using an ambiguous field name will raise an error. * The max_level and min_level attributes of yt data objects now correctly update the state of the underlying data objects when set. In addition we have added an example to the cookbook that shows how to downsample AMR data using this functionality. * It is now possible to customize the formatting of labels for ion species fields. Rather than using the default spectroscopic notation, one can call ds.set_field_label_format("ionization_label", "plus_minus") to use the more traditional notation where ionization state is indicated with + and - symbols.

1 0

Version number on master?
by Nathan Goldbaum 15 Oct '18

15 Oct '18

Hi all, I'm cutting the 3.5 release and realized we didn't have a discussion about what the version number on master should be after we release 3.5. Our next planned release is 4.0, we're not currently planning on doing a 3.6 release. That said, i think the version number on master should be 3.6.dev0. This doesn't indicate that we're going to release 3.6, it just makes it easier to track since the version number on the yt-4.0 branch is already 4.0.dev0. I don't think it makes sense to have two branches where that's the version number. Does anyone have any objections to setting it up like that? Hopefully we'll be able to merge the yt-4.0 branch into master before too long and not have to deal with the distinction. There are still a few tech debt things I want to clear up before merging, but we're in a much better position than we were before Ash's GSOC project at the beginning of the summer. -Nathan

4 6

Fwd: [Cython] Cython 0.29 final released
by Nathan Goldbaum 14 Oct '18

14 Oct '18

Lots of new features here! We should also investigate the impact of the coming default language level change. ---------- Forwarded message --------- From: Stefan Behnel <stefan_ml(a)behnel.de> Date: Sun, Oct 14, 2018 at 11:33 AM Subject: [Cython] Cython 0.29 final released To: Cython-users <cython-users(a)googlegroups.com>, Cython-devel < cython-devel(a)python.org> Dear Cythonistas, after half a year of development, many community pull requests and a lot of feedback and good ideas in online discussions and at conferences, I'm proud to release Cython 0.29. This is a major feature release that comes with many great improvements and several important bug fixes. See the long list of changes below. Download: https://pypi.org/project/Cython/0.29/ https://github.com/cython/cython/releases/tag/0.29 Changelog: https://github.com/cython/cython/blob/0.29/CHANGES.rst Foresight: Given that Cython has been in critical production use all over the world for several years, but never found the perfect time for a 1.0 version bump, we designate this to become the last 0.x release of the project and decided to skip the 1.0 release which the 0.x series has long represented anyway. Planning has already started [1] for the next major release, titled "3.0". It will finally switch the default Cython language level from Py2 to Py3, to match what users expect from a Python compiler these days without additional options or configuration. Cython 2 code will continue to be supported as before with the directive "language_level=2", although the future default (called "3str", Py3 with str literals) is already available in the 0.29 release, so users might prefer switching to that. We're happy to hear your feedback on the proposed changes. [1] https://github.com/cython/cython/milestone/58 Have fun, Stefan 0.29 (2018-10-14) ================= Features added -------------- * PEP-489 multi-phase module initialisation has been enabled again. Module reloads in other subinterpreters raise an exception to prevent corruption of the static module state. * A set of ``mypy`` compatible PEP-484 declarations were added for Cython's C data types to integrate with static analysers in typed Python code. They are available in the ``Cython/Shadow.pyi`` module and describe the types in the special ``cython`` module that can be used for typing in Python code. Original patch by Julian Gethmann. (Github issue #1965) * Memoryviews are supported in PEP-484/526 style type declarations. (Github issue #2529) * ``(a)cython.nogil`` is supported as a C-function decorator in Python code. (Github issue #2557) * Raising exceptions from nogil code will automatically acquire the GIL, instead of requiring an explicit ``with gil`` block. * C++ functions can now be declared as potentially raising both C++ and Python exceptions, so that Cython can handle both correctly. (Github issue #2615) * ``cython.inline()`` supports a direct ``language_level`` keyword argument that was previously only available via a directive. * A new language level name ``3str`` was added that mostly corresponds to language level 3, but keeps unprefixed string literals as type 'str' in both Py2 and Py3, and the builtin 'str' type unchanged. This will become the default in the next Cython release and is meant to help user code a) transition more easily to this new default and b) migrate to Python 3 source code semantics without making support for Python 2.x difficult. * In CPython 3.6 and later, looking up globals in the module dict is almost as fast as looking up C globals. (Github issue #2313) * For a Python subclass of an extension type, repeated method calls to non-overridden cpdef methods can avoid the attribute lookup in Py3.6+, which makes them 4x faster. (Github issue #2313) * (In-)equality comparisons of objects to integer literals are faster. (Github issue #2188) * Some internal and 1-argument method calls are faster. * Modules that cimport many external extension types from other Cython modules execute less import requests during module initialisation. * Constant tuples and slices are deduplicated and only created once per module. (Github issue #2292) * The coverage plugin considers more C file extensions such as ``.cc`` and ``.cxx``. (Github issue #2266) * The ``cythonize`` command accepts compile time variable values (as set by ``DEF``) through the new ``-E`` option. Patch by Jerome Kieffer. (Github issue #2315) * ``pyximport`` can import from namespace packages. Patch by Prakhar Goel. (Github issue #2294) * Some missing numpy and CPython C-API declarations were added. Patch by John Kirkham. (Github issues #2523, #2520, #2537) * Declarations for the ``pylifecycle`` C-API functions were added in a new .pxd file ``cpython.pylifecycle``. * The Pythran support was updated to work with the latest Pythran 0.8.7. Original patch by Adrien Guinet. (Github issue #2600) * ``%a`` is included in the string formatting types that are optimised into f-strings. In this case, it is also automatically mapped to ``%r`` in Python 2.x. * New C macro ``CYTHON_HEX_VERSION`` to access Cython's version in the same style as ``PY_HEX_VERSION``. * Constants in ``libc.math`` are now declared as ``const`` to simplify their handling. * An additional ``check_size`` clause was added to the ``ctypedef class`` name specification to allow suppressing warnings when importing modules with backwards-compatible ``PyTypeObject`` size changes. Patch by Matti Picus. (Github issue #2627) Bugs fixed ---------- * The exception handling in generators and coroutines under CPython 3.7 was adapted to the newly introduced exception stack. Users of Cython 0.28 who want to support Python 3.7 are encouraged to upgrade to 0.29 to avoid potentially incorrect error reporting and tracebacks. (Github issue #1958) * Crash when importing a module under Stackless Python that was built for CPython. Patch by Anselm Kruis. (Github issue #2534) * 2-value slicing of typed sequences failed if the start or stop index was None. Patch by Christian Gibson. (Github issue #2508) * Multiplied string literals lost their factor when they are part of another constant expression (e.g. 'x' * 10 + 'y' => 'xy'). * String formatting with the '%' operator didn't call the special ``__rmod__()`` method if the right side is a string subclass that implements it. (Python issue 28598) * The directive ``language_level=3`` did not apply to the first token in the source file. (Github issue #2230) * Overriding cpdef methods did not work in Python subclasses with slots. Note that this can have a performance impact on calls from Cython code. (Github issue #1771) * Fix declarations of builtin or C types using strings in pure python mode. (Github issue #2046) * Generator expressions and lambdas failed to compile in ``@cfunc`` functions. (Github issue #459) * Global names with ``const`` types were not excluded from star-import assignments which could lead to invalid C code. (Github issue #2621) * Several internal function signatures were fixed that lead to warnings in gcc-8. (Github issue #2363) * The numpy helper functions ``set_array_base()`` and ``get_array_base()`` were adapted to the current numpy C-API recommendations. Patch by Matti Picus. (Github issue #2528) * Some NumPy related code was updated to avoid deprecated API usage. Original patch by jbrockmendel. (Github issue #2559) * Several C++ STL declarations were extended and corrected. Patch by Valentin Valls. (Github issue #2207) * C lines of the module init function were unconditionally not reported in exception stack traces. Patch by Jeroen Demeyer. (Github issue #2492) * When PEP-489 support is enabled, reloading the module overwrote any static module state. It now raises an exception instead, given that reloading is not actually supported. * Object-returning, C++ exception throwing functions were not checking that the return value was non-null. Original patch by Matt Wozniski (Github Issue #2603) * The source file encoding detection could get confused if the ``c_string_encoding`` directive appeared within the first two lines. (Github issue #2632) * Cython generated modules no longer emit a warning during import when the size of the NumPy array type is larger than what was found at compile. Instead, this is assumed to be a backwards compatible change on NumPy side. Other changes ------------- * Cython now emits a warning when no ``language_level`` (2, 3 or '3str') is set explicitly, neither as a ``cythonize()`` option nor as a compiler directive. This is meant to prepare the transition of the default language level from currently Py2 to Py3, since that is what most new users will expect these days. The future default will, however, not enforce unicode literals, because this has proven a major obstacle in the support for both Python 2.x and 3.x. The next major release is intended to make this change, so that it will parse all code that does not request a specific language level as Python 3 code, but with ``str`` literals. The language level 2 will continue to be supported for an indefinite time. * The documentation was restructured, cleaned up and examples are now tested. The NumPy tutorial was also rewritten to simplify the running example. Contributed by Gabriel de Marmiesse. (Github issue #2245) * Cython compiles less of its own modules at build time to reduce the installed package size to about half of its previous size. This makes the compiler slightly slower, by about 5-7%. _______________________________________________ cython-devel mailing list cython-devel(a)python.org https://mail.python.org/mailman/listinfo/cython-devel

1 0

yt_astro_analysis citable
by Britton Smith 12 Oct '18

12 Oct '18

Hi everyone, The zenodo entry for yt_astro_analysis is up at: https://zenodo.org/record/1458961#.W7_VIXFKhTY For those on the author list, please have a look and send me any corrections to your affiliation or ORCID. For those not on the author list, it's still not too late to get yourself on there. Just send me your name and ORCID (optional). It will, in fact, never be too late. Note, there are still a few more things to do before the official release announcement, so that's why you haven't seen that yet. Britton

4 9

yt 3.5 release
by Nathan Goldbaum 10 Oct '18

10 Oct '18

Hi all, I think I'm ready to cut a 3.5 release (https://youtu.be/yciX2meIkXI?t=3). Does anyone have any objections to me cutting a release? Britton I know you wanted to coordinate the release for yt_astro_analysis, is there anything you need from any of us to help with that process? Are there still blockers for yt_astro_analysis? I will draft the release notes this afternoon, once that PR gets merged we can cut a release. It will probably take a day or two to build wheels and conda-forge packages, and then I will send out the release announcement. -Nathan

2 1

Save the date: yt dev workshop
by Matthew Turk 08 Oct '18

08 Oct '18

Hi folks, Nathan and I are pleased to send out a "Save the Date" for a yt development workshop. We're aiming for some combination of March 4, 5, 6 (with travel on Sunday) at the University of Illinois, and we should have some participant support costs. More info will be forthcoming soon, but I wanted to get this out there ASAP. -Matt

1 0

authors for yt_astro_analysis citation
by Britton Smith 04 Oct '18

04 Oct '18

Hi everyone, Following up on this conversation <https://mail.python.org/mm3/archives/list/yt-dev@python.org/message/GXOI3M3…> from a few weeks ago, I will be creating an entry on zenodo.org for citing the yt_astro_analysis package. If you've made commits to the package, you should have received an email inviting you to be on the author list. If not, please let me know. In addition, anyone else here who feels they've contributed to this package in any way (mailing list discussion, reporting issues, etc.) is welcome and encouraged to join the author list. If you'd like to appear on the author list, simply reply to this email (directly to me, please) and say so. No justification required. If possible, please include your ORCID so I don't have to look them all up myself. If you don't know what that is or don't have one, considering visiting orcid.org. There is no deadline for replying as it is possible to add new authors at anytime. Thanks! Britton

1 0

Unstructured mesh support
by Chris Richardson 03 Oct '18

03 Oct '18

Dear yt devs, I am a developer on the FEniCS Project (fenicsproject.org), which is a Finite Element package. I was talking to Nathan Goldbaum about visualisation of our meshes and solutions. I've got a certain amount to work with yt, but I would like to discuss taking it further, if any of you are interested. Best wishes, Chris -- Chris Richardson, EPSRC Research Software Engineer BP Institute, Madingley Road, Cambridge CB3 0EZ UK.

2 1

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

yt-dev October 2018