Hey everyone!
Those of you who attended or followed one of the last two winter sprints
in Leysin might know me, the others probably won't. I'm (still) a
Master's student writing my thesis on PyPy's current issue with
cross-heap cycles when using cpyext. The main point is, that they are
not collected and stay around as floating garbage, even if they are not
reachable any more. Correct me if I'm wrong, but I didn't notice anyone
else working on that topic (and/or committing something) …
[View More]since I've
picked it up two years ago (yeah, I know, that long ago...).
I saw that you would be working on "cpyext performance and completeness"
during the current winter sprint this week and thought that this might
also concern my thesis. So I thought I'll give you an update.
I recently pushed my current (non-optimized, breaking-some-tests, but
more or less working) implementation of the "CPython-style" GC-extension
for cpyext. It is still in an early stage and not all cases (legacy and
non-legacy-finalizers, weakrefs) are handled. However, I picked up some
pace during the last couple of weeks and I'm determined to finish this
implementation during the following weeks (some quirks with the tests
have been haunting me, but I think I figured most of them out by now).
After that, I will implement a second alternative implementation (to
compare to, mostly for the sake of my thesis), which will take some more
time. I also added a couple of fancy test cases (the so called
"dot-tests"), so that we can test complex object graphs a little bit
easier (and also because it was kind of cool to have another language
inside PyPy/RPython, even if it was only for the tests...), with more
test cases to come (the current ones are a bit messy). None of the new
changes concerning low-latency applications are currently integrated,
but that should not be too hard.
I guess it won't make much sense for me to join you at this year's
winter sprint, especially as I won't be able to get there before
Thursday, but I might be able to join you over the IRC channel (or some
other form of communication if you like). If there is anything that is
worth discussing please let me know! Also feel free to comment on my
code, but beware that I might change some things once I try to do some
optimizations (probably not many, but at least fix the worst issues) and
make it a little bit more readable. You can find my code on the
cpyext-gc-cycle
branch.<https://bitbucket.org/pypy/pypy/commits/branch/cpyext-gc-cycle>
Looking forward to hearing from you!
Greetings,
Stefan
[View Less]
Now that the re fixes are in for py3.7, I think our primary branch for
development should be py3.7, with backports to py3.6 if needed. PEP 494
for python 3.6 has that version in a security-fixes-only stage, so we
should adopt a similar strategy (although some bug-fixes may be needed,
especially around utf8 problems that are still cropping up).
Toward that end, I have updated the buildbot configuration to build
nightlies off the py3.7 branch instead of the py3.6 branch. So if you
wish to …
[View More]obtain a newer build of py3.6, you will need to manually trigger
the buildbots.
This does not change the status of the default branch.
Please let me know if this was too fast and you feel we should continue
to use py3.6 as our primary branch for development.
Matti
[View Less]
I have uploaded release candidates for the second release candidate PyPy
7.3.3rc2 to https://buildbot.pypy.org/pypy/
The hashes can be found at
https://foss.heptapod.net/pypy/pypy.org/-/blob/branch/default/pages/downloa…
The draft release note is at
https://doc.pypy.org/en/latest/release-v7.3.3.html.
I have decided to declare the python3.7 support "beta" level since we
did get much feedback with issues with the previous "alpha" release. The
conda migration is progressing and is the …
[View More]major motivating factor for
this release. There are some slight packaging/c-api bugs that were
exposed when trying to build the conda packages. We now can run
scikit-learn and better support pybind11.
Please try it out. Suggestions to improve the release note are also welcome.
If there are no issues I intend to release this on Friday Nov 20, unless
there is demand that we ship the _hpy_universal module (which still need
some tweaks) in the release.
Matti
[View Less]
Here are PyPy things I want to do in the near future:
- once the dust settles from the 7.3.3 release, merge the win64 branch
to default
- continue work to merge the win64-py3.6 branch to py3.6
- continue work on the rpython3 branch to see if we can get "cd
pypy/doc; make html" to work with python3, for the future day when
readthedocs drops python2 support (the idea is to make parts of rpython
support both python2 and python3). Another solution for this may be to
simplify the docs …
[View More]building so it does not import half of rpython just to
document the options.
At what point do we stop releasing python3.6? It is under long-term
CPython support (no more binaries) and will be dropped from the next
round of releases by the scientific python stack. I would like to move
to py3.7 as the main branch of python development, but we should have
clear criteria for deciding 3.7 is no longer beta. Here are a few things
worth doing:
- get it into conda by submitting a new conda recipe a PR along the
lines of this comment
https://github.com/conda-forge/conda-forge.github.io/issues/867#issuecommen….
Once that is merged, conda will begin updating the ~1000 packages that
already work for pypy3.6 ot pypy3.7. You can checkout the migration
status at https://conda-forge.org/status/#pypy
- finish the py3.7-rsre branch
Is that enough?
Once we drop 3.6, we should start a py3.8 branch. At what point would we
consider it mature enough that we would want to make noise about asking
for outside contributors to pitch in and make it work? Who else besides
me would be up for reviewing and merging PRs?
Matti
N.B.
Thanks to Ronan for a review of this mail before it went out here.
Here are some interesting PyPy tidbits I saw lately:
- run pypy in the cloud on binder (click on the "launch binder" badge)
https://gist.github.com/bollwyvl/b5443c87c58b2e04ac5faba4d62ff232
- someone made a youtube video where they show 10 minutes about PyPy on
a macOSx: they read parts of the website and show 'hello world'
https://www.reddit.com/r/PyPy/comments/jreiud/made_this_tutorial_on_pypy/
[View Less]
I have uploaded release candidates for PyPy 7.3.3rc1 to
https://buildbot.pypy.org/pypy/
The hashes can be found at
https://foss.heptapod.net/pypy/pypy.org/-/blob/branch/default/pages/downloa…
The draft release note is at
https://doc.pypy.org/en/latest/release-v7.3.3.html.
I have decided to declare the python3.7 support "beta" level since we
did get much feedback with issues with the previous "alpha" release. The
conda migration is progressing and is the major motivating factor for
…
[View More]this release. There are some slight packaging/c-api bugs that were
exposed when trying to build the conda packages. We now can run
scikit-learn and better support pybind11.
Please try it out. Suggestions to improve the release note are also welcome.
Matti
[View Less]