release seems ready
It seems we have a release, version ad5a4e55fa8e. Is there a reason to wait? buildbots http://buildbot.pypy.org/summary?branch=release-5.x release notice http://doc.pypy.org/en/latest/release-5.0.0.html Hopefully we can release 5.1 once s360-x lands on default Matti
yay! can we call it rc1? if noone objects we'll make rc1 the release say in 24 or 48h On Tue, Mar 8, 2016 at 1:49 PM, matti picus <matti.picus@gmail.com> wrote:
It seems we have a release, version ad5a4e55fa8e. Is there a reason to wait? buildbots http://buildbot.pypy.org/summary?branch=release-5.x release notice http://doc.pypy.org/en/latest/release-5.0.0.html
Hopefully we can release 5.1 once s360-x lands on default Matti
_______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
We could package it and upload as rc1, but version_info will not have rc1 unless we rerun the builds. Confusing. I prefer to apologize if we get it wrong and release a 5.0.1 bugfix Matti On Tuesday, 8 March 2016, Maciej Fijalkowski <fijall@gmail.com> wrote:
yay!
can we call it rc1? if noone objects we'll make rc1 the release say in 24 or 48h
Hi Matti, On 8 March 2016 at 15:15, matti picus <matti.picus@gmail.com> wrote:
We could package it and upload as rc1, but version_info will not have rc1 unless we rerun the builds. Confusing. I prefer to apologize if we get it wrong and release a 5.0.1 bugfix
+1. Go ahead as far as I'm concerned. About the release notice: "As a result, lxml with its cython compiled component passes all tests on PyPy" is not clear until the next official lxml is released. The current lxml 3.5.0 still contains a partially buggy workaround that tries to make it work on previous versions of cpyext. The trunk version at https://github.com/lxml/lxml has got this code removed, and that's the version that works. I'll make the ppc releases once the other releases are out. A bientôt, Armin
Did the lxml project indicate they will provide a new release soon that incorporates these fixes? I tried to build the latest development code from source, but run into many issues (lxml build server down, source package missing the pre-generated C code etc. etc.), and customer company policy wouldn't allow using a development version in production anyway. The lxml 3.5.0 does not install with pypy-5.0.0 (it used to with pypy-4.0.1, though it was too buggy to be useful), and the lxml-cffi no longer installs. On 08-03-16 15:26, Armin Rigo wrote:
Hi Matti,
On 8 March 2016 at 15:15, matti picus <matti.picus@gmail.com> wrote:
We could package it and upload as rc1, but version_info will not have rc1 unless we rerun the builds. Confusing. I prefer to apologize if we get it wrong and release a 5.0.1 bugfix +1. Go ahead as far as I'm concerned.
About the release notice: "As a result, lxml with its cython compiled component passes all tests on PyPy" is not clear until the next official lxml is released. The current lxml 3.5.0 still contains a partially buggy workaround that tries to make it work on previous versions of cpyext. The trunk version at https://github.com/lxml/lxml has got this code removed, and that's the version that works.
I'll make the ppc releases once the other releases are out.
A bientôt,
Armin _______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
There is a new lxml release as of today, unfortunately there is an issue: https://bitbucket.org/pypy/pypy/issues/2260/pypy-500-dumps-core-with-lxml-36... On 16-03-16 18:07, Armin Rigo wrote:
Hi Pim,
On 16 March 2016 at 17:32, Pim van der Eijk (Lists) <lists@sonnenglanz.net> wrote:
Did the lxml project indicate they will provide a new release soon that incorporates these fixes? You'll have to ask on the lxml mailing list.
Armin
Hi, On 17 March 2016 at 16:13, Pim van der Eijk (Lists) <lists@sonnenglanz.net> wrote:
There is a new lxml release as of today, unfortunately there is an issue: https://bitbucket.org/pypy/pypy/issues/2260/pypy-500-dumps-core-with-lxml-36...
Yes, it's what we get when both sides (lxml and pypy) are half-hearted about supporting the other. The lxml tests seem to pass, but that may be because they are small. Many bigger and longer-running processes seem to crash like that. I'm investigating. A bientôt, Armin.
Hi again, On 17 March 2016 at 17:27, Armin Rigo <arigo@tunes.org> wrote:
On 17 March 2016 at 16:13, Pim van der Eijk (Lists) <lists@sonnenglanz.net> wrote:
There is a new lxml release as of today, unfortunately there is an issue: https://bitbucket.org/pypy/pypy/issues/2260/pypy-500-dumps-core-with-lxml-36...
Yes, it's what we get when both sides (lxml and pypy) are half-hearted about supporting the other. The lxml tests seem to pass, but that may be because they are small. Many bigger and longer-running processes seem to crash like that. I'm investigating.
Fixed in 0173cdbbbacc, which then seems to work with lxml even on these larger examples. I'd love some more testing before we do the 5.0.1 bugfix release. Please try with a version of PyPy on the "release-5.x" branch recent enough to contain a09a60a9c381; an Ubuntu precompiled version is here: http://buildbot.pypy.org/nightly/release-5.x/pypy-c-jit-83125-a09a60a9c381-l... A bientôt, Armin.
Hi, I did some tests and there are no crashes. However, compared to CPython 2.7.10 there are some serious issues: - For my test programs (the script in the issue on BitBucket is derived from one of them), PyPy is much slower. script A: 256 seconds in PyPy versus 78 seconds in CPython script B: 9.73 seconds in PyPy versus 2.6 in Cpython - Memory use continues to grow up to over 80% at which time where my laptop starts swapping, whereas with CPython usage is never more than 4%. - Perhaps caused by the above, there are occasional freezes of several seconds in which nothing seems to happen, although CPU usage is still 100%. Kind Regards, Pim On 18-03-16 12:13, Armin Rigo wrote:
Hi again,
On 17 March 2016 at 16:13, Pim van der Eijk (Lists) <lists@sonnenglanz.net> wrote:
There is a new lxml release as of today, unfortunately there is an issue: https://bitbucket.org/pypy/pypy/issues/2260/pypy-500-dumps-core-with-lxml-36... Yes, it's what we get when both sides (lxml and pypy) are half-hearted about supporting the other. The lxml tests seem to pass, but that may be because they are small. Many bigger and longer-running processes seem to crash like that. I'm investigating. Fixed in 0173cdbbbacc, which then seems to work with lxml even on
On 17 March 2016 at 17:27, Armin Rigo <arigo@tunes.org> wrote: these larger examples. I'd love some more testing before we do the 5.0.1 bugfix release. Please try with a version of PyPy on the "release-5.x" branch recent enough to contain a09a60a9c381; an Ubuntu precompiled version is here:
http://buildbot.pypy.org/nightly/release-5.x/pypy-c-jit-83125-a09a60a9c381-l...
A bientôt,
Armin.
Hi Pim, On 18 March 2016 at 13:25, Pim van der Eijk (Lists) <lists@sonnenglanz.net> wrote:
- For my test programs (the script in the issue on BitBucket is derived from one of them), PyPy is much slower.
If you're comparing the speed of scripts that have a large amount of crossings of the cpyext layer (i.e. crossings between Python code and CPython C extension code), then yes, it's expected to be much slower. The speed improved a lot recently, which means it is now *much slower* instead of *very, very much slower*. It makes no sense, now or in the future, to use PyPy in the hope to speed up a script that does _only_ lxml stuff with almost no Python code running in-between.
- Memory use continues to grow up to over 80% at which time where my laptop starts swapping, whereas with CPython usage is never more than 4%.
This is more annoying. Can you give us a way to reproduce this? Armin
On 18-03-16 14:57, Armin Rigo wrote:
- Memory use continues to grow up to over 80% at which time where my laptop starts swapping, whereas with CPython usage is never more than 4%. This is more annoying. Can you give us a way to reproduce this?
It already happens with the script I attached to the original issue, which you already have: https://bitbucket.org/pypy/pypy/issues/2260/pypy-500-dumps-core-with-lxml-36...
Hi Pim, On 18 March 2016 at 15:08, Pim van der Eijk (Lists) <lists@sonnenglanz.net> wrote:
- Memory use continues to grow up to over 80% at which time where my laptop starts swapping, whereas with CPython usage is never more than 4%.
This is more annoying. Can you give us a way to reproduce this?
It already happens with the script I attached to the original issue, which you already have: https://bitbucket.org/pypy/pypy/issues/2260/pypy-500-dumps-core-with-lxml-36...
Ok, partially reproduced. With CPython it grows continously too, but only up to 1.2GB and then it finishes. With PyPy it grows faster up to 22GB. If I add some "gc.collect()" executed every few seconds, then PyPy only grows up to 1.7GB. I added "add_memory_pressure=True" to some chosen mallocs inside cpyext, and it seems to be enough to fix the problem. Now PyPy grows up to 1.7GB even without any gc.collect(). Yay! (changeset 9137853fd0ec, grafted to release-5.x too) A bientôt, Armin.
Hi Armin, On 18-03-16 18:52, Armin Rigo wrote:
Hi Pim,
- Memory use continues to grow up to over 80% at which time where my laptop starts swapping, whereas with CPython usage is never more than 4%. This is more annoying. Can you give us a way to reproduce this? It already happens with the script I attached to the original issue, which you already have: https://bitbucket.org/pypy/pypy/issues/2260/pypy-500-dumps-core-with-lxml-36... Ok, partially reproduced. With CPython it grows continously too, but only up to 1.2GB and then it finishes. With PyPy it grows faster up to 22GB. If I add some "gc.collect()" executed every few seconds,
On 18 March 2016 at 15:08, Pim van der Eijk (Lists) <lists@sonnenglanz.net> wrote: then PyPy only grows up to 1.7GB.
I added "add_memory_pressure=True" to some chosen mallocs inside cpyext, and it seems to be enough to fix the problem. Now PyPy grows up to 1.7GB even without any gc.collect(). Yay! (changeset 9137853fd0ec, grafted to release-5.x too)
I retested and confirm that the library works and memory use is now like CPython, which is great. It is still slower than CPython, for reasons you explained before, but that is because my test script heavily uses of lxml. In larger applications where lxml processing is a smaller part of the overall functionality, the PyPy speed-up of regular Python code could well compensate for this. Many thanks, Pim
A bientôt,
Armin.
I'm ok with making it official 5.0. We can always do 5.0.1 if there are problems On Tue, Mar 8, 2016 at 4:15 PM, matti picus <matti.picus@gmail.com> wrote:
We could package it and upload as rc1, but version_info will not have rc1 unless we rerun the builds. Confusing. I prefer to apologize if we get it wrong and release a 5.0.1 bugfix Matti
On Tuesday, 8 March 2016, Maciej Fijalkowski <fijall@gmail.com> wrote:
yay!
can we call it rc1? if noone objects we'll make rc1 the release say in 24 or 48h
btw, should we mention packages.pypy.org? On Tue, Mar 8, 2016 at 4:36 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
I'm ok with making it official 5.0. We can always do 5.0.1 if there are problems
On Tue, Mar 8, 2016 at 4:15 PM, matti picus <matti.picus@gmail.com> wrote:
We could package it and upload as rc1, but version_info will not have rc1 unless we rerun the builds. Confusing. I prefer to apologize if we get it wrong and release a 5.0.1 bugfix Matti
On Tuesday, 8 March 2016, Maciej Fijalkowski <fijall@gmail.com> wrote:
yay!
can we call it rc1? if noone objects we'll make rc1 the release say in 24 or 48h
Hi, On 8 March 2016 at 15:42, Maciej Fijalkowski <fijall@gmail.com> wrote:
btw, should we mention packages.pypy.org?
I would do so but only under two conditions: * it reports a post-cpyext-fixes result: which packages run or don't run now, ideally on the current "release 5.0" branch, but at least after the merge of the cpyext-gc-support-2 branch * we quickly review and fix the few manual comments, notably lxml's (we no longer recommend lxml-cffi). A bientôt, Armin
Cool, I'm happy to do the suggested fixes. We rerun it every release usually, changes by hand are done earlier. Should I start a run on the current release branch? On Tue, Mar 8, 2016 at 4:46 PM, Armin Rigo <arigo@tunes.org> wrote:
Hi,
On 8 March 2016 at 15:42, Maciej Fijalkowski <fijall@gmail.com> wrote:
btw, should we mention packages.pypy.org?
I would do so but only under two conditions:
* it reports a post-cpyext-fixes result: which packages run or don't run now, ideally on the current "release 5.0" branch, but at least after the merge of the cpyext-gc-support-2 branch
* we quickly review and fix the few manual comments, notably lxml's (we no longer recommend lxml-cffi).
A bientôt,
Armin
in other words, it shows the last release of pypy, not "trunk" On Tue, Mar 8, 2016 at 4:52 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Cool, I'm happy to do the suggested fixes.
We rerun it every release usually, changes by hand are done earlier. Should I start a run on the current release branch?
On Tue, Mar 8, 2016 at 4:46 PM, Armin Rigo <arigo@tunes.org> wrote:
Hi,
On 8 March 2016 at 15:42, Maciej Fijalkowski <fijall@gmail.com> wrote:
btw, should we mention packages.pypy.org?
I would do so but only under two conditions:
* it reports a post-cpyext-fixes result: which packages run or don't run now, ideally on the current "release 5.0" branch, but at least after the merge of the cpyext-gc-support-2 branch
* we quickly review and fix the few manual comments, notably lxml's (we no longer recommend lxml-cffi).
A bientôt,
Armin
ugh, btw, it seems someone broke embedding (as advertised, probably the cffi embedding still works) On Tue, Mar 8, 2016 at 4:53 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
in other words, it shows the last release of pypy, not "trunk"
On Tue, Mar 8, 2016 at 4:52 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Cool, I'm happy to do the suggested fixes.
We rerun it every release usually, changes by hand are done earlier. Should I start a run on the current release branch?
On Tue, Mar 8, 2016 at 4:46 PM, Armin Rigo <arigo@tunes.org> wrote:
Hi,
On 8 March 2016 at 15:42, Maciej Fijalkowski <fijall@gmail.com> wrote:
btw, should we mention packages.pypy.org?
I would do so but only under two conditions:
* it reports a post-cpyext-fixes result: which packages run or don't run now, ideally on the current "release 5.0" branch, but at least after the merge of the cpyext-gc-support-2 branch
* we quickly review and fix the few manual comments, notably lxml's (we no longer recommend lxml-cffi).
A bientôt,
Armin
I am going to test it out , quite interesting release. On Tue, Mar 8, 2016 at 8:12 PM Maciej Fijalkowski <fijall@gmail.com> wrote:
yay!
can we call it rc1? if noone objects we'll make rc1 the release say in 24 or 48h
On Tue, Mar 8, 2016 at 1:49 PM, matti picus <matti.picus@gmail.com> wrote:
It seems we have a release, version ad5a4e55fa8e. Is there a reason to wait? buildbots http://buildbot.pypy.org/summary?branch=release-5.x release notice http://doc.pypy.org/en/latest/release-5.0.0.html
Hopefully we can release 5.1 once s360-x lands on default Matti
_______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
_______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
participants (5)
-
Armin Rigo -
Maciej Fijalkowski -
matti picus -
Phyo Arkar -
Pim van der Eijk (Lists)