Hi, I ran into a crash on lxml 4.3.1 (released not too long ago it seems), pulled in via pip, on travis. We had not pinned my dependency version, so we went from 4.3.0 to 4.3.1, and all our builds started failing. This can be reproduced deterministically (on travis and also on my local linux machine), by doing: git clone git@github.com:WebAudio/web-audio-api.git git clone https://github.com/tabatkins/bikeshed.git pip install --editable $PWD/bikeshed bikeshed update cd web-audio-api bikeshed --print=plain -f spec Let me know if I can help debugging this, I've posted some background info on https://github.com/WebAudio/web-audio-api/pull/1830#issuecomment-465229675 Thanks, Paul.
Hi, thanks for the report. Paul Adenot schrieb am 19.02.19 um 18:31:
I ran into a crash on lxml 4.3.1 (released not too long ago it seems), pulled in via pip, on travis. We had not pinned my dependency version, so we went from 4.3.0 to 4.3.1, and all our builds started failing.
This can be reproduced deterministically (on travis and also on my local linux machine), by doing:
git clone git@github.com:WebAudio/web-audio-api.git git clone https://github.com/tabatkins/bikeshed.git pip install --editable $PWD/bikeshed bikeshed update cd web-audio-api bikeshed --print=plain -f spec
Let me know if I can help debugging this, I've posted some background info on https://github.com/WebAudio/web-audio-api/pull/1830#issuecomment-465229675
I tried this and I can reproduce it. Don't know yet why it crashes, so it would help if you could strip this down to a shorter snippet that I could use as a regression test. There was an important internal change in 4.3.1 that might have caused this: https://bugs.launchpad.net/lxml/+bug/1814522 It generally affects the appending of nodes to other documents. It's supposedly safer now than before, but as your example shows, there are still issues with it that are not covered by the test suite. Stefan
Stefan, Unfortunately, I'm just a user of this tool (bikeshed), and I just happened to crash the document I'm editing. I don't now much about what bikeshed does, but I know it crashes always at the same location when parsing the document. I wouldn't know how to start in getting you a smaller test case, sorry. I ran it in rr [0] on Linux, I seem to recall it was some sort of ref-counting issue, when re-parenting nodes from a fragment to another fragment or something like this (I don't know much about lxml I was just trying to understand what was going on). I might be able to get you back traces or that kind of thing, though, if that would be helpful. In any case, bikeshed has now pinned 4.3.0 unfortunately, because this crash affect a large number of users. 4.3.1 seem to have introduced this indeed, as you mention. Thanks, Paul. [0]: https://rr-project.org/, this makes this kind of complex bug very easy to debug, when you understand the code at least. On Thu, Feb 28, 2019 at 9:09 AM Stefan Behnel <stefan_ml@behnel.de> wrote:
Hi,
thanks for the report.
Paul Adenot schrieb am 19.02.19 um 18:31:
I ran into a crash on lxml 4.3.1 (released not too long ago it seems), pulled in via pip, on travis. We had not pinned my dependency version, so we went from 4.3.0 to 4.3.1, and all our builds started failing.
This can be reproduced deterministically (on travis and also on my local linux machine), by doing:
git clone git@github.com:WebAudio/web-audio-api.git git clone https://github.com/tabatkins/bikeshed.git pip install --editable $PWD/bikeshed bikeshed update cd web-audio-api bikeshed --print=plain -f spec
Let me know if I can help debugging this, I've posted some background info on
https://github.com/WebAudio/web-audio-api/pull/1830#issuecomment-465229675
I tried this and I can reproduce it. Don't know yet why it crashes, so it would help if you could strip this down to a shorter snippet that I could use as a regression test.
There was an important internal change in 4.3.1 that might have caused this:
https://bugs.launchpad.net/lxml/+bug/1814522
It generally affects the appending of nodes to other documents. It's supposedly safer now than before, but as your example shows, there are still issues with it that are not covered by the test suite.
Stefan _________________________________________________________________ Mailing list for the lxml Python XML toolkit - http://lxml.de/ lxml@lxml.de https://mailman-mail5.webfaction.com/listinfo/lxml
I just released 4.3.2 which should fix this. It's more conservative in its tree cleanup operations which could end up being done in an incorrect order in 4.3.1. Stefan Paul Adenot schrieb am 28.02.19 um 09:37:
Stefan,
Unfortunately, I'm just a user of this tool (bikeshed), and I just happened to crash the document I'm editing. I don't now much about what bikeshed does, but I know it crashes always at the same location when parsing the document. I wouldn't know how to start in getting you a smaller test case, sorry.
I ran it in rr [0] on Linux, I seem to recall it was some sort of ref-counting issue, when re-parenting nodes from a fragment to another fragment or something like this (I don't know much about lxml I was just trying to understand what was going on). I might be able to get you back traces or that kind of thing, though, if that would be helpful.
In any case, bikeshed has now pinned 4.3.0 unfortunately, because this crash affect a large number of users. 4.3.1 seem to have introduced this indeed, as you mention.
Thanks, Paul.
[0]: https://rr-project.org/, this makes this kind of complex bug very easy to debug, when you understand the code at least.
On Thu, Feb 28, 2019 at 9:09 AM Stefan Behnel <stefan_ml@behnel.de> wrote:
Hi,
thanks for the report.
Paul Adenot schrieb am 19.02.19 um 18:31:
I ran into a crash on lxml 4.3.1 (released not too long ago it seems), pulled in via pip, on travis. We had not pinned my dependency version, so we went from 4.3.0 to 4.3.1, and all our builds started failing.
This can be reproduced deterministically (on travis and also on my local linux machine), by doing:
git clone git@github.com:WebAudio/web-audio-api.git git clone https://github.com/tabatkins/bikeshed.git pip install --editable $PWD/bikeshed bikeshed update cd web-audio-api bikeshed --print=plain -f spec
Let me know if I can help debugging this, I've posted some background info on
https://github.com/WebAudio/web-audio-api/pull/1830#issuecomment-465229675
I tried this and I can reproduce it. Don't know yet why it crashes, so it would help if you could strip this down to a shorter snippet that I could use as a regression test.
There was an important internal change in 4.3.1 that might have caused this:
https://bugs.launchpad.net/lxml/+bug/1814522
It generally affects the appending of nodes to other documents. It's supposedly safer now than before, but as your example shows, there are still issues with it that are not covered by the test suite.
Stefan _________________________________________________________________ Mailing list for the lxml Python XML toolkit - http://lxml.de/ lxml@lxml.de https://mailman-mail5.webfaction.com/listinfo/lxml
Thanks, I confirm that this new dot release fixes the issue, both on the CI and locally. Cheers, Paul. On Thu, Feb 28, 2019 at 10:40 PM Stefan Behnel <stefan_ml@behnel.de> wrote:
I just released 4.3.2 which should fix this. It's more conservative in its tree cleanup operations which could end up being done in an incorrect order in 4.3.1.
Stefan
Paul Adenot schrieb am 28.02.19 um 09:37:
Stefan,
Unfortunately, I'm just a user of this tool (bikeshed), and I just happened to crash the document I'm editing. I don't now much about what bikeshed does, but I know it crashes always at the same location when parsing the document. I wouldn't know how to start in getting you a smaller test case, sorry.
I ran it in rr [0] on Linux, I seem to recall it was some sort of ref-counting issue, when re-parenting nodes from a fragment to another fragment or something like this (I don't know much about lxml I was just trying to understand what was going on). I might be able to get you back traces or that kind of thing, though, if that would be helpful.
In any case, bikeshed has now pinned 4.3.0 unfortunately, because this crash affect a large number of users. 4.3.1 seem to have introduced this indeed, as you mention.
Thanks, Paul.
[0]: https://rr-project.org/, this makes this kind of complex bug very easy to debug, when you understand the code at least.
On Thu, Feb 28, 2019 at 9:09 AM Stefan Behnel <stefan_ml@behnel.de> wrote:
Hi,
thanks for the report.
Paul Adenot schrieb am 19.02.19 um 18:31:
I ran into a crash on lxml 4.3.1 (released not too long ago it seems), pulled in via pip, on travis. We had not pinned my dependency version, so we went from 4.3.0 to 4.3.1, and all our builds started failing.
This can be reproduced deterministically (on travis and also on my local linux machine), by doing:
git clone git@github.com:WebAudio/web-audio-api.git git clone https://github.com/tabatkins/bikeshed.git pip install --editable $PWD/bikeshed bikeshed update cd web-audio-api bikeshed --print=plain -f spec
Let me know if I can help debugging this, I've posted some background info on
https://github.com/WebAudio/web-audio-api/pull/1830#issuecomment-465229675
I tried this and I can reproduce it. Don't know yet why it crashes, so
it
would help if you could strip this down to a shorter snippet that I could use as a regression test.
There was an important internal change in 4.3.1 that might have caused this:
https://bugs.launchpad.net/lxml/+bug/1814522
It generally affects the appending of nodes to other documents. It's supposedly safer now than before, but as your example shows, there are still issues with it that are not covered by the test suite.
Stefan _________________________________________________________________ Mailing list for the lxml Python XML toolkit - http://lxml.de/ lxml@lxml.de https://mailman-mail5.webfaction.com/listinfo/lxml
_________________________________________________________________ Mailing list for the lxml Python XML toolkit - http://lxml.de/ lxml@lxml.de https://mailman-mail5.webfaction.com/listinfo/lxml
participants (2)
-
Paul Adenot -
Stefan Behnel