Hi, Since I have a couple of projects building on nevow that I'd like to keep maintaining into the post-python2 world (and at least one of them contains a broader user base that I can't easily get to port templates and such), I've started porting nevow to python3. What's on https://github.com/msdemlei/nevow runs essentially all the unit tests that weren't broken on python2 except for those related to livepages. I've not tried to port any python2 code to this now, though, since I find the current situation where lots of byte strings leak through from twisted.web quite unsatisfying. I've tried to formulate something like a preliminary policy in https://github.com/msdemlei/nevow/blob/master/doc/strings_and_bytes.rst, but I'm deeply unhappy about most of this. Ideally, I'd like to fairly exclusively expose actual strings to nevow client code unless they actually deal with binary data (as in static.File, and of course request.write needs to be able to deal with this). This will probably mean isolating client code from twisted.web.Requests a bit more. So: (1) Is anyone else still interested in this? (2) Is anyone at all interested in the livepages stuff (I'm not, and dropping it would certainly make porting and maintaining the package a lot easier)? (3) If I went ahead with this and actually tried to get a python3-nevow into Debian (which I'd really, really like to see): How is everyone's feeling about keeping the name nevow even if serious functional changes (such as dropping livepages) were to come about? [What I care about most is keeping the templating system including the render_ and data_ methods]. Thanks, Markus
Hi,
Since I have a couple of projects building on nevow that I'd like to keep maintaining into the post-python2 world (and at least one of them contains a broader user base that I can't easily get to port templates and such), I've started porting nevow to python3. What's on https://github.com/msdemlei/nevow runs essentially all the unit tests that weren't broken on python2 except for those related to livepages.
I've not tried to port any python2 code to this now, though, since I find the current situation where lots of byte strings leak through from twisted.web quite unsatisfying. I've tried to formulate something like a preliminary policy in https://github.com/msdemlei/nevow/blob/master/doc/strings_and_bytes.rst, but I'm deeply unhappy about most of this.
Ideally, I'd like to fairly exclusively expose actual strings to nevow client code unless they actually deal with binary data (as in static.File, and of course request.write needs to be able to deal with this). This will probably mean isolating client code from twisted.web.Requests a bit more.
So:
(1) Is anyone else still interested in this? I'm very much interested in keeping Nevow alive, it's a the center of quite a few
Aloha On 01/23/2018 12:03 AM, Markus Demleitner wrote: projects of mine.
(2) Is anyone at all interested in the livepages stuff (I'm not, and dropping it would certainly make porting and maintaining the package a lot easier)? Definitely, Livepage is my bread and butter, although a rework of the Livepage part might be be beneficial, I just never found the time to do it and the code just flawlessly does what it advertises. (3) If I went ahead with this and actually tried to get a python3-nevow into Debian (which I'd really, really like to see): How is everyone's feeling about keeping the name nevow even if serious functional changes (such as dropping livepages) were to come about? [What I care about most is keeping the templating system including the render_ and data_ methods]. I have no association to the name but ripping Nevow apart and still name it Nevow would be at least misleading.
Mahalo, Werner
Thanks,
Markus
_______________________________________________ Twisted-web mailing list Twisted-web@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
Hi Werner, On Tue, Jan 23, 2018 at 01:20:12PM -1000, Werner Thie wrote:
(2) Is anyone at all interested in the livepages stuff (I'm not, and dropping it would certainly make porting and maintaining the package a lot easier)? Definitely, Livepage is my bread and butter, although a rework of the Livepage part might be be beneficial, I just never found the time to do it and the code just flawlessly does what it advertises.
A major reason I've not really touched livepage is that I've never dealt with it in any code I've written. So, I'm not even sure I should be worried about tests failing because node (i.e., js) on my box doesn't accept the -f flag (any more?) as the test suite apparently expects. Do you think I could charm you into trying your hand at porting the livepage part? I'll happily help out with whatever experience I've gathered in the process, of course. -- Markus
Aloha Markus Fair enough, I assume you forked Nevow, is the code accessible publicly? The testing framework on the JS side is a bit outdated, the main question on how to re-implement decent JS testing similar to the Python side remains unanswered. I did a few experiments lately by using slimerjs for fully automated end user testing, which might be one avenue or go much lighter by providing a node framework with node.js being much better supported at this time than JS interpreters at that time nevow was born. Maybe Tristan could weigh in this discussion? Mahalo, Werner On 01/23/2018 10:59 PM, Markus Demleitner wrote:
Hi Werner,
On Tue, Jan 23, 2018 at 01:20:12PM -1000, Werner Thie wrote:
(2) Is anyone at all interested in the livepages stuff (I'm not, and dropping it would certainly make porting and maintaining the package a lot easier)? Definitely, Livepage is my bread and butter, although a rework of the Livepage part might be be beneficial, I just never found the time to do it and the code just flawlessly does what it advertises. A major reason I've not really touched livepage is that I've never dealt with it in any code I've written. So, I'm not even sure I should be worried about tests failing because node (i.e., js) on my box doesn't accept the -f flag (any more?) as the test suite apparently expects.
Do you think I could charm you into trying your hand at porting the livepage part? I'll happily help out with whatever experience I've gathered in the process, of course.
-- Markus
_______________________________________________ Twisted-web mailing list Twisted-web@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
Hi Werner, On Thu, Jan 25, 2018 at 10:09:24AM -1000, Werner Thie wrote:
Fair enough, I assume you forked Nevow, is the code accessible publicly?
It's on https://github.com/msdemlei/nevow.git. -- Markus
Aloha Markus just spent a few minutessetting up a python3 venv and have trial working through the tests. Observations so far twisted pip installation fails with Building wheels for collected packages: twisted Running setup.py bdist_wheel for twisted ... error Complete output from command /home/wthie/proj/nevowport/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-u8jezyy7/twisted/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpb5r0x53kpip-wheel- --python-tag cp36: usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or: -c --help [cmd1 cmd2 ...] or: -c --help-commands or: -c cmd --help error: invalid command 'bdist_wheel' ---------------------------------------- Failed building wheel for twisted Running setup.py clean for twisted But trial seemed to start up ok, so I proceeded to install your nevow fork with python setup.py install and then ran trial on nevow and formless with trial test.nevow.test Ran 856 tests in 1.333s FAILED (skips=22, expectedFailures=2, failures=1, errors=4, successes=827) trial formless.test Ran 18 tests in 0.019s FAILED (errors=6, successes=12) I hope this is in sync with your experience, if not please advise. Mahalo, Werner On 01/26/2018 06:41 AM, Markus Demleitner wrote:
Hi Werner,
On Thu, Jan 25, 2018 at 10:09:24AM -1000, Werner Thie wrote:
Fair enough, I assume you forked Nevow, is the code accessible publicly? It's on https://github.com/msdemlei/nevow.git.
-- Markus
_______________________________________________ Twisted-web mailing list Twisted-web@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
On Jan 23, 2018, at 2:03 AM, Markus Demleitner <msdemlei@ari.uni-heidelberg.de> wrote:
Hi,
Since I have a couple of projects building on nevow that I'd like to keep maintaining into the post-python2 world (and at least one of them contains a broader user base that I can't easily get to port templates and such), I've started porting nevow to python3. What's on https://github.com/msdemlei/nevow runs essentially all the unit tests that weren't broken on python2 except for those related to livepages.
Hi Markus!
I've not tried to port any python2 code to this now, though, since I find the current situation where lots of byte strings leak through from twisted.web quite unsatisfying. I've tried to formulate something like a preliminary policy in https://github.com/msdemlei/nevow/blob/master/doc/strings_and_bytes.rst, but I'm deeply unhappy about most of this.
Yeah, IRequest is an unfortunate interface. Work is ongoing to design a more thoughtful one and gradually migrate to it. I think https://github.com/wsanchez/txrequest <https://github.com/wsanchez/txrequest> is getting pretty complete, and from there the hope is that it will get integrated into Klein and then upstreamed into Twisted.
Ideally, I'd like to fairly exclusively expose actual strings to
"actual" meaning text?
nevow client code unless they actually deal with binary data (as in static.File, and of course request.write needs to be able to deal with this). This will probably mean isolating client code from twisted.web.Requests a bit more.
So:
(1) Is anyone else still interested in this?
I can't speak to anyone else, but the fact that Nevow has a duplicated resource model is kind of unfortunate; that's why we're hoping to just replace the Twisted one eventually. (But in an upward-compatible way, with comprehensive interoperability and a smooth deprecation transition, not web2 all over again...)
(2) Is anyone at all interested in the livepages stuff (I'm not, and dropping it would certainly make porting and maintaining the package a lot easier)?
If you're going to drop livepage, is there any reason you're not just migrating to twisted.web.template? Is there any way t.w.t could be made into an easier migration path?
(3) If I went ahead with this and actually tried to get a python3-nevow into Debian (which I'd really, really like to see): How is everyone's feeling about keeping the name nevow even if serious functional changes (such as dropping livepages) were to come about? [What I care about most is keeping the templating system including the render_ and data_ methods].
If you're going to do a fork, please come up with a new name. Using the existing one is confusing, it causes namespace conflicts (what happens when Nevow itself gets some upstream patches that port it to python 3 from someone else?) and it causes import conflicts. That said, if you are interested in porting Nevow the same way that Twisted is being ported (i.e. there's one set of sources, but `pip install` on python 3 only installs the modules ported thus far) then that would certainly be a way to invest the effort in just porting the parts that you need, and letting someone else port LivePage later :). I'd be happy to make you a maintainer if you're interested in doing this - not a lot of other activity on Nevow right now! -glyph
Hi Glyph, On Tue, Jan 23, 2018 at 11:51:44PM -0800, Glyph wrote:
On Jan 23, 2018, at 2:03 AM, Markus Demleitner <msdemlei@ari.uni-heidelberg.de> wrote: through from twisted.web quite unsatisfying. I've tried to formulate something like a preliminary policy in https://github.com/msdemlei/nevow/blob/master/doc/strings_and_bytes.rst, but I'm deeply unhappy about most of this.
Yeah, IRequest is an unfortunate interface. Work is ongoing to design a more thoughtful one and gradually migrate to it. I think https://github.com/wsanchez/txrequest <https://github.com/wsanchez/txrequest> is getting pretty complete, and from there the hope is that it will get integrated into Klein and then upstreamed into Twisted.
...which of course looks like a very long migration path, very likely stretching far beyond python2's EOL if my typical deployment is on Debian stable... I wonder if one couldn't arrange for inevow.IRequest to adapt to essentially this (perhaps with a bit more glue code to keep as compatible with existing code as possible).
Ideally, I'd like to fairly exclusively expose actual strings to
"actual" meaning text?
Yes: str.
(2) Is anyone at all interested in the livepages stuff (I'm not, and dropping it would certainly make porting and maintaining the package a lot easier)?
If you're going to drop livepage, is there any reason you're not just migrating to twisted.web.template? Is there any way t.w.t could be made into an easier migration path?
I have to admit I've not really investigated t.w.t in any detail; one thing that actually does matter -- because templates of my users will silently fail with it -- is the change of the XML namespace for the nevow elements and attributes. I suppose one could work around it on template load, though. Hm -- this might be an interesting alternative to keeping all of nevow. Of course, that still doesn't help against the insiduous breakage when suddenly code needs to deal with byte strings coming out of request.args.keys(), path components, and headers; that's a bit less of a direct concern for me since it's much less deployer-visible for my applications, but if I have to write some compatibility layer, why not have nevow be it?
(3) If I went ahead with this and actually tried to get a python3-nevow into Debian (which I'd really, really like to see): How is everyone's feeling about keeping the name nevow even if serious functional changes (such as dropping livepages) were to come about? [What I care about most is keeping the templating system including the render_ and data_ methods].
If you're going to do a fork, please come up with a new name. Using the existing one is confusing, it causes namespace conflicts (what happens when Nevow itself gets some upstream patches that port it to python 3 from someone else?) and it causes import conflicts.
Fair enough (a bit more on this in my response to Werner).
That said, if you are interested in porting Nevow the same way that Twisted is being ported (i.e. there's one set of sources, but `pip install` on python 3 only installs the modules ported thus far)
Ouch -- given my experiences while porting it would seem a major chore to keep python2 compatibity. But then I've not tried, and I may be wrong. If I actually wanted to follow https://twistedmatrix.com/trac/wiki/Plan/Python3 by the letter, that's certainly far beyond what I can reasonably commit in terms of resources.
then that would certainly be a way to invest the effort in just porting the parts that you need, and letting someone else port LivePage later :). I'd be happy to make you a maintainer if you're interested in doing this - not a lot of other activity on Nevow right now!
I'd certainly like to make the effort I invest as worthwhile as possible; but, in particular after the experience of porting, I feel I'm not nearly as familiar with nevow's codebase (and customs in the twisted ecosystem) as I should be to confidently work as maintainer. Before I commit to anything, let me try to work out if I can get nevow to the point where it doesn't cause extra porting pains in the python3 migration of dependent code. Thanks, Markus
On Wed, Jan 24, 2018, at 12:54 AM, Markus Demleitner wrote:
Hi Glyph,
On Tue, Jan 23, 2018 at 11:51:44PM -0800, Glyph wrote:
On Jan 23, 2018, at 2:03 AM, Markus Demleitner <msdemlei@ari.uni-heidelberg.de> wrote: through from twisted.web quite unsatisfying. I've tried to formulate something like a preliminary policy in https://github.com/msdemlei/nevow/blob/master/doc/strings_and_bytes.rst, but I'm deeply unhappy about most of this.
Yeah, IRequest is an unfortunate interface. Work is ongoing to design a more thoughtful one and gradually migrate to it. I think https://github.com/wsanchez/txrequest <https://github.com/wsanchez/txrequest> is getting pretty complete, and from there the hope is that it will get integrated into Klein and then upstreamed into Twisted.
...which of course looks like a very long migration path, very likely stretching far beyond python2's EOL if my typical deployment is on Debian stable...
Shorter the more people we have working on it :-).
I wonder if one couldn't arrange for inevow.IRequest to adapt to essentially this (perhaps with a bit more glue code to keep as compatible with existing code as possible).
This would be great.
Ideally, I'd like to fairly exclusively expose actual strings to
"actual" meaning text?
Yes: str.
(2) Is anyone at all interested in the livepages stuff (I'm not, and dropping it would certainly make porting and maintaining the package a lot easier)?
If you're going to drop livepage, is there any reason you're not just migrating to twisted.web.template? Is there any way t.w.t could be made into an easier migration path?
I have to admit I've not really investigated t.w.t in any detail; one thing that actually does matter -- because templates of my users will silently fail with it -- is the change of the XML namespace for the nevow elements and attributes.
I suppose one could work around it on template load, though. Hm -- this might be an interesting alternative to keeping all of nevow.
I don't pretend to have a complete solution here - unfortunately I've ported very few projects *from* nevow *to* t.w.t so almost no work has gone into migration or compatibility. But I do think there's definitely something to investigate.
Of course, that still doesn't help against the insiduous breakage when suddenly code needs to deal with byte strings coming out of request.args.keys(), path components, and headers; that's a bit less of a direct concern for me since it's much less deployer-visible for my applications, but if I have to write some compatibility layer, why not have nevow be it?
This is largely about aligning as much effort as possible in one place :). Our goals are not exactly the same of course, you want Nevow+Py3, and most Klein interest right now is primarily around APIs rather than HTML, but this is exactly why extracting the request-processing fix to the lowest shared level is important :).
(3) If I went ahead with this and actually tried to get a python3-nevow into Debian (which I'd really, really like to see): How is everyone's feeling about keeping the name nevow even if serious functional changes (such as dropping livepages) were to come about? [What I care about most is keeping the templating system including the render_ and data_ methods].
If you're going to do a fork, please come up with a new name. Using the existing one is confusing, it causes namespace conflicts (what happens when Nevow itself gets some upstream patches that port it to python 3 from someone else?) and it causes import conflicts.
Fair enough (a bit more on this in my response to Werner).
I do hope we can figure out a way to work together! Although if a fork is necessary there will definitely be no hard feelings - the logistics on that are hard.
That said, if you are interested in porting Nevow the same way that Twisted is being ported (i.e. there's one set of sources, but `pip install` on python 3 only installs the modules ported thus far)
Ouch -- given my experiences while porting it would seem a major chore to keep python2 compatibity. But then I've not tried, and I may be wrong. If I actually wanted to follow https://twistedmatrix.com/trac/wiki/Plan/Python3 by the letter, that's certainly far beyond what I can reasonably commit in terms of resources.
Doing this at the very start of the project (Python2.5 + Python3.2) was agonizing; barely practical. But today (Python2.7 + Python3.4) is mostly quite pleasant.
then that would certainly be a way to invest the effort in just porting the parts that you need, and letting someone else port LivePage later :). I'd be happy to make you a maintainer if you're interested in doing this - not a lot of other activity on Nevow right now!
I'd certainly like to make the effort I invest as worthwhile as possible; but, in particular after the experience of porting, I feel I'm not nearly as familiar with nevow's codebase (and customs in the twisted ecosystem) as I should be to confidently work as maintainer.
There's really only one way to become familiar with these things - jump right in! :)
Before I commit to anything, let me try to work out if I can get nevow to the point where it doesn't cause extra porting pains in the python3 migration of dependent code.
I don't want to pressure you into any commitments you're uncomfortable with; as you said, just wanting to making the investment of your effort worthwhile. Sounds like your mind is in the right place though; let me know when you have completed your investigations :). -g
participants (3)
-
Glyph -
Markus Demleitner -
Werner Thie