Mailman 3 - Twisted-web

Listening on 2 addresses
by Axel Rau 02 Apr '19

02 Apr '19

I have a simple web application with the plugin below. It works fine with IPv4 or IPv6. The host has 2 addresses (IPv4 + IPv6 = dual stack). How can I extend the plugin to let my server listen on both addresses? Please advice, Axel PS: The plugin: from zope.interface import implementer from twisted.application.service import IServiceMaker from twisted.application import internet from twisted.plugin import IPlugin from twisted.python import usage from meteo import meteo_factory class Options(usage.Options): optParameters = [ ['port', 'p', 80, 'The port number to listen on.'], ['ipv4_address', '4', None, 'The IPv4 address to listen on.'], ['ipv6_address', '6', None, 'The IPv6 address to listen on.'], ['host', 'l', None, 'The hostname where to listen at']] @implementer(IServiceMaker, IPlugin) class MeteoServiceMaker(object): tapname = 'meteo' description = 'A web application for the meteo package' options = Options def makeService(self, options): """ Constructs a TCP server from a factory defined in meteo.py """ return internet.TCPServer( int(options['port']), meteo_factory, interface=options['ipv6_address']) serviceMaker = MeteoServiceMaker() --- PGP-Key:29E99DD6 ☀ computing @ chaos claudius

2 4

Twisted Klein idle connection timeout
by John Aherne 06 Mar '19

06 Mar '19

Klein defaults to the twisted http timeout of 60 seconds. But in twisted web Site setup you can define a timeout parameter to change this. e.g. factory = Site(root, timeout=300) I can't see any option to add a parameter in Klein to change this. Is there no option for this or have I just missed the obvious setup point. Thanks for any info. -- *John Aherne* *www.rocs.co.uk <http://www.rocs.co.uk>* 020 7223 7567

2 2

Non-Blocking I/O?
by darren＠ontrenet.com 22 Jan '19

22 Jan '19

Hi, I currently use Twisted Web as a WSGI container for my Flask application. I'm trying to get good concurrent user requests but running into some hiccups. I see that for each user session that hits my server, twistd creates an OS thread. This of course will not scale will with concurrent users. Is there a better way or perhaps command parameter I can use to support concurrent users with something like non-blocking IO which can handle thousands of concurrent users. thanks! Darren

2 1

Re: [Twisted-web] Twisted-web Digest, Vol 151, Issue 2
by Jason Harrison 09 Aug '18

09 Aug '18

On Thu, 9 Aug 2018 at 09:31 am Jean-Paul Calderone < exarkun(a)twistedmatrix.com> wrote: > In the code you posted, DelayedResponse is the root resource. The root > resource *always* has to provide a child resource. If someone requests > /foo then the child is "foo". If someone requests "/" then the child is > "". > > isLeaf = True is a shortcut that stops traversal and uses the current > resource as the requested resource. Without this, DelayedResponse must > have a child or it can only produce a 404. > > Maybe that explains it? If there are examples in the documentation that > have root resources with no children and without isLeaf = True then perhaps > they are also broken and should be fixed. If you see any, please point > them out. > > Thank you for the explanation! Let's see if I understand your explanation and the variation in the examples. https://twistedmatrix.com/documents/current/web/howto/web-in-60/custom-code… does not have isLeaf = True because it has: root = Resource() root.putChild(b"buy", PaymentRequired()) factory = Site(root) https://twistedmatrix.com/documents/current/web/howto/web-in-60/dynamic-con… does have isLeaf = True because it does not have a .putChild() The explanation about isLeaf could be improved by: Setting isLeaf to True indicates that ClockPage resources will never have any children. Without this, the ClockPage resources must have a child or it can only produce a 404. https://twistedmatrix.com/documents/current/web/howto/web-in-60/dynamic-dis… does not have isLeaf = True because it defines getChild for the root class Calendar So does not have isLeaf = True because it was written as an .rpy script?? http://twistedmatrix.com/documents/current/web/howto/web-in-60/asynchronous… Thank you again, -Jason

2 1

Re: [Twisted-web] Asynchronous responses example from documentation -- why import time??
by Jason Harrison 09 Aug '18

09 Aug '18

> On Wed, 8 Aug 2018 at 11:58 AM Jean-Paul Calderone < > exarkun(a)twistedmatrix.com> wrote: One possible explanation is that it has nothing to do with import time per > se. Instead, *any* change to your source file would have fixed the > problem. This could be because you had "stale" pyc files lying around > (cached bytecode the interpreter *thought* was up-to-date with your source > but was actually outdated). It could also be that your server process was > left running and was still using your old code. Then, for some reason, > after you added the import time your server got the new version of the code > (perhaps you're relying on an auto-reloader and it missed a change, for > example, or you just forgot to restart the server yourself). > > import time itself is definitely not a requirement for arbitrary Twisted > Web-based programs to return a response other than 404. > > Jean-Paul > > Hi Jean-Paul, I made many changes to the example in order to get it to work. So perhaps it's a caching issue, but I can't find any caches. Here's what I'm developing on: - OSX 10.13.6 - I'm using Sublime text, no auto reloaders afaik - python3 --version : Python 3.7.0 - pip3 list: Package Version -------------- ------- attrs 18.1.0 Automat 0.7.0 constantly 15.1.0 dlib 19.15.0 hyperlink 18.0.0 idna 2.7 incremental 17.5.0 numpy 1.15.0 pip 18.0 PyHamcrest 1.9.0 setuptools 39.2.0 six 1.11.0 Twisted 18.7.0 wheel 0.31.1 zope.interface 4.5.0 - running the server using ./asynchronous.py - I don't have any .pyc files in the script directory, possibly because I "chmod +x" the source file and included the #! at the top - I added logging and while the main program (setup of Twisted reactor) logged, the DelayedResource class did not - I added parts of the example at https://twistedmatrix.com/documents/current/web/howto/web-in-60/dynamic-con… - I changed "resource = DelayedResource()" to "resource = ClockPage()" - I changed the render_GET handler to return the Clock page example output - etc until I stopped getting a 404 - I restarted the server many many times I removed and backed out every change until I got a 404, concluding it was the "import time" line. Any other ideas?

2 2

Asynchronous responses example from documentation -- why import time??
by Jason Harrison 08 Aug '18

08 Aug '18

Hi, I am learning twisted-web by attempting each of the examples in the documentation and making any (small) changes necessary for Python 3. However when I got to the Aysnchronous Responses example I was not able to get it to work. The example is at: https://twistedmatrix.com/documents/current/web/howto/web-in-60/asynchronou… I have adapted it to be a self-contained "server" rather than a .rpy snippet but the response was always a 404. After much head scratching and comparison to other examples, I discovered that by adding "import time" to my "server" it started working. Why is "import time" required. What is happening without the "import time"? -Jason Cell: 604 644 8611 Email: drjasonharrison(a)gmail.com LinkedIn: http://www.linkedin.com/in/drjasonharrison Twitter: http://twitter.com/drjasonharrison #!/usr/bin/env python3 # https://twistedmatrix.com/documents/current/web/howto/web-in-60/asynchronou… # Asynchronous responses, # # try # - http://localhost:8888 import sys from twisted.web.server import Site from twisted.web.resource import Resource from twisted.internet import reactor, endpoints from twisted.web.resource import NoResource from twisted.web.server import NOT_DONE_YET from twisted.python import log import time #??? log.startLogging(sys.stdout) observer = log.PythonLoggingObserver() observer.start() class DelayedResource(Resource): def _delayedRender(self, request): log.msg("_delayedRender request {request!r}", request = request) request.write(b"<html><body>Sorry to keep you waiting.</body></html>") request.finish() def render_GET(self, request): log.msg("get request {request!r}", request = request) reactor.callLater(5, self._delayedRender, request) return NOT_DONE_YET resource = DelayedResource() factory = Site(resource) endpoint = endpoints.TCP4ServerEndpoint(reactor, 8888) endpoint.listen(factory) log.msg("running reactor") reactor.run()

2 1

Changing the order of the request header lines that get sent.
by richard77＠airmail.cc 30 Jun '18

30 Jun '18

Hello subscribers I am using scrapy 1.4.0 which makes use of twisted 17.9.0 to send and receive HTTP requests and responses (https://github.com/scrapy/scrapy/blob/master/scrapy/core/downloader/handler… ScrapyAgent.download_request()). I would like to influence the order in which the request header lines get sent. With scrapy the request header is always handled as a python dictionary and therefore can not be sorted. So I was looking at the twisted source code to find the place where it sends the request header and discontinues to use a dictionary (where I would be able to modify the order at this place). Unfortunately I couldn't find it till now. Agent.request() gets the request headers as a dictionary and then returns self._requestWithEndpoint() with the headers dictionary in https://github.com/twisted/twisted/blob/trunk/src/twisted/web/client.py on line 1655. _AgentBase._requestWithEndpoint() on line 1419 then seems to send the request but where can I change the order of the headers? Hopefully you can help me with the problem.

1 0

Nevow and python 3
by Markus Demleitner 28 Jan '18

28 Jan '18

Hi, Since I have a couple of projects building on nevow that I'd like to keep maintaining into the post-python2 world (and at least one of them contains a broader user base that I can't easily get to port templates and such), I've started porting nevow to python3. What's on https://github.com/msdemlei/nevow runs essentially all the unit tests that weren't broken on python2 except for those related to livepages. I've not tried to port any python2 code to this now, though, since I find the current situation where lots of byte strings leak through from twisted.web quite unsatisfying. I've tried to formulate something like a preliminary policy in https://github.com/msdemlei/nevow/blob/master/doc/strings_and_bytes.rst, but I'm deeply unhappy about most of this. Ideally, I'd like to fairly exclusively expose actual strings to nevow client code unless they actually deal with binary data (as in static.File, and of course request.write needs to be able to deal with this). This will probably mean isolating client code from twisted.web.Requests a bit more. So: (1) Is anyone else still interested in this? (2) Is anyone at all interested in the livepages stuff (I'm not, and dropping it would certainly make porting and maintaining the package a lot easier)? (3) If I went ahead with this and actually tried to get a python3-nevow into Debian (which I'd really, really like to see): How is everyone's feeling about keeping the name nevow even if serious functional changes (such as dropping livepages) were to come about? [What I care about most is keeping the templating system including the render_ and data_ methods]. Thanks, Markus

3 8

Behaviour on Stopping Reactor
by ROHIT LADDHA 24 Dec '17

24 Dec '17

Hi, I used following two libraries for udp listener *from twisted.internet.protocol import DatagramProtocol* *from twisted.internet import reactor* When I do reactor.stop(), it processes all the buffered datagram packets. That's fine but it should close the socket connection as well so that it doesn't listen for any new packet. What's happening in my case is that socket is getting a lot of packets continuously. If *reactor.stop()* is called, the process doesn't exit because it's still getting packets from the socket connection. Is this behaviour kept intentionally or we can change it? ‌Regards Rohit Laddha

1 0

Twisted Application(Daemon) with multiple clients
by Daniel Doron 23 Oct '17

23 Oct '17

Hi, Sorry in advance for sending this to both Twisted mailing list. i was not sure for which was it was most appropriate. I have a design in which I would like to have a Twisted Application which would periodically connect to a REST Server to pull configuration information, i.e. as client and upon receiving that information and processing would communicate spawn 1 or more clients connecting to local process via Unix sockets to send them configuration information. I think I understand how the standard approach would be: 1. create a function that would be called with callLater/LoopingCall 2. in that function create my client, connect to the REST Server, get info etc... So: 1. Do you think my approach is correct? Essentially as explained I need a service/daemon to periodically connect ot one server to pull information and then connect to other local process to forward them the relevant configuration. 2. How do I turn this into a twisted plugin? (daemon) Thanks -Daniel

2 1