[Twisted-Python] How do I run a twisted server directly
I am going step by step through the Twisted From Scratch documentation. At step labeled twistd, there is this listing:
http://twistedmatrix.com/documents/current/howto/listings/finger/finger11.py
At the end it uses:
application = service.Application('finger', uid=1, gid=1) factory = FingerFactory(moshez='Happy and well') internet.TCPServer(79, factory).setServiceParent( service.IServiceCollection(application))
Where as the previous code uses:
reactor.listenTCP(1079, FingerFactory(prefix='http://livejournal.com/~')) reactor.run()
And then gets into twistd, which is great. But I have two questions, one is how do I run this code directly (i.e., without twistd), and how do I use a debugger on code running under twistd.
Maybe the answer to my first question in look at twistd in an editor and figure it out, and the answer to the second question is run twistd in the debugger. To be honest I haven't looked at twistd, as I am already suffering for mind overload trying to understand the bits and pieces of this framework. Pretty cool stuff.
On a different unrelated subject, I am going to need to act as both a server and a client for SOAP. Can someone point me in the right direction on how to do this in twisted.
Thanks, Alex
On Fri, 2004-05-14 at 15:56, Alexander May wrote:
And then gets into twistd, which is great. But I have two questions, one is how do I run this code directly (i.e., without twistd), and how do I use a debugger on code running under twistd.
You can run twistd with -b option to run under debugger (see twistd --help).
Thanks, I'll give that a try. How about my first question, running it directly without twistd. Something along the lines of:
if '__main__' == __name__: # do something to start application directly
Thanks again, Alex
-----Original Message----- From: twisted-python-bounces@twistedmatrix.com [mailto:twisted-python-bounces@twistedmatrix.com] On Behalf Of Itamar Shtull-Trauring Sent: Friday, May 14, 2004 4:03 PM To: Twisted discussion stuff Subject: Re: [Twisted-Python] How do I run a twisted server directly
On Fri, 2004-05-14 at 15:56, Alexander May wrote:
And then gets into twistd, which is great. But I have two questions, one is how do I run this code directly (i.e., without twistd), and how do I use a debugger on code running under twistd.
You can run twistd with -b option to run under debugger (see twistd --help).
Alexander May wrote:
Thanks, I'll give that a try. How about my first question, running it directly without twistd. Something along the lines of:
if '__main__' == __name__: # do something to start application directly
Basically, Don't Do That. It's unlikely there's a good reason to. If you just want to learn how it works under the hood, then read twistd's source: twisted/scripts/twistd.py. twisted/application/app.py is also relevant.
Basically, Don't Do That. It's unlikely there's a good reason to.
I'd like to use my IDE to debug rather than the python debugger.
Also, I ran twistd.py and got "Please use twistw on windows, not twistd" (I'm on windows sometimes). I then ran twistw.py and nothing happened. I looked at the file, and there were only class and function definitions. I added "run()" at the end of the file and it seems to work. I'm using the version 1.2.0 which was marked as "Twisted Stable Release" on the download page.
Alex
-----Original Message----- From: twisted-python-bounces@twistedmatrix.com [mailto:twisted-python-bounces@twistedmatrix.com] On Behalf Of Christopher Armstrong Sent: Friday, May 14, 2004 4:33 PM To: Twisted discussion stuff Subject: Re: [Twisted-Python] How do I run a twisted server directly
Alexander May wrote:
Thanks, I'll give that a try. How about my first question, running it directly without twistd. Something along the lines of:
if '__main__' == __name__: # do something to start application directly
Basically, Don't Do That. It's unlikely there's a good reason to. If you just want to learn how it works under the hood, then read twistd's source: twisted/scripts/twistd.py. twisted/application/app.py is also relevant.
-- Twisted | Christopher Armstrong: International Man of Twistery Radix | Release Manager, Twisted Project ---------+ http://radix.twistedmatrix.com/
Alexander May wrote:
Also, I ran twistd.py and got "Please use twistw on windows, not twistd" (I'm on windows sometimes). I then ran twistw.py and nothing happened. I looked at the file, and there were only class and function definitions. I added "run()" at the end of the file and it seems to work. I'm using the version 1.2.0 which was marked as "Twisted Stable Release" on the download page.
You should be calling C:\Python23\scripts\twistd (or something like that), not Lib\twisted\scripts\twistd.py. When you install the windows package for Twisted, there should be a start menu item that runs a shell with a properly set up environment so you can just type "twistd" or "mktap" and so on and it'll DTRT.
Oops, thanks. Any hope debugging in my IDE instead of pdb?
-----Original Message----- From: twisted-python-bounces@twistedmatrix.com [mailto:twisted-python-bounces@twistedmatrix.com] On Behalf Of Christopher Armstrong Sent: Friday, May 14, 2004 8:25 PM To: Twisted discussion stuff Subject: Re: [Twisted-Python] How do I run a twisted server directly
Alexander May wrote:
Also, I ran twistd.py and got "Please use twistw on windows, not twistd" (I'm on windows sometimes). I then ran twistw.py and nothing happened. I looked at the file, and there were only class and function definitions. I added "run()" at the end of the file and it seems to work. I'm using the version 1.2.0 which was marked as "Twisted Stable Release" on the download page.
You should be calling C:\Python23\scripts\twistd (or something like that), not Lib\twisted\scripts\twistd.py. When you install the windows package for Twisted, there should be a start menu item that runs a shell with a properly set up environment so you can just type "twistd" or "mktap" and so on and it'll DTRT.
I've use pythonwin for a while and I'm giving Komodo try right now. I'll try Python.NET soon too.
I cribbed from twistw and did this:
if '__main__' == __name__: from twisted.application import app import sys
class ServerOptions(app.ServerOptions): optFlags = [['nodaemon','y', "(for backwards compatability)."]]
app.startApplication(application, 0) app.runReactorWithLogging(ServerOptions(), sys.stdout, sys.stderr)
Which seems to work. I can start my application from the debugger and hit breakpoints, etc.
Presumably I've committed all sort of horrors writing this code. Please feel free to enlighten me to the error of my ways. Also, I'm curious what is the function of the following line of code (marked with a *) is which I saw in twistw:
def runApp(config): passphrase = app.getPassphrase(config['encrypted']) app.installReactor(config['reactor']) application = app.getApplication(config, passphrase) oldstdout = sys.stdout oldstderr = sys.stderr startLogging(config['logfile']) app.initialLog() os.chdir(config['rundir']) service.IService(application).privilegedStartService() app.startApplication(application, not config['no_save']) * app.startApplication(internet.TimerService(0.1, lambda:None), 0) app.runReactorWithLogging(config, oldstdout, oldstderr) app.reportProfile(config['report-profile'], service.IProcess(application).processName) log.msg("Server Shut Down.")
* do nothing frequently?
-----Original Message----- From: twisted-python-bounces@twistedmatrix.com [mailto:twisted-python-bounces@twistedmatrix.com] On Behalf Of Robert Church Sent: Friday, May 14, 2004 9:43 PM To: Twisted discussion stuff Subject: Re: [Twisted-Python] How do I run a twisted server directly
Alexander May wrote:
Oops, thanks. Any hope debugging in my IDE instead of pdb?
What's your IDE?
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On May 14, 2004, at 10:02 PM, Alexander May wrote:
I've use pythonwin for a while and I'm giving Komodo try right now. I'll try Python.NET soon too.
<snip>
Which seems to work. I can start my application from the debugger and hit breakpoints, etc.
Presumably I've committed all sort of horrors writing this code. Please feel free to enlighten me to the error of my ways. Also, I'm curious what is the function of the following line of code (marked with a *) is which I saw in twistw:
<snip>
Why are you assuming that you can't use twistd and komodo's debugger at the same time? The last time I looked at Komodo for debugging (which admittedly was at least two years ago) importing the debugging module at any time installed the trace hook and queried Komodo for breakpoints. Then, any time the trace hook wanted to break into the debugger, it made a synchronous, blocking call over xmlrpc into the Komodo process. The Komodo process would then delay 'returning' from that xmlrpc call until you pressed step or continue in the ui.
You should just be able to import the komodo debugging shim module at the top of your .tac file, start it using twistd, and debug just fine.
dp
Why are you assuming that you can't use twistd and komodo's debugger at
the same time?
I'm new to both twisted and Komodo, and never even considered that such on option existed. Thanks for the info, I'll see if I can manage it. Is there a similar mechanism for other IDEs?
Alex
-----Original Message----- From: twisted-python-bounces@twistedmatrix.com [mailto:twisted-python-bounces@twistedmatrix.com] On Behalf Of Donovan Preston Sent: Saturday, May 15, 2004 12:00 AM To: Twisted discussion stuff Subject: Re: [Twisted-Python] How do I run a twisted server directly
On May 14, 2004, at 10:02 PM, Alexander May wrote:
I've use pythonwin for a while and I'm giving Komodo try right now. I'll try Python.NET soon too.
<snip>
Which seems to work. I can start my application from the debugger and hit breakpoints, etc.
Presumably I've committed all sort of horrors writing this code. Please feel free to enlighten me to the error of my ways. Also, I'm curious what is the function of the following line of code (marked with a *) is which I saw in twistw:
<snip>
Why are you assuming that you can't use twistd and komodo's debugger at the same time? The last time I looked at Komodo for debugging (which admittedly was at least two years ago) importing the debugging module at any time installed the trace hook and queried Komodo for breakpoints. Then, any time the trace hook wanted to break into the debugger, it made a synchronous, blocking call over xmlrpc into the Komodo process. The Komodo process would then delay 'returning' from that xmlrpc call until you pressed step or continue in the ui.
You should just be able to import the komodo debugging shim module at the top of your .tac file, start it using twistd, and debug just fine.
dp
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
You should just be able to import the komodo debugging shim module at the
top of your .tac file, start it using twistd, and debug just fine.
You mentioned that it has been a while, but does callkomodo.py sound right? Or is the "komodo debugging shim module" something else? I've tried a few things, none of which have worked well. Of course it is way past my bedtime so I'm no longer thinking straight.
Also using this technique, will there be an issue when I move my code into production? Is the import meaningless if I'm not running under the debugger? Or will I have to remember to remove the import line for the debug module. Or can I conditional import it using __debug__ and pass the -O flag through when using twistd? I apologize for my lack of knowledge; I've stayed happily ignorant of the workings of the debugger until now.
What does everybody else here use for a development environment/debugger when writing twisted or twisted apps? Any IDE users out there? I don't want to waste more time than I already have just setting up my environment.
Alex
-----Original Message----- From: twisted-python-bounces@twistedmatrix.com [mailto:twisted-python-bounces@twistedmatrix.com] On Behalf Of Donovan Preston Sent: Saturday, May 15, 2004 12:00 AM To: Twisted discussion stuff Subject: Re: [Twisted-Python] How do I run a twisted server directly
On May 14, 2004, at 10:02 PM, Alexander May wrote:
I've use pythonwin for a while and I'm giving Komodo try right now. I'll try Python.NET soon too.
<snip>
Which seems to work. I can start my application from the debugger and hit breakpoints, etc.
Presumably I've committed all sort of horrors writing this code. Please feel free to enlighten me to the error of my ways. Also, I'm curious what is the function of the following line of code (marked with a *) is which I saw in twistw:
<snip>
Why are you assuming that you can't use twistd and komodo's debugger at the same time? The last time I looked at Komodo for debugging (which admittedly was at least two years ago) importing the debugging module at any time installed the trace hook and queried Komodo for breakpoints. Then, any time the trace hook wanted to break into the debugger, it made a synchronous, blocking call over xmlrpc into the Komodo process. The Komodo process would then delay 'returning' from that xmlrpc call until you pressed step or continue in the ui.
You should just be able to import the komodo debugging shim module at the top of your .tac file, start it using twistd, and debug just fine.
dp
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On Sat, May 15, 2004 at 01:05:43AM -0400, Alexander May wrote: [...]
What does everybody else here use for a development environment/debugger when writing twisted or twisted apps? Any IDE users out there? I don't want to waste more time than I already have just setting up my environment.
I tend to simply use print statements, unit tests and pdb. My "IDE" is vim :)
I rarely find stepping through code with a debugger to be useful; I find having a log file of carefully selected print statements, and ideally automated unit tests that enable me to easily reproduce the problem without having to manually set up every time, are best.
My debugging process usually involves a bit of reasoning about what's going on, and testing hypotheses about causes by trying to trigger certain behaviours -- or more often by seeing what my selectively placed print statements do. I don't find stepping through code with a debugger to be a very efficient way to do this, because you spend most of your time stepping over code you aren't interested in.
-Andrew.
Andrew Bennetts wrote:
On Sat, May 15, 2004 at 01:05:43AM -0400, Alexander May wrote: [...]
What does everybody else here use for a development environment/debugger when writing twisted or twisted apps? Any IDE users out there? I don't want to waste more time than I already have just setting up my environment.
I tend to simply use print statements, unit tests and pdb. My "IDE" is vim :)
I use PDB. My IDE is twisted-dev.el, which runs on the popular Emacs IDE platform :).
Since many of the unit tests I run are quite time-consuming (even on very fast hardware) I do avail myself of a debugger. However, I haven't run an actual application under a debugger in years. I press F9 (twisted-dev-debug-tests) to run the current unit-test under the debugger.
What this does is simply communicate with twistd -b over standard input/output, and keep a buffer open with the code on the appropriate line.
Until somebody develops an IDE with some runtime modifications (such as smalltalk-style restartable exceptions) I doubt that I'll find any compelling reason to switch...
On Fri, 2004-05-14 at 22:02, Alexander May wrote:
I cribbed from twistw and did this:
Looks fine.
Presumably I've committed all sort of horrors writing this code. Please feel free to enlighten me to the error of my ways. Also, I'm curious what is the function of the following line of code (marked with a *) is which I saw in twistw:
- app.startApplication(internet.TimerService(0.1, lambda:None), 0)
This is so Ctrl-C will be responsive on Windows, otherwise it doesn't happen until the event loop wakes up - which may be quite a while if no networking is going on.
Thanks.
FYI - When I did this I stumbled on to something. If I have scratch.py as follows:
from twisted.application import service application = service.Application('central-server')
Where the last line consists of a single space, then I get an error when I run twistd -y scratch.py (call stack at end). When I remove the space so the last line is just a carriage return, it works fine. I found it strange. Perhaps it is a python thing I has just never stumbled on before. Not a big deal now that I know, but it drove me batty for a bit.
Also, why the preference for .tac extensions when running twistd, instead of .py?
I'm slowly working my way through the docs, so if it's RTFM, I'll understand.
Thanks again for all the help, Alex
Call stack:
Traceback (most recent call last): File "C:\Python23\scripts\twistd.py", line 36, in ? run() File "C:\Python23\Lib\site-packages\twisted\scripts\twistw.py", line 61, in run app.run(runApp, ServerOptions) File "C:\Python23\Lib\site-packages\twisted\application\app.py", line 204, in run runApp(config) File "C:\Python23\Lib\site-packages\twisted\scripts\twistw.py", line 45, in runApp application = app.getApplication(config, passphrase) --- <exception caught here> --- File "C:\Python23\Lib\site-packages\twisted\application\app.py", line 117, in getApplication application = service.loadApplication(filename, style, passphrase) File "C:\Python23\Lib\site-packages\twisted\application\service.py", line 291, in loadApplication application = sob.loadValueFromFile(filename, 'application', passphrase) File "C:\Python23\Lib\site-packages\twisted\persisted\sob.py", line 213, in loadValueFromFile exec data in d, d exceptions.SyntaxError: invalid syntax (line 5)
Failed to load application: invalid syntax (line 5)
"Alexander May" alex-news@comcast.net writes:
Where the last line consists of a single space, then I get an error when I run twistd -y scratch.py (call stack at end). When I remove the space so the last line is just a carriage return, it works fine. I found it strange. Perhaps it is a python thing I has just never stumbled on before. Not a big deal now that I know, but it drove me batty for a bit.
It sounds like an existing Python issue having to do with the internal compile function (which is also used by exec) not handling the case of a file not terminating in a newline. Programs that bring in modules as text and then compile them can run into this (it used to happen to me with the Installer package). So it's not really something related to twisted other than how it accesses the module specified with -y. I guess since as of Python 2.2, compile/exec is documented as needing the newline you might argue twistd should be taking care of that. And yes, the fact that it shows up as a bland SyntaxError is not obvious.
This got fixed as part of SourceForge bug 501622. The fix showed up in Python 2.3. (See also http://sourceforge.net/tracker/index.php?func=detail&aid=501622&grou...).
-- David
This got fixed as part of SourceForge bug 501622. The fix showed up in
Python 2.3. (See also http://sourceforge.net/tracker/index.php?func=detail&aid=501622&grou... 0&atid=105470).
Thanks for the info. I'm running Python 2.3.3. It does still appear to be an issue with python and not twisted.
C:\Documents and Settings\Alexander May\My Documents\maygold-dev\central-server>python Python 2.3.3 (#51, Dec 18 2003, 20:22:39) [MSC v.1200 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information.
c=compile ('print "hello world"\n', 'filename', 'exec') exec c
hello world
c=compile ('print "hello world"\n ', 'filename', 'exec')
Traceback (most recent call last): File "<stdin>", line 1, in ? File "filename", line 2
SyntaxError: invalid syntax
^Z
Alex
-----Original Message----- From: twisted-python-bounces@twistedmatrix.com [mailto:twisted-python-bounces@twistedmatrix.com] On Behalf Of David Bolen Sent: Monday, May 17, 2004 3:54 PM To: twisted-python@twistedmatrix.com Subject: [Twisted-Python] Re: How do I run a twisted server directly
"Alexander May" alex-news@comcast.net writes:
Where the last line consists of a single space, then I get an error when I run twistd -y scratch.py (call stack at end). When I remove the space so the last line is just a carriage return, it works fine. I
found it strange.
Perhaps it is a python thing I has just never stumbled on before. Not a big deal now that I know, but it drove me batty for a bit.
It sounds like an existing Python issue having to do with the internal compile function (which is also used by exec) not handling the case of a file not terminating in a newline. Programs that bring in modules as text and then compile them can run into this (it used to happen to me with the Installer package). So it's not really something related to twisted other than how it accesses the module specified with -y. I guess since as of Python 2.2, compile/exec is documented as needing the newline you might argue twistd should be taking care of that. And yes, the fact that it shows up as a bland SyntaxError is not obvious.
This got fixed as part of SourceForge bug 501622. The fix showed up in Python 2.3. (See also http://sourceforge.net/tracker/index.php?func=detail&aid=501622&grou... 0&atid=105470).
-- David
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
participants (8)
-
Alexander May -
Andrew Bennetts -
Christopher Armstrong -
David Bolen -
Donovan Preston -
Glyph Lefkowitz -
Itamar Shtull-Trauring -
Robert Church