Mailman 3 March 2004 - Twisted

[Twisted-Python] Twisted Python 0.7.0 Prerelease Testing
by Glyph Lefkowitz Aug. 5, 2021

Aug. 5, 2021

hi there, folks: I'd really like to release 0.7.0 but I would like it to be at least a little bit tested before I do so. Could those of you with CVS trees check everything out and see if it performs as advertised? Deeper bugs than that will have to wait for the next release, but I'd at least like to know if it works for someone other than me. Thanks. ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |… [View More]

4 5

[Twisted-Python] Scalability of an rss-aggregator
by Valentino Volonghi aka Dialtone April 2, 2004

April 2, 2004

Hi all, attached you will find my rss-aggregator made with twisted. It's really fast although when I tried with 745 feeds I got some problems. When the download reached 300 parsed feeds (more or less) it locked till I pressed Ctrl+C and then it processed the remaining 340 feeds in less than 30 seconds... I think that my design has at least an issue but I cannot find it so easily and I hope someone on this list can help me to improve it. The "script" is heavily commented. BTW When it … [View More]finishes (with all 740 feeds) it reports an awesome 330 seconds which is an impressive time, less than half a second for each feed, and It downloads more than 50Mb of feeds from the net (with 745 feeds to download). Thx for your help. -- Valentino Volonghi aka Dialtone Linux User #310274, Gentoo Proud User X Python Newsreader developer http://sourceforge.net/projects/xpn/ from twisted.internet import reactor, protocol, defer from twisted.web import client import feedparser, time, out rss_feeds = out.rss_feed # This is the default site list #rss_feeds = [('http://www.nongnu.org/straw/news.rss','straw'), # ('http://googlenews.74d.com/rss/google_it.rss','google'), # ('http://www.pythonware.com/daily/rss.xml','pythonware'), # ('http://www.theinquirer.net/inquirer.rss','inq'), # ('http://www.groklaw.net/backend/GrokLaw.rdf','grok'), # ('http://www.livejournal.com/users/moshez/data/rss','zadka'), # ('http://www.pythonware.com/news.rdf','pwn')] # michele(a)berthold.com INTER_QUERY_TIME = 300 class FeederProtocol(object): def __init__(self): self.parsed = 0 # This dict structure will be the following: # { 'URL': (TIMESTAMP, value) } self.cache = {} def gotError(self, data=None, extra_args=None): # An Error as occurred, print traceback infos and go on print data self.parsed += 1 print "="*20 print "Trying to go on..." def getFeeds(self, where=None): #print "getting feeds" # This is to get the feeds we want if not where: # We don't have a database, then we use the local # variabile rss_feeds return rss_feeds else: return None def memoize(self, feed, site=None, extra=None): # extra is the second element of each tuple of rss_feeds # site is the address of the feed, also the first element of each tuple # of rss_feeds print "Memoizing",site,"..." self.cache.setdefault(site, (time.time(),feed)) return feed def stopWorking(self, data=None): print "Closing connection number %d..."%(self.parsed,) print "-"*20 # This is here only for testing. When a protocol/interface will be # created to communicate with this rss-aggregator server, we won't need # to die after we parsed some feeds just one time. self.parsed += 1 if self.parsed >= len(rss_feeds): print "Closing all..." #for i in self.cache: # print i print time.time()-tp #reactor.stop() def getPageFromMemory(self, key=None): #print "getting from memory" # Getting the second element of the tuple which is the parsed structure # of the feed at address key, the first element of the tuple is the # timestamp d = defer.succeed(self.cache.get(key,key)[1]) return d def parseFeed(self, feed): # This is self explaining :) return feedparser.parse(feed) def startDownloading(self, site): #print "Looking if",site[0],"cached...", # Try to get the tuple (TIMESTAMP, FEED_STRUCT) from the dict if it has # already been downloaded. Otherwise assign None to already_got already_got = self.cache.get(site[0], None) # Ok guys, we got it cached, let's see what we will do if already_got: # Well, it's cached, but will it be recent enough? #print "It is\n Looking if timestamp for",site[0],"is recent enough...", elapsed_time = time.time() - already_got[0] # Woooohooo it is, elapsed_time is less than INTER_QUERY_TIME so I # can get the page from the memory, recent enough if elapsed_time < INTER_QUERY_TIME: #print "It is" return self.getPageFromMemory(site[0]) else: # Uhmmm... actually it's a bit old, I'm going to get it from the # Net then, then I'll parse it and then I'll try to memoize it # again #print "Getting",site[0],"from the Net because old" return self.downloadPage(site) else: # Well... We hadn't it cached in, so we need to get it from the Net # now, It's useless to check if it's recent enough, it's not there. #print "Getting",site[0],"from the Net" return self.downloadPage(site) def downloadPage(self, site): #print "Now downloading..." # Self-explanatory d = client.getPage(site[0]) # Uncomment the following if you want to make everything crash :), since # it will save the feed on a file, but with the memoize feature it will # crash everything cuz it will break the get-->parse-->memoize chain #d = client.downloadPage(site[0],site[1]) # Parse the feed and if there's some errors call self.gotError d.addCallbacks(self.parseFeed, self.gotError) # Now memoize it, if there's some error call self.getError d.addCallbacks(self.memoize, self.gotError, site) return d def workOnPage(self, parsed_feed=None, site=None, extra_args=None, extra_key=None): print "-"*20 #print "finished retrieving" print "Feed Version:",parsed_feed.get('version','Unknown') # # Uncomment the following if you want to print the feeds # chan = parsed_feed.get('channel', None) if chan: print chan.get('title', '') #print chan.get('link', '') #print chan.get('tagline', '') #print chan.get('description','') print "-"*20 #items = parsed_feed.get('items', None) #if items: # for item in items: # print '\tTitle: ', item.get('title','') # print '\tDate: ', item.get('date', '') # print '\tLink: ', item.get('link', '') # print '\tDescription: ', item.get('description', '') # print '\tSummary: ', item.get('summary','') # print "-"*20 #print "got",site #print "="*40 def start(self, data=None): # Here we gather all the urls for the feeds #self.factory.tries += 1 for feed in self.getFeeds(): # Now we start telling the reactor that it has # to get all the feeds one by one... d = self.startDownloading(feed) # The it will pass the result of # startDownloading to workOnPage (this is hidden in twisted) # together with the feed url (just to use some extra infos # in the workOnPage method) d.addCallbacks(self.workOnPage, self.gotError, feed) # And when the for loop is ended we put # stopWorking on the callback for the last # feed gathered d.addCallbacks(self.stopWorking, self.gotError) # This is to try the memoize feature #if self.factory.tries<3: # d.addCallback(self.start) class FeederFactory(protocol.ClientFactory): protocol = FeederProtocol() def __init__(self): # tries is used to make more connection to use the # memoizing feature #self.tries = 0 # Here we put in the FeederProtocol instance a reference to # FeederFactory under the name of self.factory (seen from protocol) self.protocol.factory = self self.protocol.start() f = FeederFactory() tp = time.time() reactor.run() [View Less]

3 17

[Twisted-Python] help ssh forwarding.
by Oisin Mulvihill April 1, 2004

April 1, 2004

Hi, I have a series of linux based internet kiosks at a variety of sites. I need to be able to connect back into the kiosk from the main server. However direct connections aren't possible as kiosks are behind there own router/firewall. The solution to this is to forward port 22 on the kiosk to a port on the server. I was thinking of just spawning the command "ssh -R11122:my_server:22 user@my_server" from the kiosk, when ever I need to connect to a machine. However I'm hoping to do this with … [View More]

2 1

[Twisted-Python] help ssh forwarding.
by Oisin Mulvihill April 1, 2004

April 1, 2004

Sorry If you get two of these, I sent the last one with the wrong email account. Hi, I have a series of linux based internet kiosks at a variety of sites. I need to be able to connect back into the kiosk from the main server. However direct connections aren't possible as kiosks are behind there own router/firewall. The solution to this is to forward port 22 on the kiosk to a port on the server. I was thinking of just spawning the command "ssh -R11122:my_server:22 user@my_server" from the … [View More]

3 4

[Twisted-Python] adbapi results through xmlrpc
by Zach Thompson March 31, 2004

March 31, 2004

Hello, I'm trying to connect 2 scripts (a server and client) xmlrpc. The server just needs to run a db query and return the result set to the client for display. Here's part of the server class: def xmlrpc_get_count(self): from twisted.enterprise import adbapi pool = adbapi.ConnectionPool("pyPgSQL.PgSQL", database="zach") return pool.runQuery("select count(*) from asdf") And part of the client class: def OnButton(self, event): from twisted.… [View More]

3 3

Re: [Twisted-Python] adbapi results through xmlrpc
by dave＠krondo.com March 31, 2004

March 31, 2004

Returning a deferred from the xmlrpc method will work fine -- twisted will use the result of the deferred and not the deferred itself when sending the return xml-rpc value. The problem is that the result of the query is not serializable, probably because it contains some pgsql objects somewhere. Instead of sending the result directly, you'll probably want to add a callback that picks the count value out of the row and column. Also, the type of the count may not be a regular integer but some … [View More]

1 0

[Twisted-Python] Twisted Euphrates (Modified by Glyph Lefkowitz)
by Glyph Lefkowitz March 29, 2004

March 29, 2004

Unfortunately, I've been out of the loop on Twisted development for some time. During the course of the PyCon sprints, I've been catching up with what's been going on and trying to do some planning for future directions in development. Yesterday Itamar and I had our quarterly screaming argument about obscure details, and it didn't come to blows, which I believe means that spring is right around the corner! The major issue we've been discussing is splitting up Twisted into multiple … [View More]packages. Twisted has been encountering some growing pains, due to a mismatch between the way it's developed and the way it's distributed. We (T.M.Labs) are really a community of developers working on several interrelated projects with different goals, stability, and release cycles, but we are distributing one monolithic system, with code of vastly different qualities in the same tarball and sometimes even in the same package. The primary problem that this leads to is revision coupling. This is probably old news to anyone trying to deploy a large system which relies on any part of Twisted besides the base reactor (and maybe twisted.web). Scenarios like this happen all the time: - Twisted v.N.M.0 is released. - Project P uses twisted components X and Y. (Generally X and Y are marked 'Unstable' or 'Semi-Stable', but are very useful or heavily marketed or otherwise convince unsuspecting users to trust their stability.) - Project Q uses twisted components Y and Z. - Project P v. 1 is released. - Project Q v. 1 is released. - Project P gives useful feedback to twisted developer A who then fixes several bugs in component X which are critical to the functioning of the as-yet-unreleased P v.2 - Twisted developer B working on project Q commits substantial "improvements" to component Y, which change the functionality in a way that breaks project X. - Radix prepares an alpha/bugfix release, v.N.M.1 At this point, Project Q is perfectly happy, because the most recent alpha works for their code. However, Project P is very unhappy, because there is no release which includes their bugfix but does NOT include code which forces them to upgrade some functionality in an a way which is incompatible with the previous micro-version release of Twisted which their users presumably already have installed. The release notes have to be very specific, and because nobody reads them anyway, a lot of confusion results. While there are problems with splitting Twisted into multiple projects, I think that the time has come to do so. Originally the feedback I heard from most users was that it was great to have lots of different packages available so readily; the "batteries included" philosophy does have a lot going for it. Now, most of what I'm hearing is complaints about the above scenario, not the least of which from my own employer :-). We are more often in the role of project Q than project P (although we have been in both more than once) and I'd prefer not to have a few projects who employ active Twisted developers control the release cycle based on their requirements. Because splitting the project involves moving some code around, we should take the opportunity to clean up another problem: our code has no effective separation between interface and implementation. Within the project this is fine, because we can easily refactor any uses of deprecated APIs. While we aren't going to break the existing package-structure interfaces (or even deprecate them, for the time being), this means that in the future, imports for twisted projects will probably look like: from twisted_core import reactor from twisted_web import Resource The way this will be implemented, a top-level "twisted_core" module (or possibly, package, but I would like as little nesting as possible here) will import explicitly public names from a _twisted_core implementation package. For the forseeable future, the twisted.* package hierarchy will continue to work for imports; however, considering how informally specified this interface has been, I imagine improved documentation will emerge for the newer interface packages. Personally I wanted to call these modules "twisted_web_1", and I also wanted to talk about versioning here, but amidst strenuous objections from Anthony Baxter and Itamar ST I will refrain :). Suffice it to say that I believe that regardless of many potential strategies available for versioning, a separate module for interface vs. implementation will make it possible to determine what is public and usable from a deployed application using the Python interactive prompt or reading the code. Since many packages don't have stability declarations anyway, I think this will result in many fewer unintentional uses of private code. The first package that we are going to try this on is twisted.web, because there is a major rewrite pending integration of all the fixes and redesign that has happened in the Nevow package. If this goes well, other packages will use the same naming conventions. In any case, we will be migrating packages out of the Twisted core. for example: lore, conch, manhole, and mail. As part of this process we hope to improve the website to reflect the name "Twisted" as primarily a part of "Twisted Matrix Labs", the group of developers working on these projects. Similar to tigris.org, TML is "a mid-sized open source community focused on building better tools for networked application development". Package maintainers: each project should have its own sub-site, so consider a project description and any other documentation you might want on that site. As far as our development process, we will be keeping all the code in one subversion repository and still running BuildBot over the entire system in a way similar to how we're doing it now. There are lots of possible improvements to the process, all the way up to an independent package management system using distutils: please, don't recommend any of these. We don't want to make a huge number of changes at the same time, and while this moving / renaming is fairly mechanical, it will touch almost every file in the world; we need to make sure it all works before we begin adding features based on it. [View Less]

8 12

[Twisted-Python] IOCP Reactor Problem
by Justin Johnson March 29, 2004

March 29, 2004

When I try to use the new IOCP reactor (latest from svn) I get the following errors. The only thing being started up is a nevow web site. This worked when using win32reactor. Anyone have any ideas? Let me know if more info is needed. Thanks. -Justin 2004/03/29 15:44 Central Standard Time [-] Traceback (most recent call last): File "C:\Python23\Lib\site-packages\twisted\protocols\http.py", line 554, in requestReceived self.process() File "C:\svn\… [View More]

1 0

RE: [Twisted-Python] Windows IOCP reactor Question
by Schneider, Michael March 29, 2004

March 29, 2004

If it is possible, that would be great!!!! Thanks Mike ---------------------------------------------------------------- Michael Schneider Senior Software Engineering Consultant UGS PLM Solutions - an EDS Company "The Greatest Performance Improvement Is the transitioning from a non-working state to the working state" > -----Original Message----- > From: twisted-python-admin(a)twistedmatrix.com > [mailto:twisted-python-admin@twistedmatrix.com]On Behalf Of Cory Dodt > Sent: … [View More]

1 0

[Twisted-Python] Windows IOCP reactor Question
by Schneider, Michael March 29, 2004

March 29, 2004

Thanks for the work on IOCP, I really appriecaite the efforts and will make use of it. I am an old UNIX hacker, and I am delivering a Windows project with twisted. I am about as far away from a windows programmer that you can get. I did have to get memory info from the Widows OS. I was going to use C, but then I found ctypes and the struct python C struct wrapper. It was nice to have a python only solution. As I was looking through the code in iocpcoreobject.c. I noticed that most (… [View More]maybe all) of the functionality could be accessed by the ctypes module and the Struct package. Being a newbie to windows programming, I am not comfortable making the definitive statement that it should use this, and elimintate the C code. Instead I would like to ask the question. Does anyone see a problem replacing the C code in iocpcoreobject.c with a python module that leverages ctype. I attached a simple python script that gets the memory from a windows box using ctypes. It would be very nice if no c compiler would be required to install twisted IOCP reactor from CVS. Thanks, Mike ------------------------------------------------------- #!/usr/bin/env python # Simple WMI and ctypes example # Michael Schnedier import os import time import wmi from ctypes import * from ctypes.wintypes import * import win32api class MEMORYSTATUS(Structure): _fields_ = [ ('dwLength', DWORD), ('dwMemoryLoad', DWORD), ('dwTotalPhys', DWORD), ('dwAvailPhys', DWORD), ('dwTotalPageFile', DWORD), ('dwAvailPageFile', DWORD), ('dwTotalVirtual', DWORD), ('dwAvailVirtual', DWORD), ] class ComputeNode: def __init__(self): self.memInfo = None self.lastUpdate = time.time() self.update() def genMemoryXmlStringRep(self): self.validate() xmlString = \ """<?xml version="1.0"?> <MemoryInfo platform="windows" TotalPhysicalMemory="%d" AvailablePhysicalMemory="%d" TotalPageFileSize="%d" AvailPageFileSize="%d" GetTotalVirtualMemory="%d" GetAvailableVirtualMemory="%d" /> """%( self.getTotalPhysicalMemory(), self.getAvailPhysicalMemory(), self.getTotalPageFileSize(), self.getAvailPageFile(), self.getTotalVirtualMemory(), self.getAvailVirtualMemory()) return xmlString def getTotalPhysicalMemory(self): self.validate() return self.memInfo.dwTotalPhys/1024/1024 def getAvailPhysicalMemory(self): self.validate() return self.memInfo.dwAvailPhys/1024/1024 def getTotalPageFileSize(self): self.validate() return self.memInfo.dwTotalPageFile/1024/1024 def getAvailPageFile(self): self.validate() return self.memInfo.dwAvailPageFile/1024/1024 def getTotalVirtualMemory(self): self.validate() return self.memInfo.dwTotalVirtual/1024/1024 def getAvailVirtualMemory(self): self.validate() return self.memInfo.dwAvailVirtual/1024/1024 def validate(self, deltaTrigger=60 ): deltaTime = time.time() - self.lastUpdate if (self.lastUpdate < 0) or (deltaTime > deltaTrigger): self.update() def update(self): self.lastUpdate = time.time() newStatus = MEMORYSTATUS() windll.kernel32.GlobalMemoryStatus(byref(newStatus)) self.memInfo = newStatus machineInfo = ComputeNode() print machineInfo.genMemoryXmlStringRep() Since the C code seems to be a wrapper to COM services, and registration of COM service I am curious ---------------------------------------------------------------- Michael Schneider Senior Software Engineering Consultant UGS PLM Solutions - an EDS Company "The Greatest Performance Improvement Is the transitioning from a non-working state to the working state" > [View Less]

2 1

[Twisted-Python] Re: [Twisted-commits] r10392 - Start http rewrite, don't look yet.
by Andrew Bennetts March 29, 2004

March 29, 2004

On Wed, Mar 24, 2004 at 03:52:02PM -0700, James Knight wrote: > Author: foom > Date: Wed Mar 24 15:52:02 2004 > New Revision: 10392 > > Added: > trunk/sandbox/foom/ > trunk/sandbox/foom/web/ > trunk/sandbox/foom/web/http.py > trunk/sandbox/foom/web/http_headers.py > trunk/sandbox/foom/web/test_http.py > trunk/sandbox/foom/web/test_http_headers.py > Log: > Start http rewrite, don't look yet. Now that we're using a version control system … [View More]

5 7

RE: [Twisted-Python] The right way to broadcast to your TCP/IP clients?
by Schneider, Michael March 29, 2004

March 29, 2004

I am in a similar situation. I am use twisted a grid server for each node on the computer. I need to send info like " - I am online - Job started - Job completed - Ready for job I am getting ready to prototype with jabber. Any other thought? Thanks Mike ---------------------------------------------------------------- Michael Schneider Senior Software Engineering Consultant UGS PLM Solutions - an EDS Company "The Greatest Performance Improvement Is the transitioning from a non-working … [View More]state to the working state" > -----Original Message----- > From: twisted-python-admin(a)twistedmatrix.com > [mailto:twisted-python-admin@twistedmatrix.com]On Behalf Of Steve > Freitas > Sent: Sunday, March 28, 2004 11:20 PM > To: twisted-python(a)twistedmatrix.com > Subject: [Twisted-Python] The right way to broadcast to your TCP/IP > clients? > > > Hi all, > > I'm working on a chat server-ish kinda thang, and although > the learning curve > of twisted was a bit much for my last project, I'm back again > for more > punishment. In this case, I'm looking to broadcast a message > from one client > to other connected clients. I came up with a solution, but I > wanted to run it > by you guys to see whether what I'm doing is The Right Way To > Do Things. > > Here's what I did for my simple prototype that takes any > message received from > a client and sends it to all clients (including the sender): > > ----------- > > factory = Factory() > factory.transports = [] > > class Chatterbox(Protocol): > > def connectionMade(self): > # Make a copy of the new connection's transport and > # stick it somewhere all the Chatterbox instances can > # get to it. > self.factory.transports.append(self.transport) > > def dataReceived(self, data): > # Every time a piece of data comes in, just write it back > # out to all the transports, ergo, clients. > for transport in self.factory.transports: > transport.write(data) > > def connectionLost(self, reason): > # When a client disconnects, delete the copy of their > # transport. Bet this'll be slow w/ lotso clients. > self.factory.transports.remove(self.transport) > > factory.protocol = Chatterbox > reactor.listenTCP(8008, factory) > reactor.run() > > ----------- > > What do you think? Is this a properly twisted way to do > things? Strike anyone > as needlessly inefficient? I'm motivated to make it as fast > as possible, so > despite what Knuth said, any proferred optimizations would be > great to hear. > (On a related note, I intend to put the for loop into a map, > but I haven't > yet embarked on language optimizations, and this question's > more about > Twisted.) > > Also, I'm obviously, and perhaps incorrectly, assuming that > once a transport > object is created, it never gets changed. If it does, the > copy I made won't > be updated -- is this a problem? > > A definite problem with this method is that it sends the data > right back to > the sender, which actually isn't desirable for my > application. Trouble is, to > avoid it I think I'd have to do this... > > def dataReceived(self, data): > for transport in self.factory.transports: > if transport != self.transport: > transport.write(data) > > Seems an inefficient way to do things, as I'd like this > server to handle > thousands of clients. > > Anyway, any help would be appreciated. :-) > > Thanks, > > Steve > > _______________________________________________ > Twisted-Python mailing list > Twisted-Python(a)twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > [View Less]

1 0

[Twisted-Python] Two Suggestions and one Question
by Paul Boehm March 28, 2004

March 28, 2004

1) Interface Based Service Discovery "hey, is there a Service running that provides IStatistics, or can you start one. If so, give me a reference" 2) Move deprecated interfaces into seperate files (makes both reading the source, and the generated documentation more pleasurable), that are (possibly conditionally) imported via from *. 3) How's the status on pyprotocols and logging module integration? regards, paul

1 0

[Twisted-Python] SFTP Server and Cred Example?
by Justin Johnson March 27, 2004

March 27, 2004

Does anyone have an example of how to use twisted.conch.ssh.filetransfer.FileTransferServer in conjunction with cred?

3 3

[Twisted-Python] Multiple Cred Checkers and Avatar Types
by Andy Gayton March 26, 2004

March 26, 2004

Hey all, Is it possible to tie the type of avatar a realm returns to the cred checker which successfully authenticated the user? For example, if the user requesting a IConchUser interface authenticates against the unix password file the realm would know to return a UnixConchUser - if the authenticated against a smbpasswordish like file they realm would return a SmbishConchUser. Is this not the right way to be thinking of cred? Any advice is greatly appreciated, Andy.

1 0

[Twisted-Python] Windows IOCP reactor
by Pavel Pergamenshchik March 26, 2004

March 26, 2004

Support for Windows completion ports is now in Twisted. Unlike win32reactor, IOCP allows a Twisted application to have a nearly unlimited amount of open connections. Performance improvements and process running support will be coming shortly. You can use this reactor by installing it with: from twisted.internet import iocp iocp.install() or by using "--reactor=iocp" parameter to trial or twistd. Suggestions are welcome!

4 8

[Twisted-Python] Creating Winows Service from Twisted (2/2)
by Schneider, Michael March 25, 2004

March 25, 2004

This is the 2nd of two mail messages on creating a Twisted service on Windows. One important thing to note is that the Windows service is standalone, and no python install is required for installation. Thanks Cory, Mike How to use------------- Save this in the moonfallen dir under sandbox along with gen.bat from the previous email 1) run gen.bat 2) cd to subdirectory created 3) install service 4) start service 5) if service does not start, you need to add python packages to the include … [View More]directive. cd to the subdirectory created in the moonfallen directory, edit the include directive in setup.cfg and execute ..\fixup.bat (this bat file) repeat until finished 6) edit include portion of gen.bat, adding the python modules that you added during the fixup step. 7) the next time you run gen.bat, the windows installer should be ready to go (unless you added more python modules in your code since the last build). hope this helps, Thanks again Cory, Mike ---------------------------------------------------------------- Michael Schneider Senior Software Engineering Consultant UGS PLM Solutions - an EDS Company "The Greatest Performance Improvement Is the transitioning from a non-working state to the working state" --------------------- fixup.bat -------------------- rem python twisted service fixup rem drop this file (fixup.bat) into sandbox/moonfallen rem rem rem Have Fun!!!!!!! rem Michael Schneider rem rem rem step to bring up a service rem 1) run tap2ntsvc (see gen.bat file) rem 2) install service rem 3) If service doesn't start, or part of the application rem fails with can't find module ... rem 3a) edit setup.cvf to add the missing python module in the rem include section rem 3b) run this script rem 3c) repeat rem 4) update gen.bat with any python modules you added in step 3 rem rem rem If you are bringing in com code, It is important to realize that rem Microsoft uses different type libs for XP and 2000 rem rem You have 2 choices at this point: rem - use two installers (1 for each platform) rem - create 1 installer rem if you go for 2, just follow the instruction above for each rem installer. rem If you opt for 1 installer here is your steps rem 1) select a "master" platform to create the OS rem 2) run you test cases on this machine against the twisted rem application that you wish to create a servcie. rem This step is necessary to create the COM wrappers in rem wincom32/gen_py/GUI rem 3) follow the steps above to create a service for one platform rem NOTE: to save you time, just go ahead and add the COM wrappers rem in site-packages/win32com/gen_py/{GUID}/{all rem wrappers}.py rem rem 4) go to your other platform (XP or windows} rem 5) run test cases on this platforms rem 6) go to site-packages/win32com rem 7) Notice that the GUID is the same except for the last digit rem {GUID}x1 for windows 2000 rem {GUID}x2 for XP rem 8) zip up the {GUID} dir rem 9) copy this zip to the win32com directory of your "master" rem service build machine. rem 10) unzip, creating to {GUID} dirs, with the only differces being rem the ending digit (x1, or x2) rem 11) Add these to the gen.bat (note: if 2000 is your master rem machine, you may not have sufficient environment variable rem space to pass in all includes, if this is the case rem 11a) rebuild service for master only rem 11b) edit setup.cfg to add com wrappers from other rem system rem 11c) run this script to create a service that contains rem the com wrappers for both platforms rem rem YOU ARE DONE!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! rem cd to service directory under moonfallen dir in sandbox python setup.py py2exe python do_inno_script.py python do_inno.py [View Less]

1 0

[Twisted-Python] Sample Script to create Windows Service for Twisted App (1/2)
by Schneider, Michael March 25, 2004

March 25, 2004

First, A big thanks to Cory for writing the tap2ntsvc, and for helping me understand how it work (as well as proving to me that it can work. In exchange for all his help, I am sending two .bat files in 2 messages. Because of mail filtering, I am going to include the .bat files at the end of the messages rather then attach them. Background: I have a twisted program that leverages WMI, ctypes, and win32 calls. These all need to be included in the exe. Lessons Learned: Microsoft uses … [View More]different type libraries for XP and 2000, the python wrappers are created in the site-packages\win32com\gen_py\{GUID}1 for 2000, and site-packages\win32com\gen_py{GUID}2 for XP. You can create different installers for both, or include them all into the same service (see the comments in fixup.bat in the 2/2 email) Have fun, Mike ---------------------------------------------------------------- Michael Schneider Senior Software Engineering Consultant UGS PLM Solutions - an EDS Company "The Greatest Performance Improvement Is the transitioning from a non-working state to the working state> ------------------ gen.bat--------------------------------------------- rem ------------- BAT file to create Service for twisted app that rem uses WMI python module rem Author: Michael Scheider rem Shadow Author (Cory - he helped me get going, re and proved to me that it could work) rem Cory also wrote the tool to package the system rem in exchange for all his hard work, I am releasing this script rem under the same license as Twisted (free) rem Use this file as a roadmap, but you may need other modules, rem Note: if you are not using WMI, you can comment out the rem %WIN2000_WMI_MODULES% and the %WINXP_WMI_MODULES% from rem tap2ntsvc command line rem Twisted Functionality/Python Modules used rem rem WMI- process and disk info rem ctypes - memory info rem twisted as web server rem twisted xml-rpc rem cgi - perl and python rm rpy files rem Special thanks to Cory to get this going rem Version Info rem python2.3.3 rem win32all build 163 rem py2exe version 1.4.2 rem inno - latest version rem Lessons Learned rem rem WMI stashes the com wrappers in site- rem rem Windows 2000 wrapper location= site-packages\win32com\gen_py\565783C6-CB41-11D1-8B02-00600806D9B6x0x1x1 rem Windows XP wrapper location = site-packages\win32com\gen_py\565783C6-CB41-11D1-8B02-00600806D9B6x0x1x2 rem rem NOTE: the numbers are the same except for 2000 uses 1 and XP uses 2 rem if you want your installer to run on both you will need rem to copy the directory from the other machine and include rem both directories rem ------------------- WMI modules ------------------------ rem the directory rem set WINXP_WMI_MODULES=win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x2.ISWbemMethod,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x2.ISWbemMethodSet,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x2.ISWbemObjectPath,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x2.ISWbemObjectSet,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x2.ISWbemProperty,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x2.ISWbemPropertySet,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x2.ISWbemServicesEx,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x2.SWbemMethod,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x2.SWbemMethodSet,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x2.SWbemObjectEx,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x2.SWbemObjectPath,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x2.SWbemObjectSet,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x2.SWbemProperty,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x2.SWbemPropertySet,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x2.SWbemServicesEx,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x2.__init__,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x2.ISWbemObjectEx,wmi set WIN2000_WMI_MODULES=win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x1.ISWbemMethod,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x1.ISWbemMethodSet,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x1.ISWbemObject,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x1.ISWbemObjectPath,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x1.ISWbemObjectSet,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x1.ISWbemProperty,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x1.ISWbemPropertySet,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x1.ISWbemServices,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x1.SWbemMethod,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x1.SWbemMethodSet,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x1.SWbemObject,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x1.SWbemObjectPath,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x1.SWbemObjectSet,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x1.SWbemProperty,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x1.SWbemPropertySet,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x1.SWbemServices,win32com.gen_py.565783C6-CB41-11D1-8B02-00600806D9B6x0x1x1.__init__,wmi python tap2ntsvc --include twisted.web.server,twisted.web.resource,twisted.web.static,twisted.web.script,gridnode.webinfo,gridnode.JobRunner,gridnode.appconfig,gridnode.sendmail,twisted.internet.win32eventreactor,%WIN2000_WMI_MODULES% -r win32 d:\grid\gridnoded.tap [View Less]

1 0

[Twisted-Python] woven
by Kölle Wolfgang March 25, 2004

March 25, 2004

hello, i am a new python programmer and i have a problem. i hope someone can help me. i have started my webserver on port 80, then i will get a url('//matlink-so-su5/Weiterleiten/Index.asp?DatumZeit=02.02.2004%22:00:00&TA=Meine%20Textadresse&langtext=MeinLangtext&Kommentar=mein%20Kommentar'). i want split this url and i should insert in this template and hand this to the HTTP server. This side should be the starting side for the user. i work with python 2.2 and twisted 1.1.1 thank … [View More]you <html> <head> <title>Unbenanntes Dokument</title> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> <style type="text/css">  </style> <link href="test.css" rel="stylesheet" type="text/css"> <style type="text/css">  </style> </head> <body> <form name="form1" method="post" action="test_file_gen.rpy"> <table width="100%" border="1" cellspacing="0" cellpadding="5"> <tr bgcolor="#FFFFCC"> <th colspan="4" valign="bottom" class="arial16bold" scope="row"><div align="left"> <table width="100%" border="0" cellspacing="0" cellpadding="0"> <tr> <td valign="bottom" class="arial16boldÜberschrift1"><div align="left"></div> <div align="left"></div> <div align="left" class="Stil1"></div> <div align="left" class="Stil1"></div> <div align="left" class="Stil1"></div> <div align="left" class="Stil1"></div> <div align="left" class="arial16boldÜberschrift1 Stil1">Einträge bearbeiten</div></td> <td><div align="right"><img src="TWKc60px.gif" width="61" height="60"></div></td> </tr> </table> </div> </th> </tr> <tr> <th width="24%" bgcolor="#FFFFCC" class="arial12bold" scope="row"><div align="left"> <p>Datum</p> </div></th> <td width="45%" bgcolor="#CCCC99">   <input name="txtDatum" type="text" class="arial10" id="txtDatum" value="02.02.2004" size="15" maxlength="10"></td> <td width="16%" bgcolor="#CCCC99" class="arial12bold">AV tel. verständigt</td> <td width="15%" bgcolor="#CCCC99" class="arial12bold">   <select name="popupAVVerständigt" class="arial10" id="popupAVVerständigt"> <option>ja/nein</option> <option>ja</option> <option>nein</option> </select></td> </tr> <tr> <th bgcolor="#FFFFCC" class="arial12bold" scope="row"><div align="left">Zeit</div></th> <td bgcolor="#CCCC99">   <input name="txtZeit" type="text" class="arial10" id="txtZeit" value="22:00:00" size="10">  </td><td bgcolor="#CCCC99" class="arial12bold">Einsatz erforderlich </td> <td bgcolor="#CCCC99" class="arial12bold">   <select name="popupEinsatzErforderlich" class="arial10" id="popupEinsatzErforderlich"> <option>ja/nein</option> <option>ja</option> <option>nein</option> </select></td> </tr> <tr> <th bgcolor="#FFFFCC" class="arial12bold" scope="row"><div align="left">Strukturplatz</div></th> <td bgcolor="#CCCC99">   <input name="txtStrukturplatz" type="text" id="<% txtStrukturplatz %>" value=""></td> <td colspan="2" bgcolor="#CCCC99"> </td> </tr> <tr> <th bgcolor="#FFFFCC" class="arial12bold" scope="row"><div align="left">Erzeugungsbereich</div></th> <td bgcolor="#CCCC99">   <input name="txtErzeugungsbereich" type="text" id="txtErzeugungsbereich" value=""></td> <td colspan="2" bgcolor="#CCCC99"> </td> </tr> <tr> <th bgcolor="#FFFFCC" class="arial12bold" scope="row"><div align="left">Kraftwerksbereich</div></th> <td bgcolor="#CCCC99">   <input name="txtKraftwerksbereich" type="text" id="txtKraftwerksbereich" value=""></td> <td colspan="2" bgcolor="#CCCC99"> </td> </tr> <tr> <th bgcolor="#FFFFCC" class="arial12bold" scope="row"><div align="left">Unterstandort</div></th> <td bgcolor="#CCCC99">   <input name="txtUnterstandort" type="text" id="txtUnterstandort" value=""></td> <td colspan="2" bgcolor="#CCCC99"> </td> </tr> <tr> <th bgcolor="#FFFFCC" class="arial12bold" scope="row"><div align="left"></div> <div align="left">Technologische Adresse </div> </th> <td bgcolor="#CCCC99">   <input name="txtTechnologischeAdresse" type="text" id="txtTechnologischeAdresse" value="Meine Textadresse"></td> <td colspan="2" bgcolor="#CCCC99"> </td> </tr> <tr> <th bgcolor="#FFFFCC" class="arial12bold" scope="row"><div align="left">Meldetext</div></th> <td colspan="3" bgcolor="#CCCC99">   <input name="txtMeldetext" type="text" id="txtMeldetext" value="MeinLangtext" size="163"></td> </tr> <tr> <th bgcolor="#FFFFCC" class="arial12bold" scope="row"><div align="left">Weitergeleitet von </div></th> <td bgcolor="#CCCC99">   <select name="popupWeitergeleitetVon" size="1" id="popupWeitergeleitetVon"> <option>mike thin</option> </select></td> <td colspan="2" bgcolor="#CCCC99"> </td> </tr> <tr> <th bgcolor="#FFFFCC" class="arial12bold" scope="row"><div align="left">Kommentar Warte Mitarbeiter </div></th> <td colspan="3" bgcolor="#CCCC99">   <input name="txtKommentarWarteMitarbeiter" type="text" id="txtKommentarWarteMitarbeiter" value="mein Kommentar" size="163"></td> </tr> <tr> <th bgcolor="#FFFFCC" class="arial12bold" scope="row"><div align="left">Zur Kenntnis genommen von </div></th> <td bgcolor="#CCCC99">   <select name="select"> <option>king mike</option> </select></td> <td colspan="2" bgcolor="#CCCC99"> </td> </tr> <tr> <th bgcolor="#FFFFCC" class="arial12bold" scope="row"><div align="left">Kommentar Anlagenverantwortlichen </div></th> <td colspan="3" bgcolor="#CCCC99">   <input name="txtKommentarAnlagenverantwortlichen" type="text" id="txtKommentarAnlagenverantwortlichen" value="" size="163"></td> </tr> <tr> <th rowspan="2" bgcolor="#FFFFCC" class="arial12bold" scope="row"><div align="left"></div> <div align="left"></div></th> <td bgcolor="#CCCC99"> </td> <td colspan="2" bgcolor="#CCCC99"> </td> </tr> <tr> <td bgcolor="#CCCC99"> </td> <td colspan="2" bgcolor="#CCCC99"> </td> </tr> <tr> <th class="arial12bold" scope="row"><div align="left"> <input name="SubmitUpdate" type="submit" class="arial12bold" id="SubmitUpdate" value="update"> <input name="SubmitDelete" type="submit" class="arial12bold" id="SubmitDelete" value="delete"> </div></th> <td bgcolor="#CCCC99"> </td> <td colspan="2" bgcolor="#CCCC99"> </td> </tr> </table> <p> </p> </form> <hr align="left" width="100%"> <table width="100%" border="0" cellspacing="0" cellpadding="0"> <tr> <td width="25%"><a href="http://www.google.at">Übersicht</a></td> <td width="22%"><a href="http://www.google.at">neuer Eintrag </a></td> <td width="28%"><a href="http://www.google.at">neu/bearbeiten</a></td> <td width="25%"><a href="http://www.google.at">IMS-Auftrag</a></td> </tr> </table> <hr align="left" width="100%"> </body> </html> [View Less]

2 1

[Twisted-Python] "Scripting Language" My Arse: Using Python for Voice over IP (Shtoom)
by Eugen Leitl March 24, 2004

March 24, 2004

http://www.python.org/pycon/dc2004/papers/6/paper.txt "Scripting Language" My Arse: Using Python for Voice over IP ============================================================ [This is the paper, not the talk. The talk is yet to come] [third draft] Abstract -------- A common complaint made of Python is that it is not suitable for serious application development, and is only suitable for "scripting" or "prototyping" tasks. The Shtoom toolkit (http://divmod.org/Home/Projects/Shtoom) is a … [View More]

1 0

[Twisted-Python] JellyToNode and boolean types
by Graham Clinch March 23, 2004

March 23, 2004

I've just started having problems with JellyToNode dying on boolean types (Bug tracker issue 363), but it doesn't seem to have moved along much recently, even though there is a seemingly accepted patch; are there any plans to apply the patch? Thanks, Graham

1 0

Re: [Twisted-Python] Threading examples...
by William McLendon March 23, 2004

March 23, 2004

Thanks, So, now my example looks like this: import time from twisted.internet import reactor def aSillyBlockingMethod(x): time.sleep(2) print x reactor.callFromThread(reactor.stop) # run method in thread reactor.callInThread(aSillyBlockingMethod, "2 seconds have passed") It still just hangs... I have to go kill it from another window after getting its pid from ps. Am I doing the reactor.stop thing correctly? Basically, I have an application that talks via a http interface … [View More]to a user and I need to spawn off and do stuff from time to time. I'm still learning how all the reactor eventloops and whatnot work. I ran into a problem where I'm trying to run something via twisted.internet.protocol.ProcessProtocol and serve its results up from within a reactor.listentTCP() page handler ... but it blew up. So, I'm thinking that I could maybe just fork it off and run that way. I've been trying to figure out how the reactors() work and the threading and whatnot the last few days but am kind of finding little help in the docs and google that is at my level of understanding... seems most of the stuff is snippets and pieces that assume you already see the big picture :-( Any help here is greatly appreciated ;-) Thanks, -William >From: Andrew Bennetts <andrew-twisted(a)puzzling.org> >Reply-To: twisted-python(a)twistedmatrix.com >To: twisted-python(a)twistedmatrix.com >Subject: Re: [Twisted-Python] Threading examples... >Date: Wed, 24 Mar 2004 09:30:13 +1100 > >On Tue, Mar 23, 2004 at 12:19:01PM -0700, William McLendon wrote: > > Hi, > > > > I'm trying out some of the threading examples from the web and am >getting > > some a lockup at the end. > > > > Here's the code I'm playing with, I pulled it straight from the >examples: > > > > (source: > > http://twisted.sourceforge.net/TwistedDocs-1.2.0/howto/threading.html) > > > > > > #!/usr/bin/env python > > import time > > from twisted.internet import reactor > > > > def aSillyBlockingMethod(x): > > import time > > time.sleep(2) > > print x > > > > # run method in thread > > reactor.callInThread(aSillyBlockingMethod, "2 seconds have passed") > >This example doesn't call to reactor.stop(), so it never stops. Put a call >to "reactor.callFromThread(reactor.stop)" after the print statement, for >example, and it should shutdown cleanly. > > > Is there something I need to do that will make the thread go away? Are > > there some more examples of threading in Twisted-Python available? > >The only example I can think of is the implementation of >twisted.enterprise.adbapi, which presents an asynchronous interface to >blocking DB-API modules by running it in threads. > >-Andrew. > > >_______________________________________________ >Twisted-Python mailing list >Twisted-Python(a)twistedmatrix.com >http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python _________________________________________________________________ Is your PC infected? Get a FREE online computer virus scan from McAfee® Security. http://clinic.mcafee.com/clinic/ibuy/campaign.asp?cid=3963 [View Less]

2 1

Re: [Twisted-Python] Threading examples...
by William McLendon March 23, 2004

March 23, 2004

Hmmm... I tried something pretty close to that (though I have no callback called writeRequestAndFinish() in my example and am not sure what it's supposed to do)... but running the threads.deferToThread() routine still leaves me with the problem that the thread doesn't die. I can't kill the app without going and manually killing it via pid. Is there anything that I can do so that when aSillyBlockingMethod() is finished it just exits? Thanks, -William >From: stephan <… [View More]

1 0

[Twisted-Python] Threading examples...
by William McLendon March 23, 2004

March 23, 2004

Hi, I'm trying out some of the threading examples from the web and am getting some a lockup at the end. Here's the code I'm playing with, I pulled it straight from the examples: (source: http://twisted.sourceforge.net/TwistedDocs-1.2.0/howto/threading.html) #!/usr/bin/env python import time from twisted.internet import reactor def aSillyBlockingMethod(x): import time time.sleep(2) print x # run method in thread reactor.callInThread(aSillyBlockingMethod, "2 seconds have … [View More]

3 2

[Twisted-Python] Isolating ISFTPServer in Conch's SFTP
by Andy Gayton March 23, 2004

March 23, 2004

I'm looking to create a filesystem backend for twisted that could be used by any filelike twisted application (sftp, webdav etc). I've got a proof of concept filesystem interface working - IFileSystem .. not the best of names ;), which I'm about to try and sftp into. ISFTPServer is a nice clean interface which it'd be easy to write an adapter for from my IFileSystem - but its implementation is bolted into FileTransferServer along with a bunch of protocol stuff .. I had been vaguely … [View More]

3 6