Twisted February 2010

twisted@python.org

75 participants
69 discussions

[Twisted-Python] Twisted Python 0.7.0 Prerelease Testing
by Glyph Lefkowitz 05 Aug '21

05 Aug '21

hi there, folks: I'd really like to release 0.7.0 but I would like it to be at least a little bit tested before I do so. Could those of you with CVS trees check everything out and see if it performs as advertised? Deeper bugs than that will have to wait for the next release, but I'd at least like to know if it works for someone other than me. Thanks. ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://www.twistedmatrix.com/~glyph/

4 5

[Twisted-Python] Real Viagra Professional - for real sex pros
by Tamika 01 May '19

01 May '19

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <center> <a href="http://rqf.entercompassion.com"><img src="http://mediapix.ru/pics/685207b3dd437eff7f15c0744b5c1d3a.gif"> <title> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <link rel="stylesheet" href="http://gfx8.nnt.com/mail/12.0.1187.0824/positioning.css" /> </head> <body style="overflow:auto;"> <div id="PrintShellToolbar"> <div class="ContainerPadding"> <span class="FloatRight"> <a href= > <img alt="" src="http://gfx1.nfi.com/mail/w2/pr02/ltr/glyph_close_rest.gif" /> <Close window> <a href= > <img alt="" src="http://gfx2.czb.com/mail/w2/pr02/ltr/i_print.gif" /> <Print> </a> </span> </div> </div> <div> <div class="BorderContainerOutline ContainerPadding"> <div class="ClearBoth"> <span class="HeaderData" style="font-size:large;"> <TuPostal.com Cambia de nombre?> </span> </div> <div class="ClearBoth"> <span class="HeaderLabel"> <From:> </span> <span class="HeaderData"> <b><TuPostal.com></b> </span> </div> <div class="ClearBoth"> <span class="HeaderLabel"> <Sent:> </span> <span class="HeaderData"> <div class="ClearBoth"> <span class="HeaderLabel"> To: </span> <span class="HeaderData"> Mikelelliott (mikelelliott(a)rlo.com) </span> </div> <br /> </div> </div> <div class="MessageBody" style=""> <br /> <div id="MessageBodyText" class="ExternalClass" style="border: 1px solid #fff ;"> <pre>Hola!<br> <br>Tupostal es ahora parte de Sonico!<br> <br>Para conocer nuestro nuevo sitio, haz clic en el siguiente link:<br><a href='http://www.gwpcom/index1.php' target='_blank'>http://www.bjncom/index1.php</a><br> <br>Sonico es un sitio donde puedes...<br>- Guardar fotos ilimitadas<br>- Re encontrarte con viejos amigos<br>- Recibir novedades de tus amigos.<br> <br>Te Esperamos! Ya somos mas de 21 millones de usuarios!!<br> <br>Sonico - Amigos Conectados<br> <br> <br>PD: Ahora tambien podras guardar tus postales<br> <br> <br>------------------------------------------<br>Haz recibido este email porque te encuentras registrado en fdm.com.<br>Si no deseas recibir mas mensajes como este, por favor haz clic en el siguiente link:<br><a href='http://www.cehcom/unsubscribe.php?type=2' target='_blank'>http://www.fyocom/unsubscribe.php?type=2</a><br> <br></pre> </div> </div> </div> </body> </html>

8 10

[Twisted-Python] regarding xml elements
by Raj kumar 02 Oct '13

02 Oct '13

Hi, I have an xml file in my application, I have created an element using Example goes like this......... document.createElement("abc") and i appeneded it by using append() method. But how i can reflect this change to my xml file? and one more thing is i want to create element with some other parameters.... like............. <abc m=" " n=" "> </abc> and i have m and n values as strings with me. can anybody help me to create this element and write it to the existing xml file as a child of an existing element? Thanks in advance.. Raj.. Forgot the famous last words? Access your message archive online at http://in.messenger.yahoo.com/webmessengerpromo.php

9 14

[Twisted-Python] Get IP of connecting client in XMLRPC server
by Remy C. Cool 28 Apr '13

28 Apr '13

Hello, On 18 May 2004, the following message was posted to this mailinglist: Jp Calderone exarkun at divmod.com wrote: >Daniel Newton wrote: > > I have a simple XML-PRC server similar to the example below: > > from twisted.web import xmlrpc, server > > class Example(xmlrpc.XMLRPC): > """An example object to be published.""" > > def xmlrpc_add(self, a, b): > """Return sum of arguments.""" > return a + b def xmlrpc_whatIsMyAddress(self): return self.transport.getPeer().host > > if __name__ == '__main__': > from twisted.internet import reactor > r = Example() > reactor.listenTCP(7080, server.Site(r)) > reactor.run() > > I want to be able to get the address of the client that calls the xmlrpc > method can anyone help me with this? This solution didn't work because 'transport' isn't a property of the Example class. I'm currently in the process of changing from a customized SimpleXMLRPCServer to a twisted XMLRPC server solution and I need to insert the client IP into the attributes passed to the called xmlrpc method. Anyone who knows the answer and is willing to share the info? Regards, Remy

3 3

[Twisted-Python] http.Request and JSON
by Vincent Gulinao 25 Jan '12

25 Jan '12

How do you read JSON content from a POST using http.Request. I peeked on args and it's empty. TIA.

8 10

[Twisted-Python] multiple versions of twisted on same machine
by Pet 27 Apr '10

27 Apr '10

Hi, I'd like to upgrade twisted to the newest version and obviously first test if my application works fine with new version. What is the best (safest) way to do it? Thanks for help, Pet

6 10

[Twisted-Python] web2 -> web ToDo List
by ssteinerX＠gmail.com 09 Mar '10

09 Mar '10

While I see tickets in the tracker for web2 and others for web, I couldn't find a 'cherry pick' list of things to be moved from web2 back into web. Is there one? Thanks, S

5 6

[Twisted-Python] IMAP4 Client extrange behavior
by César García 04 Mar '10

04 Mar '10

Hello all, I have this imap4 client http://www.pastebin.com/m4e387f1a most of the code is barrowed from a IRC friend, bob_f , it works fine but as soon as more that 1 email arrrives to the inbox, it only prints the first new messages but not the others, I wonder what am I doing wrong Thanks a lot Here is the code: #!/usr/bin/env python #coding=utf-8 """ Client de IMAP4 que descarga contenido del INBOX de una cuenta en especifico para luego extraer el numero de telefono que debe venir en los correos enviados """ import StringIO import sys from Config import retCredentials from twisted.internet.task import LoopingCall from twisted.internet import protocol from twisted.internet import defer from twisted.mail import imap4 from twisted.python import util from twisted.python import log from twisted.internet import reactor debug = 1 class IMAP4Client(imap4.IMAP4Client): def serverGreeting(self,caps): """ Metodo llamado cuando el servidor contesta """ if debug: print "On serverGreeting" self.serverCapabilities = caps if self.greetDeferred is not None: d, self.greetDeferred = self.greetDeferred, None d.addCallback(self.cbLogin) d.callback(self) def cbLogin(self, proto): """ Callback to IMAP login """ if debug: print "On Login" login = self.login(self.factory.username, self.factory.password) login.addCallback(self.startPolling) def startPolling(self, proto): """ Callback to poll every x seconds """ call = LoopingCall(self.selectMailbox,mailbox="INBOX") call.start(20, now=True) def selectMailbox(self, mailbox): """ Select the mailbox to examin """ if debug: print "On selectMailbox" mailbox = self.factory.mailbox return self.select(mailbox).addCallback(self.cbSelectSuccess) def cbSelectSuccess(self, selected): """ Examine the INBOX mailbox for new mails """ if debug: print "On cbSelectSuccess" self.messageCount = selected['EXISTS'] print "Messages: ", self.messageCount unseen = selected['EXISTS'] - selected['RECENT'] if selected['RECENT'] == 0: print "No new messages" return return self.fetchMessage("%s:*" % (unseen) ).addCallback(self.cbProcMessages) def cbProcMessages(self,messages): print messages class IMAP4ClientFactory(protocol.ClientFactory): protocol = IMAP4Client def __init__(self, username, password, onConn): self.username = username self.password = password self.mailbox = 'INBOX' self.onConn = onConn def buildProtocol(self,addr): if debug: print "On buildProtocol" p = self.protocol() p.factory = self p.greetDeferred = self.onConn auth = imap4.CramMD5ClientAuthenticator(self.username) p.registerAuthenticator(auth) return p def clientConectionFailed(self, connector, reason): d, self.onConn = self.onConn, None d.errback(reason) def ebConnection(reason): log.startLogging(sys.stdout) log.err(reason) reactor.stop() PORT = 143 RESULT = "INBOX" def main(): credentials = retCredentials() hostname = credentials['server'] username = credentials['mailbox'] password = util.getPassword('IMAP4 Password: ') onConn = defer.Deferred( ).addErrback(ebConnection ) factory = IMAP4ClientFactory(username, password, onConn) reactor.connectTCP(hostname, PORT, factory) reactor.run() if __name__ == "__main__": main() -- http://celord.blogspot.com/

1 1

[Twisted-Python] debugging a memory leak
by Alec Matusis 04 Mar '10

04 Mar '10

I modified a tolerably leaking (about 40MB/day) Twisted server: when the new code was pushed, the memory leak became catastrophic (about 100MB/hr). We could tolerate 40MB/day, but the new code needs to be debugged. First, I reverted to the old version, that is leaking 40MB/day (The leak rate is actually proportional to the number of new connections per second, (which correlates with the CPU utilization of the process): if CPU as measured by $top jumps to >90%, the leak can accelerate to 50MB/hr) I took two steps to debug the leak: 1) Using guppy/heapy via manhole: >>> hp = hpy() >>> h = hp.heap() >>> h Partition of a set of 1157084 objects. Total size = 140911144 bytes. Index Count % Size % Cumulative % Kind (class / dict of class) 0 785798 68 48463680 34 48463680 34 str 1 7357 1 24660664 18 73124344 52 dict of service.TagSession 2 11735 1 12298280 9 85422624 61 dict of twisted.internet.base.DelayedCall 3 7377 1 7731096 5 93153720 66 dict of twisted.internet.tcp.Server 4 7375 1 7729000 5 100882720 72 dict of protocols.TagProtocol 5 30925 3 7174600 5 108057320 77 __builtin__.set 6 9193 1 6373336 5 114430656 81 dict (no owner) 7 15557 1 3396904 2 117827560 84 list 8 44833 4 3227976 2 121055536 86 types.BuiltinFunctionType 9 38142 3 3051360 2 124106896 88 types.MethodType <328 more rows. Type e.g. '_.more' to view.> Note that the total size of all objects is 140911144 bytes The 1st, 3d and 4th items in this list show the actual number of connected clients. I wait for about 48 hrs, and then execute the same command, and I see approximately the same Total size, of 130MB. So the total size that Heapy sees via the manhole is stable, fluctuating around 140MB. The problem is that the total RSS size of the process visible by the OS is much larger, it is 871680KB = 851MB: $ps -o pid,vsz,rss,sz,size -p 11236 PID VSZ RSS SZ SZ 11236 1303180 871680 325795 1174388 It is this total RSS size that keeps leaking at 40MB per day. As far as I understand, this means that this is not the problem with purely Python code, since Heapy shows that the total size of all Python objects is more or less constant. Is this a correct assumption? 2) So now I turn to valgrind. I am no expert in using valgrind, so what I did was based only on general logic/rough guesses. Since I cannot run this under valgrind on a production machine due to performance reasons, I recompile python on the staging machine: ./configure --enable-shared --without-pymalloc --prefix=/nail/encap/python-2.6.4-valgrind I then follow the instructions in http://svn.python.org/projects/python/trunk/Misc/README.valgrind Then I run twistd process like this: valgrind --tool=memcheck --suppressions=/nail/sys/src/Python-2.6.4/Misc/valgrind-python.supp --leak-check=full --log-file=/tmp/valgrind.log /usr/local/bin/twistd --no_save --reactor=epoll --pidfile=logs/tagserv.pid --logfile=logs/tagserv.log --python=tagserv.py The memory for the process shown by the OS is now 5x normal, and the performance is about 5x worse, since it's running inside valgrind's synthetic CPU. Because this is done on the staging box where I cannot accurately reproduce the real load, the memory leaks from simulated load seen by $ps -o rss are pretty small, about 1 to 10MB. Still, I am interested in finding out what they are. Now I encounter the problems with my understanding of how to use valgrind. This may not be the appropriate list for this, but perhaps someone could recognize the problem. When I start the server, about 240 lines is written to valgrind log file, --log-file=/tmp/valgrind.log When I shut it down, it adds another 100 lines. No matter what I do in between, it always results in the log file with exactly 343 lines. I can have server runs with a leak of 1MB, or with 10MB, but in the end, I get pretty much the same log file. Moreover, when look for all lost memory reports: $grep lost valgrind.log.1 17,352 bytes in 31 blocks are possibly lost in loss record 49 of 62 203,312 bytes in 478 blocks are possibly lost in loss record 57 of 62 definitely lost: 0 bytes in 0 blocks. possibly lost: 220,664 bytes in 509 blocks. 64 bytes in 2 blocks are definitely lost in loss record 12 of 63 17,352 bytes in 31 blocks are possibly lost in loss record 50 of 63 203,824 bytes in 479 blocks are possibly lost in loss record 58 of 63 definitely lost: 64 bytes in 2 blocks. possibly lost: 221,176 bytes in 510 blocks. 47 bytes in 1 blocks are definitely lost in loss record 8 of 63 128 bytes in 4 blocks are definitely lost in loss record 16 of 63 584 (104 direct, 480 indirect) bytes in 2 blocks are definitely lost in loss record 26 of 63 1,008 bytes in 6 blocks are definitely lost in loss record 31 of 63 22,296 bytes in 41 blocks are possibly lost in loss record 50 of 63 183,368 bytes in 381 blocks are possibly lost in loss record 59 of 63 definitely lost: 1,287 bytes in 13 blocks. indirectly lost: 480 bytes in 20 blocks. possibly lost: 205,664 bytes in 422 blocks. If I add up all those numbers, I get less than 1MB. How do I track down the 10MB leak? Are there any alternative strategies in finding this leak?

6 11

[Twisted-Python] Is AMP secure enough for the internet?
by Attila Nagy 01 Mar '10

01 Mar '10

Hello, I'm planning a data collector gateway and wondering whether Twisted's AMP would be good for the task. AMP seems to be a good fit for the job, but I'm not sure about the security. I make the client side too, but I won't operate it, bad guys can take over that side and I want to protect my side. What would I like to do: 1. authenticate and authorize connecting clients with their SSL certificates 2. securely transfer arbitrary (binary and json) data from and to the clients (both the server and client would be twisted) 3. protect the server from malicious clients I have some concerns about all three. For the first, is this OK now: http://twistedmatrix.com/pipermail/twisted-python/2007-August/015926.html ? For the second: the server should not be affected by the data which the client sends. I mean exploiting bugs in the data paths and limited protection from DoS, like abusing blocking pieces of code and therefore halting the reactor, or preventing memory overflow (if it sends 3TBs of data, it shouldn't be queued up in RAM, I should be in control about what happens in this case, or twisted should be clear measures for these) etc... And by the third I mean the above, plus for example if I have only one command defined for AMP with one string argument then no matter what happens, the client should only access that function with that argument and no other part of the program. How do you feel, is Twisted and AMP a good choice for that? Thanks,

5 5

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

Twisted February 2010