Mailman 3 June 2009 - Twisted

[Twisted-Python] Twisted Python 0.7.0 Prerelease Testing
by Glyph Lefkowitz Aug. 5, 2021

Aug. 5, 2021

hi there, folks: I'd really like to release 0.7.0 but I would like it to be at least a little bit tested before I do so. Could those of you with CVS trees check everything out and see if it performs as advertised? Deeper bugs than that will have to wait for the next release, but I'd at least like to know if it works for someone other than me. Thanks. ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |… [View More]

4 5

[Twisted-Python] Real Viagra Professional - for real sex pros
by Tamika May 1, 2019

May 1, 2019

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <center> <a href="http://rqf.entercompassion.com"><img src="http://mediapix.ru/pics/685207b3dd437eff7f15c0744b5c1d3a.gif"> <title> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <link rel="stylesheet" href="http://gfx8.nnt.com/mail/12.0.1187.0824/positioning.css" /> </head> <body style="overflow:auto;"… [View More]> <div id="PrintShellToolbar"> <div class="ContainerPadding"> <span class="FloatRight"> <a href= > <img alt="" src="http://gfx1.nfi.com/mail/w2/pr02/ltr/glyph_close_rest.gif" /> <Close window> <a href= > <img alt="" src="http://gfx2.czb.com/mail/w2/pr02/ltr/i_print.gif" /> <Print> </a> </span> </div> </div> <div> <div class="BorderContainerOutline ContainerPadding"> <div class="ClearBoth"> <span class="HeaderData" style="font-size:large;"> <TuPostal.com Cambia de nombre?> </span> </div> <div class="ClearBoth"> <span class="HeaderLabel"> <From:> </span> <span class="HeaderData"> <b><TuPostal.com></b> </span> </div> <div class="ClearBoth"> <span class="HeaderLabel"> <Sent:> </span> <span class="HeaderData"> <div class="ClearBoth"> <span class="HeaderLabel"> To: </span> <span class="HeaderData"> Mikelelliott (mikelelliott(a)rlo.com) </span> </div> <br /> </div> </div> <div class="MessageBody" style=""> <br /> <div id="MessageBodyText" class="ExternalClass" style="border: 1px solid #fff ;"> <pre>Hola!<br> <br>Tupostal es ahora parte de Sonico!<br> <br>Para conocer nuestro nuevo sitio, haz clic en el siguiente link:<br><a href='http://www.gwpcom/index1.php' target='_blank'>http://www.bjncom/index1.php</a><br> <br>Sonico es un sitio donde puedes...<br>- Guardar fotos ilimitadas<br>- Re encontrarte con viejos amigos<br>- Recibir novedades de tus amigos.<br> <br>Te Esperamos! Ya somos mas de 21 millones de usuarios!!<br> <br>Sonico - Amigos Conectados<br> <br> <br>PD: Ahora tambien podras guardar tus postales<br> <br> <br>------------------------------------------<br>Haz recibido este email porque te encuentras registrado en fdm.com.<br>Si no deseas recibir mas mensajes como este, por favor haz clic en el siguiente link:<br><a href='http://www.cehcom/unsubscribe.php?type=2' target='_blank'>http://www.fyocom/unsubscribe.php?type=2</a><br> <br></pre> </div> </div> </div> </body> </html> [View Less]

8 10

[Twisted-Python] regarding xml elements
by Raj kumar Oct. 2, 2013

Oct. 2, 2013

Hi, I have an xml file in my application, I have created an element using Example goes like this......... document.createElement("abc") and i appeneded it by using append() method. But how i can reflect this change to my xml file? and one more thing is i want to create element with some other parameters.... like............. <abc m=" " n=" "> </abc> and i have m and n values as strings with me. can anybody help me to create this element and write it to the existing xml … [View More]

9 14

[Twisted-Python] Get IP of connecting client in XMLRPC server
by Remy C. Cool April 28, 2013

April 28, 2013

Hello, On 18 May 2004, the following message was posted to this mailinglist: Jp Calderone exarkun at divmod.com wrote: >Daniel Newton wrote: > > I have a simple XML-PRC server similar to the example below: > > from twisted.web import xmlrpc, server > > class Example(xmlrpc.XMLRPC): > """An example object to be published.""" > > def xmlrpc_add(self, a, b): > """Return sum of arguments.""" > return a + b def … [View More]

3 3

[Twisted-Python] http.Request and JSON
by Vincent Gulinao Jan. 25, 2012

Jan. 25, 2012

How do you read JSON content from a POST using http.Request. I peeked on args and it's empty. TIA.

8 10

[Twisted-Python] An XMPP client with SSL support
by Pedro Sanchez Sept. 18, 2009

Sept. 18, 2009

Hello, I have a sample XMPP client with the following structure: from twisted.words.protocols.jabber import client, jid from twisted.internet import reactor ... factory = client.basicClientFactory(myJid, myPasswd) reactor.connectTCP(host,5222,factory) reactor.run() It works well, I can connect to the XMPP server. But I would appreciate if some one could give me some hints on what's required to get the client to connect using a secure SSL link. Thanks, -- Pedro

3 3

[Twisted-Python] untwisting twistd
by Jared Gisin July 8, 2009

July 8, 2009

I am currently working on writing an application using the twisted framework. The problem that I have is that I do not want to use twistd. In fact, I find twistd to be inappropriate as it defaults pidfile and logfile to something specific to twisted, not my application, and it seem completely silly to have to pass in -pidfile and -logfile params when stating it up. Those parameters should be specified as part of the Application object itself, and should allow for CLI override. Further, as I … [View More]

7 13

[Twisted-Python] buildProtocol never called
by travis+ml-twisted＠subspacefield.org July 5, 2009

July 5, 2009

Hello, First let me introduce a piece of software I have written using Twisted: it's called dfd_keeper[1], and is a BSD/pf port of a generic concept I invented called the dynamic firewall daemon[2]. [1] http://www.subspacefield.org/security/dfd_keeper/ [2] http://www.subspacefield.org/security/dfd/ If you like the concept, please promote the idea; I think it's neat but there hasn't exactly been a groundswell of interest in it. In essence, it allows you to netcat or telnet to the daemon … [View More]

3 6

Re: [Twisted-Python] Need working examples of imap4 client.
by Pywinder Singh July 2, 2009

July 2, 2009

Phil, Much obliged. Was able to isolate and confirm that the real issue is that login is hanging for some reason, and causing the timeout message. I suspect SSL/TLS issues, which at least gives me a decided place to start investigating. Thanks again for the code, it was sanity inducing to say the least. ;) Phil Mayers wrote: > Pywinder Singh wrote: > > > > Ideally, I'd love to see a snipped which is able to log into an imap > > server and gets a list of mailboxes. If … [View More]

2 3

[Twisted-Python] eventlet 0.8.11 released; now with twisted support
by Denis Bilenko June 30, 2009

June 30, 2009

Some of the changes from 0.8: Eventlet can now run on top of twisted reactor. Twisted-based hub is enabled automatically if twisted.internet.reactor is imported. It is also possible to "embed" eventlet into a twisted application via eventlet.twistedutil.join_reactor. See the examples for details. http://bitbucket.org/denis/eventlet/src/tip/examples/ A new package, eventlet.twistedutil, is added that makes integration of twisted and eventlet easier. It has block_on function that allows to … [View More]wait for a Deferred to fire and it wraps twisted's Protocol in a synchronous interface. This is similar to and is inspired by Christopher Armstrong's corotwine library. Thanks to Dan Pascu for reviewing the package. Another new package, eventlet.green, was added to provide some of the standard modules that are fixed not to block other greenlets. This is an alternative to monkey-patching the socket, which is impossible to do if you are running twisted reactor. The package includes socket, httplib, urllib2. Much of the core functionality has been refactored and cleaned up, including the removal of eventlet.greenlib. This means that it is now possible to use plain greenlets without modification in eventlet, and the subclasses of greenlet instead of the old eventlet.greenlib.GreenletContext. Calling eventlet.api.get_hub().switch() now checks to see whether the current greenlet has a "switch_out" method and calls it if so, providing the same functionality that the GreenletContext.swap_out used to. The swap_in behavior can be duplicated by overriding the switch method, and the finalize functionality can be duplicated by having a try: finally: block around the greenlet's main implementation. The eventlet.backdoor module has been ported to this new scheme, although it's signature had to change slightly so existing code that used the backdoor will have to be modified. A number of bugs related to improper scheduling of switch calls has been fixed. The fixed functions and classes include api.trampoline, api.sleep, coros.event, coros.semaphore, coros.queue. Many methods of greenio.GreenSocket were fixed to make its behavior more like that of a regular socket. Thanks to Marcin Bachry for fixing GreenSocket.dup to preserve the timeout. Added proc module which provides an easy way to subscribe to coroutine's results. This makes it easy to wait for a single greenlet or for a set of greenlets to complete. wsgi.py now supports chunked transfer requests (patch by Mike Barton) The following modules were deprecated or removed because they were broken: hubs.nginx, hubs.libev, support.pycurls, support.twisteds, cancel method of coros.event class The following classes are still present but will be removed in the future version: - channel.channel (use coros.Channel) - coros.CoroutinePool (use pool.Pool) saranwrap.py now correctly closes the child process when the referring object is deleted, received some fixes to its detection of child process death, now correctly deals with the in keyword, and it is now possible to use coroutines in a non-blocking fashion in the child process. Time-based expiry added to db_pool. This adds the ability to expire connections both by idleness and also by total time open. There is also a connection timeout option. A small bug in httpd's error method was fixed. Python 2.3 is no longer supported. A number of tests was added along with a script to run all of them for all the configurations. The script generates an html page with the results. Thanks to Brian Brunswick for investigation of popen4 badness (eventlet.process) Thanks to Marcus Cavanaugh for pointing out some coros.queue(0) bugs. The twisted integration as well as many other improvements were funded by AG Projects (http://ag-projects.com), thanks! [View Less]

1 0

[Twisted-Python] Twisted meetup/discussion at EuroPython?
by Terry Jones June 30, 2009

June 30, 2009

I was thinking it would be nice to have a Twisted interest group meetup at EuroPython (June 30 - July 2) http://www.europython.eu Would people be interested in attending? Does someone more qualified want to organize it? Are any Twisted developers planning to be at EuroPython? The submissions page http://www.europython.eu/submission/ has a category for Interest Group Meeting or Discussion. I'm thinking more along the lines of the former - that it would be nice to be better connected to other … [View More]

6 5

Re: [Twisted-Python] Twisted meetup/discussion at EuroPython?
by Terry Jones June 30, 2009

June 30, 2009

Hi Richard That sounds good. You might enjoy Esteve Fernandez's talk at 11:30, which is about how we're using Twisted plus Thrift and AMQP for messaging / RPC. Terry Enviado desde mi dispositivo BlackBerry® de Orange.

1 0

[Twisted-Python] testing a tac file
by Rob Hoadley June 29, 2009

June 29, 2009

Hi, I'm curious if anyone has written a trial test to verify the correct startup of a .tac file. I have a buildbot running automated trial tests on all the services I've written but nothing that closes the loop by building up the services by running the tac file similarly to the way that you'd use trial. Ideally, I'd like to verify that the MultiService has properly started up all the services from a list defined by the test. I know this a fairly nitpicking problem. In production, It's easy … [View More]

2 1

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com June 28, 2009

June 28, 2009

1 0

[Twisted-Python] Conch sshd on windows
by James Mansion June 27, 2009

June 27, 2009

Can/does the conch sshd authenticate local or domain Windows users? James

2 1

[Twisted-Python] Unhandled error in Deferred with twisted-8.2.0, but not with twisted-2.1.0
by Rutt, Benjamin June 26, 2009

June 26, 2009

Am currently trying to upgrade to twisted-8.2.0. For some reason, twisted-8.2.0 thinks my deferred below was gc'd with an unhandled error, and tells me about it upon reactor exit. With python-2.4.2/twisted-2.1.0, no issues. With python-2.6.1/twisted-8.2.0, I see these issues. Any ideas why? Thanks. $ cat ./deferred-errors.py #!/usr/bin/env python import datetime, logging, os, random, re, shutil, socket import subprocess, sys, tempfile, threading, time, traceback from twisted.internet … [View More]

2 3

[Twisted-Python] How to disable XMLRPC logging
by Dustin Land June 26, 2009

June 26, 2009

Greetings! Currently I'm running two daemons that talk to each other over xmlrpc. One runs an implementation of internet.TCPServer with an xmlrpc.XMLRPC handler. The other simply uses web.xmlrpc.Proxy. In the logs for the first I get a flood of... 2009-06-24 15:31:35-0500 [-] 127.0.0.1 - - [24/Jun/2009:20:31:34 +0000] "POST / HTTP/1.0" 200 297 "-" "Twisted/XMLRPClib" In the second... 2009-06-24 16:44:13-0500 [-] Starting factory <twisted.web.xmlrpc._QueryFactory instance at 0x2d9afc8> … [View More]

2 2

[Twisted-Python] Twisted Trial
by Gerrat Rickert June 24, 2009

June 24, 2009

I know this is a very basic question, but I can't seem to get trial working correctly. (I'm working on Windows XP, with python 2.5.4 and twisted 8.2.0) I've created a very simple example: I have a file, 'test_example.py' It's contents are (between ======= lines): ========================================== from twisted.trial import unittest class ExampleTestCase(unittest.TestCase): def test_example(self): """A very simple test""" self.assertEquals(1,1) if __name__ ==… [View More] '__main__': import unittest unittest.main() ========================================== if I just run: 'python text_example.py' It runs the test (with python's regular unittest framework), and passes ok If I run: 'trial test_example.py' It appears to run the test with trial, but gives this traceback: Traceback (most recent call last): File "c:\python25\lib\site-packages\twisted\python\usage.py", line 241, in parseOptions self.postOptions() File "c:\python25\lib\site-packages\twisted\scripts\trial.py", line 293, in postOptions self['reporter'] = self._loadReporterByName(self['reporter']) File "c:\python25\lib\site-packages\twisted\scripts\trial.py", line 279, in _loadReporterByName for p in plugin.getPlugins(itrial.IReporter): File "c:\python25\lib\site-packages\twisted\plugin.py", line 200, in getPlugins allDropins = getCache(package) --- <exception caught here> --- File "c:\python25\lib\site-packages\twisted\plugin.py", line 165, in getCache provider = pluginModule.load() File "c:\python25\lib\site-packages\twisted\python\modules.py", line 380, in load return self.pathEntry.pythonPath.moduleLoader(self.name) File "c:\python25\lib\site-packages\twisted\python\reflect.py", line 456, in namedAny topLevelPackage = _importAndCheckStack(trialname) File "c:\python25\lib\site-packages\twisted\python\reflect.py", line 392, in _importAndCheckStack return __import__(importName) File "c:\python25\lib\site-packages\twisted\plugins\notestplugin.py", line 18, in <module> assert not TEST.running exceptions.AttributeError: 'module' object has no attribute 'running' Traceback (most recent call last): File "c:\python25\lib\site-packages\twisted\python\usage.py", line 241, in parseOptions self.postOptions() File "c:\python25\lib\site-packages\twisted\scripts\trial.py", line 293, in postOptions self['reporter'] = self._loadReporterByName(self['reporter']) File "c:\python25\lib\site-packages\twisted\scripts\trial.py", line 279, in _loadReporterByName for p in plugin.getPlugins(itrial.IReporter): File "c:\python25\lib\site-packages\twisted\plugin.py", line 200, in getPlugins allDropins = getCache(package) --- <exception caught here> --- File "c:\python25\lib\site-packages\twisted\plugin.py", line 165, in getCache provider = pluginModule.load() File "c:\python25\lib\site-packages\twisted\python\modules.py", line 380, in load return self.pathEntry.pythonPath.moduleLoader(self.name) File "c:\python25\lib\site-packages\twisted\python\reflect.py", line 456, in namedAny topLevelPackage = _importAndCheckStack(trialname) File "c:\python25\lib\site-packages\twisted\plugins\testplugin.py", line 12, in <module> from twisted.plugin import IPlugin, ITestPlugin, ITestPlugin2 exceptions.ImportError: cannot import name ITestPlugin test_example ExampleTestCase test_example ... [OK] ------------------------------------------------------------------------ ------- Ran 1 tests in 0.000s PASSED (successes=1) Could someone point out what I am doing wrong (or is this traceback "Normal")? (I found some post on the web about a possible cause being invalid .pyc files left over from a previous twisted install, so I deleted all .pyc files under site-packages\twisted - but still get this traceback) Thanks, Gerrat [View Less]

2 2

[Twisted-Python] [ANN] Foolscap-0.4.2 released
by Brian Warner June 24, 2009

June 24, 2009

I've recently released Foolscap-0.4.2 (and noticed that I hadn't sent out a release announcement since 0.2.8 over a year ago, despite there being 2 major and 4 minor releases in that time. my bad). The home page is http://foolscap.lothar.com/trac . Download this release from PyPI, or from http://foolscap.lothar.com/releases/foolscap-0.4.2.tar.gz . This release adds the "Foolscap Application Server", an easy way to configure and deploy various foolscap-based services like "upload-file" and "… [View More]run-command". Think of it as a twistd for foolscap. The demo directory includes code to use a FURL to checkout and commit to a Git repository: it's easy to set up, and the client holding this FURL gets no other access to the server machine. The full release notes (since 0.2.8) are attached below. Since I missed the last few releases, I'll mention some other features from the past year. The 0.4.0 series changes the preferred access point to "foolscap.api", begins to give up on the resource-consumption-attacks prevention code, and fixes compatibility with python-2.6. The 0.3.0 series adds an "Incident Gatherer" service to the logging code, and improves error handling (DeadReferenceError and RemoteException). Foolscap is a Twisted-friendly remote object protocol, a descendant of Perspective Broker, with improved security properties, third-party references, adaptable serialization, remote logging, and other useful features. Please visit http://foolscap.lothar.com/trac for more details. have a capable day, -Brian * Release 0.4.2 (16 Jun 2009) ** Compatibility Same as 0.4.1 ** the Foolscap Application Server The big new feature in this release is the "Foolscap Application Server". This is both a demo of what you can do with Foolscap, and an easy way to deploy a few simple services that run over secure connections. You create and start a "flappserver" on one machine, and then use the new "flappclient" on the other side. The server can contain multiple services, each with a separate FURL. You give the client a FURL for a specific service, it connects, does a job, and shuts down. See doc/flappserver.xhtml for details. Two service types are provided in this release. The first is a simple file-uploader: the holder of the FURL gets to upload arbitrary files into a specific target directory, and nowhere else. The second is a pre-configured command runner: the service is configured with a shell command, and the client gets to make it run (but doesn't get to influence anything about what gets run). The run-command service defaults to sending stdout/stderr/exitcode to the client program, which will behave as if it were the command being run (stdout and stderr appear at right time, and it exits with the same exitcode). The service can be configured to accept stdin, or to turn off stdout or stderr. The service always runs in a preconfigured working directory. To do this with SSH, you'd need to create a new keypair, then set up an authorized_keys entry to limit that pubkey to a single command, and hope that environment variables and the working directory don't cause any surprises. Implementing the fixed-directory file-uploader would probably require a specialized helper program. The flappserver provides an easy-to-configure capability-based replacement those sorts of SSH setups. The first use-case is to allow buildslaves to upload newly-created debian packages to a central repository and then trigger a package-index rebuild script. By using FURLs instead of raw SSH keys, the buildslaves will be unable to affect any .debs in other directories, or any other files on the repository host, nor will they be able to run arbitrary commands on that host. By storing the FURLs in a file and using the --furlfile argument to "flappclient", a buildbot transcript of the upload step will not leak the upload authority. ** new RemoteReference APIs RemoteReference now features two new methods. rref.isConnected() returns a boolean, True if the remote connection is currently live, False if it has been lost. This is an immediate form of the rref.notifyOnDisconnect() callback-registration mechanism, and can make certain types of publish-subscribe code easier to write. The second is rref.getLocationHints(), which returns a list of location hints as advertised by the host Tub. Most hints are a ("ipv4",host,portnum) tuple, but other types may be defined in the future. Note that this is derived from the FURL that each Tub sends with its my-reference sequence (i.e. it is entirely controlled by the Tub in which that Referenceable lives), so getLocationHints() is quite distinct from rref.getPeer() (which returns an IPv4Address or LoopbackAddress instance describing the other end of the actual network connection). getLocationHints() indicates what the other Tub wants you to use for new connections, getPeer() indicates what was used for the existing connection (which might not accept new connections due to NAT or proxy issues). getLocationHints() is meant to make it easier to write connection-status display code, for example in a server which holds connections to a number of peers. A status web page can loop over the peer RemoteReferences and display location information for each one without needing to look deep inside the hidden RemoteReferenceTracker instance to find it. ** giving up on resource-consumtion defenses Ticket #127 contains more detail, but beginning with this release, Foolscap will be slowly removing the code that attempted to prevent memory-exhaustion attacks. Doing this in a single process is just too hard, and the limits that were enforced provided more problems than protection. To this end, an internal 200-byte limit on FURL length (applied in Gifts) has been removed. Later releases will remove more code, hopefully simplifying the deserization path. ** other bugfixes Previous releases would throw an immediate exception when Tub.getReference() or Tub.connectTo() was called with an unreachable FURL (one with a corrupted or empty set of location hints). In code which walks a list of FURLs and tries to initiate connections to all of them, this synchronous exception would bypass all FURLs beyond the troublesome one. This has been improved: Tub.getReference() now always returns a Deferred, even if the connection is doomed to fail because of a bad FURL. These problems are now indicated by a Deferred that errbacks instead of a synchronous exception. * Release 0.4.1 (22 May 2009) ** Compatibility Same as 0.4.0 ** Bug fixes The new RemoteException class was not stringifiable under python2.4 (i.e. str(RemoteException(f)) would raise an AttributeError), causing problems especially when callRemote errbacks attempted to record the received Failure with log.msg(failure=f). This has been fixed. * Release 0.4.0 (19 May 2009) ** Compatibility The wire protocol remains the same as before, unchanged since 0.2.6 . The main API entry point has moved to "foolscap.api": e.g. you should do "from foolscap.api import Tub" instead of "from foolscap import Tub". Importing symbols directly from the "foolscap" module is now deprecated. (this makes it easier to reorganize the internal structure of Foolscap without causing circular dependencies). (#122) A near-future release (probably 0.4.1) will add proper DeprecationWarnings-raising wrappers to all classes and functions in foolscap/__init__.py . The next major release (probably 0.5.0) will remove these symbols from foolscap/__init__.py altogether. Logging functions are still meant to be imported from foolscap.logging.* . ** expose-remote-exception-types (#105) Remote exception reporting is changing. Please see the new document docs/failures.xhtml for full details. This release adds a new option: tub.setOption("expose-remote-exception-types", BOOL) The default is True, which provides the same behavior as previous releases: remote exceptions are presented to look as much as possible like local exceptions. If you set it to False, then all remote exceptions will be collapsed into a single "foolscap.api.RemoteException" type, with an attribute named .failure that can be used to get more details about the remote exception. This means that callRemote will either fire its Deferred with a regular value, or errback with one of three exception types from foolscap.api: DeadReferenceError, Violation, or RemoteException. (When the option is True, it could errback with any exception type, limited only by what the remote side chose to raise) A future version of Foolscap may change the default value of this option. We're not sure yet: we need more experience to see which mode is safer and easier to code with. If the default is changed, the deprecation sequence will probably be: 0.5.0: require expose-remote-exception-types to be set 0.6.0: change the default to False, stop requiring the option to be set 0.7.0: remove the option ** major bugs fixed: Shared references now work after a Violation (#104) The tubid returned by rref.getSturdyRef() is now reliable (#84) Foolscap should work with python-2.6: Decimal usage fixed, sha/md5 deprecation warnings fixed, import of 'sets' still causes a warning. (#118, #121) Foolscap finally uses new-style classes everywhere (#96) bin/flogtool might work better on windows now (#108) logfiles now store library versions and process IDs (#80, #97) The "flogtool web-viewer" tool listens at a URL of "/" instead of "/welcome", making it slightly easier to use (#120) You can now setOption() on both log-gatherer-furl and log-gatherer-furlfile on the same Tub. Previously this caused an error. (#114) * Release 0.3.2 (14 Oct 2008) ** Compatibility: same as 0.2.6 Incident classifier functions (introduced in 0.3.0) have been changed: if you have written custom functions for an Incident Gatherer, you will need to modify them upon upgrading to this release. ** Logging Changes The log.msg counter now uses a regular Python integer/bigint. The counter in 0.3.1 used itertools.count(), which, despite its documentation, stores the counter in a C signed int, and thus throws an exception when the message number exceeds 2**31-1 . This exception would pretty much kill any program which ran long enough to emit this many messages, a situation which was observed in a busy production server with an uptime of about three or four weeks. The 0.3.2 counter will be promoted to a bigint when necessary, removing this limitation. (ticket #99) The Incident-Gatherer now imports classification functions from files named 'classify_*.py' in the gatherer's directory. This effectively adds "classifier plugins". The signature of the functions has changed since the 0.3.0 release, making them easier to use. If you have written custom functions (and edited the gatherer.tac file to activate them, using gs.add_classifier()), you will need to modify the functions to take a single 'trigger' argument. These same 'classify_*.py' plugins are used by a new "flogtool classify-incident" subcommand, which can be pointed at an incident file, and performs the same kind of classification as the Incident Gatherer. (#102). The logfiles produced by the "flogtool tail" command and the internal incident-reporter now include the PID of the reporting process. This can be seen by passing the --verbose option to "flogtool dump", and will be made more visible in later releases. (#80). The "flogtool web-viewer" tool now marks Incident triggers (#79), and features a "Reload Logfile" button to re-read the logfile on disk (#103). This is most useful when running unit tests, in conjunction with the FLOGFILE= environment variable. ** Other Changes When running unit tests, if the #62 bug is encountered (pyopenssl >= 0.7 and twisted <= 8.1.0 and selectreactor), the test process will emit a warning and pause for ten seconds to give the operator a chance to halt the test and re-run it with --reactor=poll. This may help to reduce the confusion of a hanging+failing test run. The xfer-client.py example tool (in doc/listings/) has been made more useable, by calling os.path.expanduser() on its input files, and by doing sys.exit(1) on failure (instead of hanging), so that external programs can react appropriately. * Release 0.3.1 (03 Sep 2008) ** Compatibility: same as 0.2.6 ** callRemote API changes: DeadReferenceError All partitioning exceptions are now mapped to DeadReferenceError. Previously there were three separate exceptions that might indicate a network partition: DeadReferenceError, ConnectionLost, and ConnectionDone. (a network partition is when one party cannot reach the other party, due to a variety of reasons: temporary network failure, the remote program being shut down, the remote host being taken offline, etc). This means that, if you want to send a message and don't care whether that message makes it to the recipient or not (but you *do* still care if the recipient raises an exception during processing of that message), you can set up the Deferred chain like this: d = rref.callRemote("message", args) d.addCallback(self.handle_response) d.addErrback(lambda f: f.trap(foolscap.DeadReferenceError)) d.addErrback(log.err) The first d.addErrback will use f.trap to catch DeadReferenceError, but will pass other exceptions through to the log.err() errback. This will cause DeadReferenceError to be ignored, but other errors to be logged. DeadReferenceError will be signalled in any of the following situations: 1: the TCP connection was lost before callRemote was invoked 2: the connection was lost after the request was sent, but before the response was received 3: when the active connection is dropped because a duplicate connection was established. This can occur when two programs are simultaneously connecting to each other. ** logging improvements *** bridge foolscap logs into twistd.log By calling foolscap.logging.log.bridgeLogsToTwisted(), or by setting the $FLOGTOTWISTED environment variable (to anything), a subset of Foolscap log events will be copied into the Twisted logging system. The default filter will not copy events below the log.OPERATIONAL level, nor will it copy internal foolscap events (i.e. those with a facility name that starts with "foolscap"). This mechanism is careful to avoid loops, so it is safe to use both bridgeLogsToTwisted() and bridgeTwistedLogs() at the same time. The events that are copied into the Twisted logging system will typically show up in the twistd.log file (for applications that are run under twistd). An alternate filter function can be passed to bridgeLogsToTwisted(). This feature provides a human-readable on-disk record of significant events, using a traditional one-line-per-event all-text sequential logging structure. It does not record parent/child relationships, structured event data, or causality information. *** Incident Gatherer improvements If an Incident occurs while a previous Incident is still being recorded (i.e. during the "trailing log period"), the two will be folded together. Specifically, the second event will not trigger a new Incident, but will be recorded in the first Incident as a normal log event. This serves to address some performance problems we've seen when incident triggers occur in clusters, which used to cause dozens of simultaneous Incident Recorders to swing into action. The Incident Gatherer has been changed to only fetch one Incident at a time (per publishing application), to avoid overloading the app with a large outbound TCP queue. The Incident Gatherer has also been changed to scan the classified/* output files and reclassify any stored incidents it has that are not mentioned in one of these files. This means that you can update the classification functions (to add a function for some previously unknown type of incident), delete the classified/unknown file, then restart the incident gatherer, and it will only reclassify the previously-unknown incidents. This makes it much easier to iteratively develop classification functions. *** Application Version data The table of application versions, previously displayed only by the 'flogtool tail' command, is now recorded in the header of both Incidents and the 'flogtool tail --save-to' output file. The API to add application versions has changed: now programs should call foolscap.logging.app_versions.add_version(name, verstr). * Release 0.3.0 (04 Aug 2008) ** Compatibility: same as 0.2.6 The wire-level protocol remains the same as other recent releases. The new incident-gatherer will only work with applications that use Foolscap 0.3.0 or later. ** logging improvements The "incident gatherer" has finally been implemented. This is a service, like the log-gatherer, which subscribes to an application's logport and collects incident reports: each is a dump of accumulated log messages, triggered by some special event (such as those above a certain severity threshold). The "flogtool create-incident-gatherer" command is used to create one, and twistd is used to start it. Please see doc/logging.xhtml for more details. The incident publishing API was changed to support the incident-gatherer. The incident-gatherer will only work with logports using foolscap 0.3.0 or newer. The log-publishing API was changed slightly, to encourage the use of subscription.unsubscribe() rather than publisher.unsubscribe(subscription). The old API remains in place for backwards compatibility with log-gatherers running foolscap 0.2.9 or earlier. The Tub.setOption("log-gatherer-furlfile") can accept a file with multiple FURLs, one per line, instead of just a single FURL. This makes the application contact multiple log gatherers, offering its logport to each independently, e.g. to connect to both a log-gatherer and an incident-gatherer. ** API Additions RemoteReferences now have a getRemoteTubID() method, which returns a string (base32-encoded) representing the secure Tub ID of the remote end of the connection. For any given Tub ID, only the possessor of the matching private key should be able to provide a RemoteReference for which getRemoteTubID() will return that value. I'm not yet sure if getRemoteTubID() is a good idea or not (the traditional object-capability model discourages making access-control decisions on the basis of "who", instead these decisions should be controlled by "what": what objects do they have access to). This method is intended for use by application code that needs to use TubIDs as an index into a table of some sort. It is used by Tahoe to securely compute shared cryptographic secrets for each remote server (by hashing the TubID together with some other string). Note that the rref.getSturdyRef() call (which has been present in Foolscap since forever) is *not* secure: the remote application controls all parts of the sturdy ref FURL, including the tubid. A future version of foolscap may remedy this. ** Bug fixes The log-gatherer FURL can now be set before Tub.setLocation (the connection request will be deferred until setLocation is called), and getLogPort/getLogPortFURL cannot be called until after setLocation. These two changes, in combination, resolve a problem (#55) in which the gatherer connection could be made before the logport was ready, causing the log-gatherer to fail to subscribe to receive log events. ** Dependent Libraries Foolscap uses PyOpenSSL for all of its cryptographic routines. A bug (#62) has been found in which the current version of Twisted (8.1.0) and the current version of PyOpenSSL (0.7) interact badly, causing Foolscap's unit tests to fail. This problem will affect application code as well (specifically, Tub.stopService will hang forever). The problem only appears to affect the selectreactor, so the current recommended workaround is to run unit tests (and applications that need to shut down Tubs) with --reactor=poll (or whatever other reactor is appropriate for the platform, perhaps iocp). A less-desireable workaround is to downgrade PyOpenSSL to 0.6, or Twisted to something older. The Twisted maintainers are aware of the problem and intend to fix it in an upcoming Twisted release. * Release 0.2.9 (02 Jul 2008) ** Compatibility: exactly the same as 0.2.6 ** logging bugs fixed The foolscap.logging.log.setLogDir() option would throw an exception if the directory already existed, making it unsuitable for use in an application which is expected to be run multiple times. This has been fixed. ** logging improvements 'flogtool tail' now displays the process ID and version information about the remote process. The tool will tolerate older versions of foolscap which do not offer the get_pid interface. (foolscap ticket #71) The remote logport now uses a size-limited queue for messages going to a gatherer or 'flogtool tail', to prevent the monitored process from using unbounded amounts of memory during overload situations (when it is generating messages faster than the receiver can handle them). This solves a runaway load problem we've seen in Tahoe production systems, in which a busy node sends log messages to a gatherer too quickly for it to absorb, using lots of memory to hold the pending messages, which causes swapping, which causes more load, making the problem worse. We frequently see an otherwise well-behaved process swell to 1.4GB due to this problem, occasionally failing due to VM exhaustion. Of course, a bounded queue means that new log events will be dropped during this overload situation. (#72) ** serialization added for the Decimal type (#50) ** debian packaging targets added for gutsy and hardy The Makefile now has 'make debian-gutsy' and 'make debian-hardy' targets. These do the same thing as 'make debian-feisty'. (#76) [View Less]

1 0

Re: [Twisted-Python] adbapi and multiple queries in single
by Vishal Shetye June 23, 2009

June 23, 2009

Phil thanks for the reply. I plan to do something similar. [sorry for not providing code snippet.] I have modified your code (please ignore queries and other errors) However I am not very much convinced with this approach, As you can see validate of ObjectGateway is a public method which also needs to be called from handleLogin. So I have to write two methods one public 'validate' and other private '_validate'. Any comments/suggestions or pointers to any existing applications/architecture … [View More]documents with similar data access? class TableGateway(object): def _getUserName(self, trans, user_id): result = trans.execute('SELECT username FROM user WHERE id = %s', user_id) return result[0][0] def _sessionUpdateQuery(self, trans, sid, username): trans.execute('UPDATE session SET username = %s WHERE sid = %s', [username, sid]) def validateSid(self, txn, sid): res = txn.execute('select id from user where id=?', (sid,)) return True if res else False class ObjectGateway(object): def __init__(self, pool): self.pool = pool self.tblGw = TableGateway() def _validate(self, txn, sid): return self.tblGw.validateSid(txn, sid) def validate(self, sid): return self.pool.runInteraction(self._validate, sid) def handleLogin(self, sid, user_id): def _loginInteraction(trans): if not self._validate(trans, sid): raise "invalid sid" u = self.tblGw._getUserName(trans, user_id) self.tblGw._sessionUpdateQuery(trans, sid, username) return u return self.pool.runInteraction(_loginInteraction) Regards, vishal Date: Tue, 23 Jun 2009 10:25:16 -0400 From: Phil Christensen <phil(a)bubblehouse.org> Subject: Re: [Twisted-Python] adbapi and multiple queries in single transaction. To: Twisted general discussion <twisted-python(a)twistedmatrix.com> Message-ID: <2E70F4C7-5253-44B2-9DB8-79D8817E96F4(a)bubblehouse.org> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes On Jun 23, 2009, at 9:45 AM, Vishal Shetye wrote: >> I initially thought of putting all the queries per operation in a >> runInteraction. However this results in code-duplication as many >> queries are shared between different operations. >It seems like the easiest way to deal with this would be to make >'private' methods for all the standard queries; these methods/ >functions would accept a transaction object like you said, but the >methods themselves would only be called from an interaction, which can >supply the transaction object. >Then in each public ObjectGateway method you can just define an inner >function to serve as the interaction, calling each private query >method in turn, using the transaction object provided to that >interaction. >Here's a stupidly trivial example: class ObjectGateway(object): def __init__(self, pool): self.pool = pool def _getUserName(self, trans, user_id): result = trans.execute('SELECT username FROM user WHERE id = %s', user_id); return result[0][0] def _sessionUpdateQuery(self, trans, sid, username): trans.execute('UPDATE session SET username = %s WHERE sid = %s', [username, sid]); def handleLogin(self, sid, user_id): def _loginInteraction(trans): u = self._getUserName(trans, user_id) self._sessionUpdateQuery(trans, sid, username) return u return self.pool.runInteraction(_loginInteraction) >Other than being careful not to mess around with the instance state >during those interactions (they are running in a thread, after all), >this should be pretty straightforward. >Hope this helps, -phil DISCLAIMER ========== This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails. [View Less]

1 0

[Twisted-Python] adbapi and multiple queries in single transaction.
by Vishal Shetye June 23, 2009

June 23, 2009

Hi, I am working on an application that uses twisted and twisted.e.adbapi with sqlite in the backend. All db queries are put in separate functions in class say tableGateway.py which uses runOperation, runQuery etc as appropriate. A layer that uses it, lets say objectGateway.py, defines operations. Thus, lets say an add operation may actually call 2-3 or more functions from tableGateway.py. If my application quits or crashes in middle of an operation then it is likely that db will be … [View More]

2 1

[Twisted-Python] Fwd: SerialPort.loseConnection() in Win32, workaround
by biziap biziap June 23, 2009

June 23, 2009

Hi, I have finished the neat version of using the SerialPort in Win32. Without the bothering loseConnection tracebacks. First of all, I have to apology for the mistake in my last post. I claimed that calling the flashInput and flashOutput can be used as the workaround.I am wrong. The way I use with SerialPort is different from the standard way: SerialPort(protocol_instance, comport, reactor, baudrate) instead, I create a SerialConnect and connect the serial port by: connector = reactor.… [View More]

1 0

[Twisted-Python] twisted.web2 only can serve one connection at the same time when using scgi?
by Chris June 23, 2009

June 23, 2009

Hi everyone, I am trying to use apache2 + scgi to write a application,I found twisted.web2 supports the scgi module.I began to use this library to write some code to test how to use it,just like this: from twisted.web2 import server, channel, resource,http,http_headers from twisted.application import service, strports from twisted.internet import reactor, defer class Test(resource.Resource): def getResponse(self, result = None): rep = "The output which shall be transferred to the browser." … [View More]res = http.Response( 200, {'content-type': http_headers.MimeType('text','plain')}, rep) print "---- getResponse returning: ", res return res def render(self,request): print 'a new request is coming' d = defer.Deferred() import time d.addCallback(lambda x:x).addCallback(self.getResponse) reactor.callLater(10,d.callback,0) return d toplevel = Test() site = server.Site(toplevel) application = service.Application("demoserver") s = strports.service('tcp:8888', channel.SCGIFactory(site)) s.setServiceParent(application) in the render function,I used a defer to simulate a time-consuming operation which takes ten seconds.IF in twisted.web,I can first return a server.NOT_DONE_YET,and use request.write() and request.finish() when the time-consuming operation is done(still use defer to do this).but twisted.web2 only supports returning a iweb.IResponse.it seems returning a defer is the only way. _|| <http://python.net/crew/mwh/apidocs/twisted.web2.iweb.IResponse.html>_ here is the output: 2009-06-23 15:45:11+0800 [-] Log opened. 2009-06-23 15:45:11+0800 [-] twistd 8.2.0 (/usr/bin/python 2.5.2) starting up. 2009-06-23 15:45:11+0800 [-] reactor class: twisted.internet.selectreactor.SelectReactor. 2009-06-23 15:45:11+0800 [-] twisted.web2.channel.scgi.SCGIFactory starting on 8888 2009-06-23 15:45:11+0800 [-] Starting factory <twisted.web2.channel.scgi.SCGIFactory instance at 0xf213f8> 2009-06-23 15:45:26+0800 [SCGIChannelRequest,0,192.168.0.138] a new request is coming 2009-06-23 15:45:36+0800 [-] ---- getResponse returning: <twisted.web2.http.Response code=200, streamlen=53> 2009-06-23 15:45:36+0800 [SCGIChannelRequest,1,192.168.0.138] a new request is coming 2009-06-23 15:45:46+0800 [-] ---- getResponse returning: <twisted.web2.http.Response code=200, streamlen=53> 2009-06-23 15:45:46+0800 [SCGIChannelRequest,2,192.168.0.138] a new request is coming 2009-06-23 15:45:56+0800 [-] ---- getResponse returning: <twisted.web2.http.Response code=200, streamlen=53> I use the 3 browsers to access the URL within 1 second,but as we all see in the log,the server accepts the requests one by one,that is really not acceptable.I checked a little about the twisted.web2 source code, this render function is in a defer's callbacks chain,when I return a defer in a defer's callbacks chain,sure,the rest of the callbacks will wait for my defer to finish ,even I run it with reactor.callLater.so far,I feel I can do nothing about this,anyone can help me about this? Regards Chris [View Less]

1 0

[Twisted-Python] SIP Registrar
by Guenter Dannoritzer June 22, 2009

June 22, 2009

Hi, I am trying to run a SIP registrar. So far I used an example that comes with the Perl Net:Sip Package: http://search.cpan.org/~sullr/Net-SIP/ As I am more familiar with Python, but not yet with twisted, I tried to find a simple example that runs a SIP registrar based on twisted. I searched this mailing list and found a post about rather using the Divmod Sine sip implementation than this one. On the other side it seems like that some of this functionality is back ported into twisted. … [View More]

2 1

[Twisted-Python] adbapi : sqlite3.OperationalError : database is locked
by Gabriel Rossetti June 22, 2009

June 22, 2009

Hello everyone, I am using twisted.enterprise.adbapi with sqlite3 database adapter and "runInteraction()". I keep on getting "sqlite3.OperationalError : database is locked" errors. I understand the problem, multiple threads try to access the DB and it being already locked by a thread it raises this exception. My question is what is the standard/correct Twisted way of dealing with this? Thank you, Gabriel

2 2