Mailman 3 August 2009 - Twisted

[Twisted-Python] How to debug an AMP connection?
by Peter Westlake Aug. 10, 2009

Aug. 10, 2009

Hello, I'm having trouble with an AMP connection that doesn't fire the Deferred returned by callRemote. The AMP command copies files from the client (called a "worker" in the code below) to a server ("controller"). It sends a chunk of text at a time - 16K originally, but I tried smaller amounts too. The server appends the text to a queue to be written; it shouldn't take long, because it doesn't do the I/O synchronously, just starts a thread. What happens is that the client calls callRemote, … [View More]

4 7

[Twisted-Python] sending 'tail -f' command to a remote host using conch?
by Harsha Aug. 10, 2009

Aug. 10, 2009

Hi All, A day back I install Twisted and Conch on RHEL5. I am a novice user of Twisted software. I copied an example from dev-shed website (attached herewith). This example script uses conch to talk to a ssh deamon. This script was able to capture and return the output of simple commands such as - ls, cat etc. I mean, when I run the script to execute 'ls -al' on a remote machine it does and returns the prompt. Where as, when I run the script to execute '*tail -f /path/to/mylog.txt*' I get … [View More]the ouput of this command but the prompt never returns. When I press ^C (Control-C) I get the following run-time error. *Traceback (most recent call last): File "/usr/lib/python2.4/site-packages/Twisted-8.2.0-py2.4-linux-i686.egg/twisted/internet/tcp.py", line 520, in connectionLost protocol.connectionLost(reason) File "/usr/lib/python2.4/site-packages/Twisted-8.2.0-py2.4-linux-i686.egg/twisted/conch/ssh/transport.py", line 166, in connectionLost self.service.serviceStopped() File "/usr/lib/python2.4/site-packages/Twisted-8.2.0-py2.4-linux-i686.egg/twisted/conch/ssh/connection.py", line 59, in serviceStopped map(self.channelClosed, self.channels.values()) File "/usr/lib/python2.4/site-packages/Twisted-8.2.0-py2.4-linux-i686.egg/twisted/conch/ssh/connection.py", line 580, in channelClosed log.callWithLogger(channel, channel.closed) --- <exception caught here> --- File "/usr/lib/python2.4/site-packages/Twisted-8.2.0-py2.4-linux-i686.egg/twisted/python/log.py", line 84, in callWithLogger return callWithContext({"system": lp}, func, *args, **kw) File "/usr/lib/python2.4/site-packages/Twisted-8.2.0-py2.4-linux-i686.egg/twisted/python/log.py", line 69, in callWithContext return context.call({ILogContext: newCtx}, func, *args, **kw) File "/usr/lib/python2.4/site-packages/Twisted-8.2.0-py2.4-linux-i686.egg/twisted/python/context.py", line 59, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/usr/lib/python2.4/site-packages/Twisted-8.2.0-py2.4-linux-i686.egg/twisted/python/context.py", line 37, in callWithContext return func(*args,**kw) File "shankha.py", line 56, in closed reactor.stop() File "/usr/lib/python2.4/site-packages/Twisted-8.2.0-py2.4-linux-i686.egg/twisted/internet/base.py", line 526, in stop raise error.ReactorNotRunning( twisted.internet.error.ReactorNotRunning: Can't stop reactor that isn't running.* *I know that 'tail -f' command is expecting "Control-C" or Break to return the prompt on remote host. But my attempt to send the ^C character is failing.. I am doing somethink like this: python sshc.py <hostname> 'tail -f mylog.txt ; \x03' What is the recommended method to adopt such cases? *Cheers!! Harsha Reddy [View Less]

2 1

[Twisted-Python] twisted.positioning -- Interface, or class with method stubs?
by Laurens Van Houtven Aug. 10, 2009

Aug. 10, 2009

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi :-) As you probably know, the interfaces for t.p live in t.p.ipositioning. I still haven't quite understood why they are going to be awesome, but anyway -- I'm running into a problem where keeping them would (I think) imply a bunch of duplicated code. The thing is, t.p.nmea.NMEAAdapter needs to (at least, I think so) implement all of the methods of t.p.ipositioning.IPositioningProvider, because people will subclass t.p.nmea.NMEAAdapter and … [View More]

2 2

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com Aug. 9, 2009

Aug. 9, 2009

1 0

[Twisted-Python] Changing threadpool size in reactor
by Darren Govoni Aug. 6, 2009

Aug. 6, 2009

Hi, I could see from the docs I read how to increase the allotted threads in the reactor. I use reactor.callInThread(....) and it seems to max out at 10. I read about ThreadPool class, but need to run mine in the reactor. thanks for any tips and sorry if it was more obvious than i could see. Darren

2 1

[Twisted-Python] Adbapi issues
by Gerrat Rickert Aug. 6, 2009

Aug. 6, 2009

I'd like to use twisted's adbapi module (twisted 8.2.0 for python 2.5) with cx_Oracle, but I'm having some issues with it. Specifically: 1. It doesn't seem to reconnect (or possibly I just need enlightenment on how reconnecting works): If I tweak part of the test_adbapi.py script to work for Oracle (using a proper "conn_str", and ignoring the irrelevant parts), I get something like: class OracleTests(unittest.TestCase): """Test adbapi for Oracle""" timeout = 10 def setUp(… [View More]self): self.dbpool = adbapi.ConnectionPool('cx_Oracle', conn_str, cp_noisy=True, cp_min=1, cp_reconnect=True, cp_max=3, cp_good_sql='select * from dual', threaded=True) def test_reconnect(self): d = defer.succeed(None) d.addCallback(self._testPool_1) d.addCallback(self._testPool_2) d.addCallback(self._testPool_3) return d def _testPool_1(self, res): def _success(rslt): self.failUnless(rslt[0][0] == 'X', "Select from dual not working") d = self.dbpool.runQuery("select * from dual") d.addCallback(_success) return d def _testPool_2(self, res): self.dbpool.connections.values()[0].close() def _testPool_3(self, res): sql = "select * from dual" d = self.dbpool.runQuery(sql) def _check(row): self.failUnless(row[0][0] == 'X', " Select from dual not working ") d.addCallback(_check) return d I get this traceback: [ERROR]: test_oracle.OracleTests.test_reconnect Traceback (most recent call last): File "C:\Python25\Lib\site-packages\twisted\python\threadpool.py", line 210, i n _worker result = context.call(ctx, function, *args, **kwargs) File "C:\Python25\Lib\site-packages\twisted\python\context.py", line 59, in ca llWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "C:\Python25\Lib\site-packages\twisted\python\context.py", line 37, in ca llWithContext return func(*args,**kw) File "c:\python25\lib\site-packages\twisted\enterprise\adbapi.py", line 429, i n _runInteraction result = interaction(trans, *args, **kw) File "c:\python25\lib\site-packages\twisted\enterprise\adbapi.py", line 443, i n _runQuery trans.execute(*args, **kw) cx_Oracle.InterfaceError: not connected ...am I doing something wrong, or is this something specific to Oracle? I apologize for the seemingly rhetorical question, but does this actually work for other databases (I don't have any others installed, so trial just skips most of the tests when I run test_adbapi.py)? 2. ...since I can't get it to reconnect properly, perhaps I could just close the connections in the old connection pool, and create a new one... dbpool = adbapi.ConnectionPool('cx_Oracle', conn_str, cp_noisy=True, cp_min=1, cp_reconnect=True, cp_max=3, cp_good_sql='select * from dual', threaded=True) dbpool.close() Traceback (most recent call last): File "<stdin>", line 1, in <module> File "c:\python25\lib\site-packages\twisted\enterprise\adbapi.py", line 359, i n close if self.shutdownID: AttributeError: ConnectionPool instance has no attribute 'shutdownID' ...shouldn't shutdownID be initialized in the class, or at least in __init__ ? 3. ...as stated in PEP 249, under fetchall(), "... Note that the cursor's arraysize attribute can affect the performance of this operation." In the PEP, cursor.arraysize defaults to 1, which results in absolutely terrible performance when retrieving a large number of rows from the database (empirically tested). Since twisted has already gone through the trouble of wrapping simple calls to the database (like it's runQuery), it would be ideal if this was an optional parameter that could be passed in. It doesn't look possible to do this in the current adbapi module (or perhaps, I might just need enlightening again): def _runQuery(self, trans, *args, **kw): trans.execute(*args, **kw) return trans.fetchall() ...perhaps it could be something more like: def _runQuery(self, trans, *args, **kw): if kw.has_key('arraysize'): trans._cursor.arraysize = kw['arraysize'] trans.execute(*args, **kw) return trans.fetchall() 4. Timeouts. ...well, since Deferred.setTimeout is deprecated, and we can't cancel deferreds, most protocols (or asynchronous "mechanisms" if protocol isn't the right term here) should probably have a timeout mechanism. (...or is everyone looking at me like I'm from outer-space?) Is there a canonical way of timing out a connection? If I were to do a dbpool.runQuery("select some_cols from some_table"); and attach a timeout mechanism to the deferred that it returns, is there a way to drop/recycle that particular connection from the pool if it didn't respond in a timely fashion? ...or (assuming the dbpool.close() issue went away), would "best-practice" be just closing the old pool, and re-creating it? ...is this module widely used in production, or are most people using something like SqlAlchemy (or "rolling their own")? Thanks, Gerrat [View Less]

3 6

[Twisted-Python] twisted.positioning preliminary review
by Laurens Van Houtven Aug. 6, 2009

Aug. 6, 2009

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hey :-) I've been told that I need to produce a patch faster, and most of the eyesores that I was really unhappy with have been removed (no more mutable sentenceData dict \o/). As usual, comments welcome. I could produce an actual patch, but perhaps it would be more useful to do this with the Launchpad-published bzr branch? Obviously that means I can't attach a real patch to a trac ticket. If people want me to, I will. Some of the obvious … [View More]

2 2

[Twisted-Python] Really Basic clarification on defers
by John Aherne Aug. 6, 2009

Aug. 6, 2009

This is a really basic problem we are trying to decide about, We have programs that run quite happily, so far. Its main task is to receive data from port A and send it out via port B. Then receive data via port B and send it out via port A. It's pretty much like a chat setup. You just build up a list of connected clients and send data to them as required One side A receives some input from a tcp port - about 100-200 characters, and forwards it to another port B. We do not need to wait for any … [View More]response. If we get a response we pick that up through line receiver. We also run a calllater to check if we got a response on linereceiver within the timeframe specified. If not we drop the connection. Traffic coming in from port B is analysed and some subset is sent back to port A. Ignoring port A for the moment, just concentrating on port B, we have tried three options:-- 1. We set up a defer to handle the sendline to port B so that the reactor would schedule it in its own good time. No threads involved using the standard twisted setup. When we get a response through receiveline we fire the callback defer. If we timeout via callLater we fire the errback to clear the defer. In this case the defer does not seem to be doing very much 2. Now a fresh pair of eyes is looking at the code and saying why are we using a deferred for sending data to port B. We could just issue a straight sendline as part of the main code and carry on. If we get a response via linereceiver,we process it normally, otherwise we set our callLater running and timeout and lose the connection. So no deferreds required at all. It does seem to work.What we are not sure about is what penalty is incurred in terms of reliability or throughput by using sendline without a deferred. We are not too sure what the holdup will be and whether it could end up halting the show. Is it better to schedule these messages via deferreds or am I missing something obvious 3. So we then did an experiment and used defertothread to run the sendline in a separate thread with its own defer to maximise the asynchronous running of the code. So now we are running threads when one of the reasons for looking at twisted was that we could avoid threads as much as possible. The conundrum we are trying to resolve now is which option should we use. Do any of the options have a built-in problem awaiting the unwary. In theory all 3 options work. But if No 1 works well enough for our volume of traffic should we adopt that one. Or is it better to start using the defertothread option. Is there a simple answer The traffic is not large, upto a 100-200 remote devices on port B. They will send GPS data every 20 secs, and about 500 messages of about 200 bytes average throught the day. The remote devices will respond in an irregular manner without dropping the connection, so we force a disconnectf if important messages are not getting through. They are then forced to reconnect. We have looked through the code searching for enlightment and it does seem to be well documented, but the information we are looking for comes well before the doc strings. Hopefully, someone can give us some pointers in the right direction. Thanks for any help. John Aherne [View Less]

6 14

[Twisted-Python] Deferred documentation rewrite
by Edward Z. Yang Aug. 6, 2009

Aug. 6, 2009

Hello all, I have been prodded by the members of #twisted into rewriting the Deferred documentation. You can check out the plan at this ticket: http://twistedmatrix.com/trac/ticket/3943 Comments would be appreciated. Cheers, Edward P.S. If you reply on the mailing list, please CC me.

9 13

[Twisted-Python] My first wxPython App with Twisted
by Thalassery Koran Aug. 4, 2009

Aug. 4, 2009

Hi, I am creating the first GUI app (wxPython based) using Twisted. I am facing an issue which I cannot seem to solve/understand due to lack of trace or anything. I had a big app written on twisted and now I am creating a login screen to the app. The app used to work without any issues. I added the login screen and the login screen comes up. But then everything freezes up. I have to force quit the python application to get anything back. Can anyone help me solve it? … [View More]The code is as follows: import wx import os import logging from twisted.internet import wxreactor wxreactor.install() from twisted.internet import reactor #Called only after wxreactor.install() from twisted.internet.task import LoopingCall import clientapp def opj(path): """ Convert paths to the platform-specific separator """ st = apply(os.path.join, tuple(path.split('/'))) if path.startswith('/'): st = '/' + st return st class APPApp(wx.App): def OnInit(self): self.frame = APPFrame(None, -1, " APP Login", size=(200, 150)) self.panel = APPPanel(self.frame,-1) client = clientapp.setup_app() call = LoopingCall(client.run) call.start(.51) return True def main(): app = APPApp(0) reactor.registerWxApp(app) try: reactor.run() except StandardError: raise class APPPanel(wx.Panel): """ APP GUI PANEL The panel in which the user name and password is typed. @var ID_LOGIN: Login buttons id. When this button is clicked, the data is stored for logging. """ ID_LOGIN = wx.NewId() def __init__(self, parent, id): self.parent = parent wx.Panel.__init__(self, parent, id, size=(200,150)) self.quote = wx.StaticText(self, -1, "APP Login",wx.Point(60, 10)) wx.StaticText(self, wx.ID_ANY, "User Name ", wx.Point(20, 33)) wx.StaticText(self, wx.ID_ANY, "Password ", wx.Point(20, 58)) self.usernameBox = wx.TextCtrl(self, wx.ID_ANY, "", wx.Point(80, 31), wx.Size(100, wx.ID_ANY)) self.usernameBox.SetFocus() self.passwordBox = wx.TextCtrl(self, wx.ID_ANY, "", wx.Point(80, 56), wx.Size(100, wx.ID_ANY), style=wx.TE_PASSWORD) self.button =wx.Button(self, self.ID_LOGIN, "Login", wx.Point(60, 88)) wx.EVT_BUTTON(parent, self.ID_LOGIN, self.on_login) def on_login(self, event): username = self.usernameBox.GetString(0, 128) password = self.passwordBox.GetString(0, 128) if username and password: self.parent.on_login_click(username, password) class APPTaskBarIcon(wx.TaskBarIcon): ''' TaskBar Icon So that the application is always alive on the taskbar. 3 operations are possible, DoubleClick on the taskbar would bring up the window, right click will bring up options to restore or exit the tool. ''' TBMENU_RESTORE = wx.NewId() TBMENU_CLOSE = wx.NewId() def __init__(self, frame): wx.TaskBarIcon.__init__(self) self.frame = frame # Set the image icon = self.MakeIcon(self.frame.get_image()) self.SetIcon(icon, "APP") self.imgidx = 1 # bind some events self.Bind(wx.EVT_TASKBAR_LEFT_DCLICK, self.OnTaskBarActivate) self.Bind(wx.EVT_MENU, self.OnTaskBarActivate, id=self.TBMENU_RESTORE) self.Bind(wx.EVT_MENU, self.OnTaskBarClose, id=self.TBMENU_CLOSE) def CreatePopupMenu(self): """ Right Click Popup Menu This method is called by the base class when it needs to popup the menu for the default EVT_RIGHT_DOWN event. """ menu = wx.Menu() menu.Append(self.TBMENU_RESTORE, "Restore Window") menu.Append(self.TBMENU_CLOSE, "Exit APP") return menu def MakeIcon(self, img): """ Return the generated Icon. This function is required because various platforms have different requirements for the icon size. """ if "wxMSW" in wx.PlatformInfo: img = img.Scale(16, 16) elif "wxGTK" in wx.PlatformInfo: img = img.Scale(22, 22) icon = wx.IconFromBitmap(img.ConvertToBitmap() ) return icon def OnTaskBarActivate(self, evt): ''' Activation code - say DoubleClick etc. It raises the window ''' if self.frame.IsIconized(): self.frame.Iconize(False) if not self.frame.IsShown(): self.frame.Show(True) self.frame.Raise() def OnTaskBarClose(self, evt): ''' On right click and close, the frame itself is closed ''' wx.CallAfter(self.frame.Close) class APPFrame(wx.Frame): def __init__(self, parent, id, title, size=None): wx.Frame.__init__(self, parent, id, title, size=size, pos=wx.Point(400, 300)) icon = self.get_icon() self.SetIcon(icon) self.tbicon = APPTaskBarIcon(self) self.Show(True) self.Bind(wx.EVT_CLOSE, self.OnCloseWindow) self.Bind(wx.EVT_ICONIZE, self.OnIconify) def on_login_click(self, username, password): self.Hide() def OnCloseWindow(self, event): if self.tbicon is not None: self.tbicon.Destroy() event.Skip() def OnIconify(self, event): self.Hide() def get_image(self): img= wx.Image(opj('client.ico'), wx.BITMAP_TYPE_ICO) return img def get_icon(self): img = self.get_image() icon = wx.IconFromBitmap(img.ConvertToBitmap() ) return icon if __name__ == '__main__': main() -------------- If somebody can help me out, I would be much obliged. [View Less]

2 1

Re: [Twisted-Python] Deferred documentation rewrite
by Andrew Francis Aug. 4, 2009

Aug. 4, 2009

Hi Edward: Message: 2 Date: Mon, 03 Aug 2009 18:00:07 -0400 From: "Edward Z. Yang" <ezyang(a)MIT.EDU> Subject: Re: [Twisted-Python] Deferred documentation rewrite To: Twisted general discussion <twisted-python(a)twistedmatrix.com> Message-ID: <1249336297-sup-7263@javelin> Content-Type: text/plain; charset=UTF-8 >Why callbacks? > - Asynchronous interaction to synchronous interaction > - Delocalized execution (the parser example) > - High level functions in … [View More]

2 1

[Twisted-Python] OS for Twisted Framework to run on
by vitaly＠synapticvision.com Aug. 4, 2009

Aug. 4, 2009

hi, is there any significant reason/advantage to run the Twisted Framework on Linux or on FreeBSD? Appreciate the help.

2 1

[Twisted-Python] Twisted.web.client.getPage with proxy?
by disappearedng Aug. 3, 2009

Aug. 3, 2009

Hello world, 1) I have a proxy server running on my computer on port 8888. It's listening on localhost. I am wondering whether Twisted.web.client.getPage has some sort of proxy kwargs that can do this for me. ( I checked the source and I really doubt it has something like that) What are my alternatives? I have looked at low-level alternatives, and ProxyClient doesn't really seem to fit my situation. 2) I sometimes have 2 proxy servers running. I know urllib has a method which allows you to … [View More]

1 0

[Twisted-Python] Questions about adding documentation
by Jessica McKellar Aug. 3, 2009

Aug. 3, 2009

Hi everyone, I would like to help Twisted by adding documentation or reviewing existing documentation. However, I don't see many (any?) unassigned tickets regarding documentation of specific items, and because I am quite new to Twisted this makes it hard for me to determine where you wish I would focus my attention. Where should I focus my attention? Want to open some tickets for me to claim? Is adding to API docs more important than updating the examples and howtos? It looks like to add to … [View More]

15 40

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com Aug. 2, 2009

Aug. 2, 2009

1 0

[Twisted-Python] ANNOUNCING Tahoe, the Lofty-Atmospheric Filesystem, v1.5
by Zooko Wilcox-O'Hearn Aug. 2, 2009

Aug. 2, 2009

Dear people of the twisted mailing list: This release announcement is relevant to this list because Tahoe-LAFS is built on Twisted's networking and concurrency framework. Also, we strive to emulate the UQDS. Regards, Zooko The Tahoe-LAFS team is pleased to announce the immediate availability of version 1.5 of Tahoe, the Lofty Atmospheric File System. Tahoe-LAFS is the first cloud storage technology which offers security and privacy in the sense that the cloud storage service provider … [View More]itself can't read or alter your data. Here is the one-page explanation of its unique security and fault-tolerance properties: http://allmydata.org/source/tahoe/trunk/docs/about.html This release is the successor to v1.4.1, which was released April 13, 2009 [1]. This is a major new release, improving the user interface and performance and fixing a few bugs, and adding ports to OpenBSD, NetBSD, ArchLinux, NixOS, and embedded systems built on ARM CPUs. See the NEWS file [2] for more information. In addition to the functionality of Tahoe-LAFS itself, a crop of related projects have sprung up to extend it and to integrate it into operating systems and applications. These include frontends for Windows, Macintosh, JavaScript, and iPhone, and plugins for duplicity, bzr, Hadoop, and TiddlyWiki, and more. See the Related Projects page on the wiki [3]. COMPATIBILITY Version 1.5 is fully compatible with the version 1 series of Tahoe-LAFS. Files written by v1.5 clients can be read by clients of all versions back to v1.0. v1.5 clients can read files produced by clients of all versions since v1.0. v1.5 servers can serve clients of all versions back to v1.0 and v1.5 clients can use servers of all versions back to v1.0. This is the sixth release in the version 1 series. The version 1 series of Tahoe-LAFS will be actively supported and maintained for the forseeable future, and future versions of Tahoe-LAFS will retain the ability to read and write files compatible with Tahoe-LAFS v1. The version 1 series of Tahoe-LAFS is the basis of the consumer backup product from Allmydata, Inc. -- http://allmydata.com . WHAT IS IT GOOD FOR? With Tahoe-LAFS, you can distribute your filesystem across a set of servers, such that if some of them fail or even turn out to be malicious, the entire filesystem continues to be available. You can share your files with other users, using a simple and flexible access control scheme. We believe that the combination of erasure coding, strong encryption, Free/Open Source Software and careful engineering make Tahoe-LAFS safer than RAID, removable drive, tape, on-line backup or other Cloud storage systems. This software comes with extensive tests, and there are no known security flaws which would compromise confidentiality or data integrity in typical use. (For all currently known issues please see the known_issues.txt file [4].) LICENCE You may use this package under the GNU General Public License, version 2 or, at your option, any later version. See the file "COPYING.GPL" [5] for the terms of the GNU General Public License, version 2. You may use this package under the Transitive Grace Period Public Licence, version 1 or, at your option, any later version. (The Transitive Grace Period Public Licence has requirements similar to the GPL except that it allows you to wait for up to twelve months after you redistribute a derived work before releasing the source code of your derived work.) See the file "COPYING.TGPPL.html" [6] for the terms of the Transitive Grace Period Public Licence, version 1. (You may choose to use this package under the terms of either licence, at your option.) INSTALLATION Tahoe-LAFS works on Linux, Mac OS X, Windows, Cygwin, Solaris, *BSD, and probably most other systems. Start with "docs/install.html" [7]. HACKING AND COMMUNITY Please join us on the mailing list [8]. Patches are gratefully accepted -- the RoadMap page [9] shows the next improvements that we plan to make and CREDITS [10] lists the names of people who've contributed to the project. The Dev page [11] contains resources for hackers. SPONSORSHIP Tahoe-LAFS was originally developed thanks to the sponsorship of Allmydata, Inc. [12], a provider of commercial backup services. Allmydata, Inc. created the Tahoe-LAFS project and contributed hardware, software, ideas, bug reports, suggestions, demands, and money (employing several Tahoe-LAFS hackers and instructing them to spend part of their work time on this Free Software project). Also they awarded customized t-shirts to hackers who found security flaws in Tahoe-LAFS (see http://hacktahoe.org ). After discontinuing funding of Tahoe-LAFS R&D in early 2009, Allmydata, Inc. has continued to provide servers, co-lo space and bandwidth to the open source project. Thank you to Allmydata, Inc. for their generous and public-spirited support. This is the second release of Tahoe-LAFS which was created solely as a labor of love by volunteers; developer time is no longer funded by allmydata.com (see [13] for details). Zooko Wilcox-O'Hearn on behalf of the Tahoe-LAFS team Special acknowledgment goes to Brian Warner, whose superb engineering skills and dedication are primarily responsible for the Tahoe implementation, and significantly responsible for the Tahoe design as well, not to mention most of the docs and tests. Tahoe-LAFS wouldn't exist without him. August 1, 2009 Boulder, Colorado, USA P.S. Just kidding about that acronym. "LAFS" actually stands for "Lightweight Authorization File System". Or possibly for "Least-Authority File System". There is no truth to the rumour that it actually stands for "Long-lived Axe-tolerant File System". [1] http://allmydata.org/trac/tahoe/browser/relnotes.txt?rev=3853 [2] http://allmydata.org/trac/tahoe/browser/NEWS?rev=4033 [3] http://allmydata.org/trac/tahoe/wiki/RelatedProjects [4] http://allmydata.org/trac/tahoe/browser/docs/known_issues.txt [5] http://allmydata.org/trac/tahoe/browser/COPYING.GPL [6] http://allmydata.org/source/tahoe/trunk/COPYING.TGPPL.html [7] http://allmydata.org/source/tahoe/trunk/docs/install.html [8] http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev [9] http://allmydata.org/trac/tahoe/roadmap [10] http://allmydata.org/trac/tahoe/browser/CREDITS?rev=4035 [11] http://allmydata.org/trac/tahoe/wiki/Dev [12] http://allmydata.com [13] http://allmydata.org/pipermail/tahoe-dev/2009-March/001461.html [View Less]

1 0

[Twisted-Python] what's a workaround for Ticket #3472 (sendmail has no timeout)?
by Dave Britton Aug. 2, 2009

Aug. 2, 2009

Trying to implement a custom mail service I have run into the problem that this ticket describes: ==== Ticket #3472 (new defect ) Opened 10 months ago Last modified 10 months ago twisted.mail.smtp sendmail() should [have] parameters to be passed for the retry and timeout logic supported by SMTPClientFactory: The method: def sendmail() spawns SMTPSenderFactories without a default timeout value. These cause factories to wait forever if there is no response from the remote server. On prolonged … [View More]

2 1

[Twisted-Python] AssertionError: '' for <module '__init__' from 'nfscraper\__init__.pyc'> not in import cache
by Konrads Smelkovs Aug. 1, 2009

Aug. 1, 2009

Hello, I am running latest Twisted Python release 8.2.0 on Windows / Python 2.6. When running trial on my unittests, I get an AssertionError not in import cache: C:\Documents and Settings\Konrads\My Documents\nfscraper\src>tree /f Folder PATH listing Volume serial number is A899-B4C8 C:. └───nfscraper │ httpclient.py │ httpclient.pyc │ interfaces.py │ interfaces.pyc │ scraper.py │ scraper.pyc │ settings.py │ settings.pyc │ util.py │… [View More] util.pyc │ __init__.py │ __init__.pyc │ └───test test_scrapers.py test_scrapers.pyc __init__.py __init__.pyc C:\Documents and Settings\Konrads\My Documents\nfscraper\src>set PYTHONPATH="C:\ Documents and Settings\Konrads\My Documents\nfscraper\src" C:\Documents and Settings\Konrads\My Documents\nfscraper\src>trial nfscraper Traceback (most recent call last): File "C:\Python26\Scripts\trial.py", line 24, in <module> run() File "C:\Python26\lib\site-packages\twisted\scripts\trial.py", line 357, in ru n suite = _getSuite(config) File "C:\Python26\lib\site-packages\twisted\scripts\trial.py", line 315, in _g etSuite return loader.loadByNames(config['tests'], recurse) File "C:\Python26\lib\site-packages\twisted\trial\runner.py", line 647, in loa dByNames for thing in set(things)] File "C:\Python26\lib\site-packages\twisted\trial\runner.py", line 604, in loa dAnything return self.loadPackage(thing, recurse) File "C:\Python26\lib\site-packages\twisted\trial\runner.py", line 543, in loa dPackage pkgobj = modules.getModule(package.__name__) File "C:\Python26\lib\site-packages\twisted\python\modules.py", line 741, in g etModule return theSystemPath[moduleName] File "C:\Python26\lib\site-packages\twisted\python\modules.py", line 677, in _ _getitem__ self._findEntryPathString(moduleObject)), File "C:\Python26\lib\site-packages\twisted\python\modules.py", line 630, in _ findEntryPathString rval, modobj, pformat(self.importerCache)) AssertionError: '' for <module '__init__' from 'nfscraper\__init__.pyc'> not in import cache {'"C:\\Documents and Settings\\Konrads\\My Documents\\nfscraper\\sr c"': <imp.NullImporter object at 0x00A8F028>, 'C:\\Documents and Settings\\Konrads\\My Documents\\nfscraper\\src\\"C:\\Docume nts and Settings\\Konrads\\My Documents\\nfscraper\\src"': <imp.NullImporter obj ect at 0x00A8F488>, 'C:\\Documents and Settings\\Konrads\\My Documents\\nfscraper\\src\\"C:\\Docume nts and Settings\\Konrads\\My Documents\\nfscraper\\src"\\twisted\\plugins': <im p.NullImporter object at 0x00A8F528>, 'C:\\Python26': None, 'C:\\Python26\\DLLs': None, 'C:\\Python26\\DLLs\\twisted\\plugins': <imp.NullImporter object at 0x00A8F538> , 'C:\\Python26\\Scripts': None, 'C:\\Python26\\Scripts\\trial.py': <imp.NullImporter object at 0x00A8F498>, 'C:\\Python26\\Scripts\\twisted\\plugins': <imp.NullImporter object at 0x00A8F5 00>, 'C:\\Python26\\lib': None, 'C:\\Python26\\lib\\ctypes': None, 'C:\\Python26\\lib\\distutils': None, 'C:\\Python26\\lib\\encodings': None, 'C:\\Python26\\lib\\lib-tk': None, 'C:\\Python26\\lib\\lib-tk\\twisted\\plugins': <imp.NullImporter object at 0x00 A8F550>, 'C:\\Python26\\lib\\logging': None, 'C:\\Python26\\lib\\plat-win': <imp.NullImporter object at 0x00A8F490>, 'C:\\Python26\\lib\\plat-win\\twisted\\plugins': <imp.NullImporter object at 0x 00A8F548>, 'C:\\Python26\\lib\\site-packages': None, 'C:\\Python26\\lib\\site-packages\\PIL': None, 'C:\\Python26\\lib\\site-packages\\PIL\\twisted\\plugins': <imp.NullImporter ob ject at 0x00A8F560>, 'C:\\Python26\\lib\\site-packages\\Pythonwin': None, 'C:\\Python26\\lib\\site-packages\\Pythonwin\\twisted\\plugins': <imp.NullImpor ter object at 0x00A8F578>, 'C:\\Python26\\lib\\site-packages\\beautifulsoup-3.1.0.1-py2.6.egg': <zipimport er object "C:\Python26\lib\site-packages\beautifulsoup-3.1.0.1-py2.6.egg">, 'C:\\Python26\\lib\\site-packages\\beautifulsoup-3.1.0.1-py2.6.egg\\twisted\\pl ugins': <zipimporter object "C:\Python26\lib\site-packages\beautifulsoup-3.1.0.1 -py2.6.egg\twisted\plugins\">, 'C:\\Python26\\lib\\site-packages\\django-1.1_beta_1-py2.6.egg': None, 'C:\\Python26\\lib\\site-packages\\django-1.1_beta_1-py2.6.egg\\twisted\\plugin s': <imp.NullImporter object at 0x00A8F508>, 'C:\\Python26\\lib\\site-packages\\django_authopenid-1.0.1-py2.6.egg': None, 'C:\\Python26\\lib\\site-packages\\django_authopenid-1.0.1-py2.6.egg\\twisted\\ plugins': <imp.NullImporter object at 0x00A8F510>, 'C:\\Python26\\lib\\site-packages\\django_registration-0.7-py2.6.egg': <zipimpo rter object "C:\Python26\lib\site-packages\django_registration-0.7-py2.6.egg">, 'C:\\Python26\\lib\\site-packages\\django_registration-0.7-py2.6.egg\\twisted\\ plugins': <zipimporter object "C:\Python26\lib\site-packages\django_registration -0.7-py2.6.egg\twisted\plugins\">, 'C:\\Python26\\lib\\site-packages\\docutils-0.5-py2.6.egg': None, 'C:\\Python26\\lib\\site-packages\\docutils-0.5-py2.6.egg\\twisted\\plugins': < imp.NullImporter object at 0x00A8F520>, 'C:\\Python26\\lib\\site-packages\\epydoc-3.0.1-py2.6-win32.egg': None, 'C:\\Python26\\lib\\site-packages\\epydoc-3.0.1-py2.6-win32.egg\\twisted\\plugi ns': <imp.NullImporter object at 0x00A8F518>, 'C:\\Python26\\lib\\site-packages\\python_openid-2.2.4-py2.6.egg': <zipimporter object "C:\Python26\lib\site-packages\python_openid-2.2.4-py2.6.egg">, 'C:\\Python26\\lib\\site-packages\\python_openid-2.2.4-py2.6.egg\\twisted\\plug ins': <zipimporter object "C:\Python26\lib\site-packages\python_openid-2.2.4-py2 .6.egg\twisted\plugins\">, 'C:\\Python26\\lib\\site-packages\\setuptools-0.6c9-py2.6.egg': <zipimporter ob ject "C:\Python26\lib\site-packages\setuptools-0.6c9-py2.6.egg">, 'C:\\Python26\\lib\\site-packages\\setuptools-0.6c9-py2.6.egg\\twisted\\plugins ': <zipimporter object "C:\Python26\lib\site-packages\setuptools-0.6c9-py2.6.egg \twisted\plugins\">, 'C:\\Python26\\lib\\site-packages\\twisted': None, 'C:\\Python26\\lib\\site-packages\\twisted\\application': None, 'C:\\Python26\\lib\\site-packages\\twisted\\cred': None, 'C:\\Python26\\lib\\site-packages\\twisted\\internet': None, 'C:\\Python26\\lib\\site-packages\\twisted\\lore': None, 'C:\\Python26\\lib\\site-packages\\twisted\\lore\\scripts': None, 'C:\\Python26\\lib\\site-packages\\twisted\\persisted': None, 'C:\\Python26\\lib\\site-packages\\twisted\\plugins': None, 'C:\\Python26\\lib\\site-packages\\twisted\\python': None, 'C:\\Python26\\lib\\site-packages\\twisted\\scripts': None, 'C:\\Python26\\lib\\site-packages\\twisted\\trial': None, 'C:\\Python26\\lib\\site-packages\\twisted\\web': None, 'C:\\Python26\\lib\\site-packages\\twisted\\web2': None, 'C:\\Python26\\lib\\site-packages\\twisted\\words': None, 'C:\\Python26\\lib\\site-packages\\win32': None, 'C:\\Python26\\lib\\site-packages\\win32\\lib': None, 'C:\\Python26\\lib\\site-packages\\win32\\lib\\twisted\\plugins': <imp.NullImpo rter object at 0x00A8F570>, 'C:\\Python26\\lib\\site-packages\\win32\\twisted\\plugins': <imp.NullImporter object at 0x00A8F568>, 'C:\\Python26\\lib\\site-packages\\zope': None, 'C:\\Python26\\lib\\site-packages\\zope\\interface': None, 'C:\\Python26\\lib\\twisted\\plugins': <imp.NullImporter object at 0x00A8F540>, 'C:\\Python26\\twisted\\plugins': <imp.NullImporter object at 0x00A8F558>, 'C:\\WINDOWS\\system32\\python26.zip': <imp.NullImporter object at 0x00A8F030>, 'C:\\WINDOWS\\system32\\python26.zip\\twisted\\plugins': <imp.NullImporter obje ct at 0x00A8F530>} C:\Documents and Settings\Konrads\My Documents\nfscraper\src> -- Konrads Smelkovs Applied IT sorcery. [View Less]

1 1

[Twisted-Python] getPage using ssl proxy
by Konrads Smelkovs Aug. 1, 2009

Aug. 1, 2009

Hello, I am writing some scraper scripts and need to pass them through an intercepting proxy. getPage does not support a proxy argument and this code I found on internet won't work with SSL proxy (stalls indefinitely): def getPage(url, contextFactory=None, *args, **kwargs): scheme, host, port, path = _parse(url) factory = HTTPClientFactory(url, *args, **kwargs) if 0: # use a proxy host, port = 'localhost', 8080 factory.path = url if scheme == 'https': … [View More]

1 1

Re: [Twisted-Python] [Fwd: Re: Deferred documentation rewrite]
by Edward Z. Yang Aug. 1, 2009

Aug. 1, 2009

Excerpts from Itamar Shtull-Trauring's message of Fri Jul 31 22:26:29 -0400 2009: > The problem with this is that it perpetuates the misunderstanding the > Deferreds *make* things asynchronous, even with the intro that says > otherwise. I think it's better to assume already asynchronous code, > handling the transition from synchronous to async in an intro event loop > howto. Either way, the function that the first segment of the new docs do belong somewhere. The documentation … [View More]that traditionally served this purpose has been removed. As for perpetuating the misunderstanding of Deferreds making things asynchronous, I completely agree! However, I think this is something that can be fixed by spelling out the distinction between "writing asynchronous code" and "interacting with asynchronous code", and not just omitting the important paradigm shift that comes with sync->async. > A better comparative exposition might be with normal callbacks, e.g.: > > "def foo(x, gotResultCallback): pass" vs. "def foo(x): # return > Deferred". > > At the very least having that async but callbacky version in the middle > helps understanding. I briefly gloss on this, but I agree that this is an important point that could be further expanded. We could have implemented asynchronous mechanisms using normal callbacks, but we decided to use Deferreds instead. It's not clear to me if the common case of confusion of Deferreds occurs in people who "know callbacks" but "don't know Deferreds." As an incoming developer who was familiar with asynchronous programming, my primary problem was the ill-defined behavior of callback chains (which I resolved by hunkering down and reading the source code) rather than any fundamental misunderstanding of what Deferreds were supposed to do. > It also omits half the story: how you *create* Deferreds. There should > be a section on that as well. I agree. In fact, it might be worth making the document a little longer to address this point, because I realize now that even if you're not writing asynchronous code, you'll often need to baton Deferreds to make the execution flow work the way you want them to. > An example involving a parser, where you just wave your hands about who > pushes data in to the parser exactly (so no need to go into event loop > details), may work well. In particular, the object that wants the result > of the parsing wants to get parse errors, *not* whoever pushes data in. > Often it's same object, but not always. Deferreds help with that. That's a good distinction. I'll see how I can work that in. Cheers, Edward [View Less]

2 1