Mailman 3 May 2006 - Twisted

[Twisted-Python] Question on pamauth.py
by Terry Jones 03 May '06

03 May '06

I'm planning to use twisted.cred and just went to look for ways to implement a back end credentials checker that I could plug in. I soon ran across mentions of pamauth.py, and was pleasantly surprised to find it sitting in my Twisted-2.2.0/twisted/cred/pamauth.py But, I don't see a mention of how to use it. Most of what's on the web shows pamauth.py as living under twisted.conch, which I take to be an old code layout. This: $ find . -name '*.html' -print0 | xargs -0 grep -i pamauth at the top of my Twisted-Sumo tree gets me no hits. I see a little use of pamauth test/test_newcred.py, but it's not terribly enlightening. Is there somewhere I can read about how to use this? A bit more generally, is using pamauth considered roughly best practice? Are there other credentials checkers in wide use (and yes, I know /etc/password is in wide use :-)) If I want to build a web site where users authenticate themselves, I presumably will be setting a cookie on the browser and authentication will just happen when the browser visits. (I'm happy to hear more details on how this is best done). But what if a user is trying to run a command line app or a remote script that's talking to the server via twisted.pb? Presumably some form of credential is stored locally and transmitted with the connection. Is the passed credential in this case stored (supposing it is even stored on disk, as opposed to being stored in the user's head and entered on the command line) in a way that's totally orthogonal to cookies, or does Twisted have yet more functionality to deal with this? One final one: I imagine it's a standard pattern that people want to implement users and groups, with some form of permissions model in which users attempt actions and their privileges are checked (with fallback to their groups' permission if the user does not have explicit permission). Can anyone point me to Twisted code that does something like this? Sorry for so many questions. I don't expect anyone to start writing code for me, I plan to do that myself. But it's nice to not reinvent wheels, that's the point after all. So having just discovered the lightly (or un-) documented pamauth.py, I'm can't help but wonder what more there might be. Regards, Terry

3 2

[Twisted-Python] Database tests and Buildbots
by Jonathan Lange 03 May '06

03 May '06

Hello, I'm trying to reproduce #1630 using the buildbots and I'm not having any luck. Apparently none of the database software is installed on any of the buildbots. All the tests are skipped with 'Cannot access db'. Can the buildbot maintainers please configure their slaves to have the appropriate software installed. Otherwise, we can't really continue supporting twisted.enterprise. cheers, jml

2 3

[Twisted-Python] Twisted + Xapian WriteableDatabase leaves db_lock file on exit.
by jarrod roberson 03 May '06

03 May '06

Twisted Sumo 2006-02-12 xapian and xapian bindings 0.9.2 svn 6532 I have written a Twisted service to write to a xapian database. I am using WriteableDatabase using DB_CREATE_OR_OPEN. I am also using twistd and the application plumbing to start the server. I have tried everything but no matter how I stop the server I can't get it to shutdown xapian cleanly. I have implemented stopService() on my service and do a .flush() and have even tried setting the db instance to None and tried using del self.db as well. I even added a reactor.addSystemEventTrigger() and tried "before", "during" and "after" "shutdown" and nothing seems to work. I stop the service with CTRL-C, kill -TERM `cat twistd.pid' and just plain kill 'cat twistd.pid' It leaves the db_lock file everytime and I don't know what else it might not be cleaning up correctly either. Has anyone gotten a WritableDatabase to work with Twisted? If so, how do I stop Twisted so it will shutdown xapian cleanly?

1 0

[Twisted-Python] Re: The End, also: documentation
by Ed Suominen 03 May '06

03 May '06

Mary Gardiner wrote: > On Wed, Apr 26, 2006, Ed Suominen wrote: > >>I'm thinking about suggesting (gasp!) myself as your replacement. > > ... > >>Comments welcome. Any positive ones will likely be viewed with suspicion >>and will raise pangs of regret at my making this rash posting. The >>negative ones will likely be viewed with relief, at least by the >>aforementioned logical, responsible part... > > > You'll notice I've reassigned some tickets to you and will continue to > do so over the next few days. I'm not sure how your logical and > responsible part will feel about this... you can grab me on IRC as > hypatia for clarification of anything I filed... > > -Mary The aforementioned logical and responsible part is swamped with work just now and horrified that I made the offer. But I suppose it's done and I'll start picking things up when the flood recedes in a week or two. Best regards, Ed

2 1

[Twisted-Python] What is the meaning of "threadable.whenThreaded is deprecated Use application-level logic instead"
by jmp 02 May '06

02 May '06

Helo , Vhat is the meaning of "Use application-level logic instead" ? import twisted.internet.threads /usr/lib/python2.4/site-packages/twisted/python/context.py:74: DeprecationWarning: threadable.whenThreaded is deprecated. Use application-level logic instead. threadable.whenThreaded(initThreads)

3 4

[Twisted-Python] zope.interface changes
by Jim Fulton 01 May '06

01 May '06

I hope this is the right list to post to. Twisted-dev seems more aptly named, but it doesn't seem to get a lot of traffic. I've recently refactored the adapter-registration implementation in zope.interface. There were a number of reasons for this, including performance and making it possible, in the future, to implement a capability similar to Python's super builtin for calling overridden adaptera. The performance gains for the simple-adapter lookup used by Twisted was about 200%. For Zope, the changes led to about a 15-30% performance improvement. I expect the new interface version to be released in June. While working on this, I discovered that Twisted was bypassing the public API to try to determne of there was an adapter registered: for interfaceClass in interfaceClasses: factory = self.get(origInterface).selfImplied.get(interfaceClass, {}).get('') where self is an adapter registry. This is in python/components.py. I've added an adapter-registry API specifically for Twisted's use: def registered(required, provided, name=u''): """Return the component registered for the given interfaces and name Unlike the lookup method, this methods won't retrieve components registered for more specific required interfaces or less specific provided interfaces. If no component was registered exactly for the given interfaces and name, then None is returned. """ I've also added a backward-compatibility hack: def get(self, _): class XXXTwistedFakeOut: selfImplied = {} return XXXTwistedFakeOut This makes Zope's tests pass. Not sure about Twisted's. Obviously, it would be good for Twisted to move to the new public API. In the future, if you need a new API, please feel free to ask. :) Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

1 0

[Twisted-Python] ProducerConsumerProxy problem
by Haris Andrianakis 01 May '06

01 May '06

Hi, I'm trying to setup a ProducerConsumerProxy between a protocols.basic.FileSender producer and a TCP Protocol transport. I have the following code: fileObj = file('test.txt','r') proxy = protocols.pcp.ProducerConsumerProxy(self.transport) producer = protocols.basic.FileSender() dd = producer.beginFileTransfer(fileObj, proxy) dd.addCallback(self.uploadDone) dd.addErrback(self.uploadError) proxy.registerProducer(producer, False) The problem is that only 16KB of data are transfered and then the transfer pauses. Also if I use BasicProducerConsumerProxy no data at all gets transfered. I think that the proxy doesn't call resumeProducing() on the FileSender producer and thats why the transfer seems to halt. Any suggestions ?

2 2

[Twisted-Python] OT: job offer
by Uwe C. Schroeder 01 May '06

01 May '06

Hope nobody minds a job offer..... We have developed an application for the insurance market that is based on: wxPython for the GUI twisted pb as network layer certainly python :-) reportlab/OpenOffice/pyUno for printing zope/plone as alternative web interface. the database backend is postgresql. We're currently in the process of setting up a company to market this software. The software targets a niche market in the insurance business and has been in production for over 3 years with a limited number of clients. We're looking for developers, preferably in the greater San Francisco Bay Area but not a necessity, who have as much experience as possible in any/all of the above technologies. We have certain projects that only need partial knowledge (I'm not disclosing specifics here, I much rather list requirements for the different projects): 1) Solely Plone based development. This basically only requires Plone and postgresql experience. 2) The printing backend needs work. Anyone with good XSLT/RML experience is welcome to apply. Intention is to write a Open Document Format to RML exporter for OpenOffice. We will contribute the exporter back to the community which would benefit the OpenOffice and the reportlab projects. 3) Someone with a solid knowledge of twisted pb (not yet new pb) and wxPython experience. There are technical obstacles to the "twisted/wxPython"-marriage and you should be aware of them. 4) anyone with an open mind and good python experience. SQL database knowledge is pretty much a must for all of the above. We're currently using postgresql, but Oracle and DB2 are not out of the question. If you have commercial insurance knowledge it's even better, but I rather focus on programming skills than insurance knowledge - since the later is much easier to acquire. All of the above projects have the potential to turn into a lucrative permanent occupation. Salary and terms are negotiable. There will be several programmers involved, thus team- and communication skills are a plus. If you're interested please drop me an email. Best UC -- Open Source Solutions 4U, LLC 1618 Kelly St Phone: +1 707 568 3056 Santa Rosa, CA 95401 Cell: +1 650 302 2405 United States Fax: +1 707 568 6416

1 0

[Twisted-Python] Status of components, protocols, etc in Twisted
by Terry Jones 01 May '06

01 May '06

I'm trying to better understand componentized architectures, protocols, adaptors etc., as done in Python. I've been reading around and have a few questions about the state of play. It would be good to understand where things are, and where they're heading. It seems there are (or were) 3 efforts to provide components, protocols, adaptors, etc.: Zope, Twisted, PyProtocols. It looks like Twisted is now using the zope module to do this. But I also ran across discussion (on this list, in Feb 2004, I think), about possibly migrating zope's use of components to PyProtocols. Did that happen? I do like the sound of PyProtocols (and PEAK in general), but as someone else said, and as Philip wrote, the documentation isn't really focused on easing in a newbie. A couple of points on those docs, I think it would be good if someone set out how you would go about building something small (i.e., small concrete examples, not "big" more abstract ones). Yes, that someone could be me, agreed. Also, the docs say that it is intended that _all_ uses of type(), isinstance() etc., can/should be replaced. A few use cases would be helpful - how are we supposed to replace those things? I do have the impression that one should have a clean componentized design from the beginning and thereby never need or want to resort to introspection. If that's the case, it might be worth saying, as the docs leave one wondering if there's not perhaps some formulaic way to rewrite one's code to do the Right Thing. It's not that simple though, right? I'd also be happy to use the standard Twisted (i.e., zope) approach too. Do people have an opinion about which is better to use? Does it make much difference if you're just building a simple app with a handful of components you'd like to be able to swap in & out easily? I think the Twisted evolution of finger pages are really good, with a couple of minor weak points (e.g., highlighting new code from example to example). It's amazing what you can get done in a couple of hundred lines of Twisted code. BTW, there's a mention of __implements__ whereas implements() is used, which I guess is a hang over from an earlier version? This is on http://twistedmatrix.com/projects/core/documentation/howto/tutorial/compone… I guess someone will look for others. Grepping __implements__ gets a lot of hits in the distributed Twisted-2.2.0 docs (e.g., in doc/howto/*.html). Anyway, thanks for all the docs and code. I'm happy to put some work into making improvements where I can. Terry

3 2

[Twisted-Python] newpb status?
by Phil Christensen 01 May '06

01 May '06

Just out of curiosity, I wondered where things stood with newpb. I'm eagerly awaiting its arrival; apologies if there was a list announcement I missed. I got an overview from Brian a few months back, and it got me all pumped ;-)... Is there any documentation available yet? Even just some example code would be nice; I'd like to get into newpb, even if it's in a state of flux. Unfortunately, I can't seem to find the time to lurk in IRC that much, so i kind of need some sort of asynchronous instruction... I've certainly got no problems with reading in-code docs, but I would definitely appreciate a little bit of a starter howto or something... -phil

3 2