Mailman 3 November 2005 - Twisted

[Twisted-Python] Re: [Twisted-commits] r15062 - Upon ERROR_SEM_TIMEOUT, don't drop connection - resolves 1244
by Itamar Shtull-Trauring Nov. 8, 2005

Nov. 8, 2005

On Tue, 2005-11-08 at 10:29 -0700, Justin Johnson wrote: > Author: justinj > Date: Tue Nov 8 10:29:02 2005 > New Revision: 15062 > > Modified: > branches/justinj/issue1008/twisted/conch/test/test_text.py Was this committed by mistake? Unless the test actually outputs to terminal I don't see why the test shouldn't run.

2 1

[Twisted-Python] help with twisted and WORDS
by Max Slimmer Nov. 8, 2005

Nov. 8, 2005

I am trying to use twisted to connect with IM servers/clients. I have read all of the docs for twisted and am very impressed, it is really cool. However I have tried to get the WORDS examples to work, without much luck, and the sources don't have an over abundance of comments. I have ordered the book, hoping that will help, is there any other resources that might help expidite the learning curve, both for twisted and the WORDS interfaces. thanks, max

2 2

[Twisted-Python] Newbie - guidance required please
by Frank Millman Nov. 8, 2005

Nov. 8, 2005

Hi all, I am treading a path that I think many have trodden before. I have a requirement, I am pretty sure that Twisted will be able to solve it for me, but at this stage I am overwhelmed by the options that Twisted seems to offer, and I am worried that I will head off down the wrong path. With your indulgence, I would like to describe what I am trying to accomplish, and get some feedback as to which features of Twisted are best suited to assisting me in achieving this. I am developing a … [View More]multi-user accounting/business system. I use a database for the back end - PostgreSQL/psycopg on Linux, MS SQL Server/pywin32.odbc on MSW. I am using wxPython for the front end, but eventually I will want a web interface as well. I have developed a reasonably powerful middle layer to handle the authentication and business logic. My problem is that at present the middle layer runs on the client, each with its own connection to the database. I am sure that there are a number of problems with this architecture, but the one that concerns me particularly is that it is easy for a user with a smattering of knowledge to modify the client program to bypass my authentication and business logic, make a direct connection to the database, and generate any SQL statements they want. This makes my approach a non-starter. The solution that I have in mind is to move all of my authentication and business logic to a server program, which will be the only program that makes a connection to the database. Each client will connect to my server program, and I will have to devise some message formats for the passing of information between the client and the server. I have a little experience with multi-threaded socket programming, and I am fairly sure that I could come up with a working solution using this technique, but having read the advantages of the Twisted approach, I feel that this would be a better and more scaleable solution. Before going into detail, there is one issue I am not sure about, and that is the question of 'maintaining state'. The user selects an option from a menu - maintain a table, capture a transaction, run a report, perform an enquiry, etc. Each option is represented by a class, which is contained in a module of the same name. When an option is selected, I import the module, and instantiate the class. Each option is associated with a number of database tables, each with a number of columns. I have a generalised 'table class', which is instantiated for each table opened, and a generalised 'column class', which is instantiated for each column. A user can have more than one option running concurrently, which can result in dozens of table objects and hundreds of column objects. I was fairly relaxed about this running on each client, but I am not so sure about running them all on one server with, say, 50 or more concurrent clients. Should I worry about the number of active objects, or is it safe to assume that virtual memory will handle it, on the assumption that most of them will be idle most of the time? As I understand it, I will use Twisted in the following roles - 1. in my server program, as an interface to DB-API for all database access. 2. in my server program, to handle requests from and pass information to clients. 3. in my client program, to connect with and interact with my server program. Having read the documentation on twisted.enterprise.adbapi, it seems that '1' should be fairly straightforward. I hope there are not too many hidden complications. I assume that '2' and '3' can be discussed together, as they form both sides of a single communication channel. I have about a million questions, but I will start with three. 1. Once a client has established a connection with the server, I would prefer to keep the connection open until they log out, which could be several hours later. Is this an acceptable way of doing it, or should I keep the number of open connections down to a minimum, closing and reopening as required? 2. For authentication, the user enters a userid and password. The userid is looked up in a database table. One of the columns contains the hashed (sha) password. If the userid does not exist or the hashed password does not match, I do not allow the user to proceed. I looked at cred.py, but it seems more complicated to use this than stick to my own method. Is this a valid observation? A related question. With my present system, the authentication happens on the client, so I do not have to send the password over the network. With the new method, I will have to send it to the server. Should I be looking at using SSL for protection against sniffing, or is there an easier way? 3. Whether or not I use cred, it looks as if I should be using Perspective Broker (PB). I would normally have been thinking in terms of defining message formats for exchanging data between server and client, so the concept of defining objects that make calls on each other feels a bit strange. If I maintain the full state of the connection on the server side, I do not want to duplicate this on the client side, so I assume I must define some form of stripped down object to represent the client state, and use this to exchange information. Are there any simple examples I can look at to explain this concept in more detail? I feel I am at a crossroads. Either I can cobble something together that works, but does not fit in with the Twisted philosophy, and will therefore limit me as my application grows. Or I can get my mind more in tune with Twisted, and set up structures that will work with me as things evolve. If you have read this far, thanks for your perseverance. Any pointers or suggestions will be much appreciated. Thanks Frank Millman [View Less]

3 3

[Twisted-Python] Weird problem with XMLRPC -> ('405', 'Method Not Allowed')
by Remi Cool Nov. 8, 2005

Nov. 8, 2005

Hello, I've run into a problem with my twisted xmlrpc test server (using twisted version 2.01 or 2.1). Every time the client calls the echo function a '405 Method Not Allowed is returned'. One other thing is that pressing CTRL-C when the server is running, results in the RuntimeError: can't stop reactor that isn't running. Any ideas? Here's the server code: #!/usr/bin/env python # -*- coding: iso-8859-15 -*- """""" import sys from twisted.application import internet, service from … [View More]twisted.internet import reactor, protocol, defer from twisted.web import resource, server, static, xmlrpc from twisted.python import components, rebuild from zope.interface import Interface, implements class myXMLRPC(xmlrpc.XMLRPC): """This class contains all root xmlrpc methods and the modules subhandler.""" def __init__(self, service): xmlrpc.XMLRPC.__init__(self) self.service = service def xmlrpc_rebuild(self, modname): """Rebuild the given module""" self.service.rebuild(modname) return true def xmlrpc_echo(self, x): """Simple echo function""" return x def catchError(err): return "Internal error in server" class IowwService(components.Interface): """""" def rebuild(self, modname): """Rebuild classes in given module""" class httpResource(resource.Resource): implements(resource.IResource) def __init__(self, service): resource.Resource.__init__(self) self.service = service self.putChild('RPC2', myXMLRPC(self.service)) def render(self, request): self._clientIP = request.getClientIP() ## user = request.getUser() ## pwd = request.getPassword() ## if (user and password): return resource.Resource.render(self, request) ## else: ## request.setResponseCode(401) ## return '<html><body><h3>Must be authenticated</h3></body></html>' def render_GET(self, request): """Process HTTP GET Requests.""" return '<html><body><h3>Not Implemented</h3></body></html>' def getChild(self, path, request): """This method handles http calls""" return httpResource(self.service) components.registerAdapter(httpResource, IowwService, resource.IResource) class owwService(service.Service): """""" implements(IowwService) def rebuild(self, modname): """Rebuild classes in given module""" mod = sys.modules[modname] rebuild.rebuild(mod) def main(): """""" application = service.Application('OWW', uid=100, gid=100) s = owwService() serviceCollection = service.IServiceCollection(application) internet.TCPServer(7080, server.Site(resource.IResource(s))).setServiceParent(serviceCollection) serviceCollection.startService() try: try: print 'Running... ' reactor.run() except KeyboardInterrupt: print 'CTRL-C received, shutting down server.' except Exception, e: print 'Problem with running server.' finally: reactor.stop() serviceCollection.stopService() if __name__ == '__main__': main() And the client code: #!/usr/bin/env python from twisted.web.xmlrpc import Proxy from twisted.internet import reactor def printValue(value): print repr(value) reactor.stop() def printError(error): print 'error', error reactor.stop() proxy = Proxy('http://localhost:7080') proxy.callRemote('echo', 'hello world').addCallbacks(printValue, printError) reactor.run() [View Less]

2 3

[Twisted-Python] Layout bug in plugin.html
by Brett Viren Nov. 8, 2005

Nov. 8, 2005

Can someone with access fix http://twistedmatrix.com/projects/core/documentation/howto/plugin.html All the code examples lost their layout. Thanks, -Brett.

2 1

[Twisted-Python] IRC: getting auth of a user
by Kai Schulte Nov. 7, 2005

Nov. 7, 2005

Hello! Sorry if this is the wrong list, or has already been answered. I`m writing a little bot and I need to get the irc-auth of someone who is sending a privmsg to the bot. Probably it is easy but I can`t figure it out. So any help would be appreciated. Thanks Kai

2 6

[Twisted-Python] spawnProcess() for win32 (newbie)
by william.lewis＠uk.bnpparibas.com Nov. 7, 2005

Nov. 7, 2005

Hi, When running your "Verbose Example" in the "Using Processes" documentation (on Windows XP) I get a NotImplementedError exception raised from the spawnProcess() method in the twisted/internet/posixbase.py module. From the comments in the code it looks like Itamar has started to implement this for Windows, but it's not quite there yet. I guess my question is: when is it likely that you will have an implementation of the ProcessProtocol for win32 systems? I have installed Twisted 2.0.1 … [View More]

2 1

[Twisted-Python] Re: [Twisted-commits] r15052 - Merge forward r14926:15051 -- insomnia
by Jean-Paul Calderone Nov. 7, 2005

Nov. 7, 2005

On Mon, 07 Nov 2005 07:37:30 -0700, Jonathan Lange <jml(a)wolfwood.twistedmatrix.com> wrote: >Author: jml >Date: Mon Nov 7 07:37:28 2005 >New Revision: 15052 > > [snip] >Log: >Merge forward r14926:15051 -- insomnia > You should always create a new branch, copied from trunk, to merge forward (<http://divmod.org/trac/wiki/MergedForward>). Doing it in-place makes it much more difficult to determine which changes are yours and which changes belong to someone … [View More]

1 0

Re: [Twisted-Python] Unjelly - recursion limit reached
by Brian Warner Nov. 7, 2005

Nov. 7, 2005

> When a view_ method is called in a cacheable and the return value is a > graph of interconnected cacheables, is the return value completely > serialized at that point? Do you mean that some view_ method is being invoked on a pb.Referenceable (or pb.ViewPoint, I suppose) ? And that method is returning an instance that inherits from pb.Cacheable? And this pb.Cacheable instance holds references to other pb.Cacheables? Which may or may not have been transmitted across this wire once … [View More]already? The general answer is yes, the return value is completely serialized right then. The serialized form of the return value is transmitted in pieces as jelly recurses down the retval object graph, each new node visited causes a little bit more data to be sent. However, it does not stop for anything, and there is no way for an object being serialized to indicate that it wants to put off serialization for a while[1]. On the receiving end, the callRemote()'s Deferred will not be fired until the retval has completely finished deserialization. Intermediate objects may be constructed while deserialization is taking place, but that ought to be invisible to the caller. I'm not sure what would happen if, say, your getStateToCacheAndObserveFor method did a callRemote though the same wire. newpb has a queue to handle this sort of thing (the callRemote doesn't get transmitted until at least after the current operation has finished), but I don't know what oldpb does. If it isn't clever enough to be reentrant, the receiving end will get interleaved object state from the two operations and the results will be very very messy. getStateToCacheAndObserveFor and getStateToCopy are called from PB internals, and as a result it may not be safe to make other PB calls from there. view_* is called when the wire is in a stable state (i.e. remote methods calls are top-level objects on the wire), so I think it should be safe to make arbitrary PB calls from it. > I'm concerned that my hierarchy of cacheables received by the client > is inconsistent because it is being modified by another client view_ > call into the server before the previous hierarchy has been fully > serialized to the first client. Note that each callRemote is more-or-less atomic, and the creation of a whole object graph is more-or-less atomic, but it is entirely possible that other method calls will happen during the middle of a huge slew of RemoteCache updates. So if object A modifies a dozen Cacheables at once, and object B has a RemoteCache that is watching them, object C might sneak in a callRemote while only half of the updates have been processed. The same thing will happen if your Cacheable update method does multiple callRemotes to do its job. hope that helps somehow, -Brian [1]: in newpb, if you enable it, Slicers can return a Deferred to indicate that they want to stall serialization for a while. This raises concerns about knowing when, if ever, serialization will resume, and increases uncertainty about state coherency, because all sorts of stuff could happen by the time serialization is resumed. [View Less]

2 1

[Twisted-Python] LDAP Checker
by Manuel Aristarán Nov. 7, 2005

Nov. 7, 2005

Hi everybody, I'm learning Twisted, and already have a simple application running. I wante to experiment adding an LDAP checker to it. Is there any open source implementation of such a thing? I'm aware of LDAPtor, but I'm looking for some examples of integration with twisted.cred. Thanks, -- Manuel Aristarán <maristaran(a)gmail.com> Buenos Aires, Argentina

3 2