[Twisted-Python] PB and other distributed object protocols
Hi Gang, Preface: I'm not a protocol guru, or even a "distributed object" (DO) guru, so this message is from the point of view of a DO novice looking for insights from some of the twisted minions (/gurus) and other cybergods who hang out here. I've been monitoring Brian's work on newpb (PBng?), and I love everything about it. Except how long it's taking. :( Brian: I know you have a real job, too, so I completely understand! (Incidentally, you're the only one whose commit messages I save, because they're always so well documented.) I've read the TODO list, and pretty much everything is beyond my abilities (and time) to contribute to -- sorry! -- but is it possible to put any kind of approximate timeline on some of it? PB is what I really want to use for my project, but I need something that's close to ready-for-prime-time, and I've been waiting literally years for PB, so in the meantime I'm looking at other candidates. I would definitely prefer a "twistable" technology -- i.e., one for which some kind of async adapter can be written from twisted -- and also one that has bindings for other languages (mainly Java, since my organization is Java-heavy). I've dismissed the idea of CORBA -- although one or more of the python orbs is probably twistable, CORBA has such a negative image that I'd probably have to lie about using it, and it does have lots of weird baggage. [Aside: I always liked Bill Janssen's ILU (how could you not like a distributed object system that had bindings for both python and lisp??) and I was sad to see it go. Interesting that Bill is now in the Python community.] Over the weekend I read the docs for ICE (http://www.zeroc.com/), sort of a streamlined, WAN-optimized CORBA. It has nice features, and even support for (threaded) async operations which seems (to my novice understanding) twistable. But its type system and python binding seem very unpythonic to me, based on lots of C++-isms, etc. Ugh. I just today discovered "json-rpc", which looks pretty cool. I currently use xml-rpc in my twisted app, and I've thought that json would be a neat replacement for xml (more concise + very pythonic syntax so probably *way* more efficient python marshalling/unmarshalling). And someone finally did it -- great! Has anyone here experimented with it? Anyone thought about twistifying it? If not, I might take a stab (I'd start by looking at what Itamar [I think] did with xml-rpc). Cheers, Steve
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Stephen Waterbury wrote:
I just today discovered "json-rpc", which looks pretty cool.
Perhaps you should look at "pyro" also: http://pyro.sourceforge.net/ - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@argo.es http://www.argo.es/~jcea/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBRIX+MJlgi5GaxT1NAQLcNwQAnjuVRWuUNihHRPK/vMsijYG1986xPqV8 +NPxAoAMx2WNJYdtem4gZJQUKn5Oip1V2anZonawcoA0Oru4H366pU0EUbAQyyWH ALI6aNkD9pwLKZTCkCJSvNvbb/ZPU+mcIYuV7ydj9S5rCmo+6HM5FhUhy9cO8TrW fQ0HkfqZrno= =EvKv -----END PGP SIGNATURE-----
Jesus Cea wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Stephen Waterbury wrote:
I just today discovered "json-rpc", which looks pretty cool.
Perhaps you should look at "pyro" also:
Pyro's been around for a while. It's useful for simple things but has several deficiencies relative to PB. Steve
Pyro's been around for a while. It's useful for simple things but has several deficiencies relative to PB.
I heard of http://rpyc.wikispaces.com/ -- Lawrence http://www.oluyede.org/blog
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Stephen Waterbury wrote:
Pyro's been around for a while. It's useful for simple things but has several deficiencies relative to PB.
Details? :-). I had have running pyro for ages (patched) and I'm fairly happy. My main complain is the centralized nameserver, so I privately implemented a distributed service locator using "spread" :-). - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@argo.es http://www.argo.es/~jcea/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBRI7BEZlgi5GaxT1NAQKxDwP/Y+KzO5Ja5dH4DxNkCuo1TVBBGKuRlo8R 9UwlLzjU87wXJaMAyaG4VL2TxKdw2hNSsGDfIFboEfXQW5aTSIITiiJU6KEEBbjM zalC8Ozj0Lv61fszkRZ8KxFVclR2qn/wNC3FvTa5gppjAUpEQX4eNpheRQYvXJDr SxNxzyKxu6s= =0HQc -----END PGP SIGNATURE-----
PB is what I really want to use for my project, but I need something that's close to ready-for-prime-time, and I've been waiting literally years for PB, so in the meantime I'm looking at other candidates.
At the risk of being too obvious, we've been using PB as shipped with Twisted since 1.3.0, and it's worked very well for us. Are you saying there's specific stuff you need from newpb that's not in the current implementation ? Thomas
Thomas Vander Stichele wrote:
PB is what I really want to use for my project, but I need something that's close to ready-for-prime-time, and I've been waiting literally years for PB, so in the meantime I'm looking at other candidates.
At the risk of being too obvious, we've been using PB as shipped with Twisted since 1.3.0, and it's worked very well for us. Are you saying there's specific stuff you need from newpb that's not in the current implementation ?
Yes: forward compatibility. ;) I've written PB code, too, and that code doesn't work with newpb. Not that I would expect it to, given the fairly substantial changes Brian made in developing newpb, nor do I think it would be good to have that as a constraint on newpb. Given newpb's list of features: http://twistedmatrix.com/trac/wiki/NewPB ... what would you *not* want?? ;) But of course, you said "need". My impression is that newpb will be more scalable, which is something my app will definitely need. Also, the better serialization architecture will be important, as some of my objects can be large. And the serializability of "most inert Python types", while not essential, is very nice to have. Also, my code is interface-driven, for which newpb is better adapted (so to speak :). Steve
On Thu, 08 Jun 2006 09:30:17 -0400, Stephen Waterbury <golux@comcast.net> wrote:
Thomas Vander Stichele wrote:
At the risk of being too obvious, we've been using PB as shipped with Twisted since 1.3.0, and it's worked very well for us. Are you saying there's specific stuff you need from newpb that's not in the current implementation ?
Yes: forward compatibility. ;) I've written PB code, too, and that code doesn't work with newpb. Not that I would expect it to, given the fairly substantial changes Brian made in developing newpb, nor do I think it would be good to have that as a constraint on newpb.
I think I actually disagree. If it's going to be called "PB", it should be compatible, in some way, with the existing "PB" infrastructure. Maybe not if you wrote insanely complex custom side-effecty custom jelliers, but at least you should be able to port fairly easily. While wire-level compatibility might be desirable, I think that application-level compatibility should actually become a requirement. As you note, Steve, the only thing that the promise of a PB2 that will be totally incompatible with PB1 is doing is discouraging people from looking at either right now. If it's not going to be compatible at all, we should come up with a new name and stop calling it PB. Maybe Twisted Caps or something. That said, I don't think that pb2 should be focused on application-level compatibility *right now*. It's got a bunch of problems left to solve, and what little time Brian can spend to work on it should be spent fixing those problems. (Can we please stop calling it "newpb", by the way? Or at least, for consistency, call Twisted 2.4, "newnewnewnewnewnewnewTwisted"?) Other applications are already using PB though, and many are doing it to good effect. So let me say this as unambiguously as possible: PB1 is still supported, and until further notice, it will continue to be. PB2 is still experimental and is *not* yet supported. We might not have a lot of development bandwidth to spend on PB1, but bugs are still being fixed, deprecations removed, and APIs updated to reflect the modern state of Twisted. It can be updated and fixed incrementally while we wait for the Great Revolution of PB2. Perhaps we can even start making the various APIs converge, so that the migration process will be smoother. This isn't a new decision or announcement, just a statement of the current state of affairs. Brian has been very clear about PB2's experimental status all along. Maintenance on PB1 never stopped. JP, for example, just recently removed a bunch of grotty old-cred stuff (hallelujiah it's gone) from PB1. It just seems like because there's this new direction, everyone tacitly assumes that the old stuff will be dropped -- Twisted has an unfortunate history as such things go, but we are getting better about it.
glyph@divmod.com wrote:
On Thu, 08 Jun 2006 09:30:17 -0400, Stephen Waterbury <golux@comcast.net> wrote:
Thomas Vander Stichele wrote:
At the risk of being too obvious, we've been using PB as shipped with Twisted since 1.3.0, and it's worked very well for us. Are you saying there's specific stuff you need from newpb that's not in the current implementation ?
Yes: forward compatibility. ;) I've written PB code, too, and that code doesn't work with newpb. Not that I would expect it to, given the fairly substantial changes Brian made in developing newpb, nor do I think it would be good to have that as a constraint on newpb.
I think I actually disagree. If it's going to be called "PB", it should be compatible, in some way, with the existing "PB" infrastructure. Maybe not if you wrote insanely complex custom side-effecty custom jelliers, but at least you should be able to port fairly easily.
Glyph, thanks for (finally ;) chiming in on this one! I'm happy to hear that PB1 -> PB2 ports shouldn't be too hard ... so I'll just shut up and use PB1. :)
While wire-level compatibility might be desirable, I think that application-level compatibility should actually become a requirement. As you note, Steve, the only thing that the promise of a PB2 that will be totally incompatible with PB1 is doing is discouraging people from looking at either right now.
Well, it had definitely discouraged me from using PB1, partly because Brian's commit messages were always so cogent and PB2 sounds so great, I just thought damn, well I'll wait for that ...
Other applications are already using PB though, and many are doing it to good effect.
Thanks -- another important thing to know. (I admit, I'd probably be more au courant if I'd tune in to irc once in a while!)
So let me say this as unambiguously as possible:
PB1 is still supported, and until further notice, it will continue to be. PB2 is still experimental and is *not* yet supported. We might not have a lot of development bandwidth to spend on PB1, but bugs are still being fixed, deprecations removed, and APIs updated to reflect the modern state of Twisted.
It can be updated and fixed incrementally while we wait for the Great Revolution of PB2. Perhaps we can even start making the various APIs converge, so that the migration process will be smoother.
This isn't a new decision or announcement, just a statement of the current state of affairs.
Well, it certainly was a much-needed clarification, IMO, and it helped my decision a great deal. Thanks!
... Brian has been very clear about PB2's experimental status all along.
Agreed. What I was not obvious was how long it would remain experimental. I can bleed for a while, but it's nice to have some idea of how many pints I'll need ... ;)
... Maintenance on PB1 never stopped. JP, for example, just recently removed a bunch of grotty old-cred stuff (hallelujiah it's gone) from PB1.
Right -- thanks, JP! I do monitor commits, so I was aware that PB1 was being maintained, which is important for my decision.
It just seems like because there's this new direction, everyone tacitly assumes that the old stuff will be dropped -- Twisted has an unfortunate history as such things go, but we are getting better about it.
Also agreed -- and I'm very happy to see Twisted continuing its steady evolution, and now a mainstay among Python libraries. Kinda validates my original judgment that it would be -- thanks to the whole Twisted community for that, especially Glyph and the core minions (you know who you are :). Cheers, Steve
participants (5)
-
glyph＠divmod.com -
Jesus Cea -
Lawrence Oluyede -
Stephen Waterbury -
Thomas Vander Stichele