Mailman 3 August 2001 - Twisted

[Twisted-Python] manhole
by Chris Armstrong Aug. 9, 2001

Aug. 9, 2001

Glyph and I have talked about editing code from within manhole before. I'm not sure if he agrees with the following ideas, but they're related. I think it'd be really cool if we somehow kept a source representation of *all* of Twisted for dynamic editing at runtime (sounds cool, eh?).. Of course, this is probably far off, but here's a list of things that need to be done to accomplish this ridiculous goal. * At startup, read all of the source code from the .py files into an internal datastructure somewhere * whenever someone hacks new code from within manhole (or the telnet server (blech)), we save this too. (I actually had this idea in another context, when I wished for a special python interactive interpreter which would remember what I typed to save the code to modules later). * when we shut down the process, we must save all of this source code into the pickle as well. * offer a "source" functionality like reality's 'source' command to write out all changes to the source we've made. Yeah, I know, this seems really disgusting and memory-hogging, but it shouldn't be that bad. Actually, now that I think of it, this could be optimized some. Like, only actually retrieving source code when it's called for (that's probably the biggest optimization we can do). Now, this whole source code editing thing brings up more issues, like what do we edit source code with. Manhole is ok for some hacks, but it is of this hacker's opinion that we just call up external editors from the twisted process and pass it a temp file with a chunk of code in it. (glyph: "eggs"?) Another issue this brings up is the "stability of modifying a running process". I'd imagine only real twisted hackers would be screwing with the deep internals at runtime, but for regular "end-users" (hah), they'll probably stay close to the "edges" (very high level code). Now, this whole idea can really lead on into some crazy stuff.. Like, say, version controlling these "eggs" from *within* Twisted itself.. Thank you for listening to my crazy ramblings, -- Chris Armstrong carmstro(a)twistedmatrix.com http://twistedmatrix.com/~carmstro carmstro(a)dynup.net

1 0

[Twisted-Python] Working with Reality
by M.J. Stahl Aug. 9, 2001

Aug. 9, 2001

I would like to work towards developing Reality into a more mature environment, but (laughably) I am having trouble even getting the environment started (ie, I am recieving a traceback). Can someone please give me a general breakdown on getting TRDemo or Inheritance started. Mark

4 8

[Twisted-Python] Debian Packaging
by Moshe Zadka Aug. 9, 2001

Aug. 9, 2001

First of all, let me share with you a script I use to check whether the current CVS will generate a correct Debian package. Keep in mind that this is a test script, and is far from production quality. ---------- cut here ----- #!/bin/sh name=`dpkg-parsechangelog |grep Source | awk -F: '{print $2}'|tr -d ' '` version=`dpkg-parsechangelog |grep Version | awk -F: '{print $2}'|awk -F- '{print $1}'|tr -d ' '` build=../debian-build dir=$build/$name-$version mkdir $dir cp -ar * $dir cd $build find . -name CVS | xargs rm -rf tar czf ${name}_$version.orig.tar.gz $name-$version cd $dir dpkg-buildpackage -rfakeroot -uc -us cd .. rm -rf $name-$version ---------- cut here ----- Now, to have a well installed twisted.web server, just use this script (run it when the current directory is the toplevel CVS directory, and make sure there's a directory called "debian-build" as a sister directory) Now, go into ../debian-build install the generated twisted_*.deb Now, run /usr/bin/mktap web, or use your favourite tap file. use /usr/bin/tap2deb -t file.tap -m 'Your Name <email(a)example.com>' Go into .build, and install the generated .deb This will start your web server, and make sure it gets restarted on reboots. -- Moshe Zadka - http://moshez.geek (if you're not cool, http://moshez.org is still working)

1 0

[Twisted-Python] twisted.internet.ssl
by Itamar Shtull-Trauring Aug. 8, 2001

Aug. 8, 2001

1) You'll need pyopenssl from pyopenssl.sf.net. 2) This is totally untested. Hint: pyopenssl 0.3 segfaults if you try to write to an SSL connection before you connect(). -- itamar(at)shtull-trauring.org || itamar(at)zoteca.com http://itamarst.org || http://www.zoteca.com # System imports from OpenSSL import SSL # sibling imports import tcp class Client(tcp.Client): """I am an SSL client. """ def createContext(self): """ Create a SSL context. Subclasses may want to override. """ self.ctx = SSL.Context(SSL.SSLv23_METHOD) def createInternetSocket(self): """(internal) create an SSL socket """ sock = tcp.Client.createInternetSocket(self) return SSL.Connection(self.ctx, sock) class Port(tcp.Port): """I am an SSL server. """ def createContext(self): """ Create a SSL context. Subclasses may want to override. """ self.ctx = SSL.Context(SSL.SSLv23_METHOD) self.ctx.use_certificate_file('server.pem') self.ctx.use_privatekey_file('server.pem') def createInternetSocket(self): """(internal) create an SSL socket """ sock = tcp.Port.createInternetSocket(self) return SSL.Connection(self.ctx, sock) -----BEGIN CERTIFICATE----- MIIDBjCCAm+gAwIBAgIBATANBgkqhkiG9w0BAQQFADB7MQswCQYDVQQGEwJTRzER MA8GA1UEChMITTJDcnlwdG8xFDASBgNVBAsTC00yQ3J5cHRvIENBMSQwIgYDVQQD ExtNMkNyeXB0byBDZXJ0aWZpY2F0ZSBNYXN0ZXIxHTAbBgkqhkiG9w0BCQEWDm5n cHNAcG9zdDEuY29tMB4XDTAwMDkxMDA5NTEzMFoXDTAyMDkxMDA5NTEzMFowUzEL MAkGA1UEBhMCU0cxETAPBgNVBAoTCE0yQ3J5cHRvMRIwEAYDVQQDEwlsb2NhbGhv c3QxHTAbBgkqhkiG9w0BCQEWDm5ncHNAcG9zdDEuY29tMFwwDQYJKoZIhvcNAQEB BQADSwAwSAJBAKy+e3dulvXzV7zoTZWc5TzgApr8DmeQHTYC8ydfzH7EECe4R1Xh 5kwIzOuuFfn178FBiS84gngaNcrFi0Z5fAkCAwEAAaOCAQQwggEAMAkGA1UdEwQC MAAwLAYJYIZIAYb4QgENBB8WHU9wZW5TU0wgR2VuZXJhdGVkIENlcnRpZmljYXRl MB0GA1UdDgQWBBTPhIKSvnsmYsBVNWjj0m3M2z0qVTCBpQYDVR0jBIGdMIGagBT7 hyNp65w6kxXlxb8pUU/+7Sg4AaF/pH0wezELMAkGA1UEBhMCU0cxETAPBgNVBAoT CE0yQ3J5cHRvMRQwEgYDVQQLEwtNMkNyeXB0byBDQTEkMCIGA1UEAxMbTTJDcnlw dG8gQ2VydGlmaWNhdGUgTWFzdGVyMR0wGwYJKoZIhvcNAQkBFg5uZ3BzQHBvc3Qx LmNvbYIBADANBgkqhkiG9w0BAQQFAAOBgQA7/CqT6PoHycTdhEStWNZde7M/2Yc6 BoJuVwnW8YxGO8Sn6UJ4FeffZNcYZddSDKosw8LtPOeWoK3JINjAk5jiPQ2cww++ 7QGG/g5NDjxFZNDJP1dGiLAxPW6JXwov4v0FmdzfLOZ01jDcgQQZqEpYlgpuI5JE WUQ9Ho4EzbYCOQ== -----END CERTIFICATE----- -----BEGIN RSA PRIVATE KEY----- MIIBPAIBAAJBAKy+e3dulvXzV7zoTZWc5TzgApr8DmeQHTYC8ydfzH7EECe4R1Xh 5kwIzOuuFfn178FBiS84gngaNcrFi0Z5fAkCAwEAAQJBAIqm/bz4NA1H++Vx5Ewx OcKp3w19QSaZAwlGRtsUxrP7436QjnREM3Bm8ygU11BjkPVmtrKm6AayQfCHqJoT ZIECIQDW0BoMoL0HOYM/mrTLhaykYAVqgIeJsPjvkEhTFXWBuQIhAM3deFAvWNu4 nklUQ37XsCT2c9tmNt1LAT+slG2JOTTRAiAuXDtC/m3NYVwyHfFm+zKHRzHkClk2 HjubeEgjpj32AQIhAJqMGTaZVOwevTXvvHwNEH+vRWsAYU/gbx+OQB+7VOcBAiEA oolb6NMg/R3enNPvS1O4UU1H8wpaF77L4yiSWlE0p4w= -----END RSA PRIVATE KEY----- -----BEGIN CERTIFICATE REQUEST----- MIIBDTCBuAIBADBTMQswCQYDVQQGEwJTRzERMA8GA1UEChMITTJDcnlwdG8xEjAQ BgNVBAMTCWxvY2FsaG9zdDEdMBsGCSqGSIb3DQEJARYObmdwc0Bwb3N0MS5jb20w XDANBgkqhkiG9w0BAQEFAANLADBIAkEArL57d26W9fNXvOhNlZzlPOACmvwOZ5Ad NgLzJ1/MfsQQJ7hHVeHmTAjM664V+fXvwUGJLziCeBo1ysWLRnl8CQIDAQABoAAw DQYJKoZIhvcNAQEEBQADQQA7uqbrNTjVWpF6By5ZNPvhZ4YdFgkeXFVWi5ao/TaP Vq4BG021fJ9nlHRtr4rotpgHDX1rr+iWeHKsx4+5DRSy -----END CERTIFICATE REQUEST-----

3 2

[Twisted-Python] [URGENT] Missing packages in setup.py
by Moshe Zadka Aug. 8, 2001

Aug. 8, 2001

People, the following packages are missing from setup.py twisted.tap twisted.mail Please insert them ASAP. -- Moshe Zadka - http://moshez.geek (if you're not cool, http://moshez.org is still working)

2 2

[Twisted-Python] Daemonize...trying to reach a compromise.
by Moshe Zadka Aug. 8, 2001

Aug. 8, 2001

OK, here's where I think we stand: I add to daemonize another argument, a logfile to log to, and it is ok to check it in. Correct? -- Moshe Zadka - http://moshez.geek (if you're not cool, http://moshez.org is still working)

1 0

Re: [Twisted-Python] new function for t.i.main
by Moshe Zadka Aug. 8, 2001

Aug. 8, 2001

On Tue, 7 Aug 2001, Glyph Lefkowitz <glyph(a)twistedmatrix.com> wrote: > Daemonize's functionality is probably useful, however, there are a few > things it doesn't take into account. Insofar as it's useful, most of > twistd should probably be incorporated into it; any less is offering > incomplete functionality. No it isn't. It's offering *just* the daemonize functionality. You can have a higher-level wrapper *on top* of that which is portable, but daemonize() is useful. > After all, this is about as platform specific as you get. Yes it is. I firmly believe in platform-specific APIs which are then wrapped in platform independent APIs. I don't want anything to restrict my access to the OS > If we have > daemonize, then we probably want to have a similiar function for starting > a Windows Service. Yes, it would. This isn't an argument against daemonize. > Moreover, it would be good if we could detect which > platform we're on, then do the appropriate thing for creating a daemon on > that platform. Agreed. But not with daemonize() -- with startService(....) which would call daemonize() on UNIX. > This implies logging. "daemonize" won't work if you're still writing to > standard output after you call it. Well, it closes FD 1, so it works well enough for that. Logging is left for a higher level API. As is writing PID files (which must be done after daemonizing but before getting into the main loop) > So I'd say daemonize should at least take a logfile, and a function + > varargs, and never return. No, that's startService, another useful API. > This sounds cool, with one minor caveat. > > I don't want to give windows users the impression that they're totally > unsupported, especially any more than they already are. Fine, support them ;-) > We should make plans to have similiar windows-ize scripts, and even if we > don't have them, make it clear that we're not trying to make a > debian-specific server infrastructure. *I* don't care if my server infrastructure is Debian specific. > Also, wouldn't this "debian" functionality be very similiar for redhat, > mandrake, etc? Even solaris and MacOS X? "unixize" might be a better > name for it. No. Debian uses .deb packages. Maybe whoever does RPM chooses to redhatize or rpmize (I'm not sure: would she support SuSE? Mandrake?) Solaris uses its own unique wonderful format, as does MacOS X and AIX. I would be happy to put my /etc/init.d/ templates where everyone can use them. Everything else is distribution specific. It's not hard to have a hand install of a .tap if you have the /etc/init.d template. I want the Debian packages to be good enough that I'll feel good enough about uploading them to the official Debian archive, as well as maintainig a huge apt source on tm.com. -- Moshe Zadka - http://moshez.geek (if you're not cool, http://moshez.org is still working)

2 2

[Twisted-Python] CVS Sync Mails
by Moshe Zadka Aug. 8, 2001

Aug. 8, 2001

Can we e-mail twisted-python instead of acapnotic? *Please?* Or at least have a twisted-python-checkins(a)tm.com for that. -- Moshe Zadka - http://moshez.geek (if you're not cool, http://moshez.org is still working)

2 2

Re: [Twisted-Python] new function for t.i.main
by Moshe Zadka Aug. 8, 2001

Aug. 8, 2001

On Tue, 7 Aug 2001, Glyph Lefkowitz <glyph(a)twistedmatrix.com> wrote: > Daemonize's functionality is probably useful, however, there are a few > things it doesn't take into account. Insofar as it's useful, most of > twistd should probably be incorporated into it; any less is offering > incomplete functionality. No it isn't. It's offering *just* the daemonize functionality. You can have a higher-level wrapper *on top* of that which is portable, but daemonize() is useful. > After all, this is about as platform specific as you get. Yes it is. I firmly believe in platform-specific APIs which are then wrapped in platform independent APIs. I don't want anything to restrict my access to the OS > If we have > daemonize, then we probably want to have a similiar function for starting > a Windows Service. Yes, it would. This isn't an argument against daemonize. > Moreover, it would be good if we could detect which > platform we're on, then do the appropriate thing for creating a daemon on > that platform. Agreed. But not with daemonize() -- with startService(....) which would call daemonize() on UNIX. > This implies logging. "daemonize" won't work if you're still writing to > standard output after you call it. Well, it closes FD 1, so it works well enough for that. Logging is left for a higher level API. As is writing PID files (which must be done after daemonizing but before getting into the main loop) > So I'd say daemonize should at least take a logfile, and a function + > varargs, and never return. No, that's startService, another useful API. > This sounds cool, with one minor caveat. > > I don't want to give windows users the impression that they're totally > unsupported, especially any more than they already are. Fine, support them ;-) > We should make plans to have similiar windows-ize scripts, and even if we > don't have them, make it clear that we're not trying to make a > debian-specific server infrastructure. *I* don't care if my server infrastructure is Debian specific. > Also, wouldn't this "debian" functionality be very similiar for redhat, > mandrake, etc? Even solaris and MacOS X? "unixize" might be a better > name for it. No. Debian uses .deb packages. Maybe whoever does RPM chooses to redhatize or rpmize (I'm not sure: would she support SuSE? Mandrake?) Solaris uses its own unique wonderful format, as does MacOS X and AIX. I would be happy to put my /etc/init.d/ templates where everyone can use them. Everything else is distribution specific. It's not hard to have a hand install of a .tap if you have the /etc/init.d template. I want the Debian packages to be good enough that I'll feel good enough about uploading them to the official Debian archive, as well as maintainig a huge apt source on tm.com. -- Moshe Zadka - http://moshez.geek (if you're not cool, http://moshez.org is still working) -- Moshe Zadka - http://moshez.geek (if you're not cool, http://moshez.org is still working)

1 0

[Twisted-Python] new function for t.i.main
by Moshe Zadka Aug. 7, 2001

Aug. 7, 2001

daemonize() is cool, useful and should really be refactored out of twistd I really see no harm in providing more functionality in t.i.main Flamewars re: correct debian packaging will be left for later. def daemonize(): if os.fork(): os._exit(0) os.setsid() os.close(0) os.close(1) os.close(2) if os.fork(): os._exit(0) os.umask(0) Here's a quick thought about Debian packaging: add a script to ./bin "debianize". This script will take a .tap file and create a package named the same as the .tap file which relevant /etc/init.d/package and /var/lib/package/package.tap Perhaps even add it as a generic argument to mktap, so mktap --debian web --static /var/www will create the debian package. One caveat: version numbers. -- Moshe Zadka - http://moshez.geek (if you're not cool, http://moshez.org is still working)

2 2