[Twisted-Python] dbcred and Firebird Incompatibility
First the stuff that makes sense: --- Firebird doesn't like batched SQL statements in a single string any better than MySQL does... Also it doesn't like the Primary Key in dbcred's schema not being specified as NOT NULL In twisted_identities: identity_name varchar(64) NOT NULL PRIMARY KEY In twisted_services: service_name varchar(64) NOT NULL PRIMARY KEY --- Now for something weird... Firebird chokes in CREATE TABLE twisted_identities on the password column... by changing the name to identity_password it works?!?!?!?!? -J
On Wed, 23 Oct 2002 00:14:04 -0500, "Joshua Ritter" <ritterjoshua@msn.com> wrote:
Firebird chokes in CREATE TABLE twisted_identities on the password column... by changing the name to identity_password it works?!?!?!?!?
It's almost certainly wrong, but the first thing that occurs to me is: twistedidentities identitypassword ^ ^ ^ ^ ^ 4 8 12 16 18 twisted_identities has more than 16 alphanumeric characters in it, whereas identity_password has exactly 16. Is this a hard limit in firebird, by chance? ;-) -- | <`'> | Glyph Lefkowitz: Travelling Sorcerer | | < _/ > | Lead Developer, the Twisted project | | < ___/ > | http://www.twistedmatrix.com |
Hi, I am in the process of adding the option to compress logfiles, to twisted.python.logfile. So that Twisted doesn't come to a standstill whilst compress a large file, I made the compress function run in a seperate thread. However, if the file takes longer than a couple of seconds to compress, and rotate is called again, another thread is launched. This thread, attempts to write to the same file as the initial thread. I've had a look at the code, but I can't seem to find a provision within Twisted, to queue the threads. Is there a way to do this, or, should I be looking at something else? This only ever occurs, when the log rotation is set to occur very frequently, (i.e. every couple of seconds, or every 100 bytes), but I'd rather fix it, than impose limits on the frequency of log rotation. Thanks John
On Wed, 23 Oct 2002 13:32:21 +0100 John Abel <john.abel@pa.press.net> wrote:
Set a flag "compressionInProgress". If it is set, just remember you need to compress. When the thread finishes it should call a callback (via reactor.callFromThread, or use threads.deferToThread) and that callback will start up a new compression. -- Itamar Shtull-Trauring http://itamarst.org/ Available for Python, Twisted, Zope and Java consulting
On Wed, 23 Oct 2002 13:32:21 +0100, John Abel <john.abel@pa.press.net> wrote:
So that Twisted doesn't come to a standstill whilst compress a large file, I made the compress function run in a seperate thread.
This thread, attempts to write to the same file as the initial thread.
Why are two separate compression attempts writing to the same file? Shouldn't the file be named something along the lines of twistd.{1,2,3,4,...}.log.gz?
Yes, there is a way to do this. Twisted provides utilities to have threads interact with its mainloop, but it does not provide a full threading model: that's Python's job. Take a look at the "threading" and "Queue" modules in the standard library.
Yeah, and it's because of problems like this that we try to steer clear of threads in general in Twisted ;-). Have you considered the block-at-a-time approach that radix suggested? That way you wouldn't need to run threads at all... -- | <`'> | Glyph Lefkowitz: Travelling Sorcerer | | < _/ > | Lead Developer, the Twisted project | | < ___/ > | http://www.twistedmatrix.com |
that is odd.. I did do some tests with Firebird with enterprise and row, but have since uninstalled it :( I would guess that "password" is a key word in Firebird's SQL dialect. Maybe we should change that column name to something less likely to cause that kind of conflict. -----Original Message----- From: twisted-python-admin@twistedmatrix.com [mailto:twisted-python-admin@twistedmatrix.com]On Behalf Of Joshua Ritter Sent: Wednesday, October 23, 2002 12:14 AM To: twisted-python@twistedmatrix.com Subject: [Twisted-Python] dbcred and Firebird Incompatibility First the stuff that makes sense: --- Firebird doesn't like batched SQL statements in a single string any better than MySQL does... Also it doesn't like the Primary Key in dbcred's schema not being specified as NOT NULL In twisted_identities: identity_name varchar(64) NOT NULL PRIMARY KEY In twisted_services: service_name varchar(64) NOT NULL PRIMARY KEY --- Now for something weird... Firebird chokes in CREATE TABLE twisted_identities on the password column... by changing the name to identity_password it works?!?!?!?!? -J _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
PASSWORD is a reserved word in interbase and by extension probably firebird but not in SQL 92. ref: http://ibphoenix.com/main.nfs?a=ibphoenix&page=ibp_reserved regards, Darryl
On Wed, 23 Oct 2002 00:14:04 -0500, "Joshua Ritter" <ritterjoshua@msn.com> wrote:
Firebird chokes in CREATE TABLE twisted_identities on the password column... by changing the name to identity_password it works?!?!?!?!?
It's almost certainly wrong, but the first thing that occurs to me is: twistedidentities identitypassword ^ ^ ^ ^ ^ 4 8 12 16 18 twisted_identities has more than 16 alphanumeric characters in it, whereas identity_password has exactly 16. Is this a hard limit in firebird, by chance? ;-) -- | <`'> | Glyph Lefkowitz: Travelling Sorcerer | | < _/ > | Lead Developer, the Twisted project | | < ___/ > | http://www.twistedmatrix.com |
Hi, I am in the process of adding the option to compress logfiles, to twisted.python.logfile. So that Twisted doesn't come to a standstill whilst compress a large file, I made the compress function run in a seperate thread. However, if the file takes longer than a couple of seconds to compress, and rotate is called again, another thread is launched. This thread, attempts to write to the same file as the initial thread. I've had a look at the code, but I can't seem to find a provision within Twisted, to queue the threads. Is there a way to do this, or, should I be looking at something else? This only ever occurs, when the log rotation is set to occur very frequently, (i.e. every couple of seconds, or every 100 bytes), but I'd rather fix it, than impose limits on the frequency of log rotation. Thanks John
On Wed, 23 Oct 2002 13:32:21 +0100 John Abel <john.abel@pa.press.net> wrote:
Set a flag "compressionInProgress". If it is set, just remember you need to compress. When the thread finishes it should call a callback (via reactor.callFromThread, or use threads.deferToThread) and that callback will start up a new compression. -- Itamar Shtull-Trauring http://itamarst.org/ Available for Python, Twisted, Zope and Java consulting
On Wed, 23 Oct 2002 13:32:21 +0100, John Abel <john.abel@pa.press.net> wrote:
So that Twisted doesn't come to a standstill whilst compress a large file, I made the compress function run in a seperate thread.
This thread, attempts to write to the same file as the initial thread.
Why are two separate compression attempts writing to the same file? Shouldn't the file be named something along the lines of twistd.{1,2,3,4,...}.log.gz?
Yes, there is a way to do this. Twisted provides utilities to have threads interact with its mainloop, but it does not provide a full threading model: that's Python's job. Take a look at the "threading" and "Queue" modules in the standard library.
Yeah, and it's because of problems like this that we try to steer clear of threads in general in Twisted ;-). Have you considered the block-at-a-time approach that radix suggested? That way you wouldn't need to run threads at all... -- | <`'> | Glyph Lefkowitz: Travelling Sorcerer | | < _/ > | Lead Developer, the Twisted project | | < ___/ > | http://www.twistedmatrix.com |
that is odd.. I did do some tests with Firebird with enterprise and row, but have since uninstalled it :( I would guess that "password" is a key word in Firebird's SQL dialect. Maybe we should change that column name to something less likely to cause that kind of conflict. -----Original Message----- From: twisted-python-admin@twistedmatrix.com [mailto:twisted-python-admin@twistedmatrix.com]On Behalf Of Joshua Ritter Sent: Wednesday, October 23, 2002 12:14 AM To: twisted-python@twistedmatrix.com Subject: [Twisted-Python] dbcred and Firebird Incompatibility First the stuff that makes sense: --- Firebird doesn't like batched SQL statements in a single string any better than MySQL does... Also it doesn't like the Primary Key in dbcred's schema not being specified as NOT NULL In twisted_identities: identity_name varchar(64) NOT NULL PRIMARY KEY In twisted_services: service_name varchar(64) NOT NULL PRIMARY KEY --- Now for something weird... Firebird chokes in CREATE TABLE twisted_identities on the password column... by changing the name to identity_password it works?!?!?!?!? -J _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
PASSWORD is a reserved word in interbase and by extension probably firebird but not in SQL 92. ref: http://ibphoenix.com/main.nfs?a=ibphoenix&page=ibp_reserved regards, Darryl
participants (6)
-
Glyph Lefkowitz -
Itamar Shtull-Trauring -
John Abel -
Joshua Ritter -
Maniac - -
Sean Riley