[Twisted-Python] SELECT returns no results without conn.commit()
Hi! I'm trying to extend ppolicy (http://bimbo.fjfi.cvut.cz/ppolicy) a bit. ppolicy uses the twisted framework. On CentOS 6.5 version 8.2.0 of twisted is installed. I wrote a separate module called LookupSessionDB: http://pastebin.com/u4TMNhpd It looks up some values from a database table. It does some efforts to handle reconnect properly. If called a second time and more the SELECT statement returns no rows (line 138) if "conn.commit()" is missing in line 125. If "conn.commit()" is called after obtaining a database connection "rows = cursor.fetchall()" (line 138) returns the expected results. The module inserting the records calls "conn.commit()" after INSERT. If I execute the same SELECT statement in a separate mysql client the results are displayed. This is quite confusing. Any idea what's going on? Best regards Dirk
On Jun 3, 2014, at 6:31 AM, Franz Schwartau <franz@electromail.org> wrote:
Hi!
I'm trying to extend ppolicy (http://bimbo.fjfi.cvut.cz/ppolicy) a bit. ppolicy uses the twisted framework. On CentOS 6.5 version 8.2.0 of twisted is installed.
I should note that Twisted 8.2.0 is 6 years old at this point, so if you can't upgrade Twisted there might not be much we can do for you :).
I wrote a separate module called LookupSessionDB:
It looks up some values from a database table. It does some efforts to handle reconnect properly.
If called a second time and more the SELECT statement returns no rows (line 138) if "conn.commit()" is missing in line 125. If "conn.commit()" is called after obtaining a database connection "rows = cursor.fetchall()" (line 138) returns the expected results.
The module inserting the records calls "conn.commit()" after INSERT. If I execute the same SELECT statement in a separate mysql client the results are displayed.
This is quite confusing. Any idea what's going on?
Can you simplify this question somewhat? There's a lot of SQL in here, it's missing a required dependency ("Base") and I suspect that this is really a MySQL question and not a Twisted question. It would be helpful if you could provide a complete schema so we could run your code, in the style of http://sscce.org/. -glyph
Hi Glyph! On Tue, Jun 03, 2014 at 11:57:45AM -0700, Glyph Lefkowitz wrote:
On Jun 3, 2014, at 6:31 AM, Franz Schwartau <franz@electromail.org> wrote:
Hi!
I'm trying to extend ppolicy (http://bimbo.fjfi.cvut.cz/ppolicy) a bit. ppolicy uses the twisted framework. On CentOS 6.5 version 8.2.0 of twisted is installed.
I should note that Twisted 8.2.0 is 6 years old at this point, so if you can't upgrade Twisted there might not be much we can do for you :).
I wrote a separate module called LookupSessionDB:
It looks up some values from a database table. It does some efforts to handle reconnect properly.
If called a second time and more the SELECT statement returns no rows (line 138) if "conn.commit()" is missing in line 125. If "conn.commit()" is called after obtaining a database connection "rows = cursor.fetchall()" (line 138) returns the expected results.
The module inserting the records calls "conn.commit()" after INSERT. If I execute the same SELECT statement in a separate mysql client the results are displayed.
This is quite confusing. Any idea what's going on?
Can you simplify this question somewhat? There's a lot of SQL in here, it's missing a required dependency ("Base") and I suspect that this is really a MySQL question and not a Twisted question. It would be helpful if you could provide a complete schema so we could run your code, in the style of http://sscce.org/. -glyph
Thanks for your answer. I know twisted 8.2.0 is quite old. Unfortunatly there are no newer RPM in RHEL/CentOS or even EPEL. I could provide the complete code. But it would be laborious to set up everything. You'd need postfix, ppolicy, mysql, etc. It would be easier (for you) if I'd set up a VM. IMHO it's related to twisted. If I replace adbapi.ConnectionPool() with another connection method it works. I used mysql.connector.connect(pool_name = "ppolicy", pool_size = 5, ...) and MySQLdb.connect(). Best regards Franz
On Jun 4, 2014, at 2:03 AM, Franz Schwartau <franz@electromail.org> wrote:
I know twisted 8.2.0 is quite old. Unfortunatly there are no newer RPM in RHEL/CentOS or even EPEL.
Yeah, RHEL is kind of a bummer.
I could provide the complete code. But it would be laborious to set up everything. You'd need postfix, ppolicy, mysql, etc. It would be easier (for you) if I'd set up a VM.
If you need more than MySQL + Twisted alone to reproduce the bug, then I would suggest that perhaps you haven't tracked it down far enough to do useful investigations within Twisted... I appreciate that you're willing to set up a VM, but the interesting thing is to extract the problem into a simple example, not to get the whole ball of mud running :). -glyph
Also, might I suggest that, at least for purposes of debugging, you use a virtualenv (or a throwaway VM if you prefer) and install a copy of Twisted from this decade. Twisted is very stable, and fixes lots of bugs in every release. So you're fairly unlikely to uncover *new* problems, but very likely to fix the problem you're seeing, by doing so. Dustin On Wed, Jun 4, 2014 at 3:20 PM, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:
On Jun 4, 2014, at 2:03 AM, Franz Schwartau <franz@electromail.org> wrote:
I know twisted 8.2.0 is quite old. Unfortunatly there are no newer RPM in RHEL/CentOS or even EPEL.
Yeah, RHEL is kind of a bummer.
I could provide the complete code. But it would be laborious to set up everything. You'd need postfix, ppolicy, mysql, etc. It would be easier (for you) if I'd set up a VM.
If you need more than MySQL + Twisted alone to reproduce the bug, then I would suggest that perhaps you haven't tracked it down far enough to do useful investigations within Twisted... I appreciate that you're willing to set up a VM, but the interesting thing is to extract the problem into a simple example, not to get the whole ball of mud running :).
-glyph _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
participants (3)
-
Dustin J. Mitchell -
Franz Schwartau -
Glyph Lefkowitz