[Twisted-Python] adbapi MySQL read/write splitting
![](https://secure.gravatar.com/avatar/b5edb1094c1aa4e8a09ee0640a57f463.jpg?s=120&d=mm&r=g)
Is anyone doing read/write splitting with adbapi? Looking for advice if anyone has tackled this before. Thank you in advance. -J
![](https://secure.gravatar.com/avatar/b5edb1094c1aa4e8a09ee0640a57f463.jpg?s=120&d=mm&r=g)
That's one way of handling it. Another way is to wrap the library so it does the splitting automatically. The advantage to the latter is not making mistakes where you accidentally use the READ connection for a write. For non-async Python there are some tools like SQL Relay which will do this for you, but you have to setup SQL Relay servers and use the SQL Relay dbapi driver. MySQL Proxy will also do it, but it's not very stable. I was somewhat hoping someone had already written a wrapper for doing the splitting. Which brings me to a related question...if I were to write a wrapper to do the splitting, would wrapping MySQLdb be sufficient? Or would wrapping adbapi be the better path? -J On Tue, Aug 31, 2010 at 8:40 PM, Itamar Turner-Trauring <itamar@itamarst.org> wrote:
![](https://secure.gravatar.com/avatar/521b986cb6bb1912f71793aa76507a19.jpg?s=120&d=mm&r=g)
Used to wrap MySQLdb for something like except that I did it for sharding. It worked well on distributing load across multiple database servers. I think the guys from my former company updated the wrapper to use adbapi. - Alvin On Wed, Sep 1, 2010 at 10:54 AM, Jason J. W. Williams < jasonjwwilliams@gmail.com> wrote:
![](https://secure.gravatar.com/avatar/d7875f8cfd8ba9262bfff2bf6f6f9b35.jpg?s=120&d=mm&r=g)
On Tue, 2010-08-31 at 20:54 -0600, Jason J. W. Williams wrote:
That sounds like a bad idea. You want to send reads to the write server if you're doing so as part of a transaction that does writes, otherwise in some cases you'll end up with wrong results. For example, consider a transaction that inserts a row into a table, and then does a select to count the number of rows in that table. If you send the latter to a replicated read-only server, the result will be incorrect, since it will not include the insert (which hasn't been committed yet). So, to repeat: you should only use the read server for operations that are read-only. Which means some reads will go to the write server.
![](https://secure.gravatar.com/avatar/b5edb1094c1aa4e8a09ee0640a57f463.jpg?s=120&d=mm&r=g)
That is correct. In our case we've audited our code for transactions and all of ours are single statement trans. It's pretty typical for read/write splitting to forbid multiple-statement transactions as a cost of doing it. -J Sent via iPhone Is your e-mail Premiere? On Sep 1, 2010, at 7:15, Itamar Turner-Trauring <itamar@itamarst.org> wrote:
![](https://secure.gravatar.com/avatar/b5edb1094c1aa4e8a09ee0640a57f463.jpg?s=120&d=mm&r=g)
That's one way of handling it. Another way is to wrap the library so it does the splitting automatically. The advantage to the latter is not making mistakes where you accidentally use the READ connection for a write. For non-async Python there are some tools like SQL Relay which will do this for you, but you have to setup SQL Relay servers and use the SQL Relay dbapi driver. MySQL Proxy will also do it, but it's not very stable. I was somewhat hoping someone had already written a wrapper for doing the splitting. Which brings me to a related question...if I were to write a wrapper to do the splitting, would wrapping MySQLdb be sufficient? Or would wrapping adbapi be the better path? -J On Tue, Aug 31, 2010 at 8:40 PM, Itamar Turner-Trauring <itamar@itamarst.org> wrote:
![](https://secure.gravatar.com/avatar/521b986cb6bb1912f71793aa76507a19.jpg?s=120&d=mm&r=g)
Used to wrap MySQLdb for something like except that I did it for sharding. It worked well on distributing load across multiple database servers. I think the guys from my former company updated the wrapper to use adbapi. - Alvin On Wed, Sep 1, 2010 at 10:54 AM, Jason J. W. Williams < jasonjwwilliams@gmail.com> wrote:
![](https://secure.gravatar.com/avatar/d7875f8cfd8ba9262bfff2bf6f6f9b35.jpg?s=120&d=mm&r=g)
On Tue, 2010-08-31 at 20:54 -0600, Jason J. W. Williams wrote:
That sounds like a bad idea. You want to send reads to the write server if you're doing so as part of a transaction that does writes, otherwise in some cases you'll end up with wrong results. For example, consider a transaction that inserts a row into a table, and then does a select to count the number of rows in that table. If you send the latter to a replicated read-only server, the result will be incorrect, since it will not include the insert (which hasn't been committed yet). So, to repeat: you should only use the read server for operations that are read-only. Which means some reads will go to the write server.
![](https://secure.gravatar.com/avatar/b5edb1094c1aa4e8a09ee0640a57f463.jpg?s=120&d=mm&r=g)
That is correct. In our case we've audited our code for transactions and all of ours are single statement trans. It's pretty typical for read/write splitting to forbid multiple-statement transactions as a cost of doing it. -J Sent via iPhone Is your e-mail Premiere? On Sep 1, 2010, at 7:15, Itamar Turner-Trauring <itamar@itamarst.org> wrote:
participants (3)
-
Alvin Delagon
-
Itamar Turner-Trauring
-
Jason J. W. Williams