Convert from unsigned long long to PyLong
HI There, Maybe I should not post this in the dev group, but I think it has some relationship on the Python core. I'm using MySQLdb as the MySQL client. Recently I got a weird problem of this library. After looking into it, I suspect the problem may related to the conversion from unsigned long to PyLongObject. Here is the detail, If you are familiar with MySQLdb, the following snippet is a way to query the data from MySQL: connection = MySQLdb.connect(...) connection.autocommit(True) try: cursor = connection.cursor() if not cursor.execute(sql, values) > 0: return None row = cursor.fetchone() finally: connection.close() return row[0] Sometimes the return value of execute method would be 18446744073709552000 even there is no matched data available. I checked the source code of the library, the underlying implementation is https://github.com/farcepest/MySQLdb1/blob/master/_mysql.c#L835, static PyObject * _mysql_ConnectionObject_affected_rows( _mysql_ConnectionObject *self, PyObject *args) { if (!PyArg_ParseTuple(args, "")) return NULL; check_connection(self); return PyLong_FromUnsignedLongLong(mysql_affected_rows(&(self-> connection))); } And here is the official doc for mysql_affected_rows http://dev.mysql.com/doc/refman/5.7/en/mysql-affected-rows.html. Let me give a superficial understanding, please correct me if I were wrong. In a 64-bit system, the mysql_affected_rows is supposed to return a number of unsigned long, which means the range should be 0 ~ 2^64 (18446744073709551616), How could it be possible the function PyLong_FromUnsignedLongLong return a converted value larger than 2^64, that's what I don't understand. Does anyone have some ideas of it? The versions of the components I used: Python: 2.7.6 MySQL 5.7.11 MySQLdb 1.2.5 Thanks
So to sum this up, you claim that PyLong_FromUnsignedLongLong can somehow produce a number larger than the value range of a 64 bit number (0x10000000000000180). I have a hard time believing this. Most likely you are looking in the wrong place, mysql_affected_rows returns 2^64-1, and some Python code later adds 0x181 to that number.
I know it's hard to believe this, I wish I'm wrong. But after looking into the code for one week, I didn't find any other code change the number. I will go through them again make sure I didn't miss anything. Thanks for the reply. On Fri, Jul 22, 2016 at 5:02 PM, Stefan Ring <stefanrin@gmail.com> wrote:
So to sum this up, you claim that PyLong_FromUnsignedLongLong can somehow produce a number larger than the value range of a 64 bit number (0x10000000000000180). I have a hard time believing this.
Most likely you are looking in the wrong place, mysql_affected_rows returns 2^64-1, and some Python code later adds 0x181 to that number.
-- kenshin http://kenbeit.com Just Follow Your Heart
On Fri, Jul 22, 2016 at 3:02 AM, Stefan Ring <stefanrin@gmail.com> wrote:
So to sum this up, you claim that PyLong_FromUnsignedLongLong can somehow produce a number larger than the value range of a 64 bit number (0x10000000000000180). I have a hard time believing this.
Perhaps I misunderstood your meaning, but Python's integers (AKA "PyLong") can be bigger that a machine-native integer (e.g. 64 bits): "All integers are implemented as “long” integer objects of *arbitrary size*." (emphasis mine) (https://docs.python.org/3.5//c-api/long.html) -eric
Yes, you are right. Definitely "long" in Python can represent a number much bigger than the native. But the range of returned value from mysql_affected_rows within 0 ~ 2^64-1. No matter how it's converted, the converted value in Python also should in the range of 0 ~ 2^64 - 1. On Fri, Jul 22, 2016 at 11:50 PM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
On Fri, Jul 22, 2016 at 3:02 AM, Stefan Ring <stefanrin@gmail.com> wrote:
So to sum this up, you claim that PyLong_FromUnsignedLongLong can somehow produce a number larger than the value range of a 64 bit number (0x10000000000000180). I have a hard time believing this.
Perhaps I misunderstood your meaning, but Python's integers (AKA "PyLong") can be bigger that a machine-native integer (e.g. 64 bits):
"All integers are implemented as “long” integer objects of *arbitrary size*." (emphasis mine)
(https://docs.python.org/3.5//c-api/long.html)
-eric
-- kenshin http://kenbeit.com Just Follow Your Heart
Hey Guys, I found the mistake I made, basically I'm using a tool called Sentry to capture the exceptions. The value returned from the Python is 2^64-1, which is -1 from mysql_affected_rows. Sentry is using JSON format as the a kind of storage, apparently the MAX SAFE INTEGER <https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Obj...> is 2^53 -1. Sorry for the indeliberated report of this issue and thanks for all of you helps. On Sat, Jul 23, 2016 at 12:06 AM, Tian JiaLin <himurakenshin54@gmail.com> wrote:
Yes, you are right. Definitely "long" in Python can represent a number much bigger than the native.
But the range of returned value from mysql_affected_rows within 0 ~ 2^64-1. No matter how it's converted, the converted value in Python also should in the range of 0 ~ 2^64 - 1.
On Fri, Jul 22, 2016 at 11:50 PM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
On Fri, Jul 22, 2016 at 3:02 AM, Stefan Ring <stefanrin@gmail.com> wrote:
So to sum this up, you claim that PyLong_FromUnsignedLongLong can somehow produce a number larger than the value range of a 64 bit number (0x10000000000000180). I have a hard time believing this.
Perhaps I misunderstood your meaning, but Python's integers (AKA "PyLong") can be bigger that a machine-native integer (e.g. 64 bits):
"All integers are implemented as “long” integer objects of *arbitrary size*." (emphasis mine)
(https://docs.python.org/3.5//c-api/long.html)
-eric
-- kenshin
http://kenbeit.com Just Follow Your Heart
-- kenshin http://kenbeit.com Just Follow Your Heart
On 2016-07-23 05:23, Tian JiaLin wrote:
Hey Guys,
I found the mistake I made, basically I'm using a tool called Sentry to capture the exceptions. The value returned from the Python is 2^64-1, which is -1 from mysql_affected_rows. Sentry is using JSON format as the a kind of storage, apparently the MAX SAFE INTEGER <https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Obj...> is 2^53 -1.
[snip] JSON itself doesn't put a limit on the size of integers. That link is about how JavaScript handles JSON.
On Fri, Jul 22, 2016, at 11:50, Eric Snow wrote:
On Fri, Jul 22, 2016 at 3:02 AM, Stefan Ring <stefanrin@gmail.com> wrote:
So to sum this up, you claim that PyLong_FromUnsignedLongLong can somehow produce a number larger than the value range of a 64 bit number (0x10000000000000180). I have a hard time believing this.
Perhaps I misunderstood your meaning, but Python's integers (AKA "PyLong") can be bigger that a machine-native integer (e.g. 64 bits):
Yes, but you can't (or shouldn't be able to) get those values from a conversion from a machine-native type such as PyLong_FromUnsignedLongLong.
participants (5)
-
Eric Snow -
MRAB -
Random832 -
Stefan Ring -
Tian JiaLin