PyObject_New vs PyObject_NEW
Someone I work with recently caused a test to start asserting in VC7's instrumented free() call, using a pydebug build. He explained the change this way: "I switched from PyObject_New to PyObject_NEW, which according to it's documentation omits the check for type_object != 0 and consequently should run a little bit faster" [he doesn't ever pass 0 as the typeobject] Did he miss some other important fact about PyObject_NEW? Does the doc need to be fixed? -- Dave Abrahams Boost Consulting www.boost-consulting.com
David Abrahams wrote:
Does he use PyObject_DEL() to free the object ? -- Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Mar 12 2003)
Python/Zope Products & Consulting ... http://www.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
Python UK 2003, Oxford: 20 days left EuroPython 2003, Charleroi, Belgium: 104 days left
You can read the source code as well as I can. --Guido van Rossum (home page: http://www.python.org/~guido/)
[David Abrahams]
Did he miss some other important fact about PyObject_NEW? Does the doc need to be fixed?
[Guido]
You can read the source code as well as I can.
Possibly, but not as well as I can <wink> -- the memory API's implementation is monumentally convoluted, especially before 2.3. Speaking of which, David, which version of Python was "someone" using? Did they enable pymalloc? Did they give you a traceback (showing from where free() was called)? Was it even freeing a Python object at the time? In what code base did someone make this substitution (e.g., Python core, Boost sources, someone's own extension module, someone else's extension module)? The straight answer to your question is no. A nastier answer is that many memory mgmt screwups are shy, and can be triggered by seemingly irrelevant changes.
Tim Peters <tim.one@comcast.net> writes:
I was the one who discovered the problem, using Python 2.2.2. Curiously, "someone" missed it because he was using vc6 instead of vc7.
Did they enable pymalloc?
I don't think I did that. I don't exactly know what pymalloc is.
Did they give you a traceback (showing from where free() was called)?
I can get one for you. Here:
Was it even freeing a Python object at the time?
Yup.
Boost sources
Both answers seem to amount to "'someone' must have a bug in his code". Am I reading that correctly? -- Dave Abrahams Boost Consulting www.boost-consulting.com
[David Abrahams]
So you were using VC7. If so, using it for what? Every stick of code in question, or were you mixing VC7-compiled code with VC6-compiled code? If the latter, talk to Microsoft (by most accounts their runtime support libraries aren't compatible with each other). [traceback freeing a tuple]
Both answers seem to amount to "'someone' must have a bug in his code". Am I reading that correctly?
Yes, for the right meaning of "someone". Possibilities beyond you include Python and Microsoft. Best guess I can make based on what you haven't told us yet is that you were mixing the released Python 2.2.2 Windows core DLL (built with MSVC6) with extension code using MSVC7 C runtime libraries. Right or wrong?
Tim Peters <tim.one@comcast.net> writes:
Python was compiled with vc6, the rest with vc7. I test this combination regularly and have never seen a problem.
If the latter, talk to Microsoft (by most accounts their runtime support libraries aren't compatible with each other).
Sure, but that's only an issue if you are allocating resources in one runtime lib and deallocating in another AFAIK. There's nothing beyond memory allocation going on here, and the type object in question has a custom deallocator which goes to the same runtime that allocated it.
Totally and absolutely right. -- Dave Abrahams Boost Consulting www.boost-consulting.com
[David Abrahams]
Python was compiled with vc6, the rest with vc7. I test this combination regularly and have never seen a problem.
You have now <wink>.
See my later msg -- returning memory to a heap it wasn't obtained from is fatal enough. The object memory itself is in question here, not memory allocated *by* the object. Look at the traceback you sent if the distinction isn't clear.
David Abrahams <dave@boost-consulting.com> writes:
Sure, but that's only an issue if you are allocating resources in one runtime lib and deallocating in another AFAIK.
No. You also cannot pass struct FILE* from one C library to the other; file locking will then crash. Regards, Martin
David Abrahams wrote:
Does he use PyObject_DEL() to free the object ? -- Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Mar 12 2003)
Python/Zope Products & Consulting ... http://www.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
Python UK 2003, Oxford: 20 days left EuroPython 2003, Charleroi, Belgium: 104 days left
You can read the source code as well as I can. --Guido van Rossum (home page: http://www.python.org/~guido/)
[David Abrahams]
Did he miss some other important fact about PyObject_NEW? Does the doc need to be fixed?
[Guido]
You can read the source code as well as I can.
Possibly, but not as well as I can <wink> -- the memory API's implementation is monumentally convoluted, especially before 2.3. Speaking of which, David, which version of Python was "someone" using? Did they enable pymalloc? Did they give you a traceback (showing from where free() was called)? Was it even freeing a Python object at the time? In what code base did someone make this substitution (e.g., Python core, Boost sources, someone's own extension module, someone else's extension module)? The straight answer to your question is no. A nastier answer is that many memory mgmt screwups are shy, and can be triggered by seemingly irrelevant changes.
Tim Peters <tim.one@comcast.net> writes:
I was the one who discovered the problem, using Python 2.2.2. Curiously, "someone" missed it because he was using vc6 instead of vc7.
Did they enable pymalloc?
I don't think I did that. I don't exactly know what pymalloc is.
Did they give you a traceback (showing from where free() was called)?
I can get one for you. Here:
Was it even freeing a Python object at the time?
Yup.
Boost sources
Both answers seem to amount to "'someone' must have a bug in his code". Am I reading that correctly? -- Dave Abrahams Boost Consulting www.boost-consulting.com
[David Abrahams]
So you were using VC7. If so, using it for what? Every stick of code in question, or were you mixing VC7-compiled code with VC6-compiled code? If the latter, talk to Microsoft (by most accounts their runtime support libraries aren't compatible with each other). [traceback freeing a tuple]
Both answers seem to amount to "'someone' must have a bug in his code". Am I reading that correctly?
Yes, for the right meaning of "someone". Possibilities beyond you include Python and Microsoft. Best guess I can make based on what you haven't told us yet is that you were mixing the released Python 2.2.2 Windows core DLL (built with MSVC6) with extension code using MSVC7 C runtime libraries. Right or wrong?
Tim Peters <tim.one@comcast.net> writes:
Python was compiled with vc6, the rest with vc7. I test this combination regularly and have never seen a problem.
If the latter, talk to Microsoft (by most accounts their runtime support libraries aren't compatible with each other).
Sure, but that's only an issue if you are allocating resources in one runtime lib and deallocating in another AFAIK. There's nothing beyond memory allocation going on here, and the type object in question has a custom deallocator which goes to the same runtime that allocated it.
Totally and absolutely right. -- Dave Abrahams Boost Consulting www.boost-consulting.com
[David Abrahams]
Python was compiled with vc6, the rest with vc7. I test this combination regularly and have never seen a problem.
You have now <wink>.
See my later msg -- returning memory to a heap it wasn't obtained from is fatal enough. The object memory itself is in question here, not memory allocated *by* the object. Look at the traceback you sent if the distinction isn't clear.
David Abrahams <dave@boost-consulting.com> writes:
Sure, but that's only an issue if you are allocating resources in one runtime lib and deallocating in another AFAIK.
No. You also cannot pass struct FILE* from one C library to the other; file locking will then crash. Regards, Martin
participants (5)
-
David Abrahams -
Guido van Rossum -
M.-A. Lemburg -
martin＠v.loewis.de -
Tim Peters