The wiki page for porting to 3.x says:
http://wiki.python.org/moin/PortingExtensionModulesToPy3k """ long/int Unification
In Python 3.0, there is only one integer type. It is called int on the Python level, but actually corresponds to 2.x's long type. In the C-API, PyInt_* functions are replaced by their PyLong_* neighbors. The best course of action here is using the PyInt_* functions aliased to PyLong_* found in intobject.h. """
However, intobject.h is no longer available. The checkin message for the removal says:
""" r71697 | mark.dickinson | 2009-04-18 12:12:16 +0200 (Sat, 18 Apr 2009) | 3 lines
The comments at the top of intobject.h say that it will be removed in 3.1. Make it so. """
Since package developers are just starting to port things to 3.x and many appear to be considering supporting both 2.7 and 3.1 (including myself), I find it a bit strange that such an import aliasing header was removed in 3.1.
IMHO, that's not really a good way to encourage people to try to provide a smooth upgrade to the 3.x branch. Much to the contrary. 3.x should make it easier for developers by providing more standard helpers like the removed intobject.h header file.
M.-A. Lemburg wrote:
Since package developers are just starting to port things to 3.x and many appear to be considering supporting both 2.7 and 3.1 (including myself), I find it a bit strange that such an import aliasing header was removed in 3.1.
There's some discussion of this at http://bugs.python.org/issue7353
You might want to comment there.
Eric Smith wrote:
M.-A. Lemburg wrote:
Since package developers are just starting to port things to 3.x and many appear to be considering supporting both 2.7 and 3.1 (including myself), I find it a bit strange that such an import aliasing header was removed in 3.1.
There's some discussion of this at http://bugs.python.org/issue7353
You might want to comment there.
Thanks for pointing me to the that ticket.
Looks like Guido already commented on this, so intobject.h could be revived in some form.
Since this file would only be used by extension modules and possibly also include other helper macros, perhaps it'd be better to to rename the file to py2compat.h or something along those lines ?!
We could then also have a py2compat.c to hold corresponding C code, e.g. to provide compatibility wrappers of new APIs that implement different semantics in 3.x.
On Mon, Nov 23, 2009 at 10:44 AM, M.-A. Lemburg mal@egenix.com wrote:
Thanks for pointing me to the that ticket.
Looks like Guido already commented on this, so intobject.h could be revived in some form.
I'm wondering how a resurrected intobject.h should be used: would Linux distributors (for example) package up intobject.h with the rest of the python-devel RPM/package/whatever so that Python extension modules could just include it directly, or would it be better to encourage the extension writers to make a copy of intobject.h to add to the source for their extension?
In the first case, intobject.h would become a requirement for those extensions, so we'd presumably end up committed to either maintaining intobject.h for the lifetime of Python 3.x, or causing some pain when it does eventually get deleted.
I quite like Benjamin's suggestion (in the issue tracker) of putting it in Doc/includes.
Since this file would only be used by extension modules and possibly also include other helper macros, perhaps it'd be better to to rename the file to py2compat.h or something along those lines ?!
We could then also have a py2compat.c to hold corresponding C code, e.g. to provide compatibility wrappers of new APIs that implement different semantics in 3.x.
This is also an interesting idea.
Mark
Mark Dickinson wrote:
On Mon, Nov 23, 2009 at 10:44 AM, M.-A. Lemburg mal@egenix.com wrote:
Thanks for pointing me to the that ticket.
Looks like Guido already commented on this, so intobject.h could be revived in some form.
I'm wondering how a resurrected intobject.h should be used: would Linux distributors (for example) package up intobject.h with the rest of the python-devel RPM/package/whatever so that Python extension modules could just include it directly, or would it be better to encourage the extension writers to make a copy of intobject.h to add to the source for their extension?
In the first case, intobject.h would become a requirement for those extensions, so we'd presumably end up committed to either maintaining intobject.h for the lifetime of Python 3.x, or causing some pain when it does eventually get deleted.
I quite like Benjamin's suggestion (in the issue tracker) of putting it in Doc/includes.
I'm not sure whether Doc/ is a good location for such things.
In the past, we've put such files in Misc/, e.g. the old Makefile.pre.in based extension mechanism was shipped that way:
Extension writers would simply copy the file to their extension directory, create a Setup.in file and then have the users run "make -f Makefile.pre.in boot" to have a Makefile created which then built the extension.
However, I'm not sure whether using the copy mechanism really works out that well: it's easier to just have the file(s) in the standard Python include file locations and then have extension writers use
#include "py2compat.h"
where necessary. Extension writer can then still use a private copy if needed, but most of the time they are probably better served by using the Python provided standard one.
Since this file would only be used by extension modules and possibly also include other helper macros, perhaps it'd be better to to rename the file to py2compat.h or something along those lines ?!
We could then also have a py2compat.c to hold corresponding C code, e.g. to provide compatibility wrappers of new APIs that implement different semantics in 3.x.
This is also an interesting idea.
M.-A. Lemburg <mal <at> egenix.com> writes:
We could then also have a py2compat.c to hold corresponding C code, e.g. to provide compatibility wrappers of new APIs that implement different semantics in 3.x.
If the semantic differences are embodied in the builtin object types I wonder how you can make a compatibility wrapper (short of reimplementing the 2.x types). An int is not a long, a bytes is not a str is not a unicode.
I'm not sure concealing the differences between 2.x and 3.x behind such a wrapper is a good idea. It would be better if people became aware of / learnt about the new semantics.
Regards
Antoine.
I'm not sure concealing the differences between 2.x and 3.x behind such a wrapper is a good idea. It would be better if people became aware of / learnt about the new semantics.
+1. When porting a larger code base, I always came up with a custom set of macros, specific to the way the modules were written in the package I ported. In particular for PyString_, some of the code meant to deal with bytes, other code with text. There was no way to tell, short of reading and understanding the code. As you'll have to review the complete code, anyway, the savings you gain by using macros somebody else wrote are really minor.
Regards, Martin
IMHO, that's not really a good way to encourage people to try to provide a smooth upgrade to the 3.x branch. Much to the contrary. 3.x should make it easier for developers by providing more standard helpers like the removed intobject.h header file.
I think it's better than it sounds. The macros (unfortunately) allowed to make non-obvious mistakes. Now that they are gone, people need to really think of what precisely they want to do.
For example, consider
if (PyInt_Check(o)){ long val = PyInt_AsLong(o); // process } else if (PyLong_Check(o)) { long long val = PyLong_AsLongLong(o); // check for overflow // process }
With intobject.h, this code would continue to compile, but work incorrectly, as the second case will never be executed. It would be better to port this as
#if Py2.x if (PyInt_Check(o)){ long val = PyInt_AsLong(o); // process } else #endif if (PyLong_Check(o)) {
i.e. eliminating the int case altogether. For another example,
long foo = PyInt_AsLong(Foo);
has a hidden error in 3.x, with intobject: PyLong_AsLong might overflow, which the 2.x case doesn't.
So eliminating intobject.h likely helps avoiding subtle errors.
Regards, Martin
On Sat, Nov 21, 2009 at 11:05 AM, "Martin v. Löwis" martin@v.loewis.de wrote:
IMHO, that's not really a good way to encourage people to try to provide a smooth upgrade to the 3.x branch. Much to the contrary. 3.x should make it easier for developers by providing more standard helpers like the removed intobject.h header file.
I think it's better than it sounds. The macros (unfortunately) allowed to make non-obvious mistakes. Now that they are gone, people need to really think of what precisely they want to do.
For example, consider
if (PyInt_Check(o)){ long val = PyInt_AsLong(o); // process } else if (PyLong_Check(o)) { long long val = PyLong_AsLongLong(o); // check for overflow // process }
With intobject.h, this code would continue to compile, but work incorrectly, as the second case will never be executed. It would be better to port this as
#if Py2.x if (PyInt_Check(o)){ long val = PyInt_AsLong(o); // process } else #endif if (PyLong_Check(o)) {
i.e. eliminating the int case altogether. For another example,
long foo = PyInt_AsLong(Foo);
has a hidden error in 3.x, with intobject: PyLong_AsLong might overflow, which the 2.x case doesn't.
So eliminating intobject.h likely helps avoiding subtle errors.
FWIW, I ported gmpy to Python 3.x without using intobject.h. I'm now using the #if Py2.x ... #endif approach (almost) everywhere. The same source compiles successfully with Python 2.4 to 3.2.
Case
Regards, Martin _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/casevh%40gmail.com
"Martin v. Löwis" wrote:
IMHO, that's not really a good way to encourage people to try to provide a smooth upgrade to the 3.x branch. Much to the contrary. 3.x should make it easier for developers by providing more standard helpers like the removed intobject.h header file.
I think it's better than it sounds. The macros (unfortunately) allowed to make non-obvious mistakes. Now that they are gone, people need to really think of what precisely they want to do.
For example, consider
if (PyInt_Check(o)){ long val = PyInt_AsLong(o); // process } else if (PyLong_Check(o)) { long long val = PyLong_AsLongLong(o); // check for overflow // process }
With intobject.h, this code would continue to compile, but work incorrectly, as the second case will never be executed. It would be better to port this as
#if Py2.x if (PyInt_Check(o)){ long val = PyInt_AsLong(o); // process } else #endif if (PyLong_Check(o)) {
i.e. eliminating the int case altogether.
Sure, but that assumes that the original code already had support for Python longs, which a lot of code doesn't.
In an ideal world, developers would add that code to their extensions right away. In the real world, where developers only have limited resources available, you'll get more 3.x ports by making such ports as painless as possible while at the same time not forcing them to alienate their 2.x user base.
The long support could then be added in later releases of the extensions, giving the developers more time adapt.
For another example,
long foo = PyInt_AsLong(Foo);
has a hidden error in 3.x, with intobject: PyLong_AsLong might overflow, which the 2.x case doesn't.
That's not quite true: PyInt_AsLong(obj) will try the nb_int slot on non-integer objects which can return errors (it returns -1 and sets the error message).
So eliminating intobject.h likely helps avoiding subtle errors.
In the long run, yes. In the short run, other criteria are more important, IMHO.
In an ideal world, developers would add that code to their extensions right away. In the real world, where developers only have limited resources available, you'll get more 3.x ports by making such ports as painless as possible while at the same time not forcing them to alienate their 2.x user base.
Unfortunately, such 3.x code would be full of bugs.
For another example,
long foo = PyInt_AsLong(Foo);
has a hidden error in 3.x, with intobject: PyLong_AsLong might overflow, which the 2.x case doesn't.
That's not quite true: PyInt_AsLong(obj) will try the nb_int slot on non-integer objects which can return errors (it returns -1 and sets the error message).
However, the 2.x code will typically assume that the object is an int object, in which case PyInt_AsLong can never fail. So with intobject.h, the code will happily compile, but then fail to detect an exception at run-time - causing other difficult-to-find bugs.
Regards, Martin
participants (6)
-
"Martin v. Löwis" -
Antoine Pitrou -
Case Vanhorsen -
Eric Smith -
M.-A. Lemburg -
Mark Dickinson