Providing support files to assist 3.x extension authors
Hello, When I ported gmpy (Python to GMP multiple precision library) to Python 3.x, I began to use PyLong_AsLongAndOverflow frequently. I found the code to slightly faster and cleaner than using PyLong_AsLong and checking for overflow. I looked at making PyLong_AsLongAndOverflow available to Python 2.x. http://bugs.python.org/issue7528 includes a patch that adds PyLong_AsLongAndOverflow to Python 2.7. I also included a file (py3intcompat.c) that can be included with an extension's source code and provides PyLong_AsLongAndOverflow to earlier versions of Python 2.x. In the bug report, I suggested that py3intcompat.c could be included in the Misc directory and be made available to extension authors. This follows the precedent of pymemcompat.h. But there may be more "compatibility" files that could benefit extension authors. Mark Dickinson suggested that I bring the topic on python-dev. Several questions come to mind: 1) Is it reasonable to provide backward compatibility files (either as .h or .c) to provide support to new API calls to extension authors? 2) If yes, should they be included with the Python source or distributed as a separate entity? (2to3 and/or 3to2 projects, a Wiki page) 3) If not, and extension authors can create their own compatibility files, are there any specific attribution or copyright messages that must be included? (I assuming the compatibility was done by extracting the code for the new API and tweaking it to run on older versions of Python.) Thanks in advance for your attention, Case Van Horsen
Case Vanhorsen wrote:
Hello,
When I ported gmpy (Python to GMP multiple precision library) to Python 3.x, I began to use PyLong_AsLongAndOverflow frequently. I found the code to slightly faster and cleaner than using PyLong_AsLong and checking for overflow. I looked at making PyLong_AsLongAndOverflow available to Python 2.x. http://bugs.python.org/issue7528 includes a patch that adds PyLong_AsLongAndOverflow to Python 2.7.
I also included a file (py3intcompat.c) that can be included with an extension's source code and provides PyLong_AsLongAndOverflow to earlier versions of Python 2.x. In the bug report, I suggested that py3intcompat.c could be included in the Misc directory and be made available to extension authors. This follows the precedent of pymemcompat.h. But there may be more "compatibility" files that could benefit extension authors. Mark Dickinson suggested that I bring the topic on python-dev.
Several questions come to mind:
1) Is it reasonable to provide backward compatibility files (either as .h or .c) to provide support to new API calls to extension authors?
As a minor terminology nit, I'd call these "forward compatibility" files - they're backports of Py3k interfaces to 2.x to make it easier to write modules that will compile with both 2.x and 3.x.
2) If yes, should they be included with the Python source or distributed as a separate entity? (2to3 and/or 3to2 projects, a Wiki page)
That depends - if we're only targeting 2.7, then including the compatibility files directly in the Python source would be fine. But for a solution to be usable with extensions intended for use with 2.6 (or even 2.5) it would be necessary to distribute the forward compatibility files separately.
3) If not, and extension authors can create their own compatibility files, are there any specific attribution or copyright messages that must be included? (I assuming the compatibility was done by extracting the code for the new API and tweaking it to run on older versions of Python.)
For code extracted from the Python source, the PSF licenses would still apply. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
Several questions come to mind:
1) Is it reasonable to provide backward compatibility files (either as .h or .c) to provide support to new API calls to extension authors?
I'm skeptical. In my experience, each extension has different needs, and will also use different strategies to provide compatibility. So publishing one way as the "official" approach might be difficult. In one case, the proposed approach for compatibility actually led to incorrect code (by ignoring exceptions when extracting a long), so we would need to be fairly careful what compatibility layers we can bless as official.
2) If yes, should they be included with the Python source or distributed as a separate entity? (2to3 and/or 3to2 projects, a Wiki page)
In the way you propose it (i.e. as forward compatibility files) I fail to see the point of including them with Python 2.7. Extension authors likely want to support versions of Python before that, which now cannot be changed. So those authors would still have to include the file on their own. So a file distributed in Include of 2.7 actually hurts, as it would conflict with the copy included in packages.
3) If not, and extension authors can create their own compatibility files, are there any specific attribution or copyright messages that must be included?
If you write a compatibility file from scratch, without copying existing code, you don't need to worry at all. If you do copy parts of Python, you must follow the license, in particular clause 2. Regards, Martin
On Sun, Dec 20, 2009 at 12:49 AM, "Martin v. Löwis" <martin@v.loewis.de> wrote:
Several questions come to mind:
1) Is it reasonable to provide backward compatibility files (either as .h or .c) to provide support to new API calls to extension authors?
I'm skeptical. In my experience, each extension has different needs, and will also use different strategies to provide compatibility. So publishing one way as the "official" approach might be difficult. In one case, the proposed approach for compatibility actually led to incorrect code (by ignoring exceptions when extracting a long), so we would need to be fairly careful what compatibility layers we can bless as official.
2) If yes, should they be included with the Python source or distributed as a separate entity? (2to3 and/or 3to2 projects, a Wiki page)
In the way you propose it (i.e. as forward compatibility files) I fail to see the point of including them with Python 2.7. Extension authors likely want to support versions of Python before that, which now cannot be changed. So those authors would still have to include the file on their own.
So a file distributed in Include of 2.7 actually hurts, as it would conflict with the copy included in packages.
3) If not, and extension authors can create their own compatibility files, are there any specific attribution or copyright messages that must be included?
If you write a compatibility file from scratch, without copying existing code, you don't need to worry at all. If you do copy parts of Python, you must follow the license, in particular clause 2.
Regards, Martin
Thanks for comments. I will just maintain my own version. Case
Case Vanhorsen, 20.12.2009 01:38:
When I ported gmpy (Python to GMP multiple precision library) to Python 3.x, I began to use PyLong_AsLongAndOverflow frequently. I found the code to slightly faster and cleaner than using PyLong_AsLong and checking for overflow.
You might want to look at the code Cython generates for integer type conversions. We use specialised coercion code that gets generated on-the-fly to convert Python long/int from and to all sorts of C integer types with compile time (portability) and runtime size/value checks. Depending on your needs, this may or may not be faster than the above C-API function. Stefan
participants (4)
-
"Martin v. Löwis" -
Case Vanhorsen -
Nick Coghlan -
Stefan Behnel