Platform extension for distutils on other interpreters than CPython
Hello. I would like to have a feature on platform module (or sys or somewhere) that can tell distutils or distutils2 that this platform (be it PyPy or Jython) is not able to compile any C module. The purpose of this is to make distutils bail out in more reasonable manner than a compilation error in case this module is not going to work on anything but CPython. What do you think? Cheers, fijal
On Tue, Feb 23, 2010 at 12:50 PM, Maciej Fijalkowski <fijall@gmail.com>wrote:
I would like to have a feature on platform module (or sys or somewhere) that can tell distutils or distutils2 that this platform (be it PyPy or Jython) is not able to compile any C module. The purpose of this is to make distutils bail out in more reasonable manner than a compilation error in case this module is not going to work on anything but CPython.
Also, it would be nice if the package could tell distutils that the compilation is option, in order to support modules where the C version simply replaces functions for speed. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
On Tue, Feb 23, 2010 at 2:03 PM, Daniel Stutzbach <daniel@stutzbachenterprises.com> wrote:
On Tue, Feb 23, 2010 at 12:50 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
I would like to have a feature on platform module (or sys or somewhere) that can tell distutils or distutils2 that this platform (be it PyPy or Jython) is not able to compile any C module. The purpose of this is to make distutils bail out in more reasonable manner than a compilation error in case this module is not going to work on anything but CPython.
Also, it would be nice if the package could tell distutils that the compilation is option, in order to support modules where the C version simply replaces functions for speed.
There's an option called "optional" in the Extension class right now that will silent any compilation failures (like gcc not being there), so your project still installs. That's basically what people now can use when they have a pure Python fallback version they want to provide in case the C version cannot be built for any reason. Tarek -- Tarek Ziadé | http://ziade.org
On Tue, Feb 23, 2010 at 1:50 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Hello.
I would like to have a feature on platform module (or sys or somewhere) that can tell distutils or distutils2 that this platform (be it PyPy or Jython) is not able to compile any C module. The purpose of this is to make distutils bail out in more reasonable manner than a compilation error in case this module is not going to work on anything but CPython. FWIW this would be helpful for Jython too.
-Frank
On Tue, Feb 23, 2010 at 1:50 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Hello.
I would like to have a feature on platform module (or sys or somewhere) that can tell distutils or distutils2 that this platform (be it PyPy or Jython) is not able to compile any C module. The purpose of this is to make distutils bail out in more reasonable manner than a compilation error in case this module is not going to work on anything but CPython.
What do you think?
+1 I think we could have a global variable in sys, called "dont_compile", distutils would look at before it tris to compile stuff, exactly like how it does for pyc file (sys.dont_write_bytecode) Regards Tarek
On Tue, Feb 23, 2010 at 2:10 PM, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
On Tue, Feb 23, 2010 at 1:50 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Hello.
I would like to have a feature on platform module (or sys or somewhere) that can tell distutils or distutils2 that this platform (be it PyPy or Jython) is not able to compile any C module. The purpose of this is to make distutils bail out in more reasonable manner than a compilation error in case this module is not going to work on anything but CPython.
What do you think?
+1
I think we could have a global variable in sys, called "dont_compile", distutils would look at before it tris to compile stuff, exactly like how it does for pyc file (sys.dont_write_bytecode)
Or... wait : we already know if we are using CPython, or Jython reading sys.platform. So I could simply not trigger the compilation in case sys.platform is one of the CPythons and keep in distutils side a list of the platform names, Extension is incompatible with. That makes me wonder : why don't we have a sys.implementation variable ? (cython/jython/pypi), since we can have several values for cython in sys.platform Tarek
On Tue, Feb 23, 2010 at 2:27 PM, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
On Tue, Feb 23, 2010 at 2:10 PM, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
On Tue, Feb 23, 2010 at 1:50 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Hello.
I would like to have a feature on platform module (or sys or somewhere) that can tell distutils or distutils2 that this platform (be it PyPy or Jython) is not able to compile any C module. The purpose of this is to make distutils bail out in more reasonable manner than a compilation error in case this module is not going to work on anything but CPython.
What do you think?
+1
I think we could have a global variable in sys, called "dont_compile", distutils would look at before it tris to compile stuff, exactly like how it does for pyc file (sys.dont_write_bytecode)
Or... wait : we already know if we are using CPython, or Jython reading sys.platform.
So I could simply not trigger the compilation in case sys.platform is one of the CPythons and keep in distutils side a list of the platform names, Extension is incompatible with.
That makes me wonder : why don't we have a sys.implementation variable ? (cython/jython/pypi), since we can have several values for cython in sys.platform
Tarek
That's pypy. pypi is something else. sys.platform is not any good, since for example PyPy, and possibly any other python implementation that is not CPython, but it's not tied to any particular platform (like parrot) would say "linux2" or "win32". sys.implementation sounds good, but it'll also require a list in stdlib what's fine and what's not fine and a flag sounds like something that everyone can set, not asking to be listed in stdlib. How about sys.implementation.supports_extensions? Cheers, fijal
On Tue, Feb 23, 2010 at 2:44 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
On Tue, Feb 23, 2010 at 2:27 PM, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
On Tue, Feb 23, 2010 at 2:10 PM, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
On Tue, Feb 23, 2010 at 1:50 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Hello.
I would like to have a feature on platform module (or sys or somewhere) that can tell distutils or distutils2 that this platform (be it PyPy or Jython) is not able to compile any C module. The purpose of this is to make distutils bail out in more reasonable manner than a compilation error in case this module is not going to work on anything but CPython.
What do you think?
+1
I think we could have a global variable in sys, called "dont_compile", distutils would look at before it tris to compile stuff, exactly like how it does for pyc file (sys.dont_write_bytecode)
Or... wait : we already know if we are using CPython, or Jython reading sys.platform.
So I could simply not trigger the compilation in case sys.platform is one of the CPythons and keep in distutils side a list of the platform names, Extension is incompatible with.
That makes me wonder : why don't we have a sys.implementation variable ? (cython/jython/pypi), since we can have several values for cython in sys.platform
Tarek
That's pypy. pypi is something else. sys.platform is not any good, since for example PyPy, and possibly any other python implementation that is not CPython, but it's not tied to any particular platform (like parrot) would say "linux2" or "win32".
sys.implementation sounds good, but it'll also require a list in stdlib what's fine and what's not fine and a flag sounds like something that everyone can set, not asking to be listed in stdlib.
How about sys.implementation.supports_extensions?
I think its the other way around: You are making the assumption that, sys knows about distutils extensions. and there's no indication about the kind of extension here (C, etc..). In distutils, the Extension class works with a compiler class (ccompiler, mingwcompiler, etc) And in theory, people could add a new compiler that works under PyPy or Jython. So I think it's up to the Compiler class (through the Extension instance) to decide if the environment is suitable. For instance in the CCompiler class I can write things like: if sys.implementation != 'cython': warning.warn('Sorry I cannot compile this under %s' % sys.implementation) Tarek -- Tarek Ziadé | http://ziade.org
Tarek Ziadé wrote:
That makes me wonder : why don't we have a sys.implementation variable ? (cython/jython/pypi), since we can have several values for cython in sys.platform
You might want to try and catch up with Christian Heimes. He was going to write a PEP to add one, but must have been caught up with other things. See the thread starting at: http://mail.python.org/pipermail/python-dev/2009-October/092893.html Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
On Wed, Feb 24, 2010 at 6:35 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Tarek Ziadé wrote:
That makes me wonder : why don't we have a sys.implementation variable ? (cython/jython/pypi), since we can have several values for cython in sys.platform
Hello. So I propose to have a sys.implementation which would have string representation like "CPython" or "Jython" and have a couple of extra attributes on that. I can think about a lot of attributes, however, couple comes to mind as obvious. * supports_c_api - whether it can load and use CPython C modules * gc_strategy - probably equals "refcounting" or not, useful for some people * frame_introspection - This is mostly True for everybody except IronPython which has it as an optional command line argument. Might be useful to have it for some projects, unsure. What do you think? I'm willing to implement this for CPython and PyPy (this should be dead-simple anyway). Cheers, fijal
Maciej Fijalkowski wrote:
On Wed, Feb 24, 2010 at 6:35 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Tarek Ziadé wrote:
That makes me wonder : why don't we have a sys.implementation variable ? (cython/jython/pypi), since we can have several values for cython in sys.platform
Hello.
So I propose to have a sys.implementation which would have string representation like "CPython" or "Jython" and have a couple of extra attributes on that. I can think about a lot of attributes, however, couple comes to mind as obvious.
* supports_c_api - whether it can load and use CPython C modules * gc_strategy - probably equals "refcounting" or not, useful for some people * frame_introspection - This is mostly True for everybody except IronPython which has it as an optional command line argument. Might be useful to have it for some projects, unsure.
What do you think?
I think anything along these lines needs to be a PEP so that the developers of the different implementations all get a chance to comment and then have a firm standard to code to afterwards :) Christian was going to write one in the PEP 370 context, so it's worth following up with him to see if he ever got around to drafting anything. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
On Feb 23, 2010, at 2:10 PM, Tarek Ziadé wrote:
On Tue, Feb 23, 2010 at 1:50 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Hello.
I would like to have a feature on platform module (or sys or somewhere) that can tell distutils or distutils2 that this platform (be it PyPy or Jython) is not able to compile any C module. The purpose of this is to make distutils bail out in more reasonable manner than a compilation error in case this module is not going to work on anything but CPython.
What do you think?
+1
I think we could have a global variable in sys, called "dont_compile", distutils would look at before it tris to compile stuff, exactly like how it does for pyc file (sys.dont_write_bytecode)
Every time somebody says "let's have a global variable", God kills a kitten. If it's in sys, He bludgeons the kitten to death *with another kitten*. sys.dont_write_bytecode really ought to have been an API of an importer object somewhere; hopefully, when Brett has the time to finish the refactoring which he alluded to at the language summit, it will be. Similarly, functionally speaking this API is a good idea, but running the C compiler is distutils' job. Therefore any API which describes this functionality should be close to distutils itself.
On Wed, Feb 24, 2010 at 11:17 AM, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:
On Feb 23, 2010, at 2:10 PM, Tarek Ziadé wrote:
On Tue, Feb 23, 2010 at 1:50 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Hello.
I would like to have a feature on platform module (or sys or somewhere) that can tell distutils or distutils2 that this platform (be it PyPy or Jython) is not able to compile any C module. The purpose of this is to make distutils bail out in more reasonable manner than a compilation error in case this module is not going to work on anything but CPython.
What do you think?
+1
I think we could have a global variable in sys, called "dont_compile", distutils would look at before it tris to compile stuff, exactly like how it does for pyc file (sys.dont_write_bytecode)
Every time somebody says "let's have a global variable", God kills a kitten.
I't not a global *variable*, it's a global *constant*, unlike dont_write_bytecode.
Similarly, functionally speaking this API is a good idea, but running the C compiler is distutils' job. Therefore any API which describes this functionality should be close to distutils itself.
We're talking here about a way how interpreter can tell distutils what it can and what it can't do. Other option would be to modify distutils shipped with other python implementations, but that's a bit against goals of having a unified stdlib. Cheers, fijal
participants (6)
-
Daniel Stutzbach -
Frank Wierzbicki -
Glyph Lefkowitz -
Maciej Fijalkowski -
Nick Coghlan -
Tarek Ziadé