Mailman 3 Platform extension for distutils on other interpreters than CPython - Python-Dev

Platform extension for distutils on other interpreters than CPython

Maciej Fijalkowski

23 Feb 2010 23 Feb '10

1:50 p.m.

Hello. I would like to have a feature on platform module (or sys or somewhere) that can tell distutils or distutils2 that this platform (be it PyPy or Jython) is not able to compile any C module. The purpose of this is to make distutils bail out in more reasonable manner than a compilation error in case this module is not going to work on anything but CPython. What do you think? Cheers, fijal

Show replies by date

Daniel Stutzbach

23 Feb 23 Feb

2:03 p.m.

New subject: Platform extension for distutils on other interpreters than CPython

On Tue, Feb 23, 2010 at 12:50 PM, Maciej Fijalkowski wrote:

...

I would like to have a feature on platform module (or sys or somewhere) that can tell distutils or distutils2 that this platform (be it PyPy or Jython) is not able to compile any C module. The purpose of this is to make distutils bail out in more reasonable manner than a compilation error in case this module is not going to work on anything but CPython.

Also, it would be nice if the package could tell distutils that the compilation is option, in order to support modules where the C version simply replaces functions for speed. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com

Tarek Ziadé

2:17 p.m.

New subject: Platform extension for distutils on other interpreters than CPython

On Tue, Feb 23, 2010 at 2:03 PM, Daniel Stutzbach wrote:

...

On Tue, Feb 23, 2010 at 12:50 PM, Maciej Fijalkowski wrote:

...
I would like to have a feature on platform module (or sys or somewhere) that can tell distutils or distutils2 that this platform (be it PyPy or Jython) is not able to compile any C module. The purpose of this is to make distutils bail out in more reasonable manner than a compilation error in case this module is not going to work on anything but CPython.

Also, it would be nice if the package could tell distutils that the compilation is option, in order to support modules where the C version simply replaces functions for speed.

There's an option called "optional" in the Extension class right now that will silent any compilation failures (like gcc not being there), so your project still installs. That's basically what people now can use when they have a pure Python fallback version they want to provide in case the C version cannot be built for any reason. Tarek -- Tarek Ziadé | http://ziade.org

Frank Wierzbicki

2:05 p.m.

New subject: Platform extension for distutils on other interpreters than CPython

On Tue, Feb 23, 2010 at 1:50 PM, Maciej Fijalkowski wrote:

...

Hello.

I would like to have a feature on platform module (or sys or somewhere) that can tell distutils or distutils2 that this platform (be it PyPy or Jython) is not able to compile any C module. The purpose of this is to make distutils bail out in more reasonable manner than a compilation error in case this module is not going to work on anything but CPython. FWIW this would be helpful for Jython too.

-Frank

Tarek Ziadé

2:10 p.m.

New subject: Platform extension for distutils on other interpreters than CPython

On Tue, Feb 23, 2010 at 1:50 PM, Maciej Fijalkowski wrote:

...

Hello.

I would like to have a feature on platform module (or sys or somewhere) that can tell distutils or distutils2 that this platform (be it PyPy or Jython) is not able to compile any C module. The purpose of this is to make distutils bail out in more reasonable manner than a compilation error in case this module is not going to work on anything but CPython.

What do you think?

+1 I think we could have a global variable in sys, called "dont_compile", distutils would look at before it tris to compile stuff, exactly like how it does for pyc file (sys.dont_write_bytecode) Regards Tarek

Tarek Ziadé

2:27 p.m.

New subject: Platform extension for distutils on other interpreters than CPython

On Tue, Feb 23, 2010 at 2:10 PM, Tarek Ziadé wrote:

...

On Tue, Feb 23, 2010 at 1:50 PM, Maciej Fijalkowski wrote:

...
Hello.

I would like to have a feature on platform module (or sys or somewhere) that can tell distutils or distutils2 that this platform (be it PyPy or Jython) is not able to compile any C module. The purpose of this is to make distutils bail out in more reasonable manner than a compilation error in case this module is not going to work on anything but CPython.

What do you think?

+1

I think we could have a global variable in sys, called "dont_compile", distutils would look at before it tris to compile stuff, exactly like how it does for pyc file (sys.dont_write_bytecode)

Or... wait : we already know if we are using CPython, or Jython reading sys.platform. So I could simply not trigger the compilation in case sys.platform is one of the CPythons and keep in distutils side a list of the platform names, Extension is incompatible with. That makes me wonder : why don't we have a sys.implementation variable ? (cython/jython/pypi), since we can have several values for cython in sys.platform Tarek

Maciej Fijalkowski

2:44 p.m.

New subject: Platform extension for distutils on other interpreters than CPython

On Tue, Feb 23, 2010 at 2:27 PM, Tarek Ziadé wrote:

...

On Tue, Feb 23, 2010 at 2:10 PM, Tarek Ziadé wrote:

...
On Tue, Feb 23, 2010 at 1:50 PM, Maciej Fijalkowski wrote:

...
Hello.

I would like to have a feature on platform module (or sys or somewhere) that can tell distutils or distutils2 that this platform (be it PyPy or Jython) is not able to compile any C module. The purpose of this is to make distutils bail out in more reasonable manner than a compilation error in case this module is not going to work on anything but CPython.

What do you think?

+1

I think we could have a global variable in sys, called "dont_compile", distutils would look at before it tris to compile stuff, exactly like how it does for pyc file (sys.dont_write_bytecode)

Or... wait : we already know if we are using CPython, or Jython reading sys.platform.

So I could simply not trigger the compilation in case sys.platform is one of the CPythons and keep in distutils side a list of the platform names, Extension is incompatible with.

That makes me wonder : why don't we have a sys.implementation variable ? (cython/jython/pypi), since we can have several values for cython in sys.platform

Tarek

That's pypy. pypi is something else. sys.platform is not any good, since for example PyPy, and possibly any other python implementation that is not CPython, but it's not tied to any particular platform (like parrot) would say "linux2" or "win32". sys.implementation sounds good, but it'll also require a list in stdlib what's fine and what's not fine and a flag sounds like something that everyone can set, not asking to be listed in stdlib. How about sys.implementation.supports_extensions? Cheers, fijal

Tarek Ziadé

3:10 p.m.

New subject: Platform extension for distutils on other interpreters than CPython

On Tue, Feb 23, 2010 at 2:44 PM, Maciej Fijalkowski wrote:

...

On Tue, Feb 23, 2010 at 2:27 PM, Tarek Ziadé wrote:

...
On Tue, Feb 23, 2010 at 2:10 PM, Tarek Ziadé wrote:

...
On Tue, Feb 23, 2010 at 1:50 PM, Maciej Fijalkowski wrote:

...
Hello.

I would like to have a feature on platform module (or sys or somewhere) that can tell distutils or distutils2 that this platform (be it PyPy or Jython) is not able to compile any C module. The purpose of this is to make distutils bail out in more reasonable manner than a compilation error in case this module is not going to work on anything but CPython.

What do you think?

+1

I think we could have a global variable in sys, called "dont_compile", distutils would look at before it tris to compile stuff, exactly like how it does for pyc file (sys.dont_write_bytecode)

Or... wait : we already know if we are using CPython, or Jython reading sys.platform.

So I could simply not trigger the compilation in case sys.platform is one of the CPythons and keep in distutils side a list of the platform names, Extension is incompatible with.

That makes me wonder : why don't we have a sys.implementation variable ? (cython/jython/pypi), since we can have several values for cython in sys.platform

Tarek

That's pypy. pypi is something else. sys.platform is not any good, since for example PyPy, and possibly any other python implementation that is not CPython, but it's not tied to any particular platform (like parrot) would say "linux2" or "win32".

sys.implementation sounds good, but it'll also require a list in stdlib what's fine and what's not fine and a flag sounds like something that everyone can set, not asking to be listed in stdlib.

How about sys.implementation.supports_extensions?

I think its the other way around: You are making the assumption that, sys knows about distutils extensions. and there's no indication about the kind of extension here (C, etc..). In distutils, the Extension class works with a compiler class (ccompiler, mingwcompiler, etc) And in theory, people could add a new compiler that works under PyPy or Jython. So I think it's up to the Compiler class (through the Extension instance) to decide if the environment is suitable. For instance in the CCompiler class I can write things like: if sys.implementation != 'cython': warning.warn('Sorry I cannot compile this under %s' % sys.implementation) Tarek -- Tarek Ziadé | http://ziade.org

Nick Coghlan

24 Feb 24 Feb

6:35 a.m.

New subject: Platform extension for distutils on other interpreters than CPython

Tarek Ziadé wrote:

...

That makes me wonder : why don't we have a sys.implementation variable ? (cython/jython/pypi), since we can have several values for cython in sys.platform

You might want to try and catch up with Christian Heimes. He was going to write a PEP to add one, but must have been caught up with other things. See the thread starting at: http://mail.python.org/pipermail/python-dev/2009-October/092893.html Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

Maciej Fijalkowski

12:21 p.m.

New subject: Platform extension for distutils on other interpreters than CPython

On Wed, Feb 24, 2010 at 6:35 AM, Nick Coghlan wrote:

...

Tarek Ziadé wrote:

...
That makes me wonder : why don't we have a sys.implementation variable ? (cython/jython/pypi), since we can have several values for cython in sys.platform

Hello. So I propose to have a sys.implementation which would have string representation like "CPython" or "Jython" and have a couple of extra attributes on that. I can think about a lot of attributes, however, couple comes to mind as obvious. * supports_c_api - whether it can load and use CPython C modules * gc_strategy - probably equals "refcounting" or not, useful for some people * frame_introspection - This is mostly True for everybody except IronPython which has it as an optional command line argument. Might be useful to have it for some projects, unsure. What do you think? I'm willing to implement this for CPython and PyPy (this should be dead-simple anyway). Cheers, fijal

Nick Coghlan

25 Feb 25 Feb

7:29 a.m.

New subject: Platform extension for distutils on other interpreters than CPython

Maciej Fijalkowski wrote:

...

On Wed, Feb 24, 2010 at 6:35 AM, Nick Coghlan wrote:

...
Tarek Ziadé wrote:

...
That makes me wonder : why don't we have a sys.implementation variable ? (cython/jython/pypi), since we can have several values for cython in sys.platform

Hello.

So I propose to have a sys.implementation which would have string representation like "CPython" or "Jython" and have a couple of extra attributes on that. I can think about a lot of attributes, however, couple comes to mind as obvious.

* supports_c_api - whether it can load and use CPython C modules * gc_strategy - probably equals "refcounting" or not, useful for some people * frame_introspection - This is mostly True for everybody except IronPython which has it as an optional command line argument. Might be useful to have it for some projects, unsure.

What do you think?

I think anything along these lines needs to be a PEP so that the developers of the different implementations all get a chance to comment and then have a firm standard to code to afterwards :) Christian was going to write one in the PEP 370 context, so it's worth following up with him to see if he ever got around to drafting anything. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

Glyph Lefkowitz

24 Feb 24 Feb

11:17 a.m.

New subject: Platform extension for distutils on other interpreters than CPython

On Feb 23, 2010, at 2:10 PM, Tarek Ziadé wrote:

...

On Tue, Feb 23, 2010 at 1:50 PM, Maciej Fijalkowski wrote:

...
Hello.

I would like to have a feature on platform module (or sys or somewhere) that can tell distutils or distutils2 that this platform (be it PyPy or Jython) is not able to compile any C module. The purpose of this is to make distutils bail out in more reasonable manner than a compilation error in case this module is not going to work on anything but CPython.

What do you think?

+1

I think we could have a global variable in sys, called "dont_compile", distutils would look at before it tris to compile stuff, exactly like how it does for pyc file (sys.dont_write_bytecode)

Every time somebody says "let's have a global variable", God kills a kitten. If it's in sys, He bludgeons the kitten to death *with another kitten*. sys.dont_write_bytecode really ought to have been an API of an importer object somewhere; hopefully, when Brett has the time to finish the refactoring which he alluded to at the language summit, it will be. Similarly, functionally speaking this API is a good idea, but running the C compiler is distutils' job. Therefore any API which describes this functionality should be close to distutils itself.

Maciej Fijalkowski

3:15 p.m.

New subject: Platform extension for distutils on other interpreters than CPython

On Wed, Feb 24, 2010 at 11:17 AM, Glyph Lefkowitz wrote:

...

On Feb 23, 2010, at 2:10 PM, Tarek Ziadé wrote:

...
On Tue, Feb 23, 2010 at 1:50 PM, Maciej Fijalkowski wrote:

...
Hello.

I would like to have a feature on platform module (or sys or somewhere) that can tell distutils or distutils2 that this platform (be it PyPy or Jython) is not able to compile any C module. The purpose of this is to make distutils bail out in more reasonable manner than a compilation error in case this module is not going to work on anything but CPython.

What do you think?

+1

I think we could have a global variable in sys, called "dont_compile", distutils would look at before it tris to compile stuff, exactly like how it does for pyc file (sys.dont_write_bytecode)

Every time somebody says "let's have a global variable", God kills a kitten.

I't not a global *variable*, it's a global *constant*, unlike dont_write_bytecode.

...

Similarly, functionally speaking this API is a good idea, but running the C compiler is distutils' job. Therefore any API which describes this functionality should be close to distutils itself.

We're talking here about a way how interpreter can tell distutils what it can and what it can't do. Other option would be to modify distutils shipped with other python implementations, but that's a bit against goals of having a unified stdlib. Cheers, fijal

5171

Age (days ago)

5173

Last active (days ago)

List overview

Download

12 comments

6 participants

participants (6)

Daniel Stutzbach
Frank Wierzbicki
Glyph Lefkowitz
Maciej Fijalkowski
Nick Coghlan
Tarek Ziadé

Platform extension for distutils on other interpreters than CPython

Daniel Stutzbach

tags

participants (6)