Proposal: new "interpreter" module
I originally brought this in the discussion about removing types, but it can be separate.
Through the years, sys has gather a fare share of crud. Much of these new additions are low-level and CPython specific. I believe the sys module should strive to be an interpreter neutral, clean module. Hence, I propose the "interpreter" module. We could move all those internal functions like _getframe(), get/setrecursionlimit(), get/setcheckinterval(), etc to it. Also, if we decided to remove the types module, this would be a natural place for frames and code objects.
I look forward to your comments.
Benjamin Peterson schrieb:
I originally brought this in the discussion about removing types, but it can be separate.
Through the years, sys has gather a fare share of crud. Much of these new additions are low-level and CPython specific. I believe the sys module should strive to be an interpreter neutral, clean module. Hence, I propose the "interpreter" module. We could move all those internal functions like _getframe(), get/setrecursionlimit(), get/setcheckinterval(), etc to it. Also, if we decided to remove the types module, this would be a natural place for frames and code objects.
The idea popped up several months ago when we were discussion the future of the types module. Guido suggested the name 'pyvm'. The module was meant to contain low level interfaces to the (C)Python virtual machine.
Once the pyvm module is implemented the sys module should only contain system methods which can be implemented by all flavors of Python.
Christian
On Fri, Apr 4, 2008 at 11:27 AM, Christian Heimes christian@cheimes.de wrote:
Benjamin Peterson schrieb:
I originally brought this in the discussion about removing types, but it can be separate.
Through the years, sys has gather a fare share of crud. Much of these new additions are low-level and CPython specific. I believe the sys module should strive to be an interpreter neutral, clean module. Hence, I propose the "interpreter" module. We could move all those internal functions like _getframe(), get/setrecursionlimit(), get/setcheckinterval(), etc to it. Also, if we decided to remove the types module, this would be a natural place for frames and code objects.
The idea popped up several months ago when we were discussion the future of the types module. Guido suggested the name 'pyvm'. The module was meant to contain low level interfaces to the (C)Python virtual machine.
Once the pyvm module is implemented the sys module should only contain system methods which can be implemented by all flavors of Python.
This is exactly the type of separation I want to see. Unfortunately, because it would be a new module that means we will have to go through the typical steps needed to add it and can't rely on 2to3 and the reorg to handle it.
But I totally support moving CPython-specific stuff to another module. Probably the best thing to do is write up a proposal of what should go where, run it past here, and the write up a PEP to present to python-dev. Then we can introduce the module in 2.6/3.0. Proper deprecations will be needed in the sys module so that people can slowly transition to the new module (whether it is named interpreter, pyvm, or some other name) since the sys module can't obviously just suddenly drop some methods or attributes.
And do realize this is a chance to change the API so that if a property-like situation is better than a method call that can be used.
Sound good to people? If so, who wants to take this on?
-Brett
On Fri, Apr 4, 2008 at 5:44 AM, Brett Cannon brett@python.org wrote:
On Fri, Apr 4, 2008 at 11:27 AM, Christian Heimes christian@cheimes.de wrote:
Benjamin Peterson schrieb:
I originally brought this in the discussion about removing types, but
it can
be separate.
Through the years, sys has gather a fare share of crud. Much of these
new
additions are low-level and CPython specific. I believe the sys
module
should strive to be an interpreter neutral, clean module. Hence, I
propose
the "interpreter" module. We could move all those internal functions
like
_getframe(), get/setrecursionlimit(), get/setcheckinterval(), etc to
it.
Also, if we decided to remove the types module, this would be a
natural
place for frames and code objects.
The idea popped up several months ago when we were discussion the
future
of the types module. Guido suggested the name 'pyvm'. The module was meant to contain low level interfaces to the (C)Python virtual machine.
Once the pyvm module is implemented the sys module should only contain system methods which can be implemented by all flavors of Python.
This is exactly the type of separation I want to see. Unfortunately, because it would be a new module that means we will have to go through the typical steps needed to add it and can't rely on 2to3 and the reorg to handle it.
But I totally support moving CPython-specific stuff to another module. Probably the best thing to do is write up a proposal of what should go where, run it past here, and the write up a PEP to present to python-dev. Then we can introduce the module in 2.6/3.0. Proper deprecations will be needed in the sys module so that people can slowly transition to the new module (whether it is named interpreter, pyvm, or some other name) since the sys module can't obviously just suddenly drop some methods or attributes.
And do realize this is a chance to change the API so that if a property-like situation is better than a method call that can be used.
Sound good to people? If so, who wants to take this on?
If there is further support, I will.
-Brett
On Fri, Apr 4, 2008 at 11:27 AM, Christian Heimes christian@cheimes.de wrote:
Once the pyvm module is implemented the sys module should only contain system methods which can be implemented by all flavors of Python.
On Fri, Apr 4, 2008 at 5:44 AM, Brett Cannon brett@python.org wrote:
This is exactly the type of separation I want to see.
[snip]
Sound good to people? If so, who wants to take this on?
2008/4/4 Benjamin Peterson musiccomposition@gmail.com:
If there is further support, I will.
I don't know how much more further support you need, but I'm 100% behind the idea. If you need any more help, let me know.
Steve
On Fri, Apr 4, 2008 at 10:01 AM, Steven Bethard steven.bethard@gmail.com wrote:
On Fri, Apr 4, 2008 at 11:27 AM, Christian Heimes christian@cheimes.de wrote:
Once the pyvm module is implemented the sys module should only contain system methods which can be implemented by all flavors of Python.
On Fri, Apr 4, 2008 at 5:44 AM, Brett Cannon brett@python.org wrote:
This is exactly the type of separation I want to see.
[snip]
Sound good to people? If so, who wants to take this on?
2008/4/4 Benjamin Peterson musiccomposition@gmail.com:
If there is further support, I will.
I don't know how much more further support you need, but I'm 100% behind the idea. If you need any more help, let me know.
I'm starting the PEP now...
Steve
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
On Fri, Apr 4, 2008 at 10:27 AM, Benjamin Peterson < musiccomposition@gmail.com> wrote:
On Fri, Apr 4, 2008 at 10:01 AM, Steven Bethard steven.bethard@gmail.com wrote:
On Fri, Apr 4, 2008 at 11:27 AM, Christian Heimes christian@cheimes.de wrote:
Once the pyvm module is implemented the sys module should only
contain
system methods which can be implemented by all flavors of Python.
On Fri, Apr 4, 2008 at 5:44 AM, Brett Cannon brett@python.org wrote:
This is exactly the type of separation I want to see.
[snip]
Sound good to people? If so, who wants to take this on?
2008/4/4 Benjamin Peterson musiccomposition@gmail.com:
If there is further support, I will.
I don't know how much more further support you need, but I'm 100% behind the idea. If you need any more help, let me know.
I'm starting the PEP now...
And here it is: This is my first PEP, so do be brutal with your feedback.
PEP: XXX Title: The interpreter module Version: $Revision$ Last-Modified: $Date$ Author: Benjamin Peterson Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 4-April-2008 Python-Version: 3.0
Abstract ========
This PEP proposes a new low-level module for CPython-specific interpreter functions in order to clean out the sys module and separate general Python functionality from implementation details.
Rationale =========
The sys module currently contains functions and data that can be put into two major groups:
1. Data and functions that are available in all Python implementations and deal with the general running of a Python VM.
- display_hook, stdout, stdin, and stderr - flags and warn_options - settrace, and setprofile - traceback, excepthook - etc...
2. Data and function that affect the CPython interpreter.
- get/setrecursionlimit - get/setcheckinterval - _getframe and _current_frame - getrefcount - getdlopenflags
The second collections of items has been steadily increasing over the years causing clutter in sys. Guido has even said he doesn't recognize some of things in it [#bug-1522]_!
Other implementations have clearly struggled with what to do about the contents of sys they can't implement but must to retain compatibility. For example, Jython's sys module has dud set/getrecursionlimit functions. Moving these items items off to another module would send a clear message about what functions need and need not be implemented.
It has also been proposed that the contents of types module be distributed across the standard library [#types-removal]_; the interpreter module would provide an excellent resting place for internal types like frames and code objects.
Specification =============
A new builtin module named "interpreter" (see `Naming`_) will be added. The functions mentioned in the above list will be moved to it.
Transition Plan ===============
Once implemented in 3.x, the interpreter module will be back-ported to 2.6. Py3k warnings will be added the the sys functions it replaces.
Open Issues ===========
Naming ------
The author proposes the name "interpreter" for the new module. "pyvm" has also been suggested [#pyvm-name]_.
References ==========
.. [#bug-1522]
http://bugs.python.org/issue1522
.. [#types-removal]
http://mail.python.org/pipermail/stdlib-sig/2008-April/000172.html
.. [#pyvm-name]
http://mail.python.org/pipermail/python-3000/2007-November/011351.html
Copyright =========
This document has been placed in the public domain.
Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:
Steve
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
-- Cheers, Benjamin Peterson
On 04/04/2008, Benjamin Peterson musiccomposition@gmail.com wrote:
And here it is: This is my first PEP, so do be brutal with your feedback.
Looks reasonably good - I can't think of anything particularly brutal. You should, however, be explicit about precisely where you propose each entry in sys should go. Saying "etc..." is cheating :-)
Once you list everything, I expect you'll get a much more, shall we say, "lively" discussion!
But I've no problem with the PEP in principle.
Paul.
On Fri, Apr 4, 2008 at 1:35 PM, Paul Moore p.f.moore@gmail.com wrote:
On 04/04/2008, Benjamin Peterson musiccomposition@gmail.com wrote:
And here it is: This is my first PEP, so do be brutal with your
feedback.
Looks reasonably good - I can't think of anything particularly brutal. You should, however, be explicit about precisely where you propose each entry in sys should go. Saying "etc..." is cheating :-)
I meant everything which wasn't in the second list won't be moved. EIBTI, though!
Once you list everything, I expect you'll get a much more, shall we say, "lively" discussion!
But I've no problem with the PEP in principle.
Paul.
On Fri, Apr 4, 2008 at 10:51 AM, Benjamin Peterson musiccomposition@gmail.com wrote:
PEP: XXX Title: The interpreter module Version: $Revision$ Last-Modified: $Date$ Author: Benjamin Peterson Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 4-April-2008 Python-Version: 3.0
Abstract
This PEP proposes a new low-level module for CPython-specific interpreter functions in order to clean out the sys module and separate general Python functionality from implementation details.
Rationale
The sys module currently contains functions and data that can be put into two major groups:
- Data and functions that are available in all Python implementations and
deal with the general running of a Python VM.
- display_hook, stdout, stdin, and stderr
- flags and warn_options
- settrace, and setprofile
- traceback, excepthook
- etc...
Data and function that affect the CPython interpreter.
- get/setrecursionlimit
- get/setcheckinterval
- _getframe and _current_frame
- getrefcount
- getdlopenflags
Why getdlopenflags? That seems general to any Python implementation that opens dlls (though I believe the dl module is going away in Python 3, so it may not matter).
Also, maybe settscdump should be moved as well? Its description says to look at "Python/ceval.c"
Otherwise, it looks pretty good to me.
Steve
On Fri, Apr 4, 2008 at 4:09 PM, Steven Bethard steven.bethard@gmail.com wrote:
On Fri, Apr 4, 2008 at 10:51 AM, Benjamin Peterson musiccomposition@gmail.com wrote:
PEP: XXX Title: The interpreter module Version: $Revision$ Last-Modified: $Date$ Author: Benjamin Peterson Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 4-April-2008 Python-Version: 3.0
Abstract
This PEP proposes a new low-level module for CPython-specific
interpreter
functions in order to clean out the sys module and separate general
Python
functionality from implementation details.
Rationale
The sys module currently contains functions and data that can be put
into
two major groups:
- Data and functions that are available in all Python implementations
and
deal with the general running of a Python VM.
- display_hook, stdout, stdin, and stderr
- flags and warn_options
- settrace, and setprofile
- traceback, excepthook
- etc...
Data and function that affect the CPython interpreter.
- get/setrecursionlimit
- get/setcheckinterval
- _getframe and _current_frame
- getrefcount
- getdlopenflags
Why getdlopenflags? That seems general to any Python implementation that opens dlls (though I believe the dl module is going away in Python 3, so it may not matter).
I don't think any other implementations do open dynamic libs. Those flags don't affect the dl module; they define how Python opens C shared libs.
Also, maybe settscdump should be moved as well? Its description says to look at "Python/ceval.c"
Yes.
Otherwise, it looks pretty good to me.
Steve
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
Here's an updated version:
- display_hook, stdout, stdin, and stderr - flags and warn_options - settrace, and setprofile - traceback, excepthook - etc...
2. Data and functions that affect the CPython interpreter.
- get/setrecursionlimit - get/setcheckinterval - _getframe and _current_frame - getrefcount - get/setdlopenflags - settscdumps - api_version - winver - dllhandle - float_info
The second collections of items has been steadily increasing over the years causing clutter in sys. Guido has even said he doesn't recognize some of things in it [#bug-1522]_!
Other implementations have clearly struggled with what to do about the contents of sys they can't implement but must to retain compatibility. For example, Jython's sys module has dud set/getrecursionlimit functions. Moving these items items off to another module would send a clear message about what functions need and need not be implemented.
It has also been proposed that the contents of types module be distributed across the standard library [#types-removal]_; the interpreter module would provide an excellent resting place for internal types like frames and code objects.
Specification =============
A new builtin module named "interpreter" (see `Naming`_) will be added. The functions mentioned in the second list above will be moved to it.
Transition Plan ===============
Once implemented in 3.x, the interpreter module will be back-ported to 2.6. Py3k warnings will be added the the sys functions it replaces.
Open Issues ===========
Naming ------
The author proposes the name "interpreter" for the new module. "pyvm" has also been suggested [#pyvm-name]_.
References ==========
.. [#bug-1522]
http://bugs.python.org/issue1522
.. [#types-removal]
http://mail.python.org/pipermail/stdlib-sig/2008-April/000172.html
.. [#pyvm-name]
http://mail.python.org/pipermail/python-3000/2007-November/011351.html
Copyright =========
This document has been placed in the public domain.
Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:
On Sat, Apr 5, 2008 at 9:18 AM, Benjamin Peterson < musiccomposition@gmail.com> wrote:
On Fri, Apr 4, 2008 at 4:09 PM, Steven Bethard steven.bethard@gmail.com wrote:
On Fri, Apr 4, 2008 at 10:51 AM, Benjamin Peterson musiccomposition@gmail.com wrote:
PEP: XXX Title: The interpreter module Version: $Revision$ Last-Modified: $Date$ Author: Benjamin Peterson Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 4-April-2008 Python-Version: 3.0
Abstract
This PEP proposes a new low-level module for CPython-specific
interpreter
functions in order to clean out the sys module and separate general
Python
functionality from implementation details.
Rationale
The sys module currently contains functions and data that can be put
into
two major groups:
- Data and functions that are available in all Python implementations
and
deal with the general running of a Python VM.
- display_hook, stdout, stdin, and stderr
- flags and warn_options
- settrace, and setprofile
- traceback, excepthook
- etc...
Data and function that affect the CPython interpreter.
- get/setrecursionlimit
- get/setcheckinterval
- _getframe and _current_frame
- getrefcount
- getdlopenflags
Why getdlopenflags? That seems general to any Python implementation that opens dlls (though I believe the dl module is going away in Python 3, so it may not matter).
I don't think any other implementations do open dynamic libs. Those flags don't affect the dl module; they define how Python opens C shared libs.
Also, maybe settscdump should be moved as well? Its description says to look at "Python/ceval.c"
Yes.
Otherwise, it looks pretty good to me.
Steve
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy
-- Cheers, Benjamin Peterson
On Sat, Apr 5, 2008 at 4:58 PM, Benjamin Peterson musiccomposition@gmail.com wrote:
Here's an updated version:
- display_hook, stdout, stdin, and stderr
- flags and warn_options
- settrace, and setprofile
- traceback, excepthook
- etc...
You still need to get rid of the "etc.". Beyond being explicit, it makes it easier for people reviewing the PEP to see what they think might be better placed in the new module.
-Brett
On Sat, Apr 5, 2008 at 10:37 AM, Brett Cannon brett@python.org wrote:
On Sat, Apr 5, 2008 at 4:58 PM, Benjamin Peterson musiccomposition@gmail.com wrote:
Here's an updated version:
- display_hook, stdout, stdin, and stderr
- flags and warn_options
- settrace, and setprofile
- traceback, excepthook
- etc...
You still need to get rid of the "etc.". Beyond being explicit, it makes it easier for people reviewing the PEP to see what they think might be better placed in the new module.
PEP: XXX Title: The interpreter module Version: $Revision$ Last-Modified: $Date$ Author: Benjamin Peterson Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 4-April-2008 Python-Version: 3.0
Abstract ========
This PEP proposes a new low-level module for CPython-specific interpreter functions in order to clean out the sys module and separate general Python functionality from implementation details.
Rationale =========
The sys module currently contains functions and data that can be put into two major groups:
1. Data and functions that are available in all Python implementations and deal with the general running of a Python VM.
- argv - byteorder - builtin_module_names, path, and modules - subversion, copyright, hexversion, version, and version_info - displayhook, __displayhook__ - excepthook, __excepthook__, exc_info, and exc_clear - exec_prefix and prefix - executable - exit - flags, py3kwarning, dont_write_bytecode, and warn_options - getfilesystemencoding - get/setprofile - get/settrace - getwindowsversion - maxint and maxunicode - platform - ps1 and ps2 - stdin, stderr, stdout, __stdin__, __stderr__, __stdout__ - tracebacklimit
2. Data and functions that affect the CPython interpreter.
- get/setrecursionlimit - get/setcheckinterval - _getframe and _current_frame - getrefcount - get/setdlopenflags - settscdumps - api_version - winver - dllhandle - float_info - _compact_freelists - _clear_type_cache
The second collections of items has been steadily increasing over the years causing clutter in sys. Guido has even said he doesn't recognize some of things in it [#bug-1522]_!
Other implementations have clearly struggled with what to do about the contents of sys they can't implement but must to retain compatibility. For example, Jython's sys module has dud set/getrecursionlimit functions. Moving these items items off to another module would send a clear message about what functions need and need not be implemented.
It has also been proposed that the contents of types module be distributed across the standard library [#types-removal]_; the interpreter module would provide an excellent resting place for internal types like frames and code objects.
Specification =============
A new builtin module named "interpreter" (see `Naming`_) will be added. The functions mentioned in the second list above will be moved to it.
Transition Plan ===============
Once implemented in 3.x, the interpreter module will be back-ported to 2.6. Py3k warnings will be added the the sys functions it replaces.
Open Issues ===========
Naming ------
The author proposes the name "interpreter" for the new module. "pyvm" has also been suggested [#pyvm-name]_.
References ==========
.. [#bug-1522]
http://bugs.python.org/issue1522
.. [#types-removal]
http://mail.python.org/pipermail/stdlib-sig/2008-April/000172.html
.. [#pyvm-name]
http://mail.python.org/pipermail/python-3000/2007-November/011351.html
Copyright =========
This document has been placed in the public domain.
Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:
-Brett
On 05/04/2008, Benjamin Peterson musiccomposition@gmail.com wrote:
- Data and functions that are available in all Python implementations and deal with the general running of a Python VM.
[...]
- subversion, copyright, hexversion, version, and version_info
subversion seems specific to CPython (and in fact, to the current development processes for CPython - if we moved to a DVCS like Mercurial or Bazaar, sys.subversion would have to go, or at least change meaning significantly).
- flags, py3kwarning, dont_write_bytecode, and warn_options
dont_write_bytecode is CPython-specific, aren't they? Jython and IronPython don't use bytecode files as far as I know.
The rest looks OK to me. Paul.
On Sat, Apr 5, 2008 at 2:29 PM, Paul Moore p.f.moore@gmail.com wrote:
On 05/04/2008, Benjamin Peterson musiccomposition@gmail.com wrote:
- Data and functions that are available in all Python implementations
and
deal with the general running of a Python VM.
[...]
- subversion, copyright, hexversion, version, and version_info
subversion seems specific to CPython (and in fact, to the current development processes for CPython - if we moved to a DVCS like Mercurial or Bazaar, sys.subversion would have to go, or at least change meaning significantly).
That's true. I put it there originally because I thought of it as more general version info, but I can see it moving, too.
- flags, py3kwarning, dont_write_bytecode, and warn_options
dont_write_bytecode is CPython-specific, aren't they? Jython and IronPython don't use bytecode files as far as I know.
I can't speak for IronPython, but Jython writes compiled Java classes representing the Python. Anyway, I think this should really go in imp.
The rest looks OK to me. Paul.
On Sat, Apr 5, 2008 at 6:13 PM, Benjamin Peterson musiccomposition@gmail.com wrote:
On Sat, Apr 5, 2008 at 10:37 AM, Brett Cannon brett@python.org wrote:
On Sat, Apr 5, 2008 at 4:58 PM, Benjamin Peterson musiccomposition@gmail.com wrote:
Here's an updated version:
- display_hook, stdout, stdin, and stderr
- flags and warn_options
- settrace, and setprofile
- traceback, excepthook
- etc...
You still need to get rid of the "etc.". Beyond being explicit, it makes it easier for people reviewing the PEP to see what they think might be better placed in the new module.
PEP: XXX Title: The interpreter module Version: $Revision$ Last-Modified: $Date$ Author: Benjamin Peterson Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 4-April-2008 Python-Version: 3.0
Abstract
This PEP proposes a new low-level module for CPython-specific interpreter functions in order to clean out the sys module and separate general Python functionality from implementation details.
Rationale
The sys module currently contains functions and data that can be put into two major groups:
- Data and functions that are available in all Python implementations and
deal with the general running of a Python VM.
- argv
- byteorder
- builtin_module_names, path, and modules
builtin_module_names is an implementation detail and thus should be moved.
- subversion, copyright, hexversion, version, and version_info
Ditto for subversion.
- displayhook, __displayhook__
- excepthook, __excepthook__, exc_info, and exc_clear
- exec_prefix and prefix
- executable
- exit
- flags, py3kwarning, dont_write_bytecode, and warn_options
dont_write_bytecode might be better off in the new module, but then bytecode is part of the stdlib. I guess it depends on how the other interpreters view bytecode. They could easily just have it always set to False.
- getfilesystemencoding
- get/setprofile
- get/settrace
- getwindowsversion
- maxint and maxunicode
- platform
- ps1 and ps2
- stdin, stderr, stdout, __stdin__, __stderr__, __stdout__
- tracebacklimit
Perhaps we should have sys.interpreter or sys.vm? That way e.g. Jython no longer has to override sys.platform to specify that VM-specific changes are in effect?
-Brett
On Sat, Apr 5, 2008 at 4:04 PM, Brett Cannon brett@python.org wrote:
On Sat, Apr 5, 2008 at 6:13 PM, Benjamin Peterson musiccomposition@gmail.com wrote:
On Sat, Apr 5, 2008 at 10:37 AM, Brett Cannon brett@python.org wrote:
On Sat, Apr 5, 2008 at 4:58 PM, Benjamin Peterson musiccomposition@gmail.com wrote:
Here's an updated version:
- display_hook, stdout, stdin, and stderr
- flags and warn_options
- settrace, and setprofile
- traceback, excepthook
- etc...
You still need to get rid of the "etc.". Beyond being explicit, it makes it easier for people reviewing the PEP to see what they think might be better placed in the new module.
PEP: XXX Title: The interpreter module Version: $Revision$ Last-Modified: $Date$ Author: Benjamin Peterson Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 4-April-2008 Python-Version: 3.0
Abstract
This PEP proposes a new low-level module for CPython-specific
interpreter
functions in order to clean out the sys module and separate general
Python
functionality from implementation details.
Rationale
The sys module currently contains functions and data that can be put
into
two major groups:
- Data and functions that are available in all Python implementations
and
deal with the general running of a Python VM.
- argv
- byteorder
- builtin_module_names, path, and modules
builtin_module_names is an implementation detail and thus should be moved.
- subversion, copyright, hexversion, version, and version_info
Ditto for subversion.
- displayhook, __displayhook__
- excepthook, __excepthook__, exc_info, and exc_clear
- exec_prefix and prefix
- executable
- exit
- flags, py3kwarning, dont_write_bytecode, and warn_options
dont_write_bytecode might be better off in the new module, but then bytecode is part of the stdlib. I guess it depends on how the other interpreters view bytecode. They could easily just have it always set to False.
What do you think about moving it to imp.
- getfilesystemencoding
- get/setprofile
- get/settrace
- getwindowsversion
- maxint and maxunicode
- platform
- ps1 and ps2
- stdin, stderr, stdout, __stdin__, __stderr__, __stdout__
- tracebacklimit
Perhaps we should have sys.interpreter or sys.vm? That way e.g. Jython no longer has to override sys.platform to specify that VM-specific changes are in effect?
That's a wonderful idea. Then you wouldn't have to resort to sys.platform being "java" and such tricks.
-Brett
On Sat, Apr 5, 2008 at 11:11 PM, Benjamin Peterson musiccomposition@gmail.com wrote:
On Sat, Apr 5, 2008 at 4:04 PM, Brett Cannon brett@python.org wrote:
On Sat, Apr 5, 2008 at 6:13 PM, Benjamin Peterson
musiccomposition@gmail.com wrote:
On Sat, Apr 5, 2008 at 10:37 AM, Brett Cannon brett@python.org wrote:
On Sat, Apr 5, 2008 at 4:58 PM, Benjamin Peterson musiccomposition@gmail.com wrote:
Here's an updated version:
- display_hook, stdout, stdin, and stderr
- flags and warn_options
- settrace, and setprofile
- traceback, excepthook
- etc...
You still need to get rid of the "etc.". Beyond being explicit, it makes it easier for people reviewing the PEP to see what they think might be better placed in the new module.
PEP: XXX Title: The interpreter module Version: $Revision$ Last-Modified: $Date$ Author: Benjamin Peterson Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 4-April-2008 Python-Version: 3.0
Abstract
This PEP proposes a new low-level module for CPython-specific
interpreter
functions in order to clean out the sys module and separate general
Python
functionality from implementation details.
Rationale
The sys module currently contains functions and data that can be put
into
two major groups:
- Data and functions that are available in all Python implementations
and
deal with the general running of a Python VM.
- argv
- byteorder
- builtin_module_names, path, and modules
builtin_module_names is an implementation detail and thus should be moved.
- subversion, copyright, hexversion, version, and version_info
Ditto for subversion.
- displayhook, __displayhook__
- excepthook, __excepthook__, exc_info, and exc_clear
- exec_prefix and prefix
- executable
- exit
- flags, py3kwarning, dont_write_bytecode, and warn_options
dont_write_bytecode might be better off in the new module, but then bytecode is part of the stdlib. I guess it depends on how the other interpreters view bytecode. They could easily just have it always set to False.
What do you think about moving it to imp.
Fine by me, but some people still seem to think this is an implementation detail. Since there is a lack of a clear solution for this I say make it an Open Issue and wait until python-dev is brought into this to figure it out.
-Brett
On Sun, Apr 6, 2008 at 3:09 AM, Brett Cannon brett@python.org wrote:
On Sat, Apr 5, 2008 at 11:11 PM, Benjamin Peterson musiccomposition@gmail.com wrote:
On Sat, Apr 5, 2008 at 4:04 PM, Brett Cannon brett@python.org wrote:
On Sat, Apr 5, 2008 at 6:13 PM, Benjamin Peterson
musiccomposition@gmail.com wrote:
On Sat, Apr 5, 2008 at 10:37 AM, Brett Cannon brett@python.org
wrote:
On Sat, Apr 5, 2008 at 4:58 PM, Benjamin Peterson musiccomposition@gmail.com wrote:
Here's an updated version:
- display_hook, stdout, stdin, and stderr
- flags and warn_options
- settrace, and setprofile
- traceback, excepthook
- etc...
You still need to get rid of the "etc.". Beyond being explicit, it makes it easier for people reviewing the PEP to see what they
think
might be better placed in the new module.
PEP: XXX Title: The interpreter module Version: $Revision$ Last-Modified: $Date$ Author: Benjamin Peterson Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 4-April-2008 Python-Version: 3.0
Abstract
This PEP proposes a new low-level module for CPython-specific
interpreter
functions in order to clean out the sys module and separate general
Python
functionality from implementation details.
Rationale
The sys module currently contains functions and data that can be put
into
two major groups:
- Data and functions that are available in all Python
implementations
and
deal with the general running of a Python VM.
- argv
- byteorder
- builtin_module_names, path, and modules
builtin_module_names is an implementation detail and thus should be
moved.
- subversion, copyright, hexversion, version, and version_info
Ditto for subversion.
- displayhook, __displayhook__
- excepthook, __excepthook__, exc_info, and exc_clear
- exec_prefix and prefix
- executable
- exit
- flags, py3kwarning, dont_write_bytecode, and warn_options
dont_write_bytecode might be better off in the new module, but then bytecode is part of the stdlib. I guess it depends on how the other interpreters view bytecode. They could easily just have it always set to False.
What do you think about moving it to imp.
Fine by me, but some people still seem to think this is an implementation detail. Since there is a lack of a clear solution for this I say make it an Open Issue and wait until python-dev is brought into this to figure it out.
Ok. Here's another version:
PEP: XXX Title: Cleaning out sys and the interpreter module Version: $Revision$ Last-Modified: $Date$ Author: Benjamin Peterson Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 4-April-2008 Python-Version: 3.0
Abstract ========
This PEP proposes a new low-level module for CPython-specific interpreter functions in order to clean out the sys module and separate general Python functionality from implementation details.
Rationale =========
The sys module currently contains functions and data that can be put into two major groups:
1. Data and functions that are available in all Python implementations and deal with the general running of a Python VM.
- argv - byteorder - builtin_module_names, path, and modules - copyright, hexversion, version, and version_info - displayhook, __displayhook__ - excepthook, __excepthook__, exc_info, and exc_clear - exec_prefix and prefix - executable - exit - flags, py3kwarning, dont_write_bytecode, and warn_options - getfilesystemencoding - get/setprofile - get/settrace - getwindowsversion - maxint and maxunicode - platform - ps1 and ps2 - stdin, stderr, stdout, __stdin__, __stderr__, __stdout__ - tracebacklimit
2. Data and functions that affect the CPython interpreter.
- get/setrecursionlimit - get/setcheckinterval - _getframe and _current_frame - getrefcount - get/setdlopenflags - settscdumps - api_version - winver - dllhandle - float_info - _compact_freelists - _clear_type_cache - subversion
The second collections of items has been steadily increasing over the years causing clutter in sys. Guido has even said he doesn't recognize some of things in it [#bug-1522]_!
Other implementations have clearly struggled with what to do about the contents of sys they can't implement but must to retain compatibility. For example, Jython's sys module has dud set/getrecursionlimit functions. Moving these items items off to another module would send a clear message about what functions need and need not be implemented.
It has also been proposed that the contents of types module be distributed across the standard library [#types-removal]_; the interpreter module would provide an excellent resting place for internal types like frames and code objects.
Specification =============
A new builtin module named "interpreter" (see `Naming`_) will be added.
The second list of items above will be split into the stdlib as follows:
The interpreter module - get/setrecursionlimit - get/setcheckinterval - _getframe and _current_frame - get/setdlopenflags - settscdumps - api_version - winver - dllhandle - float_info - _clear_type_cache - subversion
The gc module: - getrefcount - _compact_freelists
Transition Plan ===============
Once implemented in 3.x, the interpreter module will be back-ported to 2.6. Py3k warnings will be added the the sys functions it replaces.
Open Issues ===========
What should move? -----------------
dont_write_bytecode ^^^^^^^^^^^^^^^^^^^^
Some believe that the writing of bytecode is an implementation detail and should be moved [#bytecode-issue]_. The counterargument is that all current, complete Python implementations do write some sort of bytecode, so it is valuable to be able to disable it. Also, if it is moved, some wish to put it in the imp module.
Naming ------
The author proposes the name "interpreter" for the new module. "pyvm" has also been suggested [#pyvm-name]_.
References ==========
.. [#bug-1522]
http://bugs.python.org/issue1522
.. [#types-removal]
http://mail.python.org/pipermail/stdlib-sig/2008-April/000172.html
.. [#bytecode-issue]
http://mail.python.org/pipermail/stdlib-sig/2008-April/000217.html
.. [#pyvm-name]
http://mail.python.org/pipermail/python-3000/2007-November/011351.html
Copyright =========
This document has been placed in the public domain.
Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:
-Brett
On Sun, Apr 6, 2008 at 2:45 PM, Benjamin Peterson musiccomposition@gmail.com wrote:
- Data and functions that are available in all Python implementations and
deal with the general running of a Python VM.
- argv
- byteorder
- builtin_module_names, path, and modules
I still think builtin_module_names should be moved. There is no guarantee other VMs have a concept of a built-in module.
- copyright, hexversion, version, and version_info
- displayhook, __displayhook__
- excepthook, __excepthook__, exc_info, and exc_clear
- exec_prefix and prefix
- executable
- exit
- flags, py3kwarning, dont_write_bytecode, and warn_options
- getfilesystemencoding
- get/setprofile
- get/settrace
- getwindowsversion
It seems silly to have Windows-specific stuff like this in the sys module, but I don't know where else they should go.
- maxint and maxunicode
- platform
- ps1 and ps2
- stdin, stderr, stdout, __stdin__, __stderr__, __stdout__
- tracebacklimit
Data and functions that affect the CPython interpreter.
- get/setrecursionlimit
- get/setcheckinterval
- _getframe and _current_frame
- getrefcount
- get/setdlopenflags
- settscdumps
- api_version
- winver
- dllhandle
- float_info
- _compact_freelists
- _clear_type_cache
- subversion
There is some stuff missing from these lists, e.g., meta_path, path_importer_cache, path_hooks, call_tracing, etc. You should do a dir() in sys and fill in the missing attributes into the list.
It's looking good, though!
-Brett
On Sun, Apr 6, 2008 at 10:56 AM, Brett Cannon brett@python.org wrote:
On Sun, Apr 6, 2008 at 2:45 PM, Benjamin Peterson musiccomposition@gmail.com wrote:
- Data and functions that are available in all Python implementations
and
deal with the general running of a Python VM.
- argv
- byteorder
- builtin_module_names, path, and modules
I still think builtin_module_names should be moved. There is no guarantee other VMs have a concept of a built-in module.
Done.
- copyright, hexversion, version, and version_info
- displayhook, __displayhook__
- excepthook, __excepthook__, exc_info, and exc_clear
- exec_prefix and prefix
- executable
- exit
- flags, py3kwarning, dont_write_bytecode, and warn_options
- getfilesystemencoding
- get/setprofile
- get/settrace
- getwindowsversion
It seems silly to have Windows-specific stuff like this in the sys module, but I don't know where else they should go.
Indeed, it does. platform is the only one I can think of.
- maxint and maxunicode
- platform
- ps1 and ps2
- stdin, stderr, stdout, __stdin__, __stderr__, __stdout__
- tracebacklimit
Data and functions that affect the CPython interpreter.
- get/setrecursionlimit
- get/setcheckinterval
- _getframe and _current_frame
- getrefcount
- get/setdlopenflags
- settscdumps
- api_version
- winver
- dllhandle
- float_info
- _compact_freelists
- _clear_type_cache
- subversion
There is some stuff missing from these lists, e.g., meta_path, path_importer_cache, path_hooks, call_tracing, etc. You should do a dir() in sys and fill in the missing attributes into the list.
I will do that. I was looking at the 2.6 docs.
It's looking good, though!
-Brett
[snip]
With more comments taken into consideration:
PEP: XXX Title: Cleaning out sys and the interpreter module Version: $Revision$ Last-Modified: $Date$ Author: Benjamin Peterson Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 4-April-2008 Python-Version: 3.0
Abstract ========
This PEP proposes a new low-level module for CPython-specific interpreter functions in order to clean out the sys module and separate general Python functionality from implementation details.
Rationale =========
The sys module currently contains functions and data that can be put into two major groups:
1. Data and functions that are available in all Python implementations and deal with the general running of a Python VM.
- argv - byteorder - path, path_hooks, meta_path, path_importer_cache, and modules - copyright, hexversion, version, and version_info - displayhook, __displayhook__ - excepthook, __excepthook__, exc_info, and exc_clear - exec_prefix and prefix - executable - exit - flags, py3kwarning, dont_write_bytecode, and warn_options - getfilesystemencoding - get/setprofile - get/settrace, call_tracing - getwindowsversion - maxint and maxunicode - platform - ps1 and ps2 - stdin, stderr, stdout, __stdin__, __stderr__, __stdout__ - tracebacklimit - intern
2. Data and functions that affect the CPython interpreter.
- get/setrecursionlimit - get/setcheckinterval - _getframe and _current_frame - getrefcount - get/setdlopenflags - settscdumps - api_version - winver - dllhandle - float_info - _compact_freelists - _clear_type_cache - subversion - builtin_module_names - callstats
The second collections of items has been steadily increasing over the years causing clutter in sys. Guido has even said he doesn't recognize some of things in it [#bug-1522]_!
Other implementations have clearly struggled with what to do about the contents of sys they can't implement but must to retain compatibility. For example, Jython's sys module has dud set/getrecursionlimit functions. Moving these items items off to another module would send a clear message about what functions need and need not be implemented.
It has also been proposed that the contents of types module be distributed across the standard library [#types-removal]_; the interpreter module would provide an excellent resting place for internal types like frames and code objects.
Specification =============
A new builtin module named "interpreter" (see `Naming`_) will be added.
The second list of items above will be split into the stdlib as follows:
The interpreter module - get/setrecursionlimit - get/setcheckinterval - _getframe and _current_frame - get/setdlopenflags - settscdumps - api_version - winver - dllhandle - float_info - _clear_type_cache - subversion - builtin_module_names - callstats
The gc module: - getrefcount - _compact_freelists
Transition Plan ===============
Once implemented in 3.x, the interpreter module will be back-ported to 2.6. Py3k warnings will be added the the sys functions it replaces.
Open Issues ===========
What should move? -----------------
dont_write_bytecode ^^^^^^^^^^^^^^^^^^^^
Some believe that the writing of bytecode is an implementation detail and should be moved [#bytecode-issue]_. The counterargument is that all current, complete Python implementations do write some sort of bytecode, so it is valuable to be able to disable it. Also, if it is moved, some wish to put it in the imp module.
Move to some to imp? --------------------
It was noted that dont_write_bytecode or maybe builtin_module_names might fit nicely in the imp module.
Naming ------
The author proposes the name "interpreter" for the new module. "pyvm" has also been suggested [#pyvm-name]_.
References ==========
.. [#bug-1522]
http://bugs.python.org/issue1522
.. [#types-removal]
http://mail.python.org/pipermail/stdlib-sig/2008-April/000172.html
.. [#bytecode-issue]
http://mail.python.org/pipermail/stdlib-sig/2008-April/000217.html
.. [#pyvm-name]
http://mail.python.org/pipermail/python-3000/2007-November/011351.html
Copyright =========
This document has been placed in the public domain.
Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:
Some comments:
The rational groups the function in two groups and this grouping makes a lot of sense.
However, your proposal later on doesn't use this rational in any way.
IMHO, it would be better to keep a "sys" module that exposes everything you have in group 1 and an implementation specific module "cpython" that exposes everything you have in group 2.
If an applications needs CPython specific features it would then import cpython. This would make things easy to see in the source code and also raise an exception on platforms where this module is not available, e.g. Jython. The same could be done for other Python implementations, e.g. have a "jython" module for Jython specific things, "ironpython" for IronPython, etc.
Introducing a new long name "interpreter" doesn't really help with resolving the problems you state in the rational and neither does putting APIs from both groups into that new module :-)
BTW: sys.intern is a CPython implementation detail and needs to go into group 2.
Le lundi 07 avril 2008 à 17:09 +0200, M.-A. Lemburg a écrit :
Introducing a new long name "interpreter" doesn't really help with resolving the problems you state in the rational and neither does putting APIs from both groups into that new module :-)
FWIW, I agree with the naming suggestion. "import cpython" makes things explicit.
Antoine Pitrou schrieb:
Le lundi 07 avril 2008 à 17:09 +0200, M.-A. Lemburg a écrit :
Introducing a new long name "interpreter" doesn't really help with resolving the problems you state in the rational and neither does putting APIs from both groups into that new module :-)
FWIW, I agree with the naming suggestion. "import cpython" makes things explicit.
Yeah, +1 for the name. Explicit is better than implicit.
Christian
On Mon, Apr 7, 2008 at 10:52 AM, Christian Heimes christian@cheimes.de wrote:
Antoine Pitrou schrieb:
Le lundi 07 avril 2008 à 17:09 +0200, M.-A. Lemburg a écrit :
Introducing a new long name "interpreter" doesn't really help with resolving the problems you state in the rational and neither does putting APIs from both groups into that new module :-)
FWIW, I agree with the naming suggestion. "import cpython" makes things explicit.
Yeah, +1 for the name. Explicit is better than implicit.
Ok! reposting relevant portions:
Specification =============
A new builtin module named "interpreter" (see `Naming`_) will be added.
The second list of items above will be split into the stdlib as follows:
The interpreter module - get/setrecursionlimit - get/setcheckinterval - _getframe and _current_frame - get/setdlopenflags - settscdumps - api_version - winver - dllhandle - float_info - _clear_type_cache - subversion - builtin_module_names - callstats - intern
The gc module: - getrefcount - _compact_freelists
Naming ------
The author proposes the name "interpreter" for the new module. "pyvm" has also been suggested [#pyvm-name]_. The name "cpython" was also well liked [#cpython-name]_.
Christian _______________________________________________ stdlib-sig mailing list stdlib-sig@python.org http://mail.python.org/mailman/listinfo/stdlib-sig
M.-A. Lemburg wrote:
Some comments:
The rational groups the function in two groups and this grouping makes a lot of sense.
However, your proposal later on doesn't use this rational in any way.
IMHO, it would be better to keep a "sys" module that exposes everything you have in group 1 and an implementation specific module "cpython" that exposes everything you have in group 2.
If an applications needs CPython specific features it would then import cpython. This would make things easy to see in the source code and also raise an exception on platforms where this module is not available, e.g. Jython. The same could be done for other Python implementations, e.g. have a "jython" module for Jython specific things, "ironpython" for IronPython, etc.
I think the idea is nice, but PyPy couldn't really expose a "pypy" module, since that is the namespace the implementation itself is living in. Maybe add a suffix to the implementation name? cpython_vm or so.
Cheers,
Carl Friedrich Bolz
On Mon, Apr 7, 2008 at 3:41 PM, Carl Friedrich Bolz cfbolz@gmx.de wrote:
M.-A. Lemburg wrote:
Some comments:
The rational groups the function in two groups and this grouping makes a lot of sense.
However, your proposal later on doesn't use this rational in any way.
IMHO, it would be better to keep a "sys" module that exposes everything you have in group 1 and an implementation specific module "cpython" that exposes everything you have in group 2.
If an applications needs CPython specific features it would then import cpython. This would make things easy to see in the source code and also raise an exception on platforms where this module is not available, e.g. Jython. The same could be done for other Python implementations, e.g. have a "jython" module for Jython specific things, "ironpython" for IronPython, etc.
I think the idea is nice, but PyPy couldn't really expose a "pypy" module, since that is the namespace the implementation itself is living in. Maybe add a suffix to the implementation name? cpython_vm or so.
You can name your implementation specific module anything you want. We're not forcing anybody else to use our naming scheme! :)
Cheers,
Carl Friedrich Bolz
On 2008-04-07 22:41, Carl Friedrich Bolz wrote:
M.-A. Lemburg wrote:
Some comments:
The rational groups the function in two groups and this grouping makes a lot of sense.
However, your proposal later on doesn't use this rational in any way.
IMHO, it would be better to keep a "sys" module that exposes everything you have in group 1 and an implementation specific module "cpython" that exposes everything you have in group 2.
If an applications needs CPython specific features it would then import cpython. This would make things easy to see in the source code and also raise an exception on platforms where this module is not available, e.g. Jython. The same could be done for other Python implementations, e.g. have a "jython" module for Jython specific things, "ironpython" for IronPython, etc.
I think the idea is nice, but PyPy couldn't really expose a "pypy" module, since that is the namespace the implementation itself is living in. Maybe add a suffix to the implementation name? cpython_vm or so.
Actually, that's not really a problem ... the pypy package will already expose all the PyPy specific APIs. Looks like a time-machine was at work ;-)
Brett Cannon wrote: [snip]
Perhaps we should have sys.interpreter or sys.vm? That way e.g. Jython no longer has to override sys.platform to specify that VM-specific changes are in effect?
FWIW, PyPy has always been using various hacks to figure out what the underlying implementation is. An official way to get at that info would be very nice.
Cheers,
Carl Friedrich Bolz
Brett Cannon schrieb:
And do realize this is a chance to change the API so that if a property-like situation is better than a method call that can be used.
Python doesn't support module properties :/
Sound good to people? If so, who wants to take this on?
I can help with the C coding but I won't have time to champion and implement everything on my own.
Christian
participants (8)
-
Antoine Pitrou -
Benjamin Peterson -
Brett Cannon -
Carl Friedrich Bolz -
Christian Heimes -
M.-A. Lemburg -
Paul Moore -
Steven Bethard